From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mga14.intel.com (mga14.intel.com [192.55.52.115]) by dpdk.org (Postfix) with ESMTP id B73415B19 for ; Tue, 27 Jan 2015 06:50:57 +0100 (CET) Received: from fmsmga003.fm.intel.com ([10.253.24.29]) by fmsmga103.fm.intel.com with ESMTP; 26 Jan 2015 21:44:51 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="4.97,862,1389772800"; d="scan'208";a="445599154" Received: from kmsmsx151.gar.corp.intel.com ([172.21.73.86]) by FMSMGA003.fm.intel.com with ESMTP; 26 Jan 2015 21:37:14 -0800 Received: from shsmsx152.ccr.corp.intel.com (10.239.6.52) by KMSMSX151.gar.corp.intel.com (172.21.73.86) with Microsoft SMTP Server (TLS) id 14.3.195.1; Tue, 27 Jan 2015 13:50:51 +0800 Received: from shsmsx101.ccr.corp.intel.com ([169.254.1.64]) by SHSMSX152.ccr.corp.intel.com ([169.254.6.129]) with mapi id 14.03.0195.001; Tue, 27 Jan 2015 13:50:50 +0800 From: "Qiu, Michael" To: Tetsuya Mukawa , "dev@dpdk.org" Thread-Topic: [dpdk-dev] [PATCH v4 00/11] Port Hotplug Framework Thread-Index: AQHQM9SRkPOzBimEUESHcClHwtkWaA== Date: Tue, 27 Jan 2015 05:50:48 +0000 Message-ID: <533710CFB86FA344BFBF2D6802E60286CBAA6D@SHSMSX101.ccr.corp.intel.com> References: <1418106629-22227-2-git-send-email-mukawa@igel.co.j> <1421664027-17971-1-git-send-email-mukawa@igel.co.jp> <533710CFB86FA344BFBF2D6802E60286CBA74A@SHSMSX101.ccr.corp.intel.com> <54C71BB7.8000509@igel.co.jp> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-originating-ip: [10.239.127.40] Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Subject: Re: [dpdk-dev] [PATCH v4 00/11] Port Hotplug Framework X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: patches and discussions about DPDK List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 27 Jan 2015 05:50:58 -0000 On 1/27/2015 1:02 PM, Tetsuya Mukawa wrote:=0A= > On 2015/01/27 12:00, Qiu, Michael wrote:=0A= >> On 1/19/2015 6:42 PM, Tetsuya Mukawa wrote:=0A= >>> This patch series adds a dynamic port hotplug framework to DPDK.=0A= >>> With the patches, DPDK apps can attach or detach ports at runtime.=0A= >>>=0A= >>> The basic concept of the port hotplug is like followings.=0A= >>> - DPDK apps must have responsibility to manage ports.=0A= >>> DPDK apps only know which ports are attached or detached at the momen= t.=0A= >>> The port hotplug framework is implemented to allow DPDK apps to manag= e ports.=0A= >>> For example, when DPDK apps call port attach function, attached port = number=0A= >>> will be returned. Also DPDK apps can detach port by port number.=0A= >>> - Kernel support is needed for attaching or detaching physical device p= orts.=0A= >>> To attach new device, the device will be recognized by kernel at firs= t and=0A= >>> controlled by kernel driver. Then user can bind the device to igb_uio= =0A= >> Here does it really need native kernel driver here? As it will be=0A= >> controlled by igb_uio.=0A= >> I think even if the device has no kernel driver is also OK.=0A= > Thanks for correcting. Yes, it should be.=0A= > How about following.=0A= >=0A= > - Kernel support is needed for attaching or detaching physical device por= ts.=0A= > To attach a new device, the device will be recognized by kernel PCI=0A= > hotplug feature at first.=0A= =0A= No, here should not explain as "kernel PCI hotplug feature" which is=0A= stand for removing or adding a PCI device from system level, it is=0A= devices related not driver.=0A= =0A= What about:=0A= =0A= - Kernel support is needed for attaching or detaching physical device=0A= ports. To attach a new physical device port, the device will be=0A= recognized by userspace directly I/O framework in kernel at first.=0A= =0A= Thanks,=0A= Michael=0A= > Then user can bind the device to igb_uio.=0A= >=0A= >> Also I have finished initial patch of passthrough driver flag in=0A= >> "struct rte_pci_device"=0A= >>=0A= >> I will send to you after I do some basic test on that, then I will send = to=0A= >> you, and you can give some comments on that.=0A= > I appreciate for your implementing.=0A= >=0A= > Thanks,=0A= > Tetsuya=0A= >=0A= >=0A= >> Thanks,=0A= >> Michael=0A= >>=0A= >>> by 'dpdk_nic_bind.py'. Finally, DPDK apps can call the port hotplug= =0A= >>> functions to attach ports.=0A= >>> For detaching, steps are vice versa.=0A= >>> - Before detach ports, ports must be stopped and closed.=0A= >>> DPDK application must call rte_eth_dev_stop() and rte_eth_dev_close()= before=0A= >>> detaching ports. These function will call finalization codes of PMDs.= =0A= >>> But so far, no PMD frees all resources allocated by initialization.= =0A= >>> It means PMDs are needed to be fixed to support the port hotplug.=0A= >>> 'RTE_PCI_DRV_DETACHABLE' is a new flag indicating a PMD supports deta= ching.=0A= >>> Without this flag, detaching will be failed.=0A= >>> - Mustn't affect legacy DPDK apps.=0A= >>> No DPDK EAL behavior is changed, if the port hotplug functions are't = called.=0A= >>> So all legacy DPDK apps can still work without modifications.=0A= >>>=0A= >>> And a few limitations.=0A= >>> - The port hotplug functions are not thread safe.=0A= >>> DPDK apps should handle it.=0A= >>> - Only support Linux and igb_uio so far.=0A= >>> BSD and VFIO is not supported. I will send VFIO patches at least, but= I don't=0A= >>> have a plan to submit BSD patch so far.=0A= >>>=0A= >>>=0A= >>> Here is port hotplug APIs.=0A= >>> -----------------------------------------------------------------------= --------=0A= >>> /**=0A= >>> * Attach a new device.=0A= >>> *=0A= >>> * @param devargs=0A= >>> * A pointer to a strings array describing the new device=0A= >>> * to be attached. The strings should be a pci address like=0A= >>> * '0000:01:00.0' or virtual device name like 'eth_pcap0'.=0A= >>> * @param port_id=0A= >>> * A pointer to a port identifier actually attached.=0A= >>> * @return=0A= >>> * 0 on success and port_id is filled, negative on error=0A= >>> */=0A= >>> int rte_eal_dev_attach(const char *devargs, uint8_t *port_id);=0A= >>>=0A= >>> /**=0A= >>> * Detach a device.=0A= >>> *=0A= >>> * @param port_id=0A= >>> * The port identifier of the device to detach.=0A= >>> * @param addr=0A= >>> * A pointer to a device name actually detached.=0A= >>> * @return=0A= >>> * 0 on success and devname is filled, negative on error=0A= >>> */=0A= >>> int rte_eal_dev_detach(uint8_t port_id, char *devname);=0A= >>> -----------------------------------------------------------------------= --------=0A= >>>=0A= >>> This patch series are for DPDK EAL. To use port hotplug function by DPD= K apps,=0A= >>> each PMD should be fixed to support 'RTE_PCI_DRV_DETACHABLE' flag. Plea= se check=0A= >>> a patch for pcap PMD.=0A= >>>=0A= >>> Also please check testpmd patch. It will show you how to fix your legac= y=0A= >>> applications to support port hotplug feature.=0A= >>>=0A= >>>=0A= >>> PATCH v4 changes=0A= >>> - Merge patches to review easier.=0A= >>> - Fix indent of 'if' statement.=0A= >>> - Fix calculation method of eal_compare_pci_addr().=0A= >>> - Fix header file declaration.=0A= >>> - Add header file to determine if hotplug can be enabled.=0A= >>> (Thanks to Qiu, Michael)=0A= >>> - Use braces with 'for' loop.=0A= >>> - Add paramerter checking.=0A= >>> - Fix sanity check code=0A= >>> - Fix comments of rte_eth_dev_type.=0A= >>> - Change function names.=0A= >>> (Thanks to Iremonger, Bernard)=0A= >>>=0A= >>> PATCH v3 changes:=0A= >>> - Fix enum definition used in rte_ethdev.c.=0A= >>> (Thanks to Zhang, Helin)=0A= >>>=0A= >>> PATCH v2 changes:=0A= >>> - Replace rte_eal_dev_attach_pdev(), rte_eal_dev_detach_pdev,=0A= >>> rte_eal_dev_attach_vdev() and rte_eal_dev_detach_vdev() to=0A= >>> rte_eal_dev_attach() and rte_eal_dev_detach().=0A= >>> - Add parameter values checking.=0A= >>> - Refashion a few functions.=0A= >>> (Thanks to Iremonger, Bernard)=0A= >>>=0A= >>> PATCH v1 Changes:=0A= >>> - Fix error checking code of librte_eth APIs.=0A= >>> - Fix issue that port from pcap PMD cannot be detached correctly.=0A= >>> - Fix issue that testpmd could hang after forwarding, if attaching and= detaching=0A= >>> is repeatedly.=0A= >>> - Fix if-condition of rte_eth_dev_get_port_by_addr().=0A= >>> (Thanks to Mark Enright)=0A= >>>=0A= >>> RFC PATCH v2 Changes:=0A= >>> - remove 'rte_eth_dev_validate_port()', and cleanup codes.=0A= >>>=0A= >>>=0A= >>> Tetsuya Mukawa (11):=0A= >>> eal/pci,ethdev: Remove assumption that port will not be detached=0A= >>> eal/pci: Consolidate pci address comparison APIs=0A= >>> ethdev: Add rte_eth_dev_free to free specified device=0A= >>> eal,ethdev: Add a function and function pointers to close ether devic= e=0A= >>> ethdev: Add functions that will be used by port hotplug functions=0A= >>> eal/linux/pci: Add functions for unmapping igb_uio resources=0A= >>> eal/pci: Add a function to remove the entry of devargs list=0A= >>> eal/pci: Cleanup pci driver initialization code=0A= >>> ethdev: Add one dev_type paramerter to rte_eth_dev_allocate=0A= >>> eal/pci: Add rte_eal_dev_attach/detach() functions=0A= >>> eal: Enable port hotplug framework in Linux=0A= >>>=0A= >>> app/test/virtual_pmd.c | 2 +-=0A= >>> config/common_linuxapp | 5 +=0A= >>> lib/librte_eal/bsdapp/eal/eal_pci.c | 16 +-=0A= >>> lib/librte_eal/common/Makefile | 1 +=0A= >>> lib/librte_eal/common/eal_common_dev.c | 273 +++++++++++++++= =0A= >>> lib/librte_eal/common/eal_common_devargs.c | 57 ++++=0A= >>> lib/librte_eal/common/eal_common_pci.c | 92 ++++-=0A= >>> lib/librte_eal/common/eal_private.h | 36 ++=0A= >>> lib/librte_eal/common/include/rte_dev.h | 33 ++=0A= >>> lib/librte_eal/common/include/rte_dev_hotplug.h | 44 +++=0A= >>> lib/librte_eal/common/include/rte_devargs.h | 18 +=0A= >>> lib/librte_eal/common/include/rte_pci.h | 76 +++++=0A= >>> lib/librte_eal/linuxapp/eal/Makefile | 1 +=0A= >>> lib/librte_eal/linuxapp/eal/eal_pci.c | 136 ++++++--=0A= >>> lib/librte_eal/linuxapp/eal/eal_pci_init.h | 8 +=0A= >>> lib/librte_eal/linuxapp/eal/eal_pci_uio.c | 67 +++-=0A= >>> lib/librte_ether/rte_ethdev.c | 424 ++++++++++++++++= ++------=0A= >>> lib/librte_ether/rte_ethdev.h | 145 +++++++-=0A= >>> lib/librte_pmd_af_packet/rte_eth_af_packet.c | 2 +-=0A= >>> lib/librte_pmd_bond/rte_eth_bond_api.c | 2 +-=0A= >>> lib/librte_pmd_pcap/rte_eth_pcap.c | 2 +-=0A= >>> lib/librte_pmd_ring/rte_eth_ring.c | 2 +-=0A= >>> lib/librte_pmd_xenvirt/rte_eth_xenvirt.c | 2 +-=0A= >>> 23 files changed, 1283 insertions(+), 161 deletions(-)=0A= >>> create mode 100644 lib/librte_eal/common/include/rte_dev_hotplug.h=0A= >>>=0A= >=0A= >=0A= =0A=