From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-pd0-f182.google.com (mail-pd0-f182.google.com [209.85.192.182]) by dpdk.org (Postfix) with ESMTP id 95F3F5B27 for ; Tue, 27 Jan 2015 08:24:26 +0100 (CET) Received: by mail-pd0-f182.google.com with SMTP id z10so17168795pdj.13 for ; Mon, 26 Jan 2015 23:24:25 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:message-id:date:from:user-agent:mime-version:to :subject:references:in-reply-to:content-type :content-transfer-encoding; bh=GRZdqmbyjnfAXXf18/5jxW3iu1IPit4EYE3NSHEw1C4=; b=bVSq5cSuM9ws22aa6wrlnZp1+NPyWdha1sLFWyifFB4kiD66zPLKH+BAZfRSGN0iHJ yFFegtrzZhzX0elxHRKE6jhZgrtotn/6DUnEUBT+99C7DJEZvGnrcOnIB0Dk+YaJ238I giSAeU/h8yt/AkShZscw97DZUZdsxD2xb70T7G8xu3qlwu2j0kXXLkAXQUZAMPdnh7PP yusjgN6ajorO5vTD+Xs2HHxtXYWX21bSnvGymBZ8fB4nsBdtXu4pmJC2VLASbdNUPSVU TbpM17Yi8OUVt3RArlm/NS57jj24uubQTK6+Btb+d+UIjE0MyNyYIe5XNJQ1nHGyCx9r FLGA== X-Gm-Message-State: ALoCoQlaieOCVpUT8UHk6hq63xiC85MHSKgPc03Ubr3iD+gm2JjUf8kUikk+Lycqp4m0C/mroV9C X-Received: by 10.66.236.1 with SMTP id uq1mr15421831pac.55.1422343465794; Mon, 26 Jan 2015 23:24:25 -0800 (PST) Received: from [10.16.129.101] (napt.igel.co.jp. [219.106.231.132]) by mx.google.com with ESMTPSA id mm9sm566272pbc.76.2015.01.26.23.24.24 (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Mon, 26 Jan 2015 23:24:25 -0800 (PST) Message-ID: <54C73D28.6040704@igel.co.jp> Date: Tue, 27 Jan 2015 16:24:24 +0900 From: Tetsuya Mukawa User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:31.0) Gecko/20100101 Thunderbird/31.4.0 MIME-Version: 1.0 To: "Qiu, Michael" , "dev@dpdk.org" References: <1418106629-22227-2-git-send-email-mukawa@igel.co.j> <1421664027-17971-1-git-send-email-mukawa@igel.co.jp> <533710CFB86FA344BFBF2D6802E60286CBA74A@SHSMSX101.ccr.corp.intel.com> <54C71BB7.8000509@igel.co.jp> <533710CFB86FA344BFBF2D6802E60286CBAA6D@SHSMSX101.ccr.corp.intel.com> In-Reply-To: <533710CFB86FA344BFBF2D6802E60286CBAA6D@SHSMSX101.ccr.corp.intel.com> Content-Type: text/plain; charset=windows-1252 Content-Transfer-Encoding: 7bit Subject: Re: [dpdk-dev] [PATCH v4 00/11] Port Hotplug Framework X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: patches and discussions about DPDK List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 27 Jan 2015 07:24:27 -0000 On 2015/01/27 14:50, Qiu, Michael wrote: > On 1/27/2015 1:02 PM, Tetsuya Mukawa wrote: >> On 2015/01/27 12:00, Qiu, Michael wrote: >>> On 1/19/2015 6:42 PM, Tetsuya Mukawa wrote: >>>> This patch series adds a dynamic port hotplug framework to DPDK. >>>> With the patches, DPDK apps can attach or detach ports at runtime. >>>> >>>> The basic concept of the port hotplug is like followings. >>>> - DPDK apps must have responsibility to manage ports. >>>> DPDK apps only know which ports are attached or detached at the moment. >>>> The port hotplug framework is implemented to allow DPDK apps to manage ports. >>>> For example, when DPDK apps call port attach function, attached port number >>>> will be returned. Also DPDK apps can detach port by port number. >>>> - Kernel support is needed for attaching or detaching physical device ports. >>>> To attach new device, the device will be recognized by kernel at first and >>>> controlled by kernel driver. Then user can bind the device to igb_uio >>> Here does it really need native kernel driver here? As it will be >>> controlled by igb_uio. >>> I think even if the device has no kernel driver is also OK. >> Thanks for correcting. Yes, it should be. >> How about following. >> >> - Kernel support is needed for attaching or detaching physical device ports. >> To attach a new device, the device will be recognized by kernel PCI >> hotplug feature at first. > No, here should not explain as "kernel PCI hotplug feature" which is > stand for removing or adding a PCI device from system level, it is > devices related not driver. > > What about: > > - Kernel support is needed for attaching or detaching physical device > ports. To attach a new physical device port, the device will be > recognized by userspace directly I/O framework in kernel at first. Thanks. I will replace like above. Thanks, Tetsuya > Thanks, > Michael >> Then user can bind the device to igb_uio. >> >>> Also I have finished initial patch of passthrough driver flag in >>> "struct rte_pci_device" >>> >>> I will send to you after I do some basic test on that, then I will send to >>> you, and you can give some comments on that. >> I appreciate for your implementing. >> >> Thanks, >> Tetsuya >> >> >>> Thanks, >>> Michael >>> >>>> by 'dpdk_nic_bind.py'. Finally, DPDK apps can call the port hotplug >>>> functions to attach ports. >>>> For detaching, steps are vice versa. >>>> - Before detach ports, ports must be stopped and closed. >>>> DPDK application must call rte_eth_dev_stop() and rte_eth_dev_close() before >>>> detaching ports. These function will call finalization codes of PMDs. >>>> But so far, no PMD frees all resources allocated by initialization. >>>> It means PMDs are needed to be fixed to support the port hotplug. >>>> 'RTE_PCI_DRV_DETACHABLE' is a new flag indicating a PMD supports detaching. >>>> Without this flag, detaching will be failed. >>>> - Mustn't affect legacy DPDK apps. >>>> No DPDK EAL behavior is changed, if the port hotplug functions are't called. >>>> So all legacy DPDK apps can still work without modifications. >>>> >>>> And a few limitations. >>>> - The port hotplug functions are not thread safe. >>>> DPDK apps should handle it. >>>> - Only support Linux and igb_uio so far. >>>> BSD and VFIO is not supported. I will send VFIO patches at least, but I don't >>>> have a plan to submit BSD patch so far. >>>> >>>> >>>> Here is port hotplug APIs. >>>> ------------------------------------------------------------------------------- >>>> /** >>>> * Attach a new device. >>>> * >>>> * @param devargs >>>> * A pointer to a strings array describing the new device >>>> * to be attached. The strings should be a pci address like >>>> * '0000:01:00.0' or virtual device name like 'eth_pcap0'. >>>> * @param port_id >>>> * A pointer to a port identifier actually attached. >>>> * @return >>>> * 0 on success and port_id is filled, negative on error >>>> */ >>>> int rte_eal_dev_attach(const char *devargs, uint8_t *port_id); >>>> >>>> /** >>>> * Detach a device. >>>> * >>>> * @param port_id >>>> * The port identifier of the device to detach. >>>> * @param addr >>>> * A pointer to a device name actually detached. >>>> * @return >>>> * 0 on success and devname is filled, negative on error >>>> */ >>>> int rte_eal_dev_detach(uint8_t port_id, char *devname); >>>> ------------------------------------------------------------------------------- >>>> >>>> This patch series are for DPDK EAL. To use port hotplug function by DPDK apps, >>>> each PMD should be fixed to support 'RTE_PCI_DRV_DETACHABLE' flag. Please check >>>> a patch for pcap PMD. >>>> >>>> Also please check testpmd patch. It will show you how to fix your legacy >>>> applications to support port hotplug feature. >>>> >>>> >>>> PATCH v4 changes >>>> - Merge patches to review easier. >>>> - Fix indent of 'if' statement. >>>> - Fix calculation method of eal_compare_pci_addr(). >>>> - Fix header file declaration. >>>> - Add header file to determine if hotplug can be enabled. >>>> (Thanks to Qiu, Michael) >>>> - Use braces with 'for' loop. >>>> - Add paramerter checking. >>>> - Fix sanity check code >>>> - Fix comments of rte_eth_dev_type. >>>> - Change function names. >>>> (Thanks to Iremonger, Bernard) >>>> >>>> PATCH v3 changes: >>>> - Fix enum definition used in rte_ethdev.c. >>>> (Thanks to Zhang, Helin) >>>> >>>> PATCH v2 changes: >>>> - Replace rte_eal_dev_attach_pdev(), rte_eal_dev_detach_pdev, >>>> rte_eal_dev_attach_vdev() and rte_eal_dev_detach_vdev() to >>>> rte_eal_dev_attach() and rte_eal_dev_detach(). >>>> - Add parameter values checking. >>>> - Refashion a few functions. >>>> (Thanks to Iremonger, Bernard) >>>> >>>> PATCH v1 Changes: >>>> - Fix error checking code of librte_eth APIs. >>>> - Fix issue that port from pcap PMD cannot be detached correctly. >>>> - Fix issue that testpmd could hang after forwarding, if attaching and detaching >>>> is repeatedly. >>>> - Fix if-condition of rte_eth_dev_get_port_by_addr(). >>>> (Thanks to Mark Enright) >>>> >>>> RFC PATCH v2 Changes: >>>> - remove 'rte_eth_dev_validate_port()', and cleanup codes. >>>> >>>> >>>> Tetsuya Mukawa (11): >>>> eal/pci,ethdev: Remove assumption that port will not be detached >>>> eal/pci: Consolidate pci address comparison APIs >>>> ethdev: Add rte_eth_dev_free to free specified device >>>> eal,ethdev: Add a function and function pointers to close ether device >>>> ethdev: Add functions that will be used by port hotplug functions >>>> eal/linux/pci: Add functions for unmapping igb_uio resources >>>> eal/pci: Add a function to remove the entry of devargs list >>>> eal/pci: Cleanup pci driver initialization code >>>> ethdev: Add one dev_type paramerter to rte_eth_dev_allocate >>>> eal/pci: Add rte_eal_dev_attach/detach() functions >>>> eal: Enable port hotplug framework in Linux >>>> >>>> app/test/virtual_pmd.c | 2 +- >>>> config/common_linuxapp | 5 + >>>> lib/librte_eal/bsdapp/eal/eal_pci.c | 16 +- >>>> lib/librte_eal/common/Makefile | 1 + >>>> lib/librte_eal/common/eal_common_dev.c | 273 +++++++++++++++ >>>> lib/librte_eal/common/eal_common_devargs.c | 57 ++++ >>>> lib/librte_eal/common/eal_common_pci.c | 92 ++++- >>>> lib/librte_eal/common/eal_private.h | 36 ++ >>>> lib/librte_eal/common/include/rte_dev.h | 33 ++ >>>> lib/librte_eal/common/include/rte_dev_hotplug.h | 44 +++ >>>> lib/librte_eal/common/include/rte_devargs.h | 18 + >>>> lib/librte_eal/common/include/rte_pci.h | 76 +++++ >>>> lib/librte_eal/linuxapp/eal/Makefile | 1 + >>>> lib/librte_eal/linuxapp/eal/eal_pci.c | 136 ++++++-- >>>> lib/librte_eal/linuxapp/eal/eal_pci_init.h | 8 + >>>> lib/librte_eal/linuxapp/eal/eal_pci_uio.c | 67 +++- >>>> lib/librte_ether/rte_ethdev.c | 424 ++++++++++++++++++------ >>>> lib/librte_ether/rte_ethdev.h | 145 +++++++- >>>> lib/librte_pmd_af_packet/rte_eth_af_packet.c | 2 +- >>>> lib/librte_pmd_bond/rte_eth_bond_api.c | 2 +- >>>> lib/librte_pmd_pcap/rte_eth_pcap.c | 2 +- >>>> lib/librte_pmd_ring/rte_eth_ring.c | 2 +- >>>> lib/librte_pmd_xenvirt/rte_eth_xenvirt.c | 2 +- >>>> 23 files changed, 1283 insertions(+), 161 deletions(-) >>>> create mode 100644 lib/librte_eal/common/include/rte_dev_hotplug.h >>>> >>