From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mga03.intel.com (mga03.intel.com [134.134.136.65]) by dpdk.org (Postfix) with ESMTP id 86D9E1BB07 for ; Wed, 11 Apr 2018 13:38:01 +0200 (CEST) X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from fmsmga001.fm.intel.com ([10.253.24.23]) by orsmga103.jf.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 11 Apr 2018 04:37:58 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.48,435,1517904000"; d="scan'208";a="46037009" Received: from jguo15x-mobl3.ccr.corp.intel.com (HELO [10.254.215.165]) ([10.254.215.165]) by fmsmga001.fm.intel.com with ESMTP; 11 Apr 2018 04:37:55 -0700 To: "Ananyev, Konstantin" , "stephen@networkplumber.org" , "Richardson, Bruce" , "Yigit, Ferruh" , "gaetan.rivet@6wind.com" , "Wu, Jingjing" , "thomas@monjalon.net" , "motih@mellanox.com" , "Van Haaren, Harry" , "Tan, Jianfeng" References: <1522779443-1932-6-git-send-email-jia.guo@intel.com> <1523012217-31146-1-git-send-email-jia.guo@intel.com> <1523012217-31146-2-git-send-email-jia.guo@intel.com> <2601191342CEEE43887BDE71AB977258AE91269E@IRSMSX102.ger.corp.intel.com> Cc: "jblunck@infradead.org" , "shreyansh.jain@nxp.com" , "dev@dpdk.org" , "Zhang, Helin" From: "Guo, Jia" Message-ID: <8afacb6c-8646-49e4-b076-4212be02b747@intel.com> Date: Wed, 11 Apr 2018 19:37:54 +0800 User-Agent: Mozilla/5.0 (Windows NT 6.3; WOW64; rv:45.0) Gecko/20100101 Thunderbird/45.4.0 MIME-Version: 1.0 In-Reply-To: <2601191342CEEE43887BDE71AB977258AE91269E@IRSMSX102.ger.corp.intel.com> Content-Type: text/plain; charset=windows-1252; format=flowed Content-Transfer-Encoding: 7bit Subject: Re: [dpdk-dev] [PATCH V19 1/4] bus/pci: introduce device hot unplug handle X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 11 Apr 2018 11:38:02 -0000 On 4/10/2018 1:47 AM, Ananyev, Konstantin wrote: > >> -----Original Message----- >> From: Guo, Jia >> Sent: Friday, April 6, 2018 11:57 AM >> To: stephen@networkplumber.org; Richardson, Bruce ; Yigit, Ferruh ; Ananyev, >> Konstantin ; gaetan.rivet@6wind.com; Wu, Jingjing ; thomas@monjalon.net; >> motih@mellanox.com; Van Haaren, Harry ; Tan, Jianfeng >> Cc: jblunck@infradead.org; shreyansh.jain@nxp.com; dev@dpdk.org; Guo, Jia ; Zhang, Helin >> Subject: [PATCH V19 1/4] bus/pci: introduce device hot unplug handle >> >> As of device hot unplug, we need some preparatory measures so that we will >> not encounter memory fault after device be plug out of the system, >> and also let we could recover the running data path but not been break. >> This allows the buses to handle device hot unplug event. >> The patch only enable the ops in pci bus, when handle device hot unplug >> event, remap a dummy memory to avoid bus read/write error. >> Other buses could accordingly implement this ops specific by themselves. >> >> Signed-off-by: Jeff Guo >> --- >> v19->v18: >> fix some typo and squeeze patch >> --- >> drivers/bus/pci/pci_common.c | 42 +++++++++++++++++++++++++++++++++ >> drivers/bus/pci/pci_common_uio.c | 32 +++++++++++++++++++++++++ >> drivers/bus/pci/private.h | 12 ++++++++++ >> lib/librte_eal/common/include/rte_bus.h | 15 ++++++++++++ >> 4 files changed, 101 insertions(+) >> >> diff --git a/drivers/bus/pci/pci_common.c b/drivers/bus/pci/pci_common.c >> index 2a00f36..09192ed 100644 >> --- a/drivers/bus/pci/pci_common.c >> +++ b/drivers/bus/pci/pci_common.c >> @@ -474,6 +474,47 @@ pci_find_device(const struct rte_device *start, rte_dev_cmp_t cmp, >> } >> >> static int >> +pci_handle_hot_unplug(struct rte_device *dev) >> +{ >> + struct rte_pci_device *pdev; >> + int ret; >> + >> + if (dev == NULL) >> + return -EINVAL; >> + >> + pdev = RTE_DEV_TO_PCI(dev); >> + >> + /* remap resources for devices */ >> + switch (pdev->kdrv) { >> + case RTE_KDRV_VFIO: >> +#ifdef VFIO_PRESENT >> + /* TODO */ >> +#endif >> + break; >> + case RTE_KDRV_IGB_UIO: >> + case RTE_KDRV_UIO_GENERIC: >> + if (rte_eal_using_phys_addrs()) { >> + /* map resources for devices that use uio */ >> + ret = pci_uio_remap_resource(pdev); >> + } >> + break; >> + case RTE_KDRV_NIC_UIO: >> + ret = pci_uio_remap_resource(pdev); >> + break; >> + default: >> + RTE_LOG(DEBUG, EAL, >> + " Not managed by a supported kernel driver, skipped\n"); >> + ret = -1; >> + break; >> + } >> + >> + if (ret != 0) >> + RTE_LOG(ERR, EAL, "failed to handle hot unplug of %s", >> + pdev->name); >> + return ret; >> +} >> + >> +static int >> pci_plug(struct rte_device *dev) >> { >> return pci_probe_all_drivers(RTE_DEV_TO_PCI(dev)); >> @@ -503,6 +544,7 @@ struct rte_pci_bus rte_pci_bus = { >> .unplug = pci_unplug, >> .parse = pci_parse, >> .get_iommu_class = rte_pci_get_iommu_class, >> + .handle_hot_unplug = pci_handle_hot_unplug, >> }, >> .device_list = TAILQ_HEAD_INITIALIZER(rte_pci_bus.device_list), >> .driver_list = TAILQ_HEAD_INITIALIZER(rte_pci_bus.driver_list), >> diff --git a/drivers/bus/pci/pci_common_uio.c b/drivers/bus/pci/pci_common_uio.c >> index 54bc20b..31a4094 100644 >> --- a/drivers/bus/pci/pci_common_uio.c >> +++ b/drivers/bus/pci/pci_common_uio.c >> @@ -146,6 +146,38 @@ pci_uio_unmap(struct mapped_pci_resource *uio_res) >> } >> } >> >> +/* remap the PCI resource of a PCI device in private virtual memory */ >> +int >> +pci_uio_remap_resource(struct rte_pci_device *dev) >> +{ >> + int i; >> + void *map_address; >> + >> + if (dev == NULL) >> + return -1; >> + >> + /* Remap all BARs */ >> + for (i = 0; i != PCI_MAX_RESOURCE; i++) { >> + /* skip empty BAR */ >> + if (dev->mem_resource[i].phys_addr == 0) >> + continue; >> + pci_unmap_resource(dev->mem_resource[i].addr, >> + (size_t)dev->mem_resource[i].len); >> + map_address = pci_map_resource( >> + dev->mem_resource[i].addr, -1, 0, >> + (size_t)dev->mem_resource[i].len, >> + MAP_ANONYMOUS | MAP_FIXED); > Wouldn't it be possible to use mremap() here? > To do munmap/mmap in one go? > since i don't thinks is need to add a new function to remap resource, so just use the exist map function with specif flag to handler it. what i want to say is if my implement have problem or what is the mremap's advantage for that, if so please let me know, thank. >> + if (map_address == MAP_FAILED) { >> + RTE_LOG(ERR, EAL, >> + "Cannot remap resource for device %s\n", >> + dev->name); >> + return -1; >> + } >> + } >> + >> + return 0; >> +} >> +