From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mga14.intel.com (mga14.intel.com [192.55.52.115]) by dpdk.org (Postfix) with ESMTP id 0C21A1BDD1 for ; Tue, 3 Jul 2018 14:12:16 +0200 (CEST) X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from fmsmga002.fm.intel.com ([10.253.24.26]) by fmsmga103.fm.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 03 Jul 2018 05:12:15 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.51,303,1526367600"; d="scan'208";a="63768522" Received: from fyigit-mobl.ger.corp.intel.com (HELO [10.237.221.83]) ([10.237.221.83]) by fmsmga002.fm.intel.com with ESMTP; 03 Jul 2018 05:12:12 -0700 To: Jeff Guo , stephen@networkplumber.org, bruce.richardson@intel.com, konstantin.ananyev@intel.com, gaetan.rivet@6wind.com, jingjing.wu@intel.com, thomas@monjalon.net, motih@mellanox.com, matan@mellanox.com, harry.van.haaren@intel.com, qi.z.zhang@intel.com, shaopeng.he@intel.com, bernard.iremonger@intel.com Cc: jblunck@infradead.org, shreyansh.jain@nxp.com, dev@dpdk.org, helin.zhang@intel.com References: <1498711073-42917-1-git-send-email-jia.guo@intel.com> <1530268248-7328-1-git-send-email-jia.guo@intel.com> <1530268248-7328-8-git-send-email-jia.guo@intel.com> From: Ferruh Yigit Openpgp: preference=signencrypt Autocrypt: addr=ferruh.yigit@intel.com; prefer-encrypt=mutual; keydata= xsFNBFXZCFABEADCujshBOAaqPZpwShdkzkyGpJ15lmxiSr3jVMqOtQS/sB3FYLT0/d3+bvy qbL9YnlbPyRvZfnP3pXiKwkRoR1RJwEo2BOf6hxdzTmLRtGtwWzI9MwrUPj6n/ldiD58VAGQ +iR1I/z9UBUN/ZMksElA2D7Jgg7vZ78iKwNnd+vLBD6I61kVrZ45Vjo3r+pPOByUBXOUlxp9 GWEKKIrJ4eogqkVNSixN16VYK7xR+5OUkBYUO+sE6etSxCr7BahMPKxH+XPlZZjKrxciaWQb +dElz3Ab4Opl+ZT/bK2huX+W+NJBEBVzjTkhjSTjcyRdxvS1gwWRuXqAml/sh+KQjPV1PPHF YK5LcqLkle+OKTCa82OvUb7cr+ALxATIZXQkgmn+zFT8UzSS3aiBBohg3BtbTIWy51jNlYdy ezUZ4UxKSsFuUTPt+JjHQBvF7WKbmNGS3fCid5Iag4tWOfZoqiCNzxApkVugltxoc6rG2TyX CmI2rP0mQ0GOsGXA3+3c1MCdQFzdIn/5tLBZyKy4F54UFo35eOX8/g7OaE+xrgY/4bZjpxC1 1pd66AAtKb3aNXpHvIfkVV6NYloo52H+FUE5ZDPNCGD0/btFGPWmWRmkPybzColTy7fmPaGz cBcEEqHK4T0aY4UJmE7Ylvg255Kz7s6wGZe6IR3N0cKNv++O7QARAQABzSVGZXJydWggWWln aXQgPGZlcnJ1aC55aWdpdEBpbnRlbC5jb20+wsF+BBMBAgAoAhsDBgsJCAcDAgYVCAIJCgsE FgIDAQIeAQIXgAUCWZR3VQUJB33WBQAKCRD5M+tD3xNhH6DWEACVhEb8q1epPwZrUDoxzu7E TS1b8tmabOmnjXZRs6+EXgUVHkp2xxkCfDmL3pa5bC0G/74aJnWjNsdvE05V1cb4YK4kRQ62 FwDQ+hlrFrwFB3PtDZk1tpkzCRHvJgnIil+0MuEh32Y57ig6hy8yO8ql7Lohyrnpfk/nNpm4 jQGEF5qEeHcEFe1AZQlPHN/STno8NZSz2nl0b2cw+cujN1krmvB52Ah/2KugQ6pprVyrGrzB c34ZQO9OsmSjJlETCZk6EZzuhfe16iqBFbOSadi9sPcJRwaUQBid+xdFWl7GQ8qC3zNPibSF HmU43yBZUqJDZlhIcl6/cFpOSjv2sDWdtjEXTDn5y/0FsuY0mFE78ItC4kCTIVk17VZoywcd fmbbnwOSWzDq7hiUYuQGkIudJw5k/A1CMsyLkoUEGN3sLfsw6KASgS4XrrmPO4UVr3mH5bP1 yC7i1OVNpzvOxtahmzm481ID8sk72GC2RktTOHb0cX+qdoiMMfYgo3wRRDYCBt6YoGYUxF1p msjocXyqToKhhnFbXLaZlVfnQ9i2i8jsj9SKig+ewC2p3lkPj6ncye9q95bzhmUeJO6sFhJg Hiz6syOMg8yCcq60j07airybAuHIDNFWk0gaWAmtHZxLObZx2PVn2nv9kLYGohFekw0AOsIW ta++5m48dnCoAc7BTQRX1ky+ARAApzQNvXvE2q1LAS+Z+ni2R13Bb1cDS1ZYq1jgpR13+OKN ipzd8MPngRJilXxBaPTErhgzR0vGcNTYhjGMSyFIHVOoBq1VbP1a0Fi/NqWzJOowo/fDfgVy K4vuitc/gCJs+2se4hdZA4EQJxVlNM51lgYDNpjPGIA43MX15OLAip73+ho6NPBMuc5qse3X pAClNhBKfENRCWN428pi3WVkT+ABRTE0taxjJNP7bb+9TQYNRqGwnGzX5/XISv44asWIQCaq vOkXSUJLd//cdVNTqtL1wreCVVR5pMXj7VIrlk07fmmJVALCmGbFr53BMb8O+8dgK2A5mitM n44d+8KdJWOwziRxcaMk/LclmZS3Iv1TERtiWt98Y9AjeAtcgYPkA3ld0BcUKONogP8pHVz1 Ed3s5rDQ91yr1S0wuAzW91fxGUO4wY+uPmxCtFVuBgd9VT9NAKTUL0qHM7CDgCnZPe0TW6Zj 8OqtdCCyAfvU9cW5xWM7Icxhde6AtPxhDSBwE8fL2ZmrDmaA4jmUKXp3i4JxRPSX84S08b+s DWXHPxy10UFU5A7EK/BEbZAKBwn9ROfm+WK+6X5xOGLoRE++OqNuUudxC1GDyLOPaqCbBCS9 +P6HsTHzxsjyJa27n4jcrcuY3P9TEcFJYSZSeSDh8mVGvugi0exnSJrrBZDyVCcAEQEAAcLB ZQQYAQIADwIbDAUCWZR1ZwUJA59cIQAKCRD5M+tD3xNhH5b+D/9XG44Ci6STdcA5RO/ur05J EE3Ux1DCHZ5V7vNAtX/8Wg4l4GZfweauXwuJ1w7Sp7fklwcNC6wsceI+EmNjGMqfIaukGetG +jBGqsQ7moOZodfXUoCK98gblKgt/BPYMVidzlGC8Q/+lZg1+o29sPnwImW+MXt/Z5az/Z17 Qc265g+p5cqJHzq6bpQdnF7Fu6btKU/kv6wJghENvgMXBuyThqsyFReJWFh2wfaKyuix3Zyj ccq7/blkhzIKmtFWgDcgaSc2UAuJU+x9nuYjihW6WobpKP/nlUDu3BIsbIq09UEke+uE/QK+ FJ8PTJkAsXOf1Bc2C0XbW4Y2hf103+YY6L8weUCBsWC5VH5VtVmeuh26ENURclwfeXhWQ9Og 77yzpTXWr5g1Z0oLpYpWPv745J4bE7pv+dzxOrFdM1xNkzY2pvXph/A8OjxZNQklDkHQ7PIB Lki5L2F4XkEOddUUQchJwzMqTPsggPDmGjgLZrqgO+s4ECZK5+nLD3HEpAbPa3JLDaScy+90 Nu1lAqPUHSnP3vYZVw85ZYm6UCxHE4VLMnnJsN09ZhsOSVR+GyP5Nyw9rT1V3lcsuH7M5Naa 2Xobn9m7l9bRCD/Ji8kG15eV1WTxx1HXVQGjdUYDI7UwegBNbwMLh17XDy+3sn/6SgcqtECA Q6pZKA2mTQxEKMLBZQQYAQIADwIbDAUCWZR3hQUJA59eRwAKCRD5M+tD3xNhH4a/D/4jLAZu UhvU1swWcNEVVCELZ0D3LOV14XcY2MXa3QOpeZ9Bgq7YYJ4S5YXK+SBQS0FkRZdjGNvlGZoG ZdpU+NsQmQFhqHGwX0IT9MeTFM8uvKgxNKGwMVcV9g0IOqwBhGHne+BFboRA9362fgGW5AYQ zT0mzzRKEoOh4r3AQvbM6kLISxo0k1ujdYiI5nj/5WoKDqxTwwfuN1uDUHsWo3tzenRmpMyU NyW3Dc+1ajvXLyo09sRRq7BnM99Rix1EGL8Qhwy+j0YAv+FuspWxUX9FxXYho5PvGLHLsHfK FYQ7x/RRbpMjkJWVfIe/xVnfvn4kz+MTA5yhvsuNi678fLwY9hBP0y4lO8Ob2IhEPdfnTuIs tFVxXuelJ9xAe5TyqP0f+fQjf1ixsBZkqOohsBXDfje0iaUpYa/OQ/BBeej0dUdg2JEu4jAC x41HpVCnP9ipLpD0fYz1d/dX0F/VY2ovW6Eba/y/ngOSAR6C+u881m7oH2l0G47MTwkaQCBA bLGXPj4TCdX3lftqt4bcBPBJ+rFAnJmRHtUuyyaewBnZ81ZU2YAptqFM1kTh+aSvMvGhfVsQ qZL2rk2OPN1hg+KXhErlbTZ6oPtLCFhSHQmuxQ4oc4U147wBTUuOdwNjtnNatUhRCp8POc+3 XphVR5G70mnca1E2vzC77z+XSlTyRA== Message-ID: Date: Tue, 3 Jul 2018 13:12:11 +0100 User-Agent: Mozilla/5.0 (Windows NT 10.0; WOW64; rv:52.0) Gecko/20100101 Thunderbird/52.8.0 MIME-Version: 1.0 In-Reply-To: <1530268248-7328-8-git-send-email-jia.guo@intel.com> Content-Type: text/plain; charset=utf-8 Content-Language: en-US Content-Transfer-Encoding: 7bit Subject: Re: [dpdk-dev] [PATCH V4 7/9] igb_uio: fix uio release issue when hot unplug X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 03 Jul 2018 12:12:17 -0000 On 6/29/2018 11:30 AM, Jeff Guo wrote: > When hot unplug device, the kernel will release the device resource in the > kernel side, such as the fd sys file will disappear, and the irq will be > released. At this time, if igb uio driver still try to release this > resource, it will cause kernel crash. On the other hand, something like > interrupt disabling do not automatically process in kernel side. If not > handler it, this redundancy and dirty thing will affect the interrupt > resource be used by other device. So the igb_uio driver have to check the > hot plug status, and the corresponding process should be taken in igb uio > driver. > > This patch propose to add structure of rte_udev_state into rte_uio_pci_dev > of igb_uio kernel driver, which will record the state of uio device, such > as probed/opened/released/removed/unplug. When detect the unexpected > removal which cause of hot unplug behavior, it will corresponding disable > interrupt resource, while for the part of releasement which kernel have > already handle, just skip it to avoid double free or null pointer kernel > crash issue. > > Signed-off-by: Jeff Guo > --- > v4->v3: > no change > --- > kernel/linux/igb_uio/igb_uio.c | 50 +++++++++++++++++++++++++++++++++++++----- > 1 file changed, 45 insertions(+), 5 deletions(-) > > diff --git a/kernel/linux/igb_uio/igb_uio.c b/kernel/linux/igb_uio/igb_uio.c > index b3233f1..d301302 100644 > --- a/kernel/linux/igb_uio/igb_uio.c > +++ b/kernel/linux/igb_uio/igb_uio.c > @@ -19,6 +19,15 @@ > > #include "compat.h" > > +/* uio pci device state */ > +enum rte_udev_state { > + RTE_UDEV_PROBED, > + RTE_UDEV_OPENNED, > + RTE_UDEV_RELEASED, > + RTE_UDEV_REMOVED, > + RTE_UDEV_UNPLUG > +}; > + > /** > * A structure describing the private information for a uio device. > */ > @@ -28,6 +37,7 @@ struct rte_uio_pci_dev { > enum rte_intr_mode mode; > struct mutex lock; > int refcnt; > + enum rte_udev_state state; > }; > > static char *intr_mode; > @@ -194,12 +204,20 @@ igbuio_pci_irqhandler(int irq, void *dev_id) > { > struct rte_uio_pci_dev *udev = (struct rte_uio_pci_dev *)dev_id; > struct uio_info *info = &udev->info; > + struct pci_dev *pdev = udev->pdev; > > /* Legacy mode need to mask in hardware */ > if (udev->mode == RTE_INTR_MODE_LEGACY && > !pci_check_and_mask_intx(udev->pdev)) > return IRQ_NONE; > > + /* check the uevent of the kobj */ > + if ((&pdev->dev.kobj)->state_remove_uevent_sent == 1) { > + dev_notice(&pdev->dev, "device:%s, sent remove uevent!\n", > + (&pdev->dev.kobj)->name); > + udev->state = RTE_UDEV_UNPLUG; > + } I guess commit log says kernel can remove device, if so do we need any locking before accessing dev? > + > uio_event_notify(info); > > /* Message signal mode, no share IRQ and automasked */ > @@ -308,7 +326,6 @@ igbuio_pci_disable_interrupts(struct rte_uio_pci_dev *udev) > #endif > } > > - > /** > * This gets called while opening uio device file. > */ > @@ -330,24 +347,33 @@ igbuio_pci_open(struct uio_info *info, struct inode *inode) > > /* enable interrupts */ > err = igbuio_pci_enable_interrupts(udev); > - mutex_unlock(&udev->lock); > if (err) { > dev_err(&dev->dev, "Enable interrupt fails\n"); > + pci_clear_master(dev); > return err; > } > + udev->state = RTE_UDEV_OPENNED; > + mutex_unlock(&udev->lock); > return 0; > } > > +/** > + * This gets called while closing uio device file. > + */ > static int > igbuio_pci_release(struct uio_info *info, struct inode *inode) > { > + > struct rte_uio_pci_dev *udev = info->priv; > struct pci_dev *dev = udev->pdev; > > + if (udev->state == RTE_UDEV_REMOVED) > + return 0; > + > mutex_lock(&udev->lock); > if (--udev->refcnt > 0) { > mutex_unlock(&udev->lock); > - return 0; > + return -1; > } > > /* disable interrupts */ > @@ -355,7 +381,7 @@ igbuio_pci_release(struct uio_info *info, struct inode *inode) > > /* stop the device from further DMA */ > pci_clear_master(dev); > - > + udev->state = RTE_UDEV_RELEASED; > mutex_unlock(&udev->lock); > return 0; > } > @@ -557,6 +583,7 @@ igbuio_pci_probe(struct pci_dev *dev, const struct pci_device_id *id) > (unsigned long long)map_dma_addr, map_addr); > } > > + udev->state = RTE_UDEV_PROBED; > return 0; > > fail_remove_group: > @@ -573,11 +600,24 @@ igbuio_pci_probe(struct pci_dev *dev, const struct pci_device_id *id) > static void > igbuio_pci_remove(struct pci_dev *dev) > { > + > struct rte_uio_pci_dev *udev = pci_get_drvdata(dev); > + int ret; > + > + /* handler hot unplug */ > + if (udev->state == RTE_UDEV_OPENNED || > + udev->state == RTE_UDEV_UNPLUG) { > + dev_notice(&dev->dev, "Unexpected removal!\n"); > + ret = igbuio_pci_release(&udev->info, NULL); > + if (ret) > + return; > + udev->state = RTE_UDEV_REMOVED; > + return; > + } > > mutex_destroy(&udev->lock); > - sysfs_remove_group(&dev->dev.kobj, &dev_attr_grp); > uio_unregister_device(&udev->info); > + sysfs_remove_group(&dev->dev.kobj, &dev_attr_grp); > igbuio_pci_release_iomem(&udev->info); > pci_disable_device(dev); > pci_set_drvdata(dev, NULL); >