From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from NAM03-BY2-obe.outbound.protection.outlook.com (mail-by2nam03on0082.outbound.protection.outlook.com [104.47.42.82]) by dpdk.org (Postfix) with ESMTP id 6D4621B2E8; Fri, 13 Oct 2017 10:12:25 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=CAVIUMNETWORKS.onmicrosoft.com; s=selector1-cavium-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version; bh=iReD+xCPL95STN7JxsMxItXcHVlT015EFuf3RNFsL84=; b=OCspCIoqyXqkDMIrZBLlF8ydKO+Igb3EAM0iqWMqb0rfQToIWuhpFjVOcSYnjRjLtDV4q6h7hRowEoy61L4suRV95PB5XRKVLvjONBKUoQuPBB9wn7zhhnrzK8eLfxEtiY6Us0LMzawbTds9ImZr30EBBStSK4FGxaFw9QTZvbc= Authentication-Results: spf=none (sender IP is ) smtp.mailfrom=Shijith.Thotton@cavium.com; Received: from localhost.localdomain (14.140.2.178) by BN6PR07MB3059.namprd07.prod.outlook.com (10.172.106.141) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384_P256) id 15.20.77.7; Fri, 13 Oct 2017 08:12:19 +0000 Date: Fri, 13 Oct 2017 13:42:06 +0530 From: Shijith Thotton To: "Wu, Jingjing" , hpatil@caviumnetworks.com Cc: "Yigit, Ferruh" , "Tan, Jianfeng" , "gregory@weka.io" , "Xing, Beilei" , "dev@dpdk.org" , "stable@dpdk.org" Message-ID: <20171013081205.GA20678@localhost.localdomain> References: <1507581083-33667-2-git-send-email-jingjing.wu@intel.com> <1507586960-35508-1-git-send-email-jingjing.wu@intel.com> <9BB6961774997848B5B42BEC655768F810E92434@SHSMSX103.ccr.corp.intel.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <9BB6961774997848B5B42BEC655768F810E92434@SHSMSX103.ccr.corp.intel.com> User-Agent: Mutt/1.5.21 (2010-09-15) X-Originating-IP: [14.140.2.178] X-ClientProxiedBy: BMXPR01CA0043.INDPRD01.PROD.OUTLOOK.COM (10.174.214.29) To BN6PR07MB3059.namprd07.prod.outlook.com (10.172.106.141) X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: f70c82a1-b7e5-4609-9c91-08d51212213b X-Microsoft-Antispam: UriScan:; BCL:0; PCL:0; RULEID:(22001)(2017030254152)(2017052603199)(201703131423075)(201703031133081)(201702281549075); SRVR:BN6PR07MB3059; X-Microsoft-Exchange-Diagnostics: 1; BN6PR07MB3059; 3:5pkdhumxrPJABbwf5pOBe/ipWadvGKyF7mUifO6qEtkhIMrnrTnfw4Rcs58DV0k1C+hbe+M+bzYM/9rFbZ/OCW0KwzSFbktyZsVMdiNwCnmxystYTTfXvPHATJAKuqsHJ+HdLtpnq4UHPkup6TFwtL9Yq0CMmY2Yl8LB2MdtH5dQIMqN4FhFrFjcd4gu303pu+7MU7w5IlwzXFhExYlDT64+Y9+c2b1yOlxAWcl0nZJ3gd6uxr9S81VKnXJievIK; 25:6G+qr8w2Hrog/SjvfR6aKmLLAS9k7nzM2JPYMbveS24Zi50QiWOlIHiO0kuxi44qZgJz/rLWTHbTH07pikSpmt7107lfzcAJdiEPjLwcvWjeG3EUxBPAsQc3RaqasT2m6GoNbTYCOUVaCAeAp6IgclCtCh15jGDbEdSqemuP5Cpw6hdkFSkFPmB3OOwah3Cae6pFpuugjnW+eVfwVnXpUVGLVsFHn4gF2Cx39wHY+kv1XqiWAuj38mhn3VZ4HPpOIay00etW6LTesXJvKic4s6sWEPlDzoOJcVNiHmmeHef4SP0GsP+YP5eTl0CplhWWk9BNNEtpQgngDF8FmKpMgg==; 31:hrASW/2XkZdM3b+U4tXJujjsiLSVXBTAXk88IKjMOocP9NUg/YrhwteF973sb5xsPK4vrM5brv7StpxknlQnwtq6VhfMdt0r8tj+t0tzBid6SRJ8U5UmM5z3zUYPYwX8wCsOJOcPYAjtYFC4TtmXGes+8VrRebGgcd0xDg+rskMy2N/OyWjGdCF/OXt9DDa42IhHyW1o11zUA/flsJJ8Ld4UQXBRsfB36spc0OS+vJ0= X-MS-TrafficTypeDiagnostic: BN6PR07MB3059: X-Microsoft-Exchange-Diagnostics: 1; BN6PR07MB3059; 20:fRbG09xm/6Lls6K2h6clU2vQHGrIr00AcFFdI8y25ByQy6H24iIMvdP0NgLcBhaBYITWJMYFinKHAQjboYZ7+iwMGvIQx3fxu6T6Z/hBUygJhLB1PXlGoZMtjcJhhW6BoR5jswU+ln1cbQSy897PJnTtrffWD7SuoEOZomjmbed7zICGn9xqX1S7UMZXZNU61Hp89v64ndggK0TpZe5VuOQ4etd1puvgP2IudX9g8GYIT8jgnChjHDvF69nLbKfEqaT+scVrfMCMxvi13wSq8Y1wqioEZ1oBv6gtZ/q6t0mYEF9acsG3/bSK7Jomm/nRiWhqw3KHIFVypSxeGMrmo9g467zT/UnnperBT6bJj+24yd9dgh33wLaONWxd0QLg/SYzScAZ9MAfLGkhS214bAYHbGH/oQDaMqqstY8KX6L7QmZ4ako7VKltu9EVHBuKo2K+r4imTgWGNM5sh7OeKBbKSVClAz99D9oCWU0zNYsBaVcwCtNhq1cD+Ip6/Q1wOrD4idiWBweR2i9vXq45K7HvY7HQreEyyuoRGi54IgLWBhLDQk0o4Z9BeG0hsH/PF5EQJea15Z2xZ9OkfvOKWCcnGMtQgMhleHPAps66XmM=; 4:vIQKvXXUHLAN7XWR/tt0LXhCXxsdxZKPZ9yTDmQJrXUaG5cFe+QpZZci4fe+d+c0BHwQBeeaPha40ANOn2ZOD/TLrfRMf70e1zzDD8SgeU+bzQq5KgQ6JdktOGSt+eTWbQa1v9CSlrmxoS6G/20txS28f00WJtpmPwE2/aNn8IhwBwnTIe4L+ihwUmLwWD9YXPLVim/j8tbYiqsHOQjhjXHdj2J0EEUXYiOza5CdX0SviyOfd8sQ0RJrF82NT/0ZBbSoYXhIjvBR0Xu0zn8owR6eNisiBaDJpEGoPHmP/rA= X-Exchange-Antispam-Report-Test: UriScan:(228905959029699); X-Microsoft-Antispam-PRVS: X-Exchange-Antispam-Report-CFA-Test: BCL:0; PCL:0; RULEID:(100000700101)(100105000095)(100000701101)(100105300095)(100000702101)(100105100095)(6040450)(2401047)(8121501046)(5005006)(3002001)(93006095)(100000703101)(100105400095)(10201501046)(6041248)(20161123564025)(20161123558100)(201703131423075)(201702281528075)(201703061421075)(201703061406153)(20161123562025)(20161123560025)(20161123555025)(6072148)(201708071742011)(100000704101)(100105200095)(100000705101)(100105500095); SRVR:BN6PR07MB3059; BCL:0; PCL:0; RULEID:(100000800101)(100110000095)(100000801101)(100110300095)(100000802101)(100110100095)(100000803101)(100110400095)(100000804101)(100110200095)(100000805101)(100110500095); SRVR:BN6PR07MB3059; X-Forefront-PRVS: 04599F3534 X-Forefront-Antispam-Report: SFV:NSPM; SFS:(10009020)(6009001)(6069001)(346002)(376002)(377454003)(13464003)(24454002)(189002)(199003)(53546010)(66066001)(47776003)(2906002)(81156014)(81166006)(72206003)(55016002)(5009440100003)(8936002)(8676002)(478600001)(2950100002)(6666003)(42882006)(54906003)(189998001)(16526018)(97736004)(229853002)(5890100001)(58126008)(6636002)(6506006)(316002)(83506001)(61506002)(16586007)(5660300001)(4326008)(106356001)(68736007)(105586002)(23726003)(76176999)(53936002)(305945005)(7736002)(9686003)(25786009)(50986999)(54356999)(101416001)(3846002)(6116002)(33656002)(50466002)(1076002)(6246003)(110426004)(18370500001); DIR:OUT; SFP:1101; SCL:1; SRVR:BN6PR07MB3059; H:localhost.localdomain; FPR:; SPF:None; PTR:InfoNoRecords; MX:1; A:1; LANG:en; Received-SPF: None (protection.outlook.com: cavium.com does not designate permitted sender hosts) X-Microsoft-Exchange-Diagnostics: =?us-ascii?Q?1; BN6PR07MB3059; 23:ZLMXpcJD6imqigK9rbCfNdoyPVIopNdVzL11JmxYb?= =?us-ascii?Q?C5Uy1n5FRvB+tNpju8YWba053K2Yj/Tj1sK6tdSzlftXyI2UCJ9dgT1cZZ3g?= =?us-ascii?Q?80mh0ccBoGtiXy1LUwNrolgEFXazpjd6gNGO0f29D6etZVIvSBKwPPrz8o7p?= =?us-ascii?Q?XmqSoAr7ZdYjkrIZlNabyewbB8y+q6VxU4q1ftSfIcCW+nWRdasUFng9+bai?= =?us-ascii?Q?SFsX2N7q5xlpmSD7YlJvFrGZeEmFXJZcakzGwP/KsuLKs2lFO8nEvOrJc8ys?= =?us-ascii?Q?WwxCr849DLQupd8o3krMEkgOVcAXyc/1BPbIopsfven3auR26athIsOAYlm0?= =?us-ascii?Q?3+CbMbfodCcJiNK50V+FSolq4WmVN/7R1pca/Q+B6BidfQTr6jWkmKjZC7PV?= =?us-ascii?Q?sR091+6qt1CNHbhYXpGlac9Dr1Rt0kQ4DLIwnplICwTySCtZayoEt2wNUCwj?= =?us-ascii?Q?H1Dt8ZK9hgCWwvRAruT6Jc9CBNyr+p6riG/g6WsGFGeSSqR/ZKkO47ROsB6W?= =?us-ascii?Q?BFs7+auExhlQqF6+GP2RBTmJAAnvEv6BNM989gPDZU24HGqt5MzAIMTCrbBT?= =?us-ascii?Q?PrOxQOAkqmhD4Ar9k5XjEbnlPeIsQpX9NiUrhl7tAMNPBldeZRFTVwYAxMiq?= =?us-ascii?Q?SBTA5QV1zt8DigAr69v+OiLaqvVuUcy3MJ7C6Tv4d2izuHpGyGH35FUtK02/?= =?us-ascii?Q?q2/qstzwCGG7x7ljkOyNLJACNPbR185sp5OTu1FLR+oIcY3nIx2hX+EpLWG8?= =?us-ascii?Q?76XQ6HYbWMSZLPB8MfVo/WAWbFXC8K0njPRRaUx8+LAYrfQXRZetAwFKv4qt?= =?us-ascii?Q?f20lzlETJ6lePVsqeZaXofJFeMroReFrCnWNeCupzSihXDhIoGKx46H0vvmm?= =?us-ascii?Q?27ZbHB8n5GzGfxEZAD7tAnbWpzu6bnZ8hF0rrvbeRJXpC19BTuQieNVHg/ew?= =?us-ascii?Q?LglEEiNnHIvk8EH1HrtRDc/19CiNyzijjOFTXcE3PNgu9nCcEscb+dQHt+Xs?= =?us-ascii?Q?moQ7Pss5ahX1AggeDmHE9crPRHdL2On8Gs1tPxYTuDLTwP6f9qGaW7t/iH9J?= =?us-ascii?Q?UrI54zjjMuTfOJEhcSEBE4BS/2Nv0mGamgSnYnW8iWKOognwadhB6yXfh1Ce?= =?us-ascii?Q?bUtSNBk2gIgRUa/54k8BCI2njBvYTcNzXE85kvc52eDtaLghJiAk3CMyhnqR?= =?us-ascii?Q?NfyEcMY7efBXdX0mKyP3ylnCmvswlDd+KBRjR73/IxCX+iC1J7LqLD4oTFMK?= =?us-ascii?Q?lBTYh3mOCG0PFJOoTC1eHew0hDOHyp6Q2W2LxaZuNwiknqA3GnCMxVwPqYuY?= =?us-ascii?Q?/38TGD/C5OQ8hEbHnZCtvnOy9fCndh/5w68yinecbrbooXqmgeVmZEuLdr/4?= =?us-ascii?Q?tGIUmW7m+k03udYBtJj2lAaYi7UmQV3WNr4iqek+xXiwYST?= X-Microsoft-Exchange-Diagnostics: 1; BN6PR07MB3059; 6:rhoxBwKWqcZetwq+BPt3HXpFix1tNoQWD3Z18Dwfhb4zdJHpLPRWfjnpgVs3Xwx26yXwvH7ItiEI0nzAaLvb4KrmQjJ+RZuFtn03pM+sFW6O0DV8zoYguxsPFYZYNzsJRGiJVmyVCHAROJpn+PKW5fSxMvyDMxcjT0ywVBd7hNOrNLZjPcZq+Ve4lC77cbGE5sozZ09nRgN1q3zgP9C1OnyG4H/pwUnG5oxEPbOGM5UsI3l48sA6oRwPh6bXe8qCCWIY+wXCFScTJoLzokGwb8/J9I4zoTx61yWbEf3t4TsSHHlK6fcbXkaRqr754+9RDdJxVhDamGR8RLibSj/vGQ==; 5:J56B0SLawXYjlStRwrWrvGZMd349J7zVpaPcDrf5HpMX42P2jIixwHaWvAafMN9BHB6eFzVqH++Ss5kKW/7rJityXo4YvTPeMopT+CQk17qtPUL14eOsFaD+8YsaO29IEItFLKvQevUGHYNcJnVYsw==; 24:ezenrHariczfTYapSvQznO+bTVdklvo6x79DMGpEvKtfV22Wl5Wttrxzj7XpsQHtgVFNLpAS+BuhodntPPfua2aqcA+fauhs+s4ZOWywqs4=; 7:GcbqgG+XATrpRL5CrC54VErH3G/gvha/nAqdkTXO7nqQa8Me/QZnnOB40QDxDvzeldyvCXeejeJS6ArXEEx8RCIDxYhg4ef9gTxeiezcTlJUEUNNXhZVn/GP6zLxWbDZpp5BY1h4gjCQn8S3iV9k2vTU20VhGzCXLEKwtSfCwFXGtsPxkyULpr/4XmnUBNMkhwCVdtLtFCUbh59oQbnw9oyVRVWkGcvSAm2+6dXPRcA= SpamDiagnosticOutput: 1:99 SpamDiagnosticMetadata: NSPM X-OriginatorOrg: caviumnetworks.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 13 Oct 2017 08:12:19.7496 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 711e4ccf-2e9b-4bcf-a551-4094005b6194 X-MS-Exchange-Transport-CrossTenantHeadersStamped: BN6PR07MB3059 Subject: Re: [dpdk-dev] [PATCH v2 2/2] igb_uio: fix interrupt enablement after FLR in VM X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 13 Oct 2017 08:12:28 -0000 On Thu, Oct 12, 2017 at 05:43:29AM +0000, Wu, Jingjing wrote: > Hi, Shjith > > Could you help to review and verify if this fix can meet your requirement? > > Thanks > Jingjing > > > -----Original Message----- > > From: Wu, Jingjing > > Sent: Tuesday, October 10, 2017 6:09 AM > > To: Yigit, Ferruh ; Tan, Jianfeng > > ; shijith.thotton@caviumnetworks.com; > > gregory@weka.io; Xing, Beilei > > Cc: dev@dpdk.org; Wu, Jingjing ; stable@dpdk.org > > Subject: [PATCH v2 2/2] igb_uio: fix interrupt enablement after FLR in VM > > > > If pass-through a VF by vfio-pci to a Qemu VM, after FLR in VM, the interrupt > > setting is not recoverd correctly to host as below: > > in VM guest: > > Capabilities: [70] MSI-X: Enable+ Count=5 Masked- in Host: > > Capabilities: [70] MSI-X: Enable+ Count=5 Masked- > > > > That was because in pci_reset_function, it first reads the PCI configure and set > > FLR reset, and then writes PCI configure as restoration. But not all the writing > > are successful to Host. > > Because vfio-pci driver doesn't allow directly write PCI MSI-X Cap. > > > > To fix this issue, we need to move the interrupt enablement from igb_uio probe > > to open device file. While it is also the similar as the behaviour in vfio_pci > > kernel module code. > > > > Fixes: b58eedfc7dd5 ("igb_uio: issue FLR during open and release of device file") > > > > Cc: stable@dpdk.org > > > > Signed-off-by: Jingjing Wu > > Signed-off-by: Jianfeng Tan Tested-by: Shijith Thotton > > --- > > v2 change: > > - fix typo > > - remove duplicated debug info > > > > lib/librte_eal/linuxapp/igb_uio/igb_uio.c | 221 ++++++++++++++++-------------- > > 1 file changed, 119 insertions(+), 102 deletions(-) > > > > diff --git a/lib/librte_eal/linuxapp/igb_uio/igb_uio.c > > b/lib/librte_eal/linuxapp/igb_uio/igb_uio.c > > index c8dd5f4..d943795 100644 > > --- a/lib/librte_eal/linuxapp/igb_uio/igb_uio.c > > +++ b/lib/librte_eal/linuxapp/igb_uio/igb_uio.c > > @@ -49,7 +49,6 @@ struct rte_uio_pci_dev { > > > > static char *intr_mode; > > static enum rte_intr_mode igbuio_intr_mode_preferred = > > RTE_INTR_MODE_MSIX; > > - > > /* sriov sysfs */ > > static ssize_t > > show_max_vfs(struct device *dev, struct device_attribute *attr, @@ -144,8 > > +143,10 @@ igbuio_pci_irqcontrol(struct uio_info *info, s32 irq_state) > > * If yes, disable it here and will be enable later. > > */ > > static irqreturn_t > > -igbuio_pci_irqhandler(int irq, struct uio_info *info) > > +igbuio_pci_irqhandler(int irq, void *dev_id) > > { > > + struct uio_device *idev = (struct uio_device *)dev_id; > > + struct uio_info *info = idev->info; > > struct rte_uio_pci_dev *udev = info->priv; > > > > /* Legacy mode need to mask in hardware */ @@ -153,10 +154,115 > > @@ igbuio_pci_irqhandler(int irq, struct uio_info *info) > > !pci_check_and_mask_intx(udev->pdev)) > > return IRQ_NONE; > > > > + uio_event_notify(info); > > + > > /* Message signal mode, no share IRQ and automasked */ > > return IRQ_HANDLED; > > } > > > > +static int > > +igbuio_pci_enable_interrupts(struct rte_uio_pci_dev *udev) { > > + int err = 0; > > +#ifndef HAVE_ALLOC_IRQ_VECTORS > > + struct msix_entry msix_entry; > > +#endif > > + > > + switch (igbuio_intr_mode_preferred) { > > + case RTE_INTR_MODE_MSIX: > > + /* Only 1 msi-x vector needed */ > > +#ifndef HAVE_ALLOC_IRQ_VECTORS > > + msix_entry.entry = 0; > > + if (pci_enable_msix(udev->pdev, &msix_entry, 1) == 0) { > > + dev_dbg(&udev->pdev->dev, "using MSI-X"); > > + udev->info.irq_flags = IRQF_NO_THREAD; > > + udev->info.irq = msix_entry.vector; > > + udev->mode = RTE_INTR_MODE_MSIX; > > + break; > > + } > > +#else > > + if (pci_alloc_irq_vectors(udev->pdev, 1, 1, PCI_IRQ_MSIX) == 1) > > { > > + dev_dbg(&udev->pdev->dev, "using MSI-X"); > > + udev->info.irq_flags = IRQF_NO_THREAD; > > + udev->info.irq = pci_irq_vector(udev->pdev, 0); > > + udev->mode = RTE_INTR_MODE_MSIX; > > + break; > > + } > > +#endif > > + > > + /* fall back to MSI */ > > + case RTE_INTR_MODE_MSI: > > +#ifndef HAVE_ALLOC_IRQ_VECTORS > > + if (pci_enable_msi(udev->pdev) == 0) { > > + dev_dbg(&udev->pdev->dev, "using MSI"); > > + udev->info.irq_flags = IRQF_NO_THREAD; > > + udev->info.irq = udev->pdev->irq; > > + udev->mode = RTE_INTR_MODE_MSI; > > + break; > > + } > > +#else > > + if (pci_alloc_irq_vectors(udev->pdev, 1, 1, PCI_IRQ_MSI) == 1) > > { > > + dev_dbg(&udev->pdev->dev, "using MSI"); > > + udev->info.irq_flags = IRQF_NO_THREAD; > > + udev->info.irq = pci_irq_vector(udev->pdev, 0); > > + udev->mode = RTE_INTR_MODE_MSI; > > + break; > > + } > > +#endif > > + /* fall back to INTX */ > > + case RTE_INTR_MODE_LEGACY: > > + if (pci_intx_mask_supported(udev->pdev)) { > > + dev_dbg(&udev->pdev->dev, "using INTX"); > > + udev->info.irq_flags = IRQF_SHARED | > > IRQF_NO_THREAD; > > + udev->info.irq = udev->pdev->irq; > > + udev->mode = RTE_INTR_MODE_LEGACY; > > + break; > > + } > > + dev_notice(&udev->pdev->dev, "PCI INTX mask not > > supported\n"); > > + /* fall back to no IRQ */ > > + case RTE_INTR_MODE_NONE: > > + udev->mode = RTE_INTR_MODE_NONE; > > + udev->info.irq = UIO_IRQ_NONE; > > + break; > > + > > + default: > > + dev_err(&udev->pdev->dev, "invalid IRQ mode %u", > > + igbuio_intr_mode_preferred); > > + udev->info.irq = UIO_IRQ_NONE; > > + err = -EINVAL; > > + } > > + > > + if (udev->info.irq != UIO_IRQ_NONE) > > + err = request_irq(udev->info.irq, igbuio_pci_irqhandler, > > + udev->info.irq_flags, udev->info.name, > > + udev->info.uio_dev); > > + dev_info(&udev->pdev->dev, "uio device registered with irq %lx\n", > > + udev->info.irq); > > + > > + return err; > > +} > > + > > +static void > > +igbuio_pci_disable_interrupts(struct rte_uio_pci_dev *udev) { > > + if (udev->info.irq) { > > + free_irq(udev->info.irq, udev->info.uio_dev); > > + udev->info.irq = 0; > > + } > > + > > +#ifndef HAVE_ALLOC_IRQ_VECTORS > > + if (udev->mode == RTE_INTR_MODE_MSIX) > > + pci_disable_msix(udev->pdev); > > + if (udev->mode == RTE_INTR_MODE_MSI) > > + pci_disable_msi(udev->pdev); > > +#else > > + if (udev->mode == RTE_INTR_MODE_MSIX || > > + udev->mode == RTE_INTR_MODE_MSI) > > + pci_free_irq_vectors(udev->pdev); > > +#endif > > +} > > + > > + > > /** > > * This gets called while opening uio device file. > > */ > > @@ -165,12 +271,19 @@ igbuio_pci_open(struct uio_info *info, struct inode > > *inode) { > > struct rte_uio_pci_dev *udev = info->priv; > > struct pci_dev *dev = udev->pdev; > > + int err; > > > > pci_reset_function(dev); > > > > /* set bus master, which was cleared by the reset function */ > > pci_set_master(dev); > > > > + /* enable interrupts */ > > + err = igbuio_pci_enable_interrupts(udev); > > + if (err) { > > + dev_err(&dev->dev, "Enable interrupt fails\n"); > > + return err; > > + } > > return 0; > > } > > > > @@ -180,6 +293,9 @@ igbuio_pci_release(struct uio_info *info, struct inode > > *inode) > > struct rte_uio_pci_dev *udev = info->priv; > > struct pci_dev *dev = udev->pdev; > > > > + /* disable interrupts */ > > + igbuio_pci_disable_interrupts(udev); > > + > > /* stop the device from further DMA */ > > pci_clear_master(dev); > > > > @@ -250,94 +366,6 @@ igbuio_pci_release_iomem(struct uio_info *info) } > > > > static int > > -igbuio_pci_enable_interrupts(struct rte_uio_pci_dev *udev) -{ > > - int err = 0; > > -#ifndef HAVE_ALLOC_IRQ_VECTORS > > - struct msix_entry msix_entry; > > -#endif > > - > > - switch (igbuio_intr_mode_preferred) { > > - case RTE_INTR_MODE_MSIX: > > - /* Only 1 msi-x vector needed */ > > -#ifndef HAVE_ALLOC_IRQ_VECTORS > > - msix_entry.entry = 0; > > - if (pci_enable_msix(udev->pdev, &msix_entry, 1) == 0) { > > - dev_dbg(&udev->pdev->dev, "using MSI-X"); > > - udev->info.irq_flags = IRQF_NO_THREAD; > > - udev->info.irq = msix_entry.vector; > > - udev->mode = RTE_INTR_MODE_MSIX; > > - break; > > - } > > -#else > > - if (pci_alloc_irq_vectors(udev->pdev, 1, 1, PCI_IRQ_MSIX) == 1) > > { > > - dev_dbg(&udev->pdev->dev, "using MSI-X"); > > - udev->info.irq_flags = IRQF_NO_THREAD; > > - udev->info.irq = pci_irq_vector(udev->pdev, 0); > > - udev->mode = RTE_INTR_MODE_MSIX; > > - break; > > - } > > -#endif > > - /* fall back to MSI */ > > - case RTE_INTR_MODE_MSI: > > -#ifndef HAVE_ALLOC_IRQ_VECTORS > > - if (pci_enable_msi(udev->pdev) == 0) { > > - dev_dbg(&udev->pdev->dev, "using MSI"); > > - udev->info.irq_flags = IRQF_NO_THREAD; > > - udev->info.irq = udev->pdev->irq; > > - udev->mode = RTE_INTR_MODE_MSI; > > - break; > > - } > > -#else > > - if (pci_alloc_irq_vectors(udev->pdev, 1, 1, PCI_IRQ_MSI) == 1) > > { > > - dev_dbg(&udev->pdev->dev, "using MSI"); > > - udev->info.irq_flags = IRQF_NO_THREAD; > > - udev->info.irq = pci_irq_vector(udev->pdev, 0); > > - udev->mode = RTE_INTR_MODE_MSI; > > - break; > > - } > > -#endif > > - /* fall back to INTX */ > > - case RTE_INTR_MODE_LEGACY: > > - if (pci_intx_mask_supported(udev->pdev)) { > > - dev_dbg(&udev->pdev->dev, "using INTX"); > > - udev->info.irq_flags = IRQF_SHARED | > > IRQF_NO_THREAD; > > - udev->info.irq = udev->pdev->irq; > > - udev->mode = RTE_INTR_MODE_LEGACY; > > - break; > > - } > > - dev_notice(&udev->pdev->dev, "PCI INTX mask not > > supported\n"); > > - /* fall back to no IRQ */ > > - case RTE_INTR_MODE_NONE: > > - udev->mode = RTE_INTR_MODE_NONE; > > - udev->info.irq = UIO_IRQ_NONE; > > - break; > > - > > - default: > > - dev_err(&udev->pdev->dev, "invalid IRQ mode %u", > > - igbuio_intr_mode_preferred); > > - err = -EINVAL; > > - } > > - > > - return err; > > -} > > - > > -static void > > -igbuio_pci_disable_interrupts(struct rte_uio_pci_dev *udev) -{ -#ifndef > > HAVE_ALLOC_IRQ_VECTORS > > - if (udev->mode == RTE_INTR_MODE_MSIX) > > - pci_disable_msix(udev->pdev); > > - if (udev->mode == RTE_INTR_MODE_MSI) > > - pci_disable_msi(udev->pdev); > > -#else > > - if (udev->mode == RTE_INTR_MODE_MSIX || > > - udev->mode == RTE_INTR_MODE_MSI) > > - pci_free_irq_vectors(udev->pdev); > > -#endif > > -} > > - > > -static int > > igbuio_setup_bars(struct pci_dev *dev, struct uio_info *info) { > > int i, iom, iop, ret; > > @@ -427,20 +455,15 @@ igbuio_pci_probe(struct pci_dev *dev, const struct > > pci_device_id *id) > > /* fill uio infos */ > > udev->info.name = "igb_uio"; > > udev->info.version = "0.1"; > > - udev->info.handler = igbuio_pci_irqhandler; > > udev->info.irqcontrol = igbuio_pci_irqcontrol; > > udev->info.open = igbuio_pci_open; > > udev->info.release = igbuio_pci_release; > > udev->info.priv = udev; > > udev->pdev = dev; > > > > - err = igbuio_pci_enable_interrupts(udev); > > - if (err != 0) > > - goto fail_release_iomem; > > - > > err = sysfs_create_group(&dev->dev.kobj, &dev_attr_grp); > > if (err != 0) > > - goto fail_disable_interrupts; > > + goto fail_release_iomem; > > > > /* register uio driver */ > > err = uio_register_device(&dev->dev, &udev->info); @@ -449,9 +472,6 > > @@ igbuio_pci_probe(struct pci_dev *dev, const struct pci_device_id *id) > > > > pci_set_drvdata(dev, udev); > > > > - dev_info(&dev->dev, "uio device registered with irq %lx\n", > > - udev->info.irq); > > - > > /* > > * Doing a harmless dma mapping for attaching the device to > > * the iommu identity mapping if kernel boots with iommu=pt. > > @@ -477,8 +497,6 @@ igbuio_pci_probe(struct pci_dev *dev, const struct > > pci_device_id *id) > > > > fail_remove_group: > > sysfs_remove_group(&dev->dev.kobj, &dev_attr_grp); > > -fail_disable_interrupts: > > - igbuio_pci_disable_interrupts(udev); > > fail_release_iomem: > > igbuio_pci_release_iomem(&udev->info); > > pci_disable_device(dev); > > @@ -495,7 +513,6 @@ igbuio_pci_remove(struct pci_dev *dev) > > > > sysfs_remove_group(&dev->dev.kobj, &dev_attr_grp); > > uio_unregister_device(&udev->info); > > - igbuio_pci_disable_interrupts(udev); > > igbuio_pci_release_iomem(&udev->info); > > pci_disable_device(dev); > > pci_set_drvdata(dev, NULL); > > -- > > 2.7.4 Hi Jingjing, This patch perfectly meets requirements as both resets are retained (open and release). Tested it with LiquidIO NIC and it works fine. I can see MSI-X re-enabled on each run with new patch. Gregory, Harish, Please verify the patch on your setup if possible. Thanks, Shijith