From: Jerin Jacob Kollanukkaran <jerinj@marvell.com>
To: "Hyong Youb Kim (hyonkim)" <hyonkim@cisco.com>,
David Marchand <david.marchand@redhat.com>,
Thomas Monjalon <thomas@monjalon.net>,
"Ferruh Yigit" <ferruh.yigit@intel.com>,
Alejandro Lucero <alejandro.lucero@netronome.com>,
Anatoly Burakov <anatoly.burakov@intel.com>
Cc: "dev@dpdk.org" <dev@dpdk.org>,
"John Daley (johndale)" <johndale@cisco.com>,
Shahed Shaikh <shshaikh@marvell.com>,
"Nithin Kumar Dabilpuram" <ndabilpuram@marvell.com>
Subject: Re: [dpdk-dev] [RFC PATCH] vfio: avoid re-installing irq handler
Date: Tue, 16 Jul 2019 06:47:09 +0000 [thread overview]
Message-ID: <BYAPR18MB24244957A9E3012E1C99A851C8CE0@BYAPR18MB2424.namprd18.prod.outlook.com> (raw)
In-Reply-To: <MWHPR11MB183961A9F226736F81FE6946BFCE0@MWHPR11MB1839.namprd11.prod.outlook.com>
> -----Original Message-----
> From: Hyong Youb Kim (hyonkim) <hyonkim@cisco.com>
> Sent: Tuesday, July 16, 2019 11:28 AM
> To: Jerin Jacob Kollanukkaran <jerinj@marvell.com>; David Marchand
> <david.marchand@redhat.com>; Thomas Monjalon
> <thomas@monjalon.net>; Ferruh Yigit <ferruh.yigit@intel.com>; Alejandro
> Lucero <alejandro.lucero@netronome.com>; Anatoly Burakov
> <anatoly.burakov@intel.com>
> Cc: dev@dpdk.org; John Daley (johndale) <johndale@cisco.com>; Shahed
> Shaikh <shshaikh@marvell.com>; Nithin Kumar Dabilpuram
> <ndabilpuram@marvell.com>
> Subject: RE: [RFC PATCH] vfio: avoid re-installing irq handler
>
> > > A rough patch for the approach mentioned earlier. It is only for
> discussion.
> > > http://mails.dpdk.org/archives/dev/2019-July/138113.html
> > >
> > > To try this out, first revert the following then apply.
> > > commit 89aac60e0be9 ("vfio: fix interrupts race condition")
> >
> > Yes. This patch has to be to reverted. It changes the existing
> > interrupt behavior and does not address the MSIX case as well.
> >
> > I think, The clean fix would be to introduce rte_intr_mask() and
> > rte_intr_unmask() by abstracting the INTX and MSIX differences And let
> > qede driver call it as needed.
> >
> > Thoughts?
>
> Hi,
Hi Hyong,
>
> You are proposing these?
> - Add rte_intr_mask_intx, rte_intr_unmask_intx.
> No APIs for masking MSI/MSI-X as vfio-pci does not support that.
> - Modify PMD irq handlers to use rte_intr_unmask_intx as necessary.
No, introduce the rte_intr_mask() and rte_intr_unmask().
For MSIX + Linux VFIO, That API can return -ENOSUP as Linux VFIO+MSIX is not supporting.
Another platform/eal may support it.
Mask and unmask is operation is known to all IRQ controllers.
So, IMO, As far as abstraction is concerned it will be good fit.
> That might be too intrusive. And too much work for the sake of INTx..
> Anyone really using/needing INTx these days? :-)
Yup. Mask needs to called only for only qede INTx. Looks like qede
Has MSIX and INTX separate handler. So this mask can go to qede INTx
>
> The following drivers call rte_intr_enable from their irq handlers. So with
> explicit rte_intr_unmask_intx, all these would need to do "if using intx,
> unmask"?
>
> atlantic, avp, axgbe, bnx2x, e1000, fm10k, ice, ixgbe, nfp, qede, sfc,
> vmxnet3
No change on these PMDs.
> And nfp seems to rely on rte_intr_enable to re-install irq handler to unmask
> a vector in MSI-X Table?
>
> if (hw->ctrl & NFP_NET_CFG_CTRL_MSIXAUTO) {
> /* If MSI-X auto-masking is used, clear the entry */
> rte_wmb();
> rte_intr_enable(&pci_dev->intr_handle);
>
> With David's patch and mine, this handler would have to first
> rte_intr_disable() and then enable, if such unmasking is really necessary..
>
> As for the semantics of rte_intr_enable/disable, I am ok as is.
> - "enable": put things in a state where NIC can send an interrupt, and
> PMD/app gets a callback.
> Whether this involves unmasking for INTx is hidden.
> - "disable": put things in a state where NIC cannot send an interrupt.
It looks OK to me. My only thought was, Since mask and unmask
is a common irq controller operation. We may not need to add
A lot of common code(Introducing a state) to hide unmask INTx.
More over as you said, There is may only handful of devices uses INTX.
IMO, mask and unmask API is good fit as eal abstraction.
But Using a separate API or hide inside eal to solve this problem is good question.
May be more thoughts from another quys will be good.
We will try to send a version with mask/unmask API to see the changes required.
>
> Regardless of vfio changes, we should probably remove rte_intr_enable
> from qede_interrupt_handler (the MSI/MSI-X interrupt handler), to make
> usage/intention clear..
Yes. Anyway this change is required.
>
> Thanks.
> -Hyong
next prev parent reply other threads:[~2019-07-16 6:47 UTC|newest]
Thread overview: 25+ messages / expand[flat|nested] mbox.gz Atom feed top
2019-07-15 16:50 Jerin Jacob Kollanukkaran
2019-07-16 5:58 ` Hyong Youb Kim (hyonkim)
2019-07-16 6:47 ` Jerin Jacob Kollanukkaran [this message]
2019-07-16 7:49 ` Hyong Youb Kim (hyonkim)
2019-07-16 9:56 ` Jerin Jacob Kollanukkaran
2019-07-16 6:46 ` [dpdk-dev] [RFC PATCH] eal: add mask and unmask interrupt apis Nithin Dabilpuram
2019-07-16 7:01 ` [dpdk-dev] [RFC PATCH v2] " Nithin Dabilpuram
2019-07-16 16:44 ` [dpdk-dev] [RFC PATCH v3 1/3] vfio: revert change that does intr eventfd setup at probe Nithin Dabilpuram
2019-07-16 16:44 ` [dpdk-dev] [RFC PATCH v3 2/3] eal: add mask and unmask interrupt APIs Nithin Dabilpuram
2019-07-17 5:55 ` Hyong Youb Kim (hyonkim)
2019-07-17 6:14 ` Jerin Jacob Kollanukkaran
2019-07-17 7:09 ` Hyong Youb Kim (hyonkim)
2019-07-17 8:03 ` Jerin Jacob Kollanukkaran
2019-07-17 8:45 ` Hyong Youb Kim (hyonkim)
2019-07-17 9:20 ` Jerin Jacob Kollanukkaran
2019-07-17 9:51 ` Hyong Youb Kim (hyonkim)
2019-07-17 10:43 ` Jerin Jacob Kollanukkaran
2019-07-17 11:06 ` Hyong Youb Kim (hyonkim)
2019-07-17 11:16 ` Jerin Jacob Kollanukkaran
2019-07-17 12:04 ` Nithin Kumar Dabilpuram
2019-07-16 16:44 ` [dpdk-dev] [RFC PATCH v3 3/3] drivers/net: use unmask API in interrupt handlers Nithin Dabilpuram
2019-07-17 6:01 ` Hyong Youb Kim (hyonkim)
2019-07-17 7:47 ` Nithin Kumar Dabilpuram
2019-07-16 20:06 ` [dpdk-dev] [RFC PATCH v3 1/3] vfio: revert change that does intr eventfd setup at probe Stephen Hemminger
-- strict thread matches above, loose matches on Subject: below --
2019-07-15 15:58 [dpdk-dev] [RFC PATCH] vfio: avoid re-installing irq handler Hyong Youb Kim
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=BYAPR18MB24244957A9E3012E1C99A851C8CE0@BYAPR18MB2424.namprd18.prod.outlook.com \
--to=jerinj@marvell.com \
--cc=alejandro.lucero@netronome.com \
--cc=anatoly.burakov@intel.com \
--cc=david.marchand@redhat.com \
--cc=dev@dpdk.org \
--cc=ferruh.yigit@intel.com \
--cc=hyonkim@cisco.com \
--cc=johndale@cisco.com \
--cc=ndabilpuram@marvell.com \
--cc=shshaikh@marvell.com \
--cc=thomas@monjalon.net \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).