DPDK patches and discussions
 help / color / mirror / Atom feed
From: "Hyong Youb Kim (hyonkim)" <hyonkim@cisco.com>
To: Nithin Dabilpuram <ndabilpuram@marvell.com>,
	David Marchand <david.marchand@redhat.com>,
	Thomas Monjalon <thomas@monjalon.net>,
	"Ferruh Yigit" <ferruh.yigit@intel.com>,
	Bruce Richardson <bruce.richardson@intel.com>
Cc: "jerinj@marvell.com" <jerinj@marvell.com>,
	"John Daley (johndale)" <johndale@cisco.com>,
	Shahed Shaikh <shshaikh@marvell.com>,
	"dev@dpdk.org" <dev@dpdk.org>
Subject: Re: [dpdk-dev] [RFC PATCH v3 2/3] eal: add mask and unmask interrupt APIs
Date: Wed, 17 Jul 2019 05:55:56 +0000	[thread overview]
Message-ID: <MWHPR11MB1839A88FBAD5C2ECCB25CE62BFC90@MWHPR11MB1839.namprd11.prod.outlook.com> (raw)
In-Reply-To: <20190716164424.16776-2-ndabilpuram@marvell.com>

> -----Original Message-----
> From: Nithin Dabilpuram <ndabilpuram@marvell.com>
> Sent: Wednesday, July 17, 2019 1:44 AM
> To: Hyong Youb Kim (hyonkim) <hyonkim@cisco.com>; David Marchand
> <david.marchand@redhat.com>; Thomas Monjalon
> <thomas@monjalon.net>; Ferruh Yigit <ferruh.yigit@intel.com>; Bruce
> Richardson <bruce.richardson@intel.com>
> Cc: jerinj@marvell.com; John Daley (johndale) <johndale@cisco.com>;
> Shahed Shaikh <shshaikh@marvell.com>; dev@dpdk.org; Nithin Dabilpuram
> <ndabilpuram@marvell.com>
> Subject: [RFC PATCH v3 2/3] eal: add mask and unmask interrupt APIs
> 
> Add new mask and unmask interrupt APIs to avoid using
> VFIO_IRQ_SET_ACTION_TRIGGER for masking/unmasking purpose for VFIO
> based handlers. This implementation is specific to Linux.
> 
> Using action trigger for masking and unmasking has below issues
> 
>  * Time consuming to do for every interrupt received as it will
>    free_irq() followed by request_irq() and all other initializations
>  * A race condition because of a window between free_irq() and
>    request_irq() with packet reception still on and device still
>    enabled and would throw warning messages like below.
>    [158764.159833] do_IRQ: 9.34 No irq handler for vector
> 
> In this patch, mask/unmask is a no-op for VFIO_MSIX/VFIO_MSI interrupts
> as they are edge triggered and kernel would not mask the interrupt before
> delivering the event to userspace.
> 
> Signed-off-by: Nithin Dabilpuram <ndabilpuram@marvell.com>
> ---
> v3:
> * Re-org patch to move driver change to 3/3
> * Add stub implementation for freebsd.
> * Fix version map file for new apis.
> v2:
> Make change in qede driver for unmask
>  lib/librte_eal/common/include/rte_interrupts.h |  23 ++++
>  lib/librte_eal/freebsd/eal/eal_interrupts.c    |  54 +++++++++
>  lib/librte_eal/linux/eal/eal_interrupts.c      | 155
> +++++++++++++++++++++++++
>  lib/librte_eal/rte_eal_version.map             |   2 +
>  4 files changed, 234 insertions(+)
> 
> diff --git a/lib/librte_eal/common/include/rte_interrupts.h
> b/lib/librte_eal/common/include/rte_interrupts.h
> index c1e912c..b0675be 100644
> --- a/lib/librte_eal/common/include/rte_interrupts.h
> +++ b/lib/librte_eal/common/include/rte_interrupts.h
> @@ -118,6 +118,29 @@ int rte_intr_enable(const struct rte_intr_handle
> *intr_handle);
>   */
>  int rte_intr_disable(const struct rte_intr_handle *intr_handle);
> 
> +/**
> + * It masks the interrupt for the specified handle.
> + *
> + * @param intr_handle
> + *  pointer to the interrupt handle.
> + *
> + * @return
> + *  - On success, zero.
> + *  - On failure, a negative value.
> + */
> +int rte_intr_mask(const struct rte_intr_handle *intr_handle);
> +
> +/**
> + * It unmasks the interrupt for the specified handle.
> + *
> + * @param intr_handle
> + *  pointer to the interrupt handle.
> + *
> + * @return
> + *  - On success, zero.
> + *  - On failure, a negative value.
> + */
> +int rte_intr_unmask(const struct rte_intr_handle *intr_handle);
>  #ifdef __cplusplus
>  }
>  #endif
[...]
> diff --git a/lib/librte_eal/linux/eal/eal_interrupts.c
> b/lib/librte_eal/linux/eal/eal_interrupts.c
> index 79ad5e8..f619981 100644
> --- a/lib/librte_eal/linux/eal/eal_interrupts.c
> +++ b/lib/librte_eal/linux/eal/eal_interrupts.c
[...]
>  int
> +rte_intr_mask(const struct rte_intr_handle *intr_handle)
> +{
> +	if (intr_handle && intr_handle->type == RTE_INTR_HANDLE_VDEV)
> +		return 0;
> +
> +	if (!intr_handle || intr_handle->fd < 0 || intr_handle->uio_cfg_fd < 0)
> +		return -1;
> +
> +	switch (intr_handle->type){
> +	/* Both masking and disabling are same for UIO */
> +	case RTE_INTR_HANDLE_UIO:
> +		if (uio_intr_disable(intr_handle))
> +			return -1;
> +		break;
> +	case RTE_INTR_HANDLE_UIO_INTX:
> +		if (uio_intx_intr_disable(intr_handle))
> +			return -1;
> +		break;
> +	/* not used at this moment */
> +	case RTE_INTR_HANDLE_ALARM:
> +		return -1;
> +#ifdef VFIO_PRESENT
> +	case RTE_INTR_HANDLE_VFIO_MSIX:
> +	case RTE_INTR_HANDLE_VFIO_MSI:
> +		return 0;

Isn't this a little confusing? It returns success, but irq is not
masked.

As is, return code 0 means...
- igb_uio: irq is masked for INTx, MSI, MSI-X
- vfio-pci + INTx: irq is masked
- vfio-pci + MSI/MSI-X: no changes

Masking is useful only for INTx, IMO...

Masking MSI/MSI-X via PCI-defined mechanisms (e.g. Mask bit in MSI-X
Table) has no practical use for drivers. Handshaking/masking/unmasking
is done via device/vendor specific ways, as needed. See all those
ack/block/unblock/credit/... mechanisms used in various drivers/NICs
to control interrupts their own way.

A long time ago in early PCIe days, the linux kernel did auto-masking
for MSI/MSI-X (i.e. mask before calling netdev irq handler). It was
soon removed as it was unnecessary overhead (expensive PIOs to NIC for
every interrupt). Windows and FreeBSD do not do auto-masking either.

Thanks.
-Hyong

> +	case RTE_INTR_HANDLE_VFIO_LEGACY:
> +		if (vfio_mask_intx(intr_handle))
> +			return -1;
> +		break;
> +#ifdef HAVE_VFIO_DEV_REQ_INTERFACE
> +	case RTE_INTR_HANDLE_VFIO_REQ:
> +		return -1;
> +#endif
> +#endif
> +	/* not used at this moment */
> +	case RTE_INTR_HANDLE_DEV_EVENT:
> +		return -1;
> +	/* unknown handle type */
> +	default:
> +		RTE_LOG(ERR, EAL,
> +			"Unknown handle type of fd %d\n",
> +					intr_handle->fd);
> +		return -1;
> +	}
> +
> +	return 0;
> +}
> +
> +int
> +rte_intr_unmask(const struct rte_intr_handle *intr_handle)
> +{
> +	if (intr_handle && intr_handle->type == RTE_INTR_HANDLE_VDEV)
> +		return 0;
> +
> +	if (!intr_handle || intr_handle->fd < 0 || intr_handle->uio_cfg_fd < 0)
> +		return -1;
> +
> +	switch (intr_handle->type){
> +	/* Both unmasking and disabling are same for UIO */
> +	case RTE_INTR_HANDLE_UIO:
> +		if (uio_intr_enable(intr_handle))
> +			return -1;
> +		break;
> +	case RTE_INTR_HANDLE_UIO_INTX:
> +		if (uio_intx_intr_enable(intr_handle))
> +			return -1;
> +		break;
> +	/* not used at this moment */
> +	case RTE_INTR_HANDLE_ALARM:
> +		return -1;
> +#ifdef VFIO_PRESENT
> +	case RTE_INTR_HANDLE_VFIO_MSIX:
> +	case RTE_INTR_HANDLE_VFIO_MSI:
> +		return 0;
> +	case RTE_INTR_HANDLE_VFIO_LEGACY:
> +		if (vfio_unmask_intx(intr_handle))
> +			return -1;
> +		break;
> +#ifdef HAVE_VFIO_DEV_REQ_INTERFACE
> +	case RTE_INTR_HANDLE_VFIO_REQ:
> +		return -1;
> +#endif
> +#endif
> +	/* not used at this moment */
> +	case RTE_INTR_HANDLE_DEV_EVENT:
> +		return -1;
> +	/* unknown handle type */
> +	default:
> +		RTE_LOG(ERR, EAL,
> +			"Unknown handle type of fd %d\n",
> +					intr_handle->fd);
> +		return -1;
> +	}
> +
> +	return 0;
> +}
> +
> +int
>  rte_intr_disable(const struct rte_intr_handle *intr_handle)
>  {
>  	if (intr_handle && intr_handle->type == RTE_INTR_HANDLE_VDEV)
> diff --git a/lib/librte_eal/rte_eal_version.map
> b/lib/librte_eal/rte_eal_version.map
> index 1892d9e..b3b2df4 100644
> --- a/lib/librte_eal/rte_eal_version.map
> +++ b/lib/librte_eal/rte_eal_version.map
> @@ -309,6 +309,8 @@ DPDK_19.08 {
>  	rte_service_lcore_attr_reset_all;
>  	rte_service_may_be_active;
>  	rte_srand;
> +	rte_intr_unmask;
> +	rte_intr_mask;
> 
>  } DPDK_19.05;
> 
> --
> 2.8.4


  reply	other threads:[~2019-07-17  5:56 UTC|newest]

Thread overview: 24+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2019-07-15 16:50 [dpdk-dev] [RFC PATCH] vfio: avoid re-installing irq handler Jerin Jacob Kollanukkaran
2019-07-16  5:58 ` Hyong Youb Kim (hyonkim)
2019-07-16  6:47   ` Jerin Jacob Kollanukkaran
2019-07-16  7:49     ` Hyong Youb Kim (hyonkim)
2019-07-16  9:56       ` Jerin Jacob Kollanukkaran
2019-07-16  6:46 ` [dpdk-dev] [RFC PATCH] eal: add mask and unmask interrupt apis Nithin Dabilpuram
2019-07-16  7:01 ` [dpdk-dev] [RFC PATCH v2] " Nithin Dabilpuram
2019-07-16 16:44 ` [dpdk-dev] [RFC PATCH v3 1/3] vfio: revert change that does intr eventfd setup at probe Nithin Dabilpuram
2019-07-16 16:44   ` [dpdk-dev] [RFC PATCH v3 2/3] eal: add mask and unmask interrupt APIs Nithin Dabilpuram
2019-07-17  5:55     ` Hyong Youb Kim (hyonkim) [this message]
2019-07-17  6:14       ` Jerin Jacob Kollanukkaran
2019-07-17  7:09         ` Hyong Youb Kim (hyonkim)
2019-07-17  8:03           ` Jerin Jacob Kollanukkaran
2019-07-17  8:45             ` Hyong Youb Kim (hyonkim)
2019-07-17  9:20               ` Jerin Jacob Kollanukkaran
2019-07-17  9:51                 ` Hyong Youb Kim (hyonkim)
2019-07-17 10:43                   ` Jerin Jacob Kollanukkaran
2019-07-17 11:06                     ` Hyong Youb Kim (hyonkim)
2019-07-17 11:16                       ` Jerin Jacob Kollanukkaran
2019-07-17 12:04                         ` Nithin Kumar Dabilpuram
2019-07-16 16:44   ` [dpdk-dev] [RFC PATCH v3 3/3] drivers/net: use unmask API in interrupt handlers Nithin Dabilpuram
2019-07-17  6:01     ` Hyong Youb Kim (hyonkim)
2019-07-17  7:47       ` Nithin Kumar Dabilpuram
2019-07-16 20:06   ` [dpdk-dev] [RFC PATCH v3 1/3] vfio: revert change that does intr eventfd setup at probe Stephen Hemminger

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=MWHPR11MB1839A88FBAD5C2ECCB25CE62BFC90@MWHPR11MB1839.namprd11.prod.outlook.com \
    --to=hyonkim@cisco.com \
    --cc=bruce.richardson@intel.com \
    --cc=david.marchand@redhat.com \
    --cc=dev@dpdk.org \
    --cc=ferruh.yigit@intel.com \
    --cc=jerinj@marvell.com \
    --cc=johndale@cisco.com \
    --cc=ndabilpuram@marvell.com \
    --cc=shshaikh@marvell.com \
    --cc=thomas@monjalon.net \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).