[dpdk-dev] [PATCH 0/2] net/mlx5 add rxq interrupt support

[dpdk-dev] [PATCH 0/2] net/mlx5 add rxq interrupt support

[Shahaf Shuler]

This patchset adds support for rxq interrupts on MLX5 PMD.
The first patch introduces changes on eal_interrupt to support external interrupt handlers.
The second patch implements the support on mlx5 PMD and demonstrate the use of the changes made in the first patch

[PATCH 1/2] eal_interrupts: support external rxq interrupt handlers
[PATCH 2/2] net/mlx5: support user space rxq interrupt event

[dpdk-dev] [PATCH 1/2] eal_interrupts: support external rxq interrupt handlers

[Shahaf Shuler]

Prior to this patch only UIO/VFIO interrupt handlers types were supported.
This patch adds support for the external interrupt handler type, allowing
external drivers to set their own fds with specific interrupt handlers.

Signed-off-by: Shahaf Shuler <shahafs@mellanox.com>
---
 lib/librte_eal/linuxapp/eal/eal_interrupts.c       | 33 ++++++++++++++--------
 .../linuxapp/eal/include/exec-env/rte_interrupts.h |  9 ++++++
 2 files changed, 30 insertions(+), 12 deletions(-)

diff --git a/lib/librte_eal/linuxapp/eal/eal_interrupts.c b/lib/librte_eal/linuxapp/eal/eal_interrupts.c
index b5b3f2b..c4465f9 100644
--- a/lib/librte_eal/linuxapp/eal/eal_interrupts.c
+++ b/lib/librte_eal/linuxapp/eal/eal_interrupts.c
@@ -937,6 +937,8 @@ static __attribute__((noreturn)) void *
 		bytes_read = sizeof(buf.vfio_intr_count);
 		break;
 #endif
+	case RTE_INTR_HANDLE_EXT:
+		return;
 	default:
 		bytes_read = 1;
 		RTE_LOG(INFO, EAL, "unexpected intr type\n");
@@ -1165,6 +1167,24 @@ static __attribute__((noreturn)) void *
 	return rc;
 }
 
+void
+rte_intr_free_epoll_fd(struct rte_intr_handle *intr_handle)
+{
+	uint32_t i;
+	struct rte_epoll_event *rev;
+
+	for (i = 0; i < intr_handle->nb_efd; i++) {
+		rev = &intr_handle->elist[i];
+		if (rev->status == RTE_EPOLL_INVALID)
+			continue;
+		if (rte_epoll_ctl(rev->epfd, EPOLL_CTL_DEL, rev->fd, rev)) {
+			/* force free if the entry valid */
+			eal_epoll_data_safe_free(rev);
+			rev->status = RTE_EPOLL_INVALID;
+		}
+	}
+}
+
 int
 rte_intr_efd_enable(struct rte_intr_handle *intr_handle, uint32_t nb_efd)
 {
@@ -1200,19 +1220,8 @@ static __attribute__((noreturn)) void *
 rte_intr_efd_disable(struct rte_intr_handle *intr_handle)
 {
 	uint32_t i;
-	struct rte_epoll_event *rev;
-
-	for (i = 0; i < intr_handle->nb_efd; i++) {
-		rev = &intr_handle->elist[i];
-		if (rev->status == RTE_EPOLL_INVALID)
-			continue;
-		if (rte_epoll_ctl(rev->epfd, EPOLL_CTL_DEL, rev->fd, rev)) {
-			/* force free if the entry valid */
-			eal_epoll_data_safe_free(rev);
-			rev->status = RTE_EPOLL_INVALID;
-		}
-	}
 
+	rte_intr_free_epoll_fd(intr_handle);
 	if (intr_handle->max_intr > intr_handle->nb_efd) {
 		for (i = 0; i < intr_handle->nb_efd; i++)
 			close(intr_handle->efds[i]);
diff --git a/lib/librte_eal/linuxapp/eal/include/exec-env/rte_interrupts.h b/lib/librte_eal/linuxapp/eal/include/exec-env/rte_interrupts.h
index d459bf4..735d307 100644
--- a/lib/librte_eal/linuxapp/eal/include/exec-env/rte_interrupts.h
+++ b/lib/librte_eal/linuxapp/eal/include/exec-env/rte_interrupts.h
@@ -171,6 +171,15 @@ struct rte_intr_handle {
 		int epfd, int op, unsigned int vec, void *data);
 
 /**
+ * It deletes registered eventfds.
+ *
+ * @param intr_handle
+ *   Pointer to the interrupt handle.
+ */
+void
+rte_intr_free_epoll_fd(struct rte_intr_handle *intr_handle);
+
+/**
  * It enables the packet I/O interrupt event if it's necessary.
  * It creates event fd for each interrupt vector when MSIX is used,
  * otherwise it multiplexes a single event fd.
-- 
1.8.3.1

Re: [dpdk-dev] [PATCH 1/2] eal_interrupts: support external rxq interrupt handlers

[Thomas Monjalon]

2017-02-08 15:57, Shahaf Shuler:
>  /**
> + * It deletes registered eventfds.
> + *
> + * @param intr_handle
> + *   Pointer to the interrupt handle.
> + */
> +void
> +rte_intr_free_epoll_fd(struct rte_intr_handle *intr_handle);

This patch looks OK except a miss in the EAL map files for this new function.
The title should be "eal/linux: support external Rx interrupt"

I think this patch should target next-net.
Please Ferruh, take care of this patchset. Thanks

Re: [dpdk-dev] [PATCH 1/2] eal_interrupts: support external rxq interrupt handlers

[Shahaf Shuler]

Thursday, March 9, 2017 8:10 PM, Thomas Monjalon:
> 2017-02-08 15:57, Shahaf Shuler:
> >  /**
> > + * It deletes registered eventfds.
> > + *
> > + * @param intr_handle
> > + *   Pointer to the interrupt handle.
> > + */
> > +void
> > +rte_intr_free_epoll_fd(struct rte_intr_handle *intr_handle);
> 
> This patch looks OK except a miss in the EAL map files for this new function.
> The title should be "eal/linux: support external Rx interrupt"
> 
> I think this patch should target next-net.
> Please Ferruh, take care of this patchset. Thanks

Ferruh, any more comments before submitting a v2? 

Re: [dpdk-dev] [PATCH 1/2] eal_interrupts: support external rxq interrupt handlers

[Ferruh Yigit]

On 3/12/2017 8:30 AM, Shahaf Shuler wrote:
> Thursday, March 9, 2017 8:10 PM, Thomas Monjalon:
>> 2017-02-08 15:57, Shahaf Shuler:
>>>  /**
>>> + * It deletes registered eventfds.
>>> + *
>>> + * @param intr_handle
>>> + *   Pointer to the interrupt handle.
>>> + */
>>> +void
>>> +rte_intr_free_epoll_fd(struct rte_intr_handle *intr_handle);
>>
>> This patch looks OK except a miss in the EAL map files for this new function.
>> The title should be "eal/linux: support external Rx interrupt"
>>
>> I think this patch should target next-net.
>> Please Ferruh, take care of this patchset. Thanks
> 
> Ferruh, any more comments before submitting a v2? 

Hi Shahaf,

No extra comments, only second patch does not cleanly apply to next-net,
please rebase it to before sending.

Thanks,
ferruh

[dpdk-dev] [PATCH 2/2] net/mlx5: support user space rxq interrupt event

[Shahaf Shuler]

Implement rxq interrupt callbacks

Signed-off-by: Shahaf Shuler <shahafs@mellanox.com>
---
 drivers/net/mlx5/Makefile       |   5 ++
 drivers/net/mlx5/mlx5.c         |   2 +
 drivers/net/mlx5/mlx5_rxq.c     | 126 +++++++++++++++++++++++++++++++++++++++-
 drivers/net/mlx5/mlx5_rxtx.c    |  73 +++++++++++++++++++++++
 drivers/net/mlx5/mlx5_rxtx.h    |   7 +++
 drivers/net/mlx5/mlx5_trigger.c |   9 +++
 6 files changed, 221 insertions(+), 1 deletion(-)

diff --git a/drivers/net/mlx5/Makefile b/drivers/net/mlx5/Makefile
index 671089c..084044a 100644
--- a/drivers/net/mlx5/Makefile
+++ b/drivers/net/mlx5/Makefile
@@ -137,6 +137,11 @@ mlx5_autoconf.h.new: $(RTE_SDK)/buildtools/auto-config-h.sh
 		/usr/include/linux/ethtool.h \
 		enum ETHTOOL_LINK_MODE_100000baseKR4_Full_BIT \
 		$(AUTOCONF_OUTPUT)
+	$Q sh -- '$<' '$@' \
+		HAVE_UPDATE_CQ_CI \
+		infiniband/mlx5_hw.h \
+		func ibv_mlx5_exp_update_cq_ci \
+		$(AUTOCONF_OUTPUT)
 
 # Create mlx5_autoconf.h or update it in case it differs from the new one.
 
diff --git a/drivers/net/mlx5/mlx5.c b/drivers/net/mlx5/mlx5.c
index 3fa555d..dbf3f6e 100644
--- a/drivers/net/mlx5/mlx5.c
+++ b/drivers/net/mlx5/mlx5.c
@@ -225,6 +225,8 @@
 	.rss_hash_update = mlx5_rss_hash_update,
 	.rss_hash_conf_get = mlx5_rss_hash_conf_get,
 	.filter_ctrl = mlx5_dev_filter_ctrl,
+	.rx_queue_intr_enable = mlx5_rx_intr_enable,
+	.rx_queue_intr_disable = mlx5_rx_intr_disable,
 };
 
 static struct {
diff --git a/drivers/net/mlx5/mlx5_rxq.c b/drivers/net/mlx5/mlx5_rxq.c
index 28e93d3..e6070a0 100644
--- a/drivers/net/mlx5/mlx5_rxq.c
+++ b/drivers/net/mlx5/mlx5_rxq.c
@@ -36,6 +36,7 @@
 #include <errno.h>
 #include <string.h>
 #include <stdint.h>
+#include <fcntl.h>
 
 /* Verbs header. */
 /* ISO C doesn't support unnamed structs/unions, disabling -pedantic. */
@@ -57,6 +58,7 @@
 #include <rte_malloc.h>
 #include <rte_ethdev.h>
 #include <rte_common.h>
+#include <rte_interrupts.h>
 #ifdef PEDANTIC
 #pragma GCC diagnostic error "-Wpedantic"
 #endif
@@ -773,6 +775,8 @@
 		claim_zero(ibv_exp_destroy_wq(rxq_ctrl->wq));
 	if (rxq_ctrl->cq != NULL)
 		claim_zero(ibv_destroy_cq(rxq_ctrl->cq));
+	if (rxq_ctrl->channel != NULL)
+		claim_zero(ibv_destroy_comp_channel(rxq_ctrl->channel));
 	if (rxq_ctrl->rd != NULL) {
 		struct ibv_exp_destroy_res_domain_attr attr = {
 			.comp_mask = 0,
@@ -1014,6 +1018,16 @@
 		      (void *)dev, strerror(ret));
 		goto error;
 	}
+	if (dev->data->dev_conf.intr_conf.rxq) {
+		tmpl.channel = ibv_create_comp_channel(priv->ctx);
+		if (tmpl.channel == NULL) {
+			dev->data->dev_conf.intr_conf.rxq = 0;
+			ret = ENOMEM;
+			ERROR("%p: Comp Channel creation failure: %s",
+			(void *)dev, strerror(ret));
+			goto error;
+		}
+	}
 	attr.cq = (struct ibv_exp_cq_init_attr){
 		.comp_mask = IBV_EXP_CQ_INIT_ATTR_RES_DOMAIN,
 		.res_domain = tmpl.rd,
@@ -1023,7 +1037,7 @@
 		attr.cq.flags |= IBV_EXP_CQ_COMPRESSED_CQE;
 		cqe_n = (desc * 2) - 1; /* Double the number of CQEs. */
 	}
-	tmpl.cq = ibv_exp_create_cq(priv->ctx, cqe_n, NULL, NULL, 0,
+	tmpl.cq = ibv_exp_create_cq(priv->ctx, cqe_n, NULL, tmpl.channel, 0,
 				    &attr.cq);
 	if (tmpl.cq == NULL) {
 		ret = ENOMEM;
@@ -1347,3 +1361,113 @@
 	rxq = (*priv->rxqs)[index];
 	return priv->dev->rx_pkt_burst(rxq, pkts, pkts_n);
 }
+
+/**
+ * Fill epoll fd list for rxq interrupts.
+ *
+ * @param priv
+ *   Private structure.
+ *
+ * @return
+ *   0 on success, negative on failure.
+ */
+int
+priv_intr_efd_enable(struct priv *priv)
+{
+	unsigned int i;
+	unsigned int rxqs_n = priv->rxqs_n;
+	unsigned int n = RTE_MIN(rxqs_n, (uint32_t)RTE_MAX_RXTX_INTR_VEC_ID);
+	struct rte_intr_handle *intr_handle = priv->dev->intr_handle;
+
+	if (n == 0)
+		return 0;
+	if (n < rxqs_n) {
+		WARN("rxqs num is larger than EAL max interrupt vector "
+		     "%u > %u unable to supprt rxq interrupts",
+		     rxqs_n, (uint32_t)RTE_MAX_RXTX_INTR_VEC_ID);
+		return -EINVAL;
+	}
+	intr_handle->type = RTE_INTR_HANDLE_EXT;
+	for (i = 0; i != n; ++i) {
+		struct rxq *rxq = (*priv->rxqs)[i];
+		struct rxq_ctrl *rxq_ctrl =
+			container_of(rxq, struct rxq_ctrl, rxq);
+		int fd = rxq_ctrl->channel->fd;
+		int flags;
+		int rc;
+
+		flags = fcntl(fd, F_GETFL);
+		rc = fcntl(fd, F_SETFL, flags | O_NONBLOCK);
+		if (rc < 0) {
+			WARN("failed to change rxq interrupt file "
+			     "descriptor %d for queue index %d", fd, i);
+			return -1;
+		}
+		intr_handle->efds[i] = fd;
+	}
+	intr_handle->nb_efd = n;
+	return 0;
+}
+
+/**
+ * Clean epoll fd list for rxq interrupts.
+ *
+ * @param priv
+ *   Private structure.
+ */
+void
+priv_intr_efd_disable(struct priv *priv)
+{
+	struct rte_intr_handle *intr_handle = priv->dev->intr_handle;
+
+	rte_intr_free_epoll_fd(intr_handle);
+}
+
+/**
+ * Create and init interrupt vector array.
+ *
+ * @param priv
+ *   Private structure.
+ *
+ * @return
+ *   0 on success, negative on failure.
+ */
+int
+priv_create_intr_vec(struct priv *priv)
+{
+	unsigned int rxqs_n = priv->rxqs_n;
+	unsigned int i;
+	struct rte_intr_handle *intr_handle = priv->dev->intr_handle;
+
+	if (rxqs_n == 0)
+		return 0;
+	intr_handle->intr_vec = (int *)
+		rte_malloc("intr_vec", rxqs_n * sizeof(int), 0);
+	if (intr_handle->intr_vec == NULL) {
+		WARN("Failed to allocate memory for intr_vec "
+		     "rxq interrupt will not be supported");
+		return -ENOMEM;
+	}
+	for (i = 0; i != rxqs_n; ++i) {
+		/* 1:1 mapping between rxq and interrupt. */
+		intr_handle->intr_vec[i] = RTE_INTR_VEC_RXTX_OFFSET + i;
+	}
+	return 0;
+}
+
+/**
+ * Destroy init interrupt vector array.
+ *
+ * @param priv
+ *   Private structure.
+ *
+ * @return
+ *   0 on success, negative on failure.
+ */
+void
+priv_destroy_intr_vec(struct priv *priv)
+{
+	struct rte_intr_handle *intr_handle = priv->dev->intr_handle;
+
+	rte_free(intr_handle->intr_vec);
+}
diff --git a/drivers/net/mlx5/mlx5_rxtx.c b/drivers/net/mlx5/mlx5_rxtx.c
index 53a7bc7..0faa018 100644
--- a/drivers/net/mlx5/mlx5_rxtx.c
+++ b/drivers/net/mlx5/mlx5_rxtx.c
@@ -1631,3 +1631,76 @@
 	(void)pkts_n;
 	return 0;
 }
+
+/**
+ * DPDK callback for rx queue interrupt enable.
+ *
+ * @param dev
+ *   Pointer to Ethernet device structure.
+ * @param rx_queue_id
+ *   RX queue number
+ *
+ * @return
+ *   0 on success, negative on failure.
+ */
+int
+mlx5_rx_intr_enable(struct rte_eth_dev *dev, uint16_t rx_queue_id)
+{
+#ifdef HAVE_UPDATE_CQ_CI
+	struct priv *priv = mlx5_get_priv(dev);
+	struct rxq *rxq = (*priv->rxqs)[rx_queue_id];
+	struct rxq_ctrl *rxq_ctrl = container_of(rxq, struct rxq_ctrl, rxq);
+	struct ibv_cq *cq = rxq_ctrl->cq;
+	uint16_t ci = rxq->cq_ci;
+	int ret = 0;
+
+	ibv_mlx5_exp_update_cq_ci(cq, ci);
+	ret = ibv_req_notify_cq(cq, 0);
+#else
+	int ret = -1;
+	(void)dev;
+	(void)rx_queue_id;
+#endif
+	if (ret)
+		WARN("unable to arm interrupt on rx queue %d", rx_queue_id);
+	return ret;
+}
+
+/**
+ * DPDK callback for rx queue interrupt disable.
+ *
+ * @param dev
+ *   Pointer to Ethernet device structure.
+ * @param rx_queue_id
+ *   RX queue number
+ *
+ * @return
+ *   0 on success, negative on failure.
+ */
+int
+mlx5_rx_intr_disable(struct rte_eth_dev *dev, uint16_t rx_queue_id)
+{
+#ifdef HAVE_UPDATE_CQ_CI
+	struct priv *priv = mlx5_get_priv(dev);
+	struct rxq *rxq = (*priv->rxqs)[rx_queue_id];
+	struct rxq_ctrl *rxq_ctrl = container_of(rxq, struct rxq_ctrl, rxq);
+	struct ibv_cq *cq = rxq_ctrl->cq;
+	struct ibv_cq *ev_cq;
+	void *ev_ctx;
+	int ret = 0;
+
+	ret = ibv_get_cq_event(cq->channel, &ev_cq, &ev_ctx);
+	if (ret || ev_cq != cq)
+		ret = -1;
+	else
+		ibv_ack_cq_events(cq, 1);
+#else
+	int ret = -1;
+	(void)dev;
+	(void)rx_queue_id;
+#endif
+	if (ret)
+		WARN("unable to disable interrupt on rx queue %d",
+		     rx_queue_id);
+	return ret;
+}
diff --git a/drivers/net/mlx5/mlx5_rxtx.h b/drivers/net/mlx5/mlx5_rxtx.h
index 302ca49..6e2c099 100644
--- a/drivers/net/mlx5/mlx5_rxtx.h
+++ b/drivers/net/mlx5/mlx5_rxtx.h
@@ -138,6 +138,7 @@ struct rxq_ctrl {
 	struct ibv_mr *mr; /* Memory Region (for mp). */
 	struct ibv_exp_wq_family *if_wq; /* WQ burst interface. */
 	struct ibv_exp_cq_family_v1 *if_cq; /* CQ interface. */
+	struct ibv_comp_channel *channel;
 	unsigned int socket; /* CPU socket ID for allocations. */
 	struct rxq rxq; /* Data path structure. */
 };
@@ -294,6 +295,10 @@ size_t priv_flow_attr(struct priv *, struct ibv_exp_flow_attr *,
 void priv_destroy_hash_rxqs(struct priv *);
 int priv_allow_flow_type(struct priv *, enum hash_rxq_flow_type);
 int priv_rehash_flows(struct priv *);
+int priv_intr_efd_enable(struct priv *priv);
+void priv_intr_efd_disable(struct priv *priv);
+int priv_create_intr_vec(struct priv *priv);
+void priv_destroy_intr_vec(struct priv *priv);
 void rxq_cleanup(struct rxq_ctrl *);
 int rxq_rehash(struct rte_eth_dev *, struct rxq_ctrl *);
 int rxq_ctrl_setup(struct rte_eth_dev *, struct rxq_ctrl *, uint16_t,
@@ -322,6 +327,8 @@ int mlx5_tx_queue_setup(struct rte_eth_dev *, uint16_t, uint16_t, unsigned int,
 uint16_t mlx5_rx_burst(void *, struct rte_mbuf **, uint16_t);
 uint16_t removed_tx_burst(void *, struct rte_mbuf **, uint16_t);
 uint16_t removed_rx_burst(void *, struct rte_mbuf **, uint16_t);
+int mlx5_rx_intr_enable(struct rte_eth_dev *dev, uint16_t rx_queue_id);
+int mlx5_rx_intr_disable(struct rte_eth_dev *dev, uint16_t rx_queue_id);
 
 /* mlx5_mr.c */
 
diff --git a/drivers/net/mlx5/mlx5_trigger.c b/drivers/net/mlx5/mlx5_trigger.c
index 30addd2..4bc5b7a 100644
--- a/drivers/net/mlx5/mlx5_trigger.c
+++ b/drivers/net/mlx5/mlx5_trigger.c
@@ -91,6 +91,11 @@
 		priv_fdir_enable(priv);
 	priv_dev_interrupt_handler_install(priv, dev);
 	err = priv_flow_start(priv);
+	if (dev->data->dev_conf.intr_conf.rxq) {
+		err = priv_intr_efd_enable(priv);
+		if (!err)
+			err = priv_create_intr_vec(priv);
+	}
 	priv_xstats_init(priv);
 	priv_unlock(priv);
 	return -err;
@@ -124,6 +129,10 @@
 	priv_fdir_disable(priv);
 	priv_flow_stop(priv);
 	priv_dev_interrupt_handler_uninstall(priv, dev);
+	if (priv->dev->data->dev_conf.intr_conf.rxq) {
+		priv_destroy_intr_vec(priv);
+		priv_intr_efd_disable(priv);
+	}
 	priv->started = 0;
 	priv_unlock(priv);
 }
-- 
1.8.3.1

Re: [dpdk-dev] [PATCH 0/2] net/mlx5 add rxq interrupt support

[Ferruh Yigit]

On 2/8/2017 1:57 PM, Shahaf Shuler wrote:
> This patchset adds support for rxq interrupts on MLX5 PMD.
> The first patch introduces changes on eal_interrupt to support external interrupt handlers.
> The second patch implements the support on mlx5 PMD and demonstrate the use of the changes made in the first patch
> 
> [PATCH 1/2] eal_interrupts: support external rxq interrupt handlers
> [PATCH 2/2] net/mlx5: support user space rxq interrupt event

Hi Shahaf,

I assume this patch is for 17.05 release, can you please confirm?

Thanks,
ferruh

Re: [dpdk-dev] [PATCH 0/2] net/mlx5 add rxq interrupt support

[Shahaf Shuler]

Thursday, February 9, 2017 4:48 PM, Ferruh Yigit:
> From: Ferruh Yigit [mailto:ferruh.yigit@intel.com]
> Sent: Thursday, February 9, 2017 4:48 PM
> To: Shahaf Shuler <shahafs@mellanox.com>; david.marchand@6wind.com;
> Adrien Mazarguil <adrien.mazarguil@6wind.com>; Nélio Laranjeiro
> <nelio.laranjeiro@6wind.com>
> Cc: dev@dpdk.org
> Subject: Re: [dpdk-dev] [PATCH 0/2] net/mlx5 add rxq interrupt support
> 
> On 2/8/2017 1:57 PM, Shahaf Shuler wrote:
> > This patchset adds support for rxq interrupts on MLX5 PMD.
> > The first patch introduces changes on eal_interrupt to support external
> interrupt handlers.
> > The second patch implements the support on mlx5 PMD and demonstrate
> > the use of the changes made in the first patch
> >
> > [PATCH 1/2] eal_interrupts: support external rxq interrupt handlers
> > [PATCH 2/2] net/mlx5: support user space rxq interrupt event
> 
> Hi Shahaf,
> 
> I assume this patch is for 17.05 release, can you please confirm?

Correct. 

> 
> Thanks,
> ferruh
> 

Re: [dpdk-dev] [PATCH 0/2] net/mlx5 add rxq interrupt support

[Shahaf Shuler]

Wednesday, February 8, 2017 3:58 PM, Shahaf Shuler:
> Subject: [dpdk-dev] [PATCH 0/2] net/mlx5 add rxq interrupt support
> 
> This patchset adds support for rxq interrupts on MLX5 PMD.
> The first patch introduces changes on eal_interrupt to support external
> interrupt handlers.
> The second patch implements the support on mlx5 PMD and demonstrate
> the use of the changes made in the first patch
> 
> [PATCH 1/2] eal_interrupts: support external rxq interrupt handlers [PATCH
> 2/2] net/mlx5: support user space rxq interrupt event

Hi,
Any comments regarding this patchset? 

Re: [dpdk-dev] [PATCH 0/2] net/mlx5 add rxq interrupt support

[Shahaf Shuler]

Tuesday, February 21, 2017 1:40 PM, Shahaf Shuler:

> Wednesday, February 8, 2017 3:58 PM, Shahaf Shuler:
> > Subject: [dpdk-dev] [PATCH 0/2] net/mlx5 add rxq interrupt support
> >
> > This patchset adds support for rxq interrupts on MLX5 PMD.
> > The first patch introduces changes on eal_interrupt to support
> > external interrupt handlers.
> > The second patch implements the support on mlx5 PMD and demonstrate
> > the use of the changes made in the first patch
> >
> > [PATCH 1/2] eal_interrupts: support external rxq interrupt handlers
> > [PATCH 2/2] net/mlx5: support user space rxq interrupt event
> 
> Hi,
> Any comments regarding this patchset?
> 
> 

Just understood Thomas is the current maintainer of EAL (maybe need to update MAINTAINERS file).

Thomas, this patch set is on the mailing list for a while. Can you please review and comment? 

[dpdk-dev] [PATCH v2 0/2] net/mlx5 add rxq interrupt support

[Shahaf Shuler]

This patchset adds support for rxq interrupts on MLX5 PMD.
The first patch introduces changes on eal_interrupt to support external interrupt handlers.
The second patch implements the support on mlx5 PMD and demonstrate the use of the changes made in the first patch

on v2:
 * add new function to EAL map file.
 * change commit title.
 * add documentation to guide and release notes.

[PATCH v2 1/2] eal/linux: support external Rx interrupt
[PATCH v2 2/2] net/mlx5: support user space rxq interrupt event

[dpdk-dev] [PATCH v2 1/2] eal/linux: support external Rx interrupt

[Shahaf Shuler]

Prior to this patch only UIO/VFIO interrupt handlers types were supported.
This patch adds support for the external interrupt handler type, allowing
external drivers to set their own fds with specific interrupt handlers.

Signed-off-by: Shahaf Shuler <shahafs@mellanox.com>
---
 lib/librte_eal/linuxapp/eal/eal_interrupts.c       | 33 ++++++++++++++--------
 .../linuxapp/eal/include/exec-env/rte_interrupts.h |  9 ++++++
 lib/librte_eal/linuxapp/eal/rte_eal_version.map    |  7 +++++
 3 files changed, 37 insertions(+), 12 deletions(-)

diff --git a/lib/librte_eal/linuxapp/eal/eal_interrupts.c b/lib/librte_eal/linuxapp/eal/eal_interrupts.c
index 92a19cb..c19eb8c 100644
--- a/lib/librte_eal/linuxapp/eal/eal_interrupts.c
+++ b/lib/librte_eal/linuxapp/eal/eal_interrupts.c
@@ -939,6 +939,8 @@ static __attribute__((noreturn)) void *
 		bytes_read = sizeof(buf.vfio_intr_count);
 		break;
 #endif
+	case RTE_INTR_HANDLE_EXT:
+		return;
 	default:
 		bytes_read = 1;
 		RTE_LOG(INFO, EAL, "unexpected intr type\n");
@@ -1167,6 +1169,24 @@ static __attribute__((noreturn)) void *
 	return rc;
 }
 
+void
+rte_intr_free_epoll_fd(struct rte_intr_handle *intr_handle)
+{
+	uint32_t i;
+	struct rte_epoll_event *rev;
+
+	for (i = 0; i < intr_handle->nb_efd; i++) {
+		rev = &intr_handle->elist[i];
+		if (rev->status == RTE_EPOLL_INVALID)
+			continue;
+		if (rte_epoll_ctl(rev->epfd, EPOLL_CTL_DEL, rev->fd, rev)) {
+			/* force free if the entry valid */
+			eal_epoll_data_safe_free(rev);
+			rev->status = RTE_EPOLL_INVALID;
+		}
+	}
+}
+
 int
 rte_intr_efd_enable(struct rte_intr_handle *intr_handle, uint32_t nb_efd)
 {
@@ -1202,19 +1222,8 @@ static __attribute__((noreturn)) void *
 rte_intr_efd_disable(struct rte_intr_handle *intr_handle)
 {
 	uint32_t i;
-	struct rte_epoll_event *rev;
-
-	for (i = 0; i < intr_handle->nb_efd; i++) {
-		rev = &intr_handle->elist[i];
-		if (rev->status == RTE_EPOLL_INVALID)
-			continue;
-		if (rte_epoll_ctl(rev->epfd, EPOLL_CTL_DEL, rev->fd, rev)) {
-			/* force free if the entry valid */
-			eal_epoll_data_safe_free(rev);
-			rev->status = RTE_EPOLL_INVALID;
-		}
-	}
 
+	rte_intr_free_epoll_fd(intr_handle);
 	if (intr_handle->max_intr > intr_handle->nb_efd) {
 		for (i = 0; i < intr_handle->nb_efd; i++)
 			close(intr_handle->efds[i]);
diff --git a/lib/librte_eal/linuxapp/eal/include/exec-env/rte_interrupts.h b/lib/librte_eal/linuxapp/eal/include/exec-env/rte_interrupts.h
index d459bf4..735d307 100644
--- a/lib/librte_eal/linuxapp/eal/include/exec-env/rte_interrupts.h
+++ b/lib/librte_eal/linuxapp/eal/include/exec-env/rte_interrupts.h
@@ -171,6 +171,15 @@ struct rte_intr_handle {
 		int epfd, int op, unsigned int vec, void *data);
 
 /**
+ * It deletes registered eventfds.
+ *
+ * @param intr_handle
+ *   Pointer to the interrupt handle.
+ */
+void
+rte_intr_free_epoll_fd(struct rte_intr_handle *intr_handle);
+
+/**
  * It enables the packet I/O interrupt event if it's necessary.
  * It creates event fd for each interrupt vector when MSIX is used,
  * otherwise it multiplexes a single event fd.
diff --git a/lib/librte_eal/linuxapp/eal/rte_eal_version.map b/lib/librte_eal/linuxapp/eal/rte_eal_version.map
index 9c134b4..905d8de 100644
--- a/lib/librte_eal/linuxapp/eal/rte_eal_version.map
+++ b/lib/librte_eal/linuxapp/eal/rte_eal_version.map
@@ -186,3 +186,10 @@ DPDK_17.02 {
 	rte_bus_unregister;
 
 } DPDK_16.11;
+
+DPDK_17.05 {
+	global:
+
+	rte_intr_free_epoll_fd;
+
+} DPDK_17.02;
-- 
1.8.3.1

Re: [dpdk-dev] [PATCH v2 1/2] eal/linux: support external Rx interrupt

[Yongseok Koh]

> On Mar 14, 2017, at 6:03 AM, Shahaf Shuler <shahafs@mellanox.com> wrote:
> 
> Prior to this patch only UIO/VFIO interrupt handlers types were supported.
> This patch adds support for the external interrupt handler type, allowing
> external drivers to set their own fds with specific interrupt handlers.
> 
> Signed-off-by: Shahaf Shuler <shahafs@mellanox.com>

Acked-by: Yongseok Koh <yskoh@mellanox.com>

Regards,
Yongseok

Re: [dpdk-dev] [PATCH v2 1/2] eal/linux: support external Rx interrupt

[Ferruh Yigit]

On 3/14/2017 4:38 PM, Yongseok Koh wrote:
> 
>> On Mar 14, 2017, at 6:03 AM, Shahaf Shuler <shahafs@mellanox.com> wrote:
>>
>> Prior to this patch only UIO/VFIO interrupt handlers types were supported.
>> This patch adds support for the external interrupt handler type, allowing
>> external drivers to set their own fds with specific interrupt handlers.
>>
>> Signed-off-by: Shahaf Shuler <shahafs@mellanox.com>
> 
> Acked-by: Yongseok Koh <yskoh@mellanox.com>

Series applied to dpdk-next-net/master, thanks.

[dpdk-dev] [PATCH v2 2/2] net/mlx5: support user space rxq interrupt event

[Shahaf Shuler]

Implement rxq interrupt callbacks

Signed-off-by: Shahaf Shuler <shahafs@mellanox.com>
---
 doc/guides/nics/features/mlx5.ini      |   1 +
 doc/guides/rel_notes/release_17_05.rst |   6 ++
 drivers/net/mlx5/Makefile              |   5 ++
 drivers/net/mlx5/mlx5.c                |   2 +
 drivers/net/mlx5/mlx5_rxq.c            | 126 ++++++++++++++++++++++++++++++++-
 drivers/net/mlx5/mlx5_rxtx.c           |  73 +++++++++++++++++++
 drivers/net/mlx5/mlx5_rxtx.h           |   7 ++
 drivers/net/mlx5/mlx5_trigger.c        |   9 +++
 8 files changed, 228 insertions(+), 1 deletion(-)

diff --git a/doc/guides/nics/features/mlx5.ini b/doc/guides/nics/features/mlx5.ini
index 1814f82..532c0ef 100644
--- a/doc/guides/nics/features/mlx5.ini
+++ b/doc/guides/nics/features/mlx5.ini
@@ -7,6 +7,7 @@
 Speed capabilities   = Y
 Link status          = Y
 Link status event    = Y
+Rx interrupt         = Y
 Queue start/stop     = Y
 MTU update           = Y
 Jumbo frame          = Y
diff --git a/doc/guides/rel_notes/release_17_05.rst b/doc/guides/rel_notes/release_17_05.rst
index 49e9640..f9f79c2 100644
--- a/doc/guides/rel_notes/release_17_05.rst
+++ b/doc/guides/rel_notes/release_17_05.rst
@@ -62,6 +62,11 @@ New Features
   Added support for Hardware TSO for tunneled and non-tunneled packets.
   Tunneling protocols supported are GRE and VXLAN.
 
+* **Added support for Rx interrupts on mlx5 driver.**
+
+  Rx queues can be armed with an interrupt which will trigger on the
+  next packet arrival.
+
 * **Added vmxnet3 version 3 support.**
 
   Added support for vmxnet3 version 3 which includes several
@@ -73,6 +78,7 @@ New Features
   * Generic flow API support for Ethernet, VLAN, IPv4, IPv6, UDP and TCP
     pattern items with QUEUE action for ingress traffic.
 
+
 Resolved Issues
 ---------------
 
diff --git a/drivers/net/mlx5/Makefile b/drivers/net/mlx5/Makefile
index 671089c..084044a 100644
--- a/drivers/net/mlx5/Makefile
+++ b/drivers/net/mlx5/Makefile
@@ -137,6 +137,11 @@ mlx5_autoconf.h.new: $(RTE_SDK)/buildtools/auto-config-h.sh
 		/usr/include/linux/ethtool.h \
 		enum ETHTOOL_LINK_MODE_100000baseKR4_Full_BIT \
 		$(AUTOCONF_OUTPUT)
+	$Q sh -- '$<' '$@' \
+		HAVE_UPDATE_CQ_CI \
+		infiniband/mlx5_hw.h \
+		func ibv_mlx5_exp_update_cq_ci \
+		$(AUTOCONF_OUTPUT)
 
 # Create mlx5_autoconf.h or update it in case it differs from the new one.
 
diff --git a/drivers/net/mlx5/mlx5.c b/drivers/net/mlx5/mlx5.c
index 6f42948..ebc7984 100644
--- a/drivers/net/mlx5/mlx5.c
+++ b/drivers/net/mlx5/mlx5.c
@@ -225,6 +225,8 @@
 	.rss_hash_update = mlx5_rss_hash_update,
 	.rss_hash_conf_get = mlx5_rss_hash_conf_get,
 	.filter_ctrl = mlx5_dev_filter_ctrl,
+	.rx_queue_intr_enable = mlx5_rx_intr_enable,
+	.rx_queue_intr_disable = mlx5_rx_intr_disable,
 };
 
 static struct {
diff --git a/drivers/net/mlx5/mlx5_rxq.c b/drivers/net/mlx5/mlx5_rxq.c
index 28e93d3..e6070a0 100644
--- a/drivers/net/mlx5/mlx5_rxq.c
+++ b/drivers/net/mlx5/mlx5_rxq.c
@@ -36,6 +36,7 @@
 #include <errno.h>
 #include <string.h>
 #include <stdint.h>
+#include <fcntl.h>
 
 /* Verbs header. */
 /* ISO C doesn't support unnamed structs/unions, disabling -pedantic. */
@@ -57,6 +58,7 @@
 #include <rte_malloc.h>
 #include <rte_ethdev.h>
 #include <rte_common.h>
+#include <rte_interrupts.h>
 #ifdef PEDANTIC
 #pragma GCC diagnostic error "-Wpedantic"
 #endif
@@ -773,6 +775,8 @@
 		claim_zero(ibv_exp_destroy_wq(rxq_ctrl->wq));
 	if (rxq_ctrl->cq != NULL)
 		claim_zero(ibv_destroy_cq(rxq_ctrl->cq));
+	if (rxq_ctrl->channel != NULL)
+		claim_zero(ibv_destroy_comp_channel(rxq_ctrl->channel));
 	if (rxq_ctrl->rd != NULL) {
 		struct ibv_exp_destroy_res_domain_attr attr = {
 			.comp_mask = 0,
@@ -1014,6 +1018,16 @@
 		      (void *)dev, strerror(ret));
 		goto error;
 	}
+	if (dev->data->dev_conf.intr_conf.rxq) {
+		tmpl.channel = ibv_create_comp_channel(priv->ctx);
+		if (tmpl.channel == NULL) {
+			dev->data->dev_conf.intr_conf.rxq = 0;
+			ret = ENOMEM;
+			ERROR("%p: Comp Channel creation failure: %s",
+			(void *)dev, strerror(ret));
+			goto error;
+		}
+	}
 	attr.cq = (struct ibv_exp_cq_init_attr){
 		.comp_mask = IBV_EXP_CQ_INIT_ATTR_RES_DOMAIN,
 		.res_domain = tmpl.rd,
@@ -1023,7 +1037,7 @@
 		attr.cq.flags |= IBV_EXP_CQ_COMPRESSED_CQE;
 		cqe_n = (desc * 2) - 1; /* Double the number of CQEs. */
 	}
-	tmpl.cq = ibv_exp_create_cq(priv->ctx, cqe_n, NULL, NULL, 0,
+	tmpl.cq = ibv_exp_create_cq(priv->ctx, cqe_n, NULL, tmpl.channel, 0,
 				    &attr.cq);
 	if (tmpl.cq == NULL) {
 		ret = ENOMEM;
@@ -1347,3 +1361,113 @@
 	rxq = (*priv->rxqs)[index];
 	return priv->dev->rx_pkt_burst(rxq, pkts, pkts_n);
 }
+
+/**
+ * Fill epoll fd list for rxq interrupts.
+ *
+ * @param priv
+ *   Private structure.
+ *
+ * @return
+ *   0 on success, negative on failure.
+ */
+int
+priv_intr_efd_enable(struct priv *priv)
+{
+	unsigned int i;
+	unsigned int rxqs_n = priv->rxqs_n;
+	unsigned int n = RTE_MIN(rxqs_n, (uint32_t)RTE_MAX_RXTX_INTR_VEC_ID);
+	struct rte_intr_handle *intr_handle = priv->dev->intr_handle;
+
+	if (n == 0)
+		return 0;
+	if (n < rxqs_n) {
+		WARN("rxqs num is larger than EAL max interrupt vector "
+		     "%u > %u unable to supprt rxq interrupts",
+		     rxqs_n, (uint32_t)RTE_MAX_RXTX_INTR_VEC_ID);
+		return -EINVAL;
+	}
+	intr_handle->type = RTE_INTR_HANDLE_EXT;
+	for (i = 0; i != n; ++i) {
+		struct rxq *rxq = (*priv->rxqs)[i];
+		struct rxq_ctrl *rxq_ctrl =
+			container_of(rxq, struct rxq_ctrl, rxq);
+		int fd = rxq_ctrl->channel->fd;
+		int flags;
+		int rc;
+
+		flags = fcntl(fd, F_GETFL);
+		rc = fcntl(fd, F_SETFL, flags | O_NONBLOCK);
+		if (rc < 0) {
+			WARN("failed to change rxq interrupt file "
+			     "descriptor %d for queue index %d", fd, i);
+			return -1;
+		}
+		intr_handle->efds[i] = fd;
+	}
+	intr_handle->nb_efd = n;
+	return 0;
+}
+
+/**
+ * Clean epoll fd list for rxq interrupts.
+ *
+ * @param priv
+ *   Private structure.
+ */
+void
+priv_intr_efd_disable(struct priv *priv)
+{
+	struct rte_intr_handle *intr_handle = priv->dev->intr_handle;
+
+	rte_intr_free_epoll_fd(intr_handle);
+}
+
+/**
+ * Create and init interrupt vector array.
+ *
+ * @param priv
+ *   Private structure.
+ *
+ * @return
+ *   0 on success, negative on failure.
+ */
+int
+priv_create_intr_vec(struct priv *priv)
+{
+	unsigned int rxqs_n = priv->rxqs_n;
+	unsigned int i;
+	struct rte_intr_handle *intr_handle = priv->dev->intr_handle;
+
+	if (rxqs_n == 0)
+		return 0;
+	intr_handle->intr_vec = (int *)
+		rte_malloc("intr_vec", rxqs_n * sizeof(int), 0);
+	if (intr_handle->intr_vec == NULL) {
+		WARN("Failed to allocate memory for intr_vec "
+		     "rxq interrupt will not be supported");
+		return -ENOMEM;
+	}
+	for (i = 0; i != rxqs_n; ++i) {
+		/* 1:1 mapping between rxq and interrupt. */
+		intr_handle->intr_vec[i] = RTE_INTR_VEC_RXTX_OFFSET + i;
+	}
+	return 0;
+}
+
+/**
+ * Destroy init interrupt vector array.
+ *
+ * @param priv
+ *   Private structure.
+ *
+ * @return
+ *   0 on success, negative on failure.
+ */
+void
+priv_destroy_intr_vec(struct priv *priv)
+{
+	struct rte_intr_handle *intr_handle = priv->dev->intr_handle;
+
+	rte_free(intr_handle->intr_vec);
+}
diff --git a/drivers/net/mlx5/mlx5_rxtx.c b/drivers/net/mlx5/mlx5_rxtx.c
index 2d33f2d..cf3abd3 100644
--- a/drivers/net/mlx5/mlx5_rxtx.c
+++ b/drivers/net/mlx5/mlx5_rxtx.c
@@ -1671,3 +1671,76 @@
 	(void)pkts_n;
 	return 0;
 }
+
+/**
+ * DPDK callback for rx queue interrupt enable.
+ *
+ * @param dev
+ *   Pointer to Ethernet device structure.
+ * @param rx_queue_id
+ *   RX queue number
+ *
+ * @return
+ *   0 on success, negative on failure.
+ */
+int
+mlx5_rx_intr_enable(struct rte_eth_dev *dev, uint16_t rx_queue_id)
+{
+#ifdef HAVE_UPDATE_CQ_CI
+	struct priv *priv = mlx5_get_priv(dev);
+	struct rxq *rxq = (*priv->rxqs)[rx_queue_id];
+	struct rxq_ctrl *rxq_ctrl = container_of(rxq, struct rxq_ctrl, rxq);
+	struct ibv_cq *cq = rxq_ctrl->cq;
+	uint16_t ci = rxq->cq_ci;
+	int ret = 0;
+
+	ibv_mlx5_exp_update_cq_ci(cq, ci);
+	ret = ibv_req_notify_cq(cq, 0);
+#else
+	int ret = -1;
+	(void)dev;
+	(void)rx_queue_id;
+#endif
+	if (ret)
+		WARN("unable to arm interrupt on rx queue %d", rx_queue_id);
+	return ret;
+}
+
+/**
+ * DPDK callback for rx queue interrupt disable.
+ *
+ * @param dev
+ *   Pointer to Ethernet device structure.
+ * @param rx_queue_id
+ *   RX queue number
+ *
+ * @return
+ *   0 on success, negative on failure.
+ */
+int
+mlx5_rx_intr_disable(struct rte_eth_dev *dev, uint16_t rx_queue_id)
+{
+#ifdef HAVE_UPDATE_CQ_CI
+	struct priv *priv = mlx5_get_priv(dev);
+	struct rxq *rxq = (*priv->rxqs)[rx_queue_id];
+	struct rxq_ctrl *rxq_ctrl = container_of(rxq, struct rxq_ctrl, rxq);
+	struct ibv_cq *cq = rxq_ctrl->cq;
+	struct ibv_cq *ev_cq;
+	void *ev_ctx;
+	int ret = 0;
+
+	ret = ibv_get_cq_event(cq->channel, &ev_cq, &ev_ctx);
+	if (ret || ev_cq != cq)
+		ret = -1;
+	else
+		ibv_ack_cq_events(cq, 1);
+#else
+	int ret = -1;
+	(void)dev;
+	(void)rx_queue_id;
+#endif
+	if (ret)
+		WARN("unable to disable interrupt on rx queue %d",
+		     rx_queue_id);
+	return ret;
+}
diff --git a/drivers/net/mlx5/mlx5_rxtx.h b/drivers/net/mlx5/mlx5_rxtx.h
index 9669564..0db810c 100644
--- a/drivers/net/mlx5/mlx5_rxtx.h
+++ b/drivers/net/mlx5/mlx5_rxtx.h
@@ -138,6 +138,7 @@ struct rxq_ctrl {
 	struct ibv_mr *mr; /* Memory Region (for mp). */
 	struct ibv_exp_wq_family *if_wq; /* WQ burst interface. */
 	struct ibv_exp_cq_family_v1 *if_cq; /* CQ interface. */
+	struct ibv_comp_channel *channel;
 	unsigned int socket; /* CPU socket ID for allocations. */
 	struct rxq rxq; /* Data path structure. */
 };
@@ -299,6 +300,10 @@ size_t priv_flow_attr(struct priv *, struct ibv_exp_flow_attr *,
 void priv_destroy_hash_rxqs(struct priv *);
 int priv_allow_flow_type(struct priv *, enum hash_rxq_flow_type);
 int priv_rehash_flows(struct priv *);
+int priv_intr_efd_enable(struct priv *priv);
+void priv_intr_efd_disable(struct priv *priv);
+int priv_create_intr_vec(struct priv *priv);
+void priv_destroy_intr_vec(struct priv *priv);
 void rxq_cleanup(struct rxq_ctrl *);
 int rxq_rehash(struct rte_eth_dev *, struct rxq_ctrl *);
 int rxq_ctrl_setup(struct rte_eth_dev *, struct rxq_ctrl *, uint16_t,
@@ -327,6 +332,8 @@ int mlx5_tx_queue_setup(struct rte_eth_dev *, uint16_t, uint16_t, unsigned int,
 uint16_t mlx5_rx_burst(void *, struct rte_mbuf **, uint16_t);
 uint16_t removed_tx_burst(void *, struct rte_mbuf **, uint16_t);
 uint16_t removed_rx_burst(void *, struct rte_mbuf **, uint16_t);
+int mlx5_rx_intr_enable(struct rte_eth_dev *dev, uint16_t rx_queue_id);
+int mlx5_rx_intr_disable(struct rte_eth_dev *dev, uint16_t rx_queue_id);
 
 /* mlx5_mr.c */
 
diff --git a/drivers/net/mlx5/mlx5_trigger.c b/drivers/net/mlx5/mlx5_trigger.c
index 0acbf28..229b05e 100644
--- a/drivers/net/mlx5/mlx5_trigger.c
+++ b/drivers/net/mlx5/mlx5_trigger.c
@@ -95,6 +95,11 @@
 		goto error;
 	}
 	priv_dev_interrupt_handler_install(priv, dev);
+	if (dev->data->dev_conf.intr_conf.rxq) {
+		err = priv_intr_efd_enable(priv);
+		if (!err)
+			err = priv_create_intr_vec(priv);
+	}
 	priv_xstats_init(priv);
 	priv_unlock(priv);
 	return 0;
@@ -135,6 +140,10 @@
 	priv_fdir_disable(priv);
 	priv_flow_stop(priv);
 	priv_dev_interrupt_handler_uninstall(priv, dev);
+	if (priv->dev->data->dev_conf.intr_conf.rxq) {
+		priv_destroy_intr_vec(priv);
+		priv_intr_efd_disable(priv);
+	}
 	priv->started = 0;
 	priv_unlock(priv);
 }
-- 
1.8.3.1

Re: [dpdk-dev] [PATCH v2 2/2] net/mlx5: support user space rxq interrupt event

[Yongseok Koh]

> On Mar 14, 2017, at 6:03 AM, Shahaf Shuler <shahafs@mellanox.com> wrote:
> 
> Implement rxq interrupt callbacks
> 
> Signed-off-by: Shahaf Shuler <shahafs@mellanox.com>

Acked-by: Yongseok Koh <yskoh@mellanox.com>

Regards,
Yongseok