DPDK patches and discussions
 help / color / mirror / Atom feed
* [dpdk-dev] [PATCH v1] net/mlx5: return EAGAIN on premature disable interrupt calls
@ 2020-07-16  5:36 Ophir Munk
  2020-07-21 14:24 ` [dpdk-dev] [PATCH v2] " Ophir Munk
  0 siblings, 1 reply; 6+ messages in thread
From: Ophir Munk @ 2020-07-16  5:36 UTC (permalink / raw)
  To: dev; +Cc: Raslan Darawsheh, Ophir Munk, Matan Azrad

RXQ interrupts under Linux are based on the epoll mechanism. An
expected order of operations is as follows:
1. Call rte_eth_dev_rx_intr_enable(), to arm the CQ for receiving events
on data input.
2. Block on rte_epoll_wait() with an array of file descriptors
representing the CQ events. Upon data arrival the kernel will signal an
input event on the corresponding CQ fd.
3. Call rte_eth_dev_rx_intr_disable() after the event was received and
continue in polling mode. The mlx5 implementation of
rte_eth_dev_rx_intr_disable() is to get the CQ event and ack it.

In practice applications may wake up from rte_epoll_wait() due to
timeout with no event to ack but still call
rte_eth_dev_rx_intr_disable() unconditionally.  In such cases the call
should return EAGAIN (since the file descriptors are non-blocked), as
opposed to EINVAL which indicates a real failure.  In case of EAGAIN the
PMD should not warn on "Unable to disable interrupt on Rx queue".

Signed-off-by: Ophir Munk <ophirmu@mellanox.com>
Acked-by: Viacheslav Ovsiienko <viacheslavo@mellanox.com>
---
 drivers/net/mlx5/mlx5_rxq.c | 14 +++++++++++---
 1 file changed, 11 insertions(+), 3 deletions(-)

diff --git a/drivers/net/mlx5/mlx5_rxq.c b/drivers/net/mlx5/mlx5_rxq.c
index b436f06..38dfb98 100644
--- a/drivers/net/mlx5/mlx5_rxq.c
+++ b/drivers/net/mlx5/mlx5_rxq.c
@@ -936,7 +936,14 @@ mlx5_rx_intr_disable(struct rte_eth_dev *dev, uint16_t rx_queue_id)
 	}
 	ret = mlx5_glue->get_cq_event(rxq_obj->channel, &ev_cq, &ev_ctx);
 	if (ret || ev_cq != rxq_obj->cq) {
-		rte_errno = EINVAL;
+		/**
+		 * For non-zero 'ret' - save the errno (may be EAGAIN which
+		 * means this function was called before receiving an event).
+		 */
+		if (ret)
+			rte_errno = errno;
+		else
+			rte_errno = EINVAL;
 		goto exit;
 	}
 	rxq_data->cq_arm_sn++;
@@ -947,8 +954,9 @@ mlx5_rx_intr_disable(struct rte_eth_dev *dev, uint16_t rx_queue_id)
 	ret = rte_errno; /* Save rte_errno before cleanup. */
 	if (rxq_obj)
 		mlx5_rxq_obj_release(rxq_obj);
-	DRV_LOG(WARNING, "port %u unable to disable interrupt on Rx queue %d",
-		dev->data->port_id, rx_queue_id);
+	if (ret != EAGAIN)
+		DRV_LOG(WARNING, "port %u unable to disable interrupt on Rx queue %d",
+			dev->data->port_id, rx_queue_id);
 	rte_errno = ret; /* Restore rte_errno. */
 	return -rte_errno;
 }
-- 
2.8.4


^ permalink raw reply	[flat|nested] 6+ messages in thread

* [dpdk-dev] [PATCH v2] return EAGAIN on premature disable interrupt calls
  2020-07-16  5:36 [dpdk-dev] [PATCH v1] net/mlx5: return EAGAIN on premature disable interrupt calls Ophir Munk
@ 2020-07-21 14:24 ` Ophir Munk
  2020-07-21 14:24   ` [dpdk-dev] [PATCH v2] net/mlx5: " Ophir Munk
  0 siblings, 1 reply; 6+ messages in thread
From: Ophir Munk @ 2020-07-21 14:24 UTC (permalink / raw)
  To: dev; +Cc: Raslan Darawsheh, Ophir Munk, Matan Azrad

v1 - Initial release (get event in Verbs only)
v2 - Handle another case of DevX get event.

Ophir Munk (1):
  net/mlx5: return EAGAIN on premature disable interrupt calls

 drivers/net/mlx5/mlx5_rxq.c | 23 ++++++++++++++---------
 1 file changed, 14 insertions(+), 9 deletions(-)

-- 
2.8.4


^ permalink raw reply	[flat|nested] 6+ messages in thread

* [dpdk-dev] [PATCH v2] net/mlx5: return EAGAIN on premature disable interrupt calls
  2020-07-21 14:24 ` [dpdk-dev] [PATCH v2] " Ophir Munk
@ 2020-07-21 14:24   ` Ophir Munk
  2020-07-21 14:41     ` [dpdk-dev] [PATCH v3] " Ophir Munk
  0 siblings, 1 reply; 6+ messages in thread
From: Ophir Munk @ 2020-07-21 14:24 UTC (permalink / raw)
  To: dev; +Cc: Raslan Darawsheh, Ophir Munk, Matan Azrad

RXQ interrupts under Linux are based on the epoll mechanism. An
expected order of operations is as follows:
1. Call rte_eth_dev_rx_intr_enable(), to arm the CQ for receiving events
on data input.
2. Block on rte_epoll_wait() with an array of file descriptors
representing the CQ events. Upon data arrival the kernel will signal an
input event on the corresponding CQ fd.
3. Call rte_eth_dev_rx_intr_disable() after the event was received and
continue in polling mode. The mlx5 implementation of
rte_eth_dev_rx_intr_disable() is to get the CQ event and ack it.

In practice applications may wake up from rte_epoll_wait() due to
timeout with no event to ack but still call
rte_eth_dev_rx_intr_disable() unconditionally.  In such cases the call
should return EAGAIN (since the file descriptors are non-blocked), as
opposed to EINVAL which indicates a real failure.  In case of EAGAIN the
PMD should not warn on "Unable to disable interrupt on Rx queue".

This commit also fixes (1) where a 0 returned value from function
devx_get_event() was considered an error.

Fixes: 19e429e5c7c2 ("net/mlx5: implement CQ for RxQ using DevX API")

Signed-off-by: Ophir Munk <ophirmu@mellanox.com>
Acked-by: Viacheslav Ovsiienko <viacheslavo@mellanox.com>
Acked-by: Raslan Darawsheh <rasland@mellanox.com>
---
 drivers/net/mlx5/mlx5_rxq.c | 23 ++++++++++++++---------
 1 file changed, 14 insertions(+), 9 deletions(-)

diff --git a/drivers/net/mlx5/mlx5_rxq.c b/drivers/net/mlx5/mlx5_rxq.c
index e6dc5ac..c78e522 100644
--- a/drivers/net/mlx5/mlx5_rxq.c
+++ b/drivers/net/mlx5/mlx5_rxq.c
@@ -1188,10 +1188,8 @@ mlx5_rx_intr_disable(struct rte_eth_dev *dev, uint16_t rx_queue_id)
 	if (rxq_obj->type == MLX5_RXQ_OBJ_TYPE_IBV) {
 		ret = mlx5_glue->get_cq_event(rxq_obj->ibv_channel, &ev_cq,
 					      &ev_ctx);
-		if (ret || ev_cq != rxq_obj->ibv_cq) {
-			rte_errno = EINVAL;
+		if (ret < 0 || ev_cq != rxq_obj->ibv_cq)
 			goto exit;
-		}
 		mlx5_glue->ack_cq_events(rxq_obj->ibv_cq, 1);
 	} else if (rxq_obj->type == MLX5_RXQ_OBJ_TYPE_DEVX_RQ) {
 #ifdef HAVE_IBV_DEVX_EVENT
@@ -1200,22 +1198,29 @@ mlx5_rx_intr_disable(struct rte_eth_dev *dev, uint16_t rx_queue_id)
 		ret = mlx5_glue->devx_get_event
 				(rxq_obj->devx_channel, event_data,
 				 sizeof(struct mlx5dv_devx_async_event_hdr));
-		if (ret <= 0 || event_data->cookie !=
-				(uint64_t)(uintptr_t)rxq_obj->devx_cq) {
-			rte_errno = EINVAL;
+		if (ret < 0 || event_data->cookie !=
+				(uint64_t)(uintptr_t)rxq_obj->devx_cq)
 			goto exit;
-		}
 #endif /* HAVE_IBV_DEVX_EVENT */
 	}
 	rxq_data->cq_arm_sn++;
 	mlx5_rxq_obj_release(rxq_obj);
 	return 0;
 exit:
+	/**
+	 * For ret < 0 save the errno (may be EAGAIN which means the get_event
+	 * function was called before receiving one).
+	 */
+	if (ret < 0)
+		rte_errno = errno;
+	else
+		rte_errno = EINVAL;
 	ret = rte_errno; /* Save rte_errno before cleanup. */
 	if (rxq_obj)
 		mlx5_rxq_obj_release(rxq_obj);
-	DRV_LOG(WARNING, "port %u unable to disable interrupt on Rx queue %d",
-		dev->data->port_id, rx_queue_id);
+	if (ret != EAGAIN)
+		DRV_LOG(WARNING, "port %u unable to disable interrupt on Rx queue %d",
+			dev->data->port_id, rx_queue_id);
 	rte_errno = ret; /* Restore rte_errno. */
 	return -rte_errno;
 }
-- 
2.8.4


^ permalink raw reply	[flat|nested] 6+ messages in thread

* [dpdk-dev] [PATCH v3] return EAGAIN on premature disable interrupt calls
  2020-07-21 14:24   ` [dpdk-dev] [PATCH v2] net/mlx5: " Ophir Munk
@ 2020-07-21 14:41     ` Ophir Munk
  2020-07-21 14:41       ` [dpdk-dev] [PATCH v3] net/mlx5: " Ophir Munk
  0 siblings, 1 reply; 6+ messages in thread
From: Ophir Munk @ 2020-07-21 14:41 UTC (permalink / raw)
  To: dev; +Cc: Raslan Darawsheh, Ophir Munk, Matan Azrad

v1 - Initial release (get event in Verbs only)
v2 - Handle another case of DevX get event.
v3 - Update commit message

Ophir Munk (1):
  net/mlx5: return EAGAIN on premature disable interrupt calls

 drivers/net/mlx5/mlx5_rxq.c | 23 ++++++++++++++---------
 1 file changed, 14 insertions(+), 9 deletions(-)

-- 
2.8.4


^ permalink raw reply	[flat|nested] 6+ messages in thread

* [dpdk-dev] [PATCH v3] net/mlx5: return EAGAIN on premature disable interrupt calls
  2020-07-21 14:41     ` [dpdk-dev] [PATCH v3] " Ophir Munk
@ 2020-07-21 14:41       ` Ophir Munk
  2020-07-22  9:11         ` Raslan Darawsheh
  0 siblings, 1 reply; 6+ messages in thread
From: Ophir Munk @ 2020-07-21 14:41 UTC (permalink / raw)
  To: dev; +Cc: Raslan Darawsheh, Ophir Munk, Matan Azrad

RXQ interrupts under Linux are based on the epoll mechanism. An
expected order of operations is as follows:
1. Call rte_eth_dev_rx_intr_enable(), to arm the CQ for receiving events
on data input.
2. Block on rte_epoll_wait() with an array of file descriptors
representing the CQ events. Upon data arrival the kernel will signal an
input event on the corresponding CQ fd.
3. Call rte_eth_dev_rx_intr_disable() after the event was received and
continue in polling mode. The mlx5 implementation of
rte_eth_dev_rx_intr_disable() is to get the CQ event and ack it.

In practice applications may wake up from rte_epoll_wait() due to
timeout with no event to ack but still call
rte_eth_dev_rx_intr_disable() unconditionally.  In such cases the call
should return EAGAIN (since the file descriptors are non-blocked), as
opposed to EINVAL which indicates a real failure.  In case of EAGAIN the
PMD should not warn on "Unable to disable interrupt on Rx queue".

This commit fixes a earlier commit where the returned value 0 from
function devx_get_event() - was considered an error.

Fixes: 19e429e5c7c2 ("net/mlx5: implement CQ for RxQ using DevX API")

Signed-off-by: Ophir Munk <ophirmu@mellanox.com>
Acked-by: Viacheslav Ovsiienko <viacheslavo@mellanox.com>
Acked-by: Raslan Darawsheh <rasland@mellanox.com>
---
 drivers/net/mlx5/mlx5_rxq.c | 23 ++++++++++++++---------
 1 file changed, 14 insertions(+), 9 deletions(-)

diff --git a/drivers/net/mlx5/mlx5_rxq.c b/drivers/net/mlx5/mlx5_rxq.c
index e6dc5ac..c78e522 100644
--- a/drivers/net/mlx5/mlx5_rxq.c
+++ b/drivers/net/mlx5/mlx5_rxq.c
@@ -1188,10 +1188,8 @@ mlx5_rx_intr_disable(struct rte_eth_dev *dev, uint16_t rx_queue_id)
 	if (rxq_obj->type == MLX5_RXQ_OBJ_TYPE_IBV) {
 		ret = mlx5_glue->get_cq_event(rxq_obj->ibv_channel, &ev_cq,
 					      &ev_ctx);
-		if (ret || ev_cq != rxq_obj->ibv_cq) {
-			rte_errno = EINVAL;
+		if (ret < 0 || ev_cq != rxq_obj->ibv_cq)
 			goto exit;
-		}
 		mlx5_glue->ack_cq_events(rxq_obj->ibv_cq, 1);
 	} else if (rxq_obj->type == MLX5_RXQ_OBJ_TYPE_DEVX_RQ) {
 #ifdef HAVE_IBV_DEVX_EVENT
@@ -1200,22 +1198,29 @@ mlx5_rx_intr_disable(struct rte_eth_dev *dev, uint16_t rx_queue_id)
 		ret = mlx5_glue->devx_get_event
 				(rxq_obj->devx_channel, event_data,
 				 sizeof(struct mlx5dv_devx_async_event_hdr));
-		if (ret <= 0 || event_data->cookie !=
-				(uint64_t)(uintptr_t)rxq_obj->devx_cq) {
-			rte_errno = EINVAL;
+		if (ret < 0 || event_data->cookie !=
+				(uint64_t)(uintptr_t)rxq_obj->devx_cq)
 			goto exit;
-		}
 #endif /* HAVE_IBV_DEVX_EVENT */
 	}
 	rxq_data->cq_arm_sn++;
 	mlx5_rxq_obj_release(rxq_obj);
 	return 0;
 exit:
+	/**
+	 * For ret < 0 save the errno (may be EAGAIN which means the get_event
+	 * function was called before receiving one).
+	 */
+	if (ret < 0)
+		rte_errno = errno;
+	else
+		rte_errno = EINVAL;
 	ret = rte_errno; /* Save rte_errno before cleanup. */
 	if (rxq_obj)
 		mlx5_rxq_obj_release(rxq_obj);
-	DRV_LOG(WARNING, "port %u unable to disable interrupt on Rx queue %d",
-		dev->data->port_id, rx_queue_id);
+	if (ret != EAGAIN)
+		DRV_LOG(WARNING, "port %u unable to disable interrupt on Rx queue %d",
+			dev->data->port_id, rx_queue_id);
 	rte_errno = ret; /* Restore rte_errno. */
 	return -rte_errno;
 }
-- 
2.8.4


^ permalink raw reply	[flat|nested] 6+ messages in thread

* Re: [dpdk-dev] [PATCH v3] net/mlx5: return EAGAIN on premature disable interrupt calls
  2020-07-21 14:41       ` [dpdk-dev] [PATCH v3] net/mlx5: " Ophir Munk
@ 2020-07-22  9:11         ` Raslan Darawsheh
  0 siblings, 0 replies; 6+ messages in thread
From: Raslan Darawsheh @ 2020-07-22  9:11 UTC (permalink / raw)
  To: Ophir Munk, dev; +Cc: Matan Azrad

Hi,

> -----Original Message-----
> From: Ophir Munk <ophirmu@mellanox.com>
> Sent: Tuesday, July 21, 2020 5:41 PM
> To: dev@dpdk.org
> Cc: Raslan Darawsheh <rasland@mellanox.com>; Ophir Munk
> <ophirmu@mellanox.com>; Matan Azrad <matan@mellanox.com>
> Subject: [PATCH v3] net/mlx5: return EAGAIN on premature disable interrupt
> calls
> 
> RXQ interrupts under Linux are based on the epoll mechanism. An
> expected order of operations is as follows:
> 1. Call rte_eth_dev_rx_intr_enable(), to arm the CQ for receiving events
> on data input.
> 2. Block on rte_epoll_wait() with an array of file descriptors
> representing the CQ events. Upon data arrival the kernel will signal an
> input event on the corresponding CQ fd.
> 3. Call rte_eth_dev_rx_intr_disable() after the event was received and
> continue in polling mode. The mlx5 implementation of
> rte_eth_dev_rx_intr_disable() is to get the CQ event and ack it.
> 
> In practice applications may wake up from rte_epoll_wait() due to
> timeout with no event to ack but still call
> rte_eth_dev_rx_intr_disable() unconditionally.  In such cases the call
> should return EAGAIN (since the file descriptors are non-blocked), as
> opposed to EINVAL which indicates a real failure.  In case of EAGAIN the
> PMD should not warn on "Unable to disable interrupt on Rx queue".
> 
> This commit fixes a earlier commit where the returned value 0 from
> function devx_get_event() - was considered an error.
> 
> Fixes: 19e429e5c7c2 ("net/mlx5: implement CQ for RxQ using DevX API")
> 
> Signed-off-by: Ophir Munk <ophirmu@mellanox.com>
> Acked-by: Viacheslav Ovsiienko <viacheslavo@mellanox.com>
> Acked-by: Raslan Darawsheh <rasland@mellanox.com>
> ---


Patch applied to next-net-mlx,

Kindest regards,
Raslan Darawsheh

^ permalink raw reply	[flat|nested] 6+ messages in thread

end of thread, other threads:[~2020-07-22  9:11 UTC | newest]

Thread overview: 6+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2020-07-16  5:36 [dpdk-dev] [PATCH v1] net/mlx5: return EAGAIN on premature disable interrupt calls Ophir Munk
2020-07-21 14:24 ` [dpdk-dev] [PATCH v2] " Ophir Munk
2020-07-21 14:24   ` [dpdk-dev] [PATCH v2] net/mlx5: " Ophir Munk
2020-07-21 14:41     ` [dpdk-dev] [PATCH v3] " Ophir Munk
2020-07-21 14:41       ` [dpdk-dev] [PATCH v3] net/mlx5: " Ophir Munk
2020-07-22  9:11         ` Raslan Darawsheh

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).