From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id 80400A053A; Mon, 27 Jul 2020 16:00:51 +0200 (CEST) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 0DBF21BFD9; Mon, 27 Jul 2020 16:00:50 +0200 (CEST) Received: from mellanox.co.il (mail-il-dmz.mellanox.com [193.47.165.129]) by dpdk.org (Postfix) with ESMTP id 2A95B1BFCF for ; Mon, 27 Jul 2020 16:00:48 +0200 (CEST) Received: from Internal Mail-Server by MTLPINE1 (envelope-from matan@mellanox.com) with SMTP; 27 Jul 2020 17:00:46 +0300 Received: from pegasus25.mtr.labs.mlnx. (pegasus25.mtr.labs.mlnx [10.210.16.10]) by labmailer.mlnx (8.13.8/8.13.8) with ESMTP id 06RE0kUx014371; Mon, 27 Jul 2020 17:00:46 +0300 From: Matan Azrad To: Maxime Coquelin Cc: dev@dpdk.org Date: Mon, 27 Jul 2020 14:00:44 +0000 Message-Id: <1595858444-126652-1-git-send-email-matan@mellanox.com> X-Mailer: git-send-email 1.8.3.1 Subject: [dpdk-dev] [PATCH] vdpa/mlx5: fix notification timing X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" The issue is relevant only for the timer event modes: 0 and 1. When the HW finishes to consume a burst of the guest Rx descriptors, it creates a CQE in the CQ. When traffic stops, the mlx5 driver arms the CQ to get a notification when a specific CQE index is created - the index to be armed is the next CQE index which should be polled by the driver. The mlx5 driver configured the kernel driver to send notification to the guest callfd in the same time of the armed CQE event. It means that the guest was notified only for each first CQE in a poll cycle, so if the driver polled CQEs of all the virtio queue available descriptors, the guest was not notified again for the rest because there was no any new CQE to trigger the guest notification. Hence, the Rx queues might be stuck when the guest didn't work with poll mode. Remove prior kernel notification, and do manual notification after CQ polling. Fixes: a9dd7275a149 ("vdpa/mlx5: optimize notification events") Signed-off-by: Matan Azrad Acked-by: Xueming Li --- drivers/vdpa/mlx5/mlx5_vdpa_event.c | 17 +++-------------- 1 file changed, 3 insertions(+), 14 deletions(-) diff --git a/drivers/vdpa/mlx5/mlx5_vdpa_event.c b/drivers/vdpa/mlx5/mlx5_vdpa_event.c index e14b380..0414c91 100644 --- a/drivers/vdpa/mlx5/mlx5_vdpa_event.c +++ b/drivers/vdpa/mlx5/mlx5_vdpa_event.c @@ -172,17 +172,6 @@ rte_errno = errno; goto error; } - if (callfd != -1 && - priv->event_mode != MLX5_VDPA_EVENT_MODE_ONLY_INTERRUPT) { - ret = mlx5_glue->devx_subscribe_devx_event_fd(priv->eventc, - callfd, - cq->cq->obj, 0); - if (ret) { - DRV_LOG(ERR, "Failed to subscribe CQE event fd."); - rte_errno = errno; - goto error; - } - } cq->callfd = callfd; /* Init CQ to ones to be in HW owner in the start. */ cq->cqes[0].op_own = MLX5_CQE_OWNER_MASK; @@ -352,11 +341,11 @@ struct mlx5_vdpa_virtq, eqp); mlx5_vdpa_cq_poll(cq); + /* Notify guest for descs consuming. */ + if (cq->callfd != -1) + eventfd_write(cq->callfd, (eventfd_t)1); if (priv->event_mode == MLX5_VDPA_EVENT_MODE_ONLY_INTERRUPT) { mlx5_vdpa_cq_arm(priv, cq); - /* Notify guest for descs consuming. */ - if (cq->callfd != -1) - eventfd_write(cq->callfd, (eventfd_t)1); return; } /* Don't arm again - timer will take control. */ -- 1.8.3.1