DPDK patches and discussions
 help / color / mirror / Atom feed
* [PATCH] vhost: fix deadlock during software live migration of VDPA in a nested virtualization environment
@ 2024-01-18 10:33 Hao Chen
  2024-01-18 14:46 ` David Marchand
  2024-01-22  3:27 ` [PATCH v2] " Hao Chen
  0 siblings, 2 replies; 6+ messages in thread
From: Hao Chen @ 2024-01-18 10:33 UTC (permalink / raw)
  To: dev; +Cc: zy, huangml, Maxime Coquelin, Chenbo Xia

In a nested virtualization environment, running dpdk vdpa in QEMU-L1 for
software live migration will result in a deadlock between dpdke-vdpa and
QEMU-L2 processes.
rte_vdpa_relay_vring_used->
__vhost_iova_to_vva->
vhost_user_iotlb_rd_unlock(vq)->
vhost_user_iotlb_miss-> send vhost message VHOST_USER_SLAVE_IOTLB_MSG to
QEMU's vdpa socket,
then call vhost_user_iotlb_rd_lock(vq) to hold the read lock `iotlb_lock`.
But there is no place to release this read lock.

QEMU L2 get the VHOST_USER_SLAVE_IOTLB_MSG,
then call vhost_user_send_device_iotlb_msg to send VHOST_USER_IOTLB_MSG
messages to dpdk-vdpa.
Dpdk vdpa will call vhost_user_iotlb_msg->
vhost_user_iotlb_cache_insert, here, will obtain the write lock
`iotlb_lock`, but the read lock `iotlb_lock` has not been released and
will block here.

This patch add lock and unlock function to fix the deadlock.

Signed-off-by: Hao Chen <chenh@yusur.tech>
---
 lib/vhost/vdpa.c | 8 +++++++-
 1 file changed, 7 insertions(+), 1 deletion(-)

diff --git a/lib/vhost/vdpa.c b/lib/vhost/vdpa.c
index 9776fc07a9..9132414209 100644
--- a/lib/vhost/vdpa.c
+++ b/lib/vhost/vdpa.c
@@ -19,6 +19,7 @@
 #include "rte_vdpa.h"
 #include "vdpa_driver.h"
 #include "vhost.h"
+#include "iotlb.h"
 
 /** Double linked list of vDPA devices. */
 TAILQ_HEAD(vdpa_device_list, rte_vdpa_device);
@@ -193,10 +194,12 @@ rte_vdpa_relay_vring_used(int vid, uint16_t qid, void *vring_m)
 			if (unlikely(nr_descs > vq->size))
 				return -1;
 
+			vhost_user_iotlb_rd_lock(vq);
 			desc_ring = (struct vring_desc *)(uintptr_t)
 				vhost_iova_to_vva(dev, vq,
 						vq->desc[desc_id].addr, &dlen,
 						VHOST_ACCESS_RO);
+			vhost_user_iotlb_rd_unlock(vq);
 			if (unlikely(!desc_ring))
 				return -1;
 
@@ -220,9 +223,12 @@ rte_vdpa_relay_vring_used(int vid, uint16_t qid, void *vring_m)
 			if (unlikely(nr_descs-- == 0))
 				goto fail;
 			desc = desc_ring[desc_id];
-			if (desc.flags & VRING_DESC_F_WRITE)
+			if (desc.flags & VRING_DESC_F_WRITE) {
+				vhost_user_iotlb_rd_lock(vq);
 				vhost_log_write_iova(dev, vq, desc.addr,
 						     desc.len);
+				vhost_user_iotlb_rd_unlock(vq);
+			}
 			desc_id = desc.next;
 		} while (desc.flags & VRING_DESC_F_NEXT);
 
-- 
2.27.0


^ permalink raw reply	[flat|nested] 6+ messages in thread

end of thread, other threads:[~2024-02-06 14:58 UTC | newest]

Thread overview: 6+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2024-01-18 10:33 [PATCH] vhost: fix deadlock during software live migration of VDPA in a nested virtualization environment Hao Chen
2024-01-18 14:46 ` David Marchand
2024-01-19  6:36   ` Hao Chen
2024-01-22  3:27 ` [PATCH v2] " Hao Chen
2024-02-05 10:00   ` Maxime Coquelin
2024-02-06 14:57   ` Maxime Coquelin

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).