From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 893B443C60 for ; Thu, 7 Mar 2024 02:34:59 +0100 (CET) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 82BC142E64; Thu, 7 Mar 2024 02:34:59 +0100 (CET) Received: from mail-wm1-f43.google.com (mail-wm1-f43.google.com [209.85.128.43]) by mails.dpdk.org (Postfix) with ESMTP id D153D42ECF for ; Thu, 7 Mar 2024 02:34:57 +0100 (CET) Received: by mail-wm1-f43.google.com with SMTP id 5b1f17b1804b1-4131018eaf8so1148295e9.3 for ; Wed, 06 Mar 2024 17:34:57 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1709775297; x=1710380097; darn=dpdk.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=wlyoU+SIiev3/ipV3qygniw7+McX0nC/DM2MS7Lzspk=; b=C1sASuKRuhqrEpf7sWgTZV4FrZ2zaf96zx+cSiRFdHeZvsfcGxy4SeIo+PEeirBbfT /DlazcYID6WKOBDXuyp6zUG7bsXPEs/n6LYEzgQ9MZpwRvk7/eqvazn0DbmVvuFbhCls +1jlqe0YLr0fAKSvj9VuKgCclN8F7iYTXCJ1y/0v1F2CCiGYF7MnzhdR/jAs7l3KeYme EcLM3iTff55vJxxxefyQGkbVIlSz4mxXrqYAtJXVkUajMUS3r2STa09o+C6orbGStt0I TvO6bvioL4KRaTccEmchi8+0ZJgSSCQ8sXVdK3OtF5HT6QQ37puZ9TDLtZ5av1qGE1en WWyQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1709775297; x=1710380097; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=wlyoU+SIiev3/ipV3qygniw7+McX0nC/DM2MS7Lzspk=; b=Ccu8A1z28CxApHumTSShzthV2AlCyPdf0yfkMC/29Ht9a4HbGPQmm6pVHvmlmQUWM4 VflSsgpMze7ek35fBNtuGCdmMyDRNm5DckMMPlBnyQf1B0QLaxMP1r58YtkdNpzCIBkf 0+g0b00xpkamxxCl42FboA/JhWO1MWFxJfcy2at5poq67HIj2VArpTcFoZmNdwtf4E4X 2kK9n/3oWnv2yBIErZy2l6Hs2rz3DDhav02uaUvL46Sg5Ha52yn5ouvQvbr3JFzg2kGG gsHtcTmh4be5Im6aF6sF714nHwAR4GnRaf1wUUBEG8fB/YYfSwVfLcISdmVQscDzZNzl 9lQg== X-Forwarded-Encrypted: i=1; AJvYcCVdLc+6Myd+vx1Ul+ncGqF8+HN4GAMaN4vPOPR8ai+lPqIAiGghmuh+FsIUdlBfA1WmGGH5HVEThzsrD2/gmVY= X-Gm-Message-State: AOJu0YykA1XUiJtCeDQGn8j1bSkw6JlsuYeAlwM8kj2XuRtRr4tcQi1Q AQ1UzCYAQIARb01hP6rubQHWyDfxgIZviquJ8rxw7wEPfY8vhYiTy/Z3IMxm X-Google-Smtp-Source: AGHT+IEAtY+SiM5m+F6Na6MLyDJpuhailHGzyVTpoxQLvUFewuE//tfdbLIwig97D6fWnkIB9TdmYg== X-Received: by 2002:a05:600c:3ac8:b0:412:e847:a346 with SMTP id d8-20020a05600c3ac800b00412e847a346mr6555453wms.33.1709775297518; Wed, 06 Mar 2024 17:34:57 -0800 (PST) Received: from localhost ([2a01:4b00:d036:ae00:7aef:1aaa:3dff:d546]) by smtp.gmail.com with ESMTPSA id ay11-20020a05600c1e0b00b00412b431eb0csm884140wmb.14.2024.03.06.17.34.56 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 06 Mar 2024 17:34:57 -0800 (PST) From: luca.boccassi@gmail.com To: Hao Chen Cc: Maxime Coquelin , dpdk stable Subject: patch 'vhost: fix deadlock during vDPA SW live migration' has been queued to stable release 22.11.5 Date: Thu, 7 Mar 2024 01:31:03 +0000 Message-Id: <20240307013159.1735343-46-luca.boccassi@gmail.com> X-Mailer: git-send-email 2.39.2 In-Reply-To: <20240307013159.1735343-1-luca.boccassi@gmail.com> References: <20240307013159.1735343-1-luca.boccassi@gmail.com> MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-BeenThere: stable@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: patches for DPDK stable branches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: stable-bounces@dpdk.org Hi, FYI, your patch has been queued to stable release 22.11.5 Note it hasn't been pushed to http://dpdk.org/browse/dpdk-stable yet. It will be pushed if I get no objections before 03/09/24. So please shout if anyone has objections. Also note that after the patch there's a diff of the upstream commit vs the patch applied to the branch. This will indicate if there was any rebasing needed to apply to the stable branch. If there were code changes for rebasing (ie: not only metadata diffs), please double check that the rebase was correctly done. Queued patches are on a temporary branch at: https://github.com/bluca/dpdk-stable This queued commit can be viewed at: https://github.com/bluca/dpdk-stable/commit/9b6bdd022bf51267805e5e252545979387be3f4d Thanks. Luca Boccassi --- >From 9b6bdd022bf51267805e5e252545979387be3f4d Mon Sep 17 00:00:00 2001 From: Hao Chen Date: Mon, 22 Jan 2024 11:27:44 +0800 Subject: [PATCH] vhost: fix deadlock during vDPA SW live migration [ upstream commit 19639c3b693bcfdc941c56d9d5bd60f65a8eeecb ] In a nested virtualization environment, running dpdk-vdpa in QEMU-L1 for software live migration will result in a deadlock between dpdk-vdpa and QEMU-L2 processes. 'rte_vdpa_relay_vring_used'-> '__vhost_iova_to_vva'-> 'vhost_user_iotlb_rd_unlock(vq)'-> 'vhost_user_iotlb_miss'-> send vhost message 'VHOST_USER_SLAVE_IOTLB_MSG' to QEMU-L2's vdpa socket, then call 'vhost_user_iotlb_rd_lock(vq)' to hold the read lock `iotlb_lock`. But there is no place to release this read lock. QEMU-L2 get the 'VHOST_USER_SLAVE_IOTLB_MSG', then call 'vhost_user_send_device_iotlb_msg' to send 'VHOST_USER_IOTLB_MSG' messages to dpdk-vdpa. dpdk-vdpa will call vhost_user_iotlb_cache_insert and will obtain the write lock `iotlb_lock`, but the read lock `iotlb_lock` has not been released and will block here. This patch add lock and unlock function to fix the deadlock. Fixes: b13ad2decc83 ("vhost: provide helpers for virtio ring relay") Signed-off-by: Hao Chen Reviewed-by: Maxime Coquelin --- lib/vhost/vdpa.c | 10 +++++++++- 1 file changed, 9 insertions(+), 1 deletion(-) diff --git a/lib/vhost/vdpa.c b/lib/vhost/vdpa.c index 577cb00a43..cf51ca957a 100644 --- a/lib/vhost/vdpa.c +++ b/lib/vhost/vdpa.c @@ -19,6 +19,7 @@ #include "rte_vdpa.h" #include "vdpa_driver.h" #include "vhost.h" +#include "iotlb.h" /** Double linked list of vDPA devices. */ TAILQ_HEAD(vdpa_device_list, rte_vdpa_device); @@ -191,17 +192,21 @@ rte_vdpa_relay_vring_used(int vid, uint16_t qid, void *vring_m) if (unlikely(nr_descs > vq->size)) return -1; + vhost_user_iotlb_rd_lock(vq); desc_ring = (struct vring_desc *)(uintptr_t) vhost_iova_to_vva(dev, vq, vq->desc[desc_id].addr, &dlen, VHOST_ACCESS_RO); + vhost_user_iotlb_rd_unlock(vq); if (unlikely(!desc_ring)) return -1; if (unlikely(dlen < vq->desc[desc_id].len)) { + vhost_user_iotlb_rd_lock(vq); idesc = vhost_alloc_copy_ind_table(dev, vq, vq->desc[desc_id].addr, vq->desc[desc_id].len); + vhost_user_iotlb_rd_unlock(vq); if (unlikely(!idesc)) return -1; @@ -218,9 +223,12 @@ rte_vdpa_relay_vring_used(int vid, uint16_t qid, void *vring_m) if (unlikely(nr_descs-- == 0)) goto fail; desc = desc_ring[desc_id]; - if (desc.flags & VRING_DESC_F_WRITE) + if (desc.flags & VRING_DESC_F_WRITE) { + vhost_user_iotlb_rd_lock(vq); vhost_log_write_iova(dev, vq, desc.addr, desc.len); + vhost_user_iotlb_rd_unlock(vq); + } desc_id = desc.next; } while (desc.flags & VRING_DESC_F_NEXT); -- 2.39.2 --- Diff of the applied patch vs upstream commit (please double-check if non-empty: --- --- - 2024-03-07 01:05:38.869547086 +0000 +++ 0046-vhost-fix-deadlock-during-vDPA-SW-live-migration.patch 2024-03-07 01:05:34.786939762 +0000 @@ -1 +1 @@ -From 19639c3b693bcfdc941c56d9d5bd60f65a8eeecb Mon Sep 17 00:00:00 2001 +From 9b6bdd022bf51267805e5e252545979387be3f4d Mon Sep 17 00:00:00 2001 @@ -5,0 +6,2 @@ +[ upstream commit 19639c3b693bcfdc941c56d9d5bd60f65a8eeecb ] + @@ -30 +31,0 @@ -Cc: stable@dpdk.org @@ -35,2 +36,2 @@ - lib/vhost/vdpa.c | 11 +++++++++-- - 1 file changed, 9 insertions(+), 2 deletions(-) + lib/vhost/vdpa.c | 10 +++++++++- + 1 file changed, 9 insertions(+), 1 deletion(-) @@ -39 +40 @@ -index 9776fc07a9..a1dd5a753b 100644 +index 577cb00a43..cf51ca957a 100644 @@ -50,9 +51 @@ -@@ -147,7 +148,6 @@ rte_vdpa_unregister_device(struct rte_vdpa_device *dev) - - int - rte_vdpa_relay_vring_used(int vid, uint16_t qid, void *vring_m) -- __rte_no_thread_safety_analysis /* FIXME: requires iotlb_lock? */ - { - struct virtio_net *dev = get_device(vid); - uint16_t idx, idx_m, desc_id; -@@ -193,17 +193,21 @@ rte_vdpa_relay_vring_used(int vid, uint16_t qid, void *vring_m) +@@ -191,17 +192,21 @@ rte_vdpa_relay_vring_used(int vid, uint16_t qid, void *vring_m) @@ -80 +73 @@ -@@ -220,9 +224,12 @@ rte_vdpa_relay_vring_used(int vid, uint16_t qid, void *vring_m) +@@ -218,9 +223,12 @@ rte_vdpa_relay_vring_used(int vid, uint16_t qid, void *vring_m)