From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id A3AC543B86 for ; Tue, 5 Mar 2024 10:53:30 +0100 (CET) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 9DA384026B; Tue, 5 Mar 2024 10:53:30 +0100 (CET) Received: from NAM10-MW2-obe.outbound.protection.outlook.com (mail-mw2nam10on2051.outbound.protection.outlook.com [40.107.94.51]) by mails.dpdk.org (Postfix) with ESMTP id 8C7CC4014F for ; Tue, 5 Mar 2024 10:53:29 +0100 (CET) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=f3usgF/LCBJQPsv9LzJjJ8WM4XQc4LfUQk5zi4xKQZl+UOk9cZMoTV6J4DYiW0srNJY8hPTE6BAWClugKdlKrVQSC31G97dCCsAbEXDwJbj1ubk+dn67fBBJqmthTmN5feGyn9zhGMyNEDXZaJiLA29iWubu0u54zvEcjzO9Atx6mghddp6B7KB3xR0h27ji8twx7sMiIUQtvPZikj8g+j+59QOX2n00P3a+rvh/mSq4hmcHACgEoT9IBzL7hL+kPm7s3NRGlxv7MMKHqAUnPMR715Knyl0/HlYqHkmPOeaXe5OnjdXgZ6y65XcIyB34dOc2NGi2yaFPkyoB12/WdA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=S0q15YSX3aklJWD3ZjPiJijWElT68Cv53WvTTurxd2A=; b=VJDoO63EODw8G9/n+jBWelj3zTP/9bcfF/32as9BO0VUcbQVX0/FuHewDiDtpoRJ7vuH0QLucV0iQBMJXKROh8CF9HJDk+LG1GjfrsPdpei93kdSQV481XeM1GjzDViXes+Wsfk6vwGKJJE4LYPQwCN4/Q8rxSU4bhK43eN1cgAwHeIHW5bfHnxToVSix4k570Zb2WKn3ogNMkFaYhwCFslnCKnNB7b6PcnqfICHmzcvBcViOYdF/Lo4tbB1PyrQNRMGrAP9CCNPJz9vwDWWEJBpcBBJ3zoIWeTC9j412dOvpgF/Iedxjhkk7F7v8l56GVBO/liUS2m3AJDRFTHYXA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 216.228.117.161) smtp.rcpttodomain=yusur.tech smtp.mailfrom=nvidia.com; dmarc=pass (p=reject sp=reject pct=100) action=none header.from=nvidia.com; dkim=none (message not signed); arc=none (0) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=S0q15YSX3aklJWD3ZjPiJijWElT68Cv53WvTTurxd2A=; b=KCG7MtCmg1S0pClNwQ9o9BBqJ/jBljCjpZTa94/jKfcLRHhqeabJzHMBcGHNkngUKZ3nmsBTCOpOoagiHyp+szvLwNTB+i4Ad5hMgboBTfA8PwWF+cWhJhzgmpzYLxlbDWnLUmyLxzNkpSZcOYTRPa8mQXbjwtYfkCfOUcqo8SmuYRNyIsL3Eq3E/kuyJe7s8umAQuR7qvO1eR9v39ZgJQLroS+yiOYqFhMzuZnGP6Xt20vRdUpcYXLCuxUtWwxpYs6wpRr/O0NtQcElCP11i6RPcZAuMJ52rhPV9M0OxECXNK++stA3ldObhxHWi7Xh4rEtqeFRV6sp9brn1ah7yQ== Received: from BYAPR21CA0029.namprd21.prod.outlook.com (2603:10b6:a03:114::39) by LV3PR12MB9233.namprd12.prod.outlook.com (2603:10b6:408:194::15) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7339.39; Tue, 5 Mar 2024 09:53:26 +0000 Received: from SJ5PEPF000001D1.namprd05.prod.outlook.com (2603:10b6:a03:114:cafe::92) by BYAPR21CA0029.outlook.office365.com (2603:10b6:a03:114::39) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7386.4 via Frontend Transport; Tue, 5 Mar 2024 09:53:26 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 216.228.117.161) smtp.mailfrom=nvidia.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=nvidia.com; Received-SPF: Pass (protection.outlook.com: domain of nvidia.com designates 216.228.117.161 as permitted sender) receiver=protection.outlook.com; client-ip=216.228.117.161; helo=mail.nvidia.com; pr=C Received: from mail.nvidia.com (216.228.117.161) by SJ5PEPF000001D1.mail.protection.outlook.com (10.167.242.53) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7362.11 via Frontend Transport; Tue, 5 Mar 2024 09:53:26 +0000 Received: from rnnvmail201.nvidia.com (10.129.68.8) by mail.nvidia.com (10.129.200.67) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.986.41; Tue, 5 Mar 2024 01:53:14 -0800 Received: from nvidia.com (10.126.231.35) by rnnvmail201.nvidia.com (10.129.68.8) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.1258.12; Tue, 5 Mar 2024 01:53:13 -0800 From: Xueming Li To: Hao Chen CC: Maxime Coquelin , dpdk stable Subject: patch 'vhost: fix deadlock during vDPA SW live migration' has been queued to stable release 23.11.1 Date: Tue, 5 Mar 2024 17:46:40 +0800 Message-ID: <20240305094757.439387-60-xuemingl@nvidia.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20240305094757.439387-1-xuemingl@nvidia.com> References: <20240305094757.439387-1-xuemingl@nvidia.com> MIME-Version: 1.0 Content-Transfer-Encoding: 8bit Content-Type: text/plain X-Originating-IP: [10.126.231.35] X-ClientProxiedBy: rnnvmail202.nvidia.com (10.129.68.7) To rnnvmail201.nvidia.com (10.129.68.8) X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: SJ5PEPF000001D1:EE_|LV3PR12MB9233:EE_ X-MS-Office365-Filtering-Correlation-Id: 2338dbc7-2d6b-4efe-4217-08dc3cfa1a4a X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: nzXf6nDNVjcXT3NI+I/jB6N1r4g7fJS7iBZKT1F02iRMuU5YoUQSHodgb68z/5nbLOLYuoXa1ZfeGH1r6S2bfIzGLLevOZ4Ss0VKUF6M8BuMTPC7vvnN9CUFu+5dC4AstYaeuPVN5iHKqMyekQikRacYVE/lqAYdXA5QXPN+nQvYjwfwLTPbZJRUM0VUOWfRaSezo8KaJ43wIsOvm0y8UUjKhqdsQPlFLXqLqTAglJB4yJ34uEWaYnQ0Oi9TxBxbUjN5RqvadvAgYA06I8wMLzIiEslD/VgD0TRZ6WKiPjbixD1mmhKVzPvnGx8QVgvj2D+91Ujyxto/OZajJGFVe9qz4C1khB7ZDOnlw36FTS4RbCk6zweT7/5Gg8ZIK4uqZgXQmMc4cnrcly5b9jOa7eDvwnWan4gx5yOea2Ml7FoSZGNeLMKVEjpqI/7prUSFNQLsfnRZa6vkrvC9/sWyP1YHzZSyd9n7Ukop+s1knb4UEsaTsL5PjsyLsYeZ6uPCVxSfRD3zUXgB2+ilo/fV/X5FDWqcT+Zb5qpmVp6fcQdaQzMbU2Rbw2eLsg/pY+oEz7PFzb8DXCzqhozq0iw4y/oNYZVRwuVBfSswubit7d4XDAoNCPLvx9L4szWWS4UY2jq+ijzdeQ+trhE/h0JfPBxKXtT2moRtFhmqqZ5phGdnf7IzkfwccbbFvsloDyL7k2ubi9zTbMI1KHxos+0wlfGsM6z2WAYyguoUOiPfJyNCZnsYDt8R9dXOT1A9SGcc X-Forefront-Antispam-Report: CIP:216.228.117.161; CTRY:US; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:mail.nvidia.com; PTR:dc6edge2.nvidia.com; CAT:NONE; SFS:(13230031)(376005)(36860700004)(82310400014); DIR:OUT; SFP:1101; X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 05 Mar 2024 09:53:26.0176 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 2338dbc7-2d6b-4efe-4217-08dc3cfa1a4a X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=43083d15-7273-40c1-b7db-39efd9ccc17a; Ip=[216.228.117.161]; Helo=[mail.nvidia.com] X-MS-Exchange-CrossTenant-AuthSource: SJ5PEPF000001D1.namprd05.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: LV3PR12MB9233 X-BeenThere: stable@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: patches for DPDK stable branches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: stable-bounces@dpdk.org Hi, FYI, your patch has been queued to stable release 23.11.1 Note it hasn't been pushed to http://dpdk.org/browse/dpdk-stable yet. It will be pushed if I get no objections before 03/31/24. So please shout if anyone has objections. Also note that after the patch there's a diff of the upstream commit vs the patch applied to the branch. This will indicate if there was any rebasing needed to apply to the stable branch. If there were code changes for rebasing (ie: not only metadata diffs), please double check that the rebase was correctly done. Queued patches are on a temporary branch at: https://git.dpdk.org/dpdk-stable/log/?h=23.11-staging This queued commit can be viewed at: https://git.dpdk.org/dpdk-stable/commit/?h=23.11-staging&id=5eb1dd92dce634a60324243f530dfb4130efe610 Thanks. Xueming Li --- >From 5eb1dd92dce634a60324243f530dfb4130efe610 Mon Sep 17 00:00:00 2001 From: Hao Chen Date: Mon, 22 Jan 2024 11:27:44 +0800 Subject: [PATCH] vhost: fix deadlock during vDPA SW live migration Cc: Xueming Li [ upstream commit 19639c3b693bcfdc941c56d9d5bd60f65a8eeecb ] In a nested virtualization environment, running dpdk-vdpa in QEMU-L1 for software live migration will result in a deadlock between dpdk-vdpa and QEMU-L2 processes. 'rte_vdpa_relay_vring_used'-> '__vhost_iova_to_vva'-> 'vhost_user_iotlb_rd_unlock(vq)'-> 'vhost_user_iotlb_miss'-> send vhost message 'VHOST_USER_SLAVE_IOTLB_MSG' to QEMU-L2's vdpa socket, then call 'vhost_user_iotlb_rd_lock(vq)' to hold the read lock `iotlb_lock`. But there is no place to release this read lock. QEMU-L2 get the 'VHOST_USER_SLAVE_IOTLB_MSG', then call 'vhost_user_send_device_iotlb_msg' to send 'VHOST_USER_IOTLB_MSG' messages to dpdk-vdpa. dpdk-vdpa will call vhost_user_iotlb_cache_insert and will obtain the write lock `iotlb_lock`, but the read lock `iotlb_lock` has not been released and will block here. This patch add lock and unlock function to fix the deadlock. Fixes: b13ad2decc83 ("vhost: provide helpers for virtio ring relay") Signed-off-by: Hao Chen Reviewed-by: Maxime Coquelin --- lib/vhost/vdpa.c | 11 +++++++++-- 1 file changed, 9 insertions(+), 2 deletions(-) diff --git a/lib/vhost/vdpa.c b/lib/vhost/vdpa.c index 219eef879c..ce4fb09859 100644 --- a/lib/vhost/vdpa.c +++ b/lib/vhost/vdpa.c @@ -19,6 +19,7 @@ #include "rte_vdpa.h" #include "vdpa_driver.h" #include "vhost.h" +#include "iotlb.h" /** Double linked list of vDPA devices. */ TAILQ_HEAD(vdpa_device_list, rte_vdpa_device); @@ -147,7 +148,6 @@ rte_vdpa_unregister_device(struct rte_vdpa_device *dev) int rte_vdpa_relay_vring_used(int vid, uint16_t qid, void *vring_m) - __rte_no_thread_safety_analysis /* FIXME: requires iotlb_lock? */ { struct virtio_net *dev = get_device(vid); uint16_t idx, idx_m, desc_id; @@ -193,17 +193,21 @@ rte_vdpa_relay_vring_used(int vid, uint16_t qid, void *vring_m) if (unlikely(nr_descs > vq->size)) return -1; + vhost_user_iotlb_rd_lock(vq); desc_ring = (struct vring_desc *)(uintptr_t) vhost_iova_to_vva(dev, vq, vq->desc[desc_id].addr, &dlen, VHOST_ACCESS_RO); + vhost_user_iotlb_rd_unlock(vq); if (unlikely(!desc_ring)) return -1; if (unlikely(dlen < vq->desc[desc_id].len)) { + vhost_user_iotlb_rd_lock(vq); idesc = vhost_alloc_copy_ind_table(dev, vq, vq->desc[desc_id].addr, vq->desc[desc_id].len); + vhost_user_iotlb_rd_unlock(vq); if (unlikely(!idesc)) return -1; @@ -220,9 +224,12 @@ rte_vdpa_relay_vring_used(int vid, uint16_t qid, void *vring_m) if (unlikely(nr_descs-- == 0)) goto fail; desc = desc_ring[desc_id]; - if (desc.flags & VRING_DESC_F_WRITE) + if (desc.flags & VRING_DESC_F_WRITE) { + vhost_user_iotlb_rd_lock(vq); vhost_log_write_iova(dev, vq, desc.addr, desc.len); + vhost_user_iotlb_rd_unlock(vq); + } desc_id = desc.next; } while (desc.flags & VRING_DESC_F_NEXT); -- 2.34.1 --- Diff of the applied patch vs upstream commit (please double-check if non-empty: --- --- - 2024-03-05 17:39:32.743697159 +0800 +++ 0059-vhost-fix-deadlock-during-vDPA-SW-live-migration.patch 2024-03-05 17:39:30.773566493 +0800 @@ -1 +1 @@ -From 19639c3b693bcfdc941c56d9d5bd60f65a8eeecb Mon Sep 17 00:00:00 2001 +From 5eb1dd92dce634a60324243f530dfb4130efe610 Mon Sep 17 00:00:00 2001 @@ -4,0 +5,3 @@ +Cc: Xueming Li + +[ upstream commit 19639c3b693bcfdc941c56d9d5bd60f65a8eeecb ] @@ -30 +32,0 @@ -Cc: stable@dpdk.org @@ -39 +41 @@ -index 9776fc07a9..a1dd5a753b 100644 +index 219eef879c..ce4fb09859 100644