From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 176F242DC7; Tue, 4 Jul 2023 04:32:52 +0200 (CEST) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 9AC5240F18; Tue, 4 Jul 2023 04:32:51 +0200 (CEST) Received: from APC01-TYZ-obe.outbound.protection.outlook.com (mail-tyzapc01on2083.outbound.protection.outlook.com [40.107.117.83]) by mails.dpdk.org (Postfix) with ESMTP id A9A3E40E03 for ; Tue, 4 Jul 2023 04:32:49 +0200 (CEST) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=hi0qcRCtAWo5VuECUiOi6j51uUDZUqg2U0e9NPKGhn0IlS2/i2Bgmsd+OqzesJyjX3VixW4Yg5iTin6BXk5YXC+z6st3Li/GPGEiohxDkEiGdlHIpJSb5HwLoTUl78q2ZFwXgXaSl9Kd0NzwEjlRUhlxuXcx0T3KcEeikvza2SORJpyLAHj6g8C2NrW8Yy9Vf52ig+dv47sGtoia0NmezAbGjSbexcysU66XL+x6nPr9hkVFpO6jcLOCm/YeEdXjBVDmv15Bla+E1MEv7ZAJnofE1bG7W+wScKzoV8GUowTVsG2hjjITeNnd+hmEBWMoDiTjUkCgntI1QaXkoEzjzA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=kf00fxX4UyDsdUBswvvjrYyEAZY6FxfpxVHL8e8RbIA=; b=Vovnl4MJ/91+cM9V3YBlZyfpjps72YSnGI9cMPaFb+am7ryxgf/PosksHZi2btM2MDNL76WiBd3dTCoxthjd6HRoTzvQnXR8Poc0RSOwsgySHXOJcS804MvOkBd7uMW3EsRndGiJY3cZSm3X9DjuKqF/AVV4gghi/1/slssaVnOtX4CHYsRIC1AtQJm6f2X2K+kK5MaSJ7pjgPguN1+mvicaWGahTun+4qLnVT5HjBgrvDWJ7rCRwN+xj4Q9aTv0nm10J4nydjeDqW4CKPeld0lNTiouWQunPagGp/6jUIPqJYp1EVuFSyVvwCa8pYMvPTBSNPLIZFFyX+2SdVL5BQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=jaguarmicro.com; dmarc=pass action=none header.from=jaguarmicro.com; dkim=pass header.d=jaguarmicro.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=jaguarmicro.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=kf00fxX4UyDsdUBswvvjrYyEAZY6FxfpxVHL8e8RbIA=; b=kfclnYgMHEAf7S5W7cfGx77XscoB8U0cF0hs02yYnuSBjJXRrfdNzWE9JAEk6jCraspNwA8dsH2wslEcGgb1d5QG1vmURZIp7E/kVRIs1B/UeBlxEltmX+ST9ZOvEmFSJFXs4/948R/sE+cIIaSoByhpQ6jt9hD5SwkroXanszV+8oVkPLUeEWRKvrYUMu5MucT3nAtDp7Ygb+hF8h/4Ux2ITzUfE0BKgVX25eZu8gEUi19t+LThaLD5PCfqtaR0n+bAGhixudJxhY0ZWnKU59y0T7dfgwjb2+gQOE2RuaQn0SEIlpI0Tyx4Qn4NHufE0iB1FYsTQc4M1edHmiL3VQ== Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=jaguarmicro.com; Received: from SI2PR06MB4752.apcprd06.prod.outlook.com (2603:1096:4:14c::14) by PUZPR06MB5936.apcprd06.prod.outlook.com (2603:1096:301:11d::13) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6544.24; Tue, 4 Jul 2023 02:32:43 +0000 Received: from SI2PR06MB4752.apcprd06.prod.outlook.com ([fe80::25f2:1ee4:9de:7b00]) by SI2PR06MB4752.apcprd06.prod.outlook.com ([fe80::25f2:1ee4:9de:7b00%3]) with mapi id 15.20.6544.024; Tue, 4 Jul 2023 02:32:43 +0000 From: Rma Ma To: dpdk-dev Cc: Maxime Coquelin , Chenbo Xia , Rma Ma Subject: [PATCH v2] vhost: add notify reply ops to fix message deadlock Date: Tue, 4 Jul 2023 10:32:28 +0800 Message-Id: <20230704023228.24879-1-rma.ma@jaguarmicro.com> X-Mailer: git-send-email 2.17.1 Content-Type: text/plain X-ClientProxiedBy: SI2PR06CA0014.apcprd06.prod.outlook.com (2603:1096:4:186::11) To SI2PR06MB4752.apcprd06.prod.outlook.com (2603:1096:4:14c::14) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: SI2PR06MB4752:EE_|PUZPR06MB5936:EE_ X-MS-Office365-Filtering-Correlation-Id: 9928d291-fab7-4012-ed75-08db7c36f1e9 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: /L/I0LKPZOUlF3EQpI+JOCnD4o1uWpv3TZ9+I0OaeZFx0fzxPKo5388Kvjs49FSOMuX/icPvW9iH4JQeOOd3J+5VuII2y3oSfRmWmKsUHD730SowaXFFKQAwNLgQpHPXYGywnXcNX6YY6D0FT+2kGuy5Rxf0d5FKCVwRk2mQ9UBvRYvPj96YgR/+O92ZjLYpj4omvy5lyrw5wK0QvwoKap+qpvZJItrtmS8490i1JJIzJGdEzy4GoaDKKl9VRFwMj1cqhoX8fVFtN4D+s010Ykl28AUXSkuwZgiZiI9OJ0iibL10WaRQSZRcUjlb8fUgyanyQadb8NiWeANrK7cMMP7i4jgcVM4jSEW7GSlNSBhbTaF95sj63xVyC8VBwFiGu3hhrrQg01kC2u1RLiCCy25uYXDNnqD95PvvG5sS5sCoq4S2bz2MEtjT9VGL68dSEDJNk/IC5rjqIZkW0IEtP0mhx73yh3cQM1p/s2fvX4c2DLxCrTySkTZcI/fKJfcq+2yUygp8h+3JtJhaLkOpktsxyi5qwpIzSZuiqvrbG0jY6uvAJsslSpT2dMwhgFpg5xdraHagbigBD/UUYbTS5TmCNwSHaAKCn+d4CgAi5TXPl/lWKLxUJHXOzxt2Tsq/ X-Forefront-Antispam-Report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:SI2PR06MB4752.apcprd06.prod.outlook.com; PTR:; CAT:NONE; SFS:(13230028)(346002)(39840400004)(136003)(396003)(366004)(376002)(451199021)(107886003)(26005)(478600001)(1076003)(6666004)(6512007)(6506007)(86362001)(2616005)(186003)(38350700002)(38100700002)(54906003)(66476007)(6916009)(66556008)(4326008)(66946007)(83380400001)(52116002)(6486002)(316002)(5660300002)(8676002)(8936002)(15650500001)(44832011)(41300700001)(2906002)(36756003); DIR:OUT; SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?us-ascii?Q?a6AkSIkeX/3Z1AQWeC4N8UCJ6o4xc9OC7q0k+ll+MuTCk1BYxy+zbSytjxfW?= =?us-ascii?Q?eq35H3m55cvbfHDn62wHEkTP4JYAduYb1e27S7tTs7yhUzGCmlebdFnIwqwQ?= =?us-ascii?Q?WzRWbQ97P6r57HN8WGuAZo9HnsWdon75drLDyjcO1KEYjx/gPPO1H0fVNHCY?= =?us-ascii?Q?r5b+YlZ3yP5An2Ey64xVGkv7ADV8lCE1d8XyUR7xhwhorP5Wl7qOMK4YPYkU?= =?us-ascii?Q?CBgeDwthn97HNx5q7zlWAYvHfp5M1ZRczUjzwEMiTMRNDdXW2jPTAQ8clOh5?= =?us-ascii?Q?HMI1Xexq6AH1QVvVoscKv9jNJyBHaOuSMIQ+La+hVSnFgOS6TLqHpGHNVmkU?= =?us-ascii?Q?hC2NvLzD0RSE83zTVytASdqITPIzX8cgjr4EbqbLizw96RJSL8qddXlwLQXB?= =?us-ascii?Q?d2F02aMuQBsvsxhcGrz/03N/NTLIffk1Xg0ffty20xPNeMcantLTISlHrKAq?= =?us-ascii?Q?HiHnmB0F0mY1o5bCDthW7LKNb1GzVf0UR5ywpxEWNUECcgaRSgvUDzdBr8rQ?= =?us-ascii?Q?d2stI4wJUgVoPGTTMaPH1L8/s+yJBbGq2G/Sa3LK0/9VFekkFC0jpJRBneZE?= =?us-ascii?Q?M0+NbliWKj0hPbLtyHjIS7863oJnQzgDSrCJibs3Cc9KVm7S05FJ57dlTFKo?= =?us-ascii?Q?7+jGULypZH/DI31zayle85rGVTU4klcHfSNySNED+7k/QFHleaLYJ45Kgwiu?= =?us-ascii?Q?PxLapNQOGMtVqzCEYaOuf0Eae7wOcYHecna1sFZUCd9kVKsgR/mkVNMoNLKD?= =?us-ascii?Q?1dFCCzZDuGLk4ZS/PnYk64aBCZ95Li2dZyTwr1x+1JzAUOXAUSdKuVZgG30X?= =?us-ascii?Q?G7gHo3jVYpmMtiSEWgiV4/j7P/IFNEA5ceVBr0sT/TrDw43110Xy4IrVG3Jw?= =?us-ascii?Q?2FkvDXg3QUi5ysX4XCi3ZhTrppOuehHE2yfKat/mCtUXYH1UKcn9AkpgJRTc?= =?us-ascii?Q?TKa6eGhjWbOrSW/+tg5foY0JNQ0/9x+5fa8k/M0HEEK9BZK7fOeh2Yzc5705?= =?us-ascii?Q?sSD4GxXEruxARIechJGphwRX9J99Ht3qa+tOgq7PkEFo6OLtsQ3cp5mIhnm7?= =?us-ascii?Q?NpRusL1410tbCzx6yZr1YaZA8xHbpnFg502zGw8BNwlBkrxWT+JUz3hSlt38?= =?us-ascii?Q?vOoJGau45t0J7mH/WMOJpqSyCOa/YxQ+ysZg9/oTeBSaylF1JWlCdmx/II2u?= =?us-ascii?Q?r6gotENku19LMZw51ct3NT0CGjCjs0+nH4UeUXBQqr3kfnKkrOpJ/wBIM9pp?= =?us-ascii?Q?OzATg4rLX/o0gxXgbUgzQDNrAJd63tFIjpgEtF3gm4Su2LHCuCRt4b8JoCSq?= =?us-ascii?Q?3jrlyyd7K1DnkQA4clyTNITS8fkeoFCx34VuI6IHi3ehkXyZOkHwOL+XUkf0?= =?us-ascii?Q?AGEs6AeKcmxdt/2bXNS8gJY72MAoEd1vtx89Si4eXd7RJfTX8RFtqNpxs5FE?= =?us-ascii?Q?MuKFSgRmVSUFodEoTnDVPFDWDT07RLGsKVVWOpRdAFBKINAJC3Li2yarHOep?= =?us-ascii?Q?MuKgXCg/6cj+VntbempsSXB6W3pGRQMXsDq4wYJlDsBlKRAaZGu2HEMw1Xz9?= =?us-ascii?Q?AUR2oWbc7My/yx8Pce3mHLOxLrFuj3nCZu9dqMYn+o9WnJ1ArjN1LtudIe6H?= =?us-ascii?Q?Vw=3D=3D?= X-OriginatorOrg: jaguarmicro.com X-MS-Exchange-CrossTenant-Network-Message-Id: 9928d291-fab7-4012-ed75-08db7c36f1e9 X-MS-Exchange-CrossTenant-AuthSource: SI2PR06MB4752.apcprd06.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 04 Jul 2023 02:32:43.3768 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 1e45a5c2-d3e1-46b3-a0e6-c5ebf6d8ba7b X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: GX7Goax4r+awTE8qSP2H1W5YGnzhZ5yrcy0rslADrdcA+YH6j8Ea5AZ4b1KxgP2bp5STgchyiREA2qiEnSa51Q== X-MS-Exchange-Transport-CrossTenantHeadersStamped: PUZPR06MB5936 X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Since backend and frontend message are synchronous in the same thread, there will be a probability of message deadlock. Consider each driver to determine whether to wait for response. Fixes: d90cf7d111ac ("vhost: support host notifier") Cc: maxime.coquelin@redhat.com Signed-off-by: Rma Ma --- v2 - fix format error in commit message --- lib/vhost/vdpa_driver.h | 3 +++ lib/vhost/vhost_user.c | 23 ++++++++++++++++++----- 2 files changed, 21 insertions(+), 5 deletions(-) diff --git a/lib/vhost/vdpa_driver.h b/lib/vhost/vdpa_driver.h index 8db4ab9f4d..3d2ea3c90e 100644 --- a/lib/vhost/vdpa_driver.h +++ b/lib/vhost/vdpa_driver.h @@ -81,6 +81,9 @@ struct rte_vdpa_dev_ops { /** get device type: net device, blk device... */ int (*get_dev_type)(struct rte_vdpa_device *dev, uint32_t *type); + + /** Get the notify reply flag */ + int (*get_notify_reply_flag)(int vid, bool *need_reply); }; /** diff --git a/lib/vhost/vhost_user.c b/lib/vhost/vhost_user.c index 901a80bbaa..aa61992939 100644 --- a/lib/vhost/vhost_user.c +++ b/lib/vhost/vhost_user.c @@ -3365,13 +3365,14 @@ rte_vhost_backend_config_change(int vid, bool need_reply) static int vhost_user_backend_set_vring_host_notifier(struct virtio_net *dev, int index, int fd, uint64_t offset, - uint64_t size) + uint64_t size, + bool need_reply) { int ret; struct vhu_msg_context ctx = { .msg = { .request.backend = VHOST_USER_BACKEND_VRING_HOST_NOTIFIER_MSG, - .flags = VHOST_USER_VERSION | VHOST_USER_NEED_REPLY, + .flags = VHOST_USER_VERSION, .size = sizeof(ctx.msg.payload.area), .payload.area = { .u64 = index & VHOST_USER_VRING_IDX_MASK, @@ -3388,7 +3389,13 @@ static int vhost_user_backend_set_vring_host_notifier(struct virtio_net *dev, ctx.fd_num = 1; } - ret = send_vhost_backend_message_process_reply(dev, &ctx); + if (!need_reply) + ret = send_vhost_backend_message(dev, &ctx); + else { + ctx.msg.flags |= VHOST_USER_NEED_REPLY; + ret = send_vhost_backend_message_process_reply(dev, &ctx); + } + if (ret < 0) VHOST_LOG_CONFIG(dev->ifname, ERR, "failed to set host notifier (%d)\n", ret); @@ -3402,6 +3409,7 @@ int rte_vhost_host_notifier_ctrl(int vid, uint16_t qid, bool enable) int vfio_device_fd, ret = 0; uint64_t offset, size; unsigned int i, q_start, q_last; + bool need_reply; dev = get_device(vid); if (!dev) @@ -3440,6 +3448,11 @@ int rte_vhost_host_notifier_ctrl(int vid, uint16_t qid, bool enable) if (vfio_device_fd < 0) return -ENOTSUP; + if (vdpa_dev->ops->get_notify_reply_flag == NULL) + need_reply = true; + else + vdpa_dev->ops->get_notify_reply_flag(vid, &need_reply); + if (enable) { for (i = q_start; i <= q_last; i++) { if (vdpa_dev->ops->get_notify_area(vid, i, &offset, @@ -3449,7 +3462,7 @@ int rte_vhost_host_notifier_ctrl(int vid, uint16_t qid, bool enable) } if (vhost_user_backend_set_vring_host_notifier(dev, i, - vfio_device_fd, offset, size) < 0) { + vfio_device_fd, offset, size, need_reply) < 0) { ret = -EFAULT; goto disable; } @@ -3458,7 +3471,7 @@ int rte_vhost_host_notifier_ctrl(int vid, uint16_t qid, bool enable) disable: for (i = q_start; i <= q_last; i++) { vhost_user_backend_set_vring_host_notifier(dev, i, -1, - 0, 0); + 0, 0, need_reply); } } -- 2.17.1