From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id ABCEF42DC7; Tue, 4 Jul 2023 04:52:18 +0200 (CEST) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 93AEB42D0E; Tue, 4 Jul 2023 04:52:18 +0200 (CEST) Received: from APC01-SG2-obe.outbound.protection.outlook.com (mail-sgaapc01on2071.outbound.protection.outlook.com [40.107.215.71]) by mails.dpdk.org (Postfix) with ESMTP id E487340E03 for ; Tue, 4 Jul 2023 04:52:16 +0200 (CEST) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=aR1qcbxLu8+aMp4wnREykaA0eiZKxHsxK3XXzhKciMlnOt/uKt4VdeBPDQ+cmYqWKh5u8nVbEKu8/TmvzKL1EDQ1/tPSAwbuODt7cJCKREAU7GzZmS5cNXu97Dl4U1Ahy+AQFuaJ0PMCelbeixaS5H3U9cKa6sru7RzM5aweFBW6JwLuNroJazOA+JDWJLQ6jBRn0ZK2TPl+J67FC3bjL2ULizG3ogf+kwvJLtJ2mjB/YIfDaFtwEki86nuhid5BgbZOEyAwbBBYS+uoVv8ciuA2WBQSSX+Ef/DTXoNc5LTHQoDSXLZSkFqZwuw1dIgVCujjIXCq9xQ3Prj6mfOLgg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=H8b75a9RT9+42eP6yqt6bxNCXKxPtFcz8nv3UEgRagQ=; b=GxO2CXAZVOVf9v5oldx9GJINf3LfkDZc/WbtrvdbH/s5AahUSeEjGf7xJN6GVLCKNtCip8e5CFh0hei37HCq+llHORInS3Lw32z+TuMX+hHHQCSZvcvXjgSutXnoKpvCaV1Zmrt0wcn5Mb6E7M8/CjtYXKEGQlYrufkRpbhCP5xmfJ7JJQHqmfEuo12qazLYmvrWKZoNLIvr2hErRrWv/pytzuwDRdyVSUROsb/Emn6XY+kx5YYJt3uHBizhN+MEeiAH7dvqMKqdUV8TxovIciIYI6/kEVES57OgDHZvuSbEGzQPHTVT86fbjrHkr3FqxIfCDhQgC8dZfC9UpoSHRg== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=jaguarmicro.com; dmarc=pass action=none header.from=jaguarmicro.com; dkim=pass header.d=jaguarmicro.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=jaguarmicro.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=H8b75a9RT9+42eP6yqt6bxNCXKxPtFcz8nv3UEgRagQ=; b=iigc8XAh+LCWxW4uDXnQOuQyxP+Ov/Dz6lbnidOl+JK/AjTWVwYIFqcFfmWvJI54go4RrPLkduxAKTQXyB74l2yIU/p9Pbe2FzfG36EsBi7LBP0liqlYuVJOWAF7dD/qxmFiKMNnA9yBcaqKSluxUkUc4ewdpqRue13BDLk3RDtMMTZ6ojqZ/+Oz+qqvntXJYwf0Sv3QmgNUJsw1vgSM1bC/T7TKP3/0Ksd8FTmyIITJjn5TgRE3m/8aHV1FfaS3LPSnmRZ3RoB2LV+3EilZwSvorJYnSw2gGiFsyZmTf8kU7BkjN4vSFtn/H9PD9YyMYkDD/CQxkCxSaXJz2dPjUA== Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=jaguarmicro.com; Received: from SI2PR06MB4752.apcprd06.prod.outlook.com (2603:1096:4:14c::14) by SG2PR06MB5335.apcprd06.prod.outlook.com (2603:1096:4:1d7::5) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6544.24; Tue, 4 Jul 2023 02:52:12 +0000 Received: from SI2PR06MB4752.apcprd06.prod.outlook.com ([fe80::25f2:1ee4:9de:7b00]) by SI2PR06MB4752.apcprd06.prod.outlook.com ([fe80::25f2:1ee4:9de:7b00%3]) with mapi id 15.20.6544.024; Tue, 4 Jul 2023 02:52:12 +0000 From: Rma Ma To: dpdk-dev Cc: Maxime Coquelin , Chenbo Xia , Rma Ma Subject: [PATCH v3] vhost: add notify reply ops to fix message deadlock Date: Tue, 4 Jul 2023 10:51:57 +0800 Message-Id: <20230704025157.26024-1-rma.ma@jaguarmicro.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20230704023228.24879-1-rma.ma@jaguarmicro.com> References: <20230704023228.24879-1-rma.ma@jaguarmicro.com> Content-Type: text/plain X-ClientProxiedBy: SI2PR02CA0008.apcprd02.prod.outlook.com (2603:1096:4:194::12) To SI2PR06MB4752.apcprd06.prod.outlook.com (2603:1096:4:14c::14) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: SI2PR06MB4752:EE_|SG2PR06MB5335:EE_ X-MS-Office365-Filtering-Correlation-Id: 55aaf7c0-4991-4912-6aba-08db7c39aa7f X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: HDXZG5L7ZYZZrc+p4AXBW4yAPxj6YUktfwXIN29QM2YLWv6M7/HTxw3SjgYdcaahM/bSf4WPqJ59zP4Nvl85QiLubLZ+ZaQzHij9PjmYKFB14NgM1ar74uCvBVT/K2DmQEkqf1xkMeDgFnunJtb8osM/2RqGV1rjVqU2fU11GvnzDMlGPMDixoiGD6D136eyajxRvNOgETV4FiRzeeiYsUknpuGanO8oXrvbLfsFT2eZXE8SFr2YGt3ubET4hwLJ8ObdngATOFXP7Jm+dmwGeczXXtNyYasjlAgcCT3mjc7a0IlE4/d6xQwWR53KYHYDUM1uIL3YMESf++w6ArZ0v94inNZVPle2D3BKOVJfvhFrtISHoKCRBbYZFjpmSnRVtDoCf+Yt510Y1zElA4mgkWcx/dw8adY1XOZjMino9QTpVco7RV1aRJ2AIytuDzRfj0Bb5Y7que4m/k5bjP/QurFEDx8xMkc1MMOQyv05ZJXMZRT6C7D0UO5+QC7znrzkdAIIEcim/pndzi/booaJXHkxbkIc5Pj2HONFNShWFsiEo/nS95yKltB+gx7lQrz8LphGFhwGfdPA2gvMtZSu0F8nJHWgZYOMS6CoeY1ukploNwjUAeqrNhnQBAAUgpGn X-Forefront-Antispam-Report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:SI2PR06MB4752.apcprd06.prod.outlook.com; PTR:; CAT:NONE; SFS:(13230028)(366004)(396003)(376002)(39840400004)(346002)(136003)(451199021)(6486002)(52116002)(38100700002)(6666004)(38350700002)(83380400001)(2616005)(186003)(36756003)(26005)(1076003)(6506007)(6512007)(107886003)(478600001)(2906002)(5660300002)(316002)(54906003)(66556008)(66476007)(6916009)(66946007)(44832011)(41300700001)(4326008)(86362001)(8676002)(8936002)(15650500001); DIR:OUT; SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: IhTvUUk49/tTKVPWqd/HUd+1Y0vLSXBm44aoJIwaLDJXRR28nbriU2LBxM3ug1e2yBunc1AgdqKqY+l5aKPuC6ZmkgeqtyxQaFXeLldwZpTFbSduLfSlfoBU+OMD4aE3eaBRw77udsyYXBpmFI0g9swynZ2x2mlZlDteJu5dNkhmaVS5WOegzx0xZnqLJUN5dW/0KRb/kYYST67rTq+0VWxYRkkm44Ah1Tny2R5zS/66IVzTljVUnILFef87jCUHliCcb4LsvMuUWMN5yzf0dhjk+rz+x43AL+qGdxQ233s1MYxuKtYPzr6GHxN243JEre3XaNEC4m8SuLalk/X8suIzh1MrGAJKE4Sv6aDG5/vQ5vMCLoLnbU1XtLDtXwdIx863n+x152UZG4xzcZvg9FRKxpmD00semlhW3yHANx8= X-OriginatorOrg: jaguarmicro.com X-MS-Exchange-CrossTenant-Network-Message-Id: 55aaf7c0-4991-4912-6aba-08db7c39aa7f X-MS-Exchange-CrossTenant-AuthSource: SI2PR06MB4752.apcprd06.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 04 Jul 2023 02:52:12.0046 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 1e45a5c2-d3e1-46b3-a0e6-c5ebf6d8ba7b X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: XpDHXYPW+MRkCCdhcrTcdViE2gka4iIpqCG5mGP4oCg3YCC+n7wiEytspI0oYjDHzzzsg6VHWnTfllS90allJA== X-MS-Exchange-Transport-CrossTenantHeadersStamped: SG2PR06MB5335 X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Since backend and frontend message are synchronous in the same thread, there will be a probability of message deadlock. Consider each driver to determine whether to wait for response. Fixes: d90cf7d111ac ("vhost: support host notifier") Cc: maxime.coquelin@redhat.com Signed-off-by: Rma Ma --- v2 - fix format error in commit message v3 - add --in-reply-to --- lib/vhost/vdpa_driver.h | 3 +++ lib/vhost/vhost_user.c | 23 ++++++++++++++++++----- 2 files changed, 21 insertions(+), 5 deletions(-) diff --git a/lib/vhost/vdpa_driver.h b/lib/vhost/vdpa_driver.h index 8db4ab9f4d..3d2ea3c90e 100644 --- a/lib/vhost/vdpa_driver.h +++ b/lib/vhost/vdpa_driver.h @@ -81,6 +81,9 @@ struct rte_vdpa_dev_ops { /** get device type: net device, blk device... */ int (*get_dev_type)(struct rte_vdpa_device *dev, uint32_t *type); + + /** Get the notify reply flag */ + int (*get_notify_reply_flag)(int vid, bool *need_reply); }; /** diff --git a/lib/vhost/vhost_user.c b/lib/vhost/vhost_user.c index 901a80bbaa..aa61992939 100644 --- a/lib/vhost/vhost_user.c +++ b/lib/vhost/vhost_user.c @@ -3365,13 +3365,14 @@ rte_vhost_backend_config_change(int vid, bool need_reply) static int vhost_user_backend_set_vring_host_notifier(struct virtio_net *dev, int index, int fd, uint64_t offset, - uint64_t size) + uint64_t size, + bool need_reply) { int ret; struct vhu_msg_context ctx = { .msg = { .request.backend = VHOST_USER_BACKEND_VRING_HOST_NOTIFIER_MSG, - .flags = VHOST_USER_VERSION | VHOST_USER_NEED_REPLY, + .flags = VHOST_USER_VERSION, .size = sizeof(ctx.msg.payload.area), .payload.area = { .u64 = index & VHOST_USER_VRING_IDX_MASK, @@ -3388,7 +3389,13 @@ static int vhost_user_backend_set_vring_host_notifier(struct virtio_net *dev, ctx.fd_num = 1; } - ret = send_vhost_backend_message_process_reply(dev, &ctx); + if (!need_reply) + ret = send_vhost_backend_message(dev, &ctx); + else { + ctx.msg.flags |= VHOST_USER_NEED_REPLY; + ret = send_vhost_backend_message_process_reply(dev, &ctx); + } + if (ret < 0) VHOST_LOG_CONFIG(dev->ifname, ERR, "failed to set host notifier (%d)\n", ret); @@ -3402,6 +3409,7 @@ int rte_vhost_host_notifier_ctrl(int vid, uint16_t qid, bool enable) int vfio_device_fd, ret = 0; uint64_t offset, size; unsigned int i, q_start, q_last; + bool need_reply; dev = get_device(vid); if (!dev) @@ -3440,6 +3448,11 @@ int rte_vhost_host_notifier_ctrl(int vid, uint16_t qid, bool enable) if (vfio_device_fd < 0) return -ENOTSUP; + if (vdpa_dev->ops->get_notify_reply_flag == NULL) + need_reply = true; + else + vdpa_dev->ops->get_notify_reply_flag(vid, &need_reply); + if (enable) { for (i = q_start; i <= q_last; i++) { if (vdpa_dev->ops->get_notify_area(vid, i, &offset, @@ -3449,7 +3462,7 @@ int rte_vhost_host_notifier_ctrl(int vid, uint16_t qid, bool enable) } if (vhost_user_backend_set_vring_host_notifier(dev, i, - vfio_device_fd, offset, size) < 0) { + vfio_device_fd, offset, size, need_reply) < 0) { ret = -EFAULT; goto disable; } @@ -3458,7 +3471,7 @@ int rte_vhost_host_notifier_ctrl(int vid, uint16_t qid, bool enable) disable: for (i = q_start; i <= q_last; i++) { vhost_user_backend_set_vring_host_notifier(dev, i, -1, - 0, 0); + 0, 0, need_reply); } } -- 2.17.1