From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 4EDA142D73; Fri, 30 Jun 2023 09:15:29 +0200 (CEST) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id EAC6042BAC; Fri, 30 Jun 2023 09:15:25 +0200 (CEST) Received: from APC01-SG2-obe.outbound.protection.outlook.com (mail-sgaapc01on2050.outbound.protection.outlook.com [40.107.215.50]) by mails.dpdk.org (Postfix) with ESMTP id D71734021F for ; Fri, 30 Jun 2023 05:07:23 +0200 (CEST) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=Wno9CTo43hrI5eMmf+g0+tOw+6vHvhCLFiTYJNMUPVMayz4maFcIieDD52w5XagknC11xXvfTK8BNyoQRmw69YCvB8ewyk57VACIurNhe8VPOLJbdpAgF4tr0GJQgkJdR2wf8p8Yr3MLU+pYV2t0KoLneEbSTHN+prJbHkz9ui2J3rKaCeM6SSpbNOMdo+u78JG9avnblpC5XmYrKR0yEjHiEgoFixFxcvjgCJw2Jf4Noky6RC1eFeQPidIMjzAIBt3NO+kUpL9p9UeQMFWaWj/2tCKyKwwWPCSV95mwlxHNWrP1AQQIvEmaSZ8hZ1eMcAU2W6wy0MfyLMsPfSyDaw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=rGXT5dp55x53GCKMub9j80jYSeASZXKO0D8QUsjHpsQ=; b=QBNDBT5AAGnlABsCc0zbGVvHjQcti9ezwgc9Z4ODibxb9NHi+eeYn5ARsXUv/pT7j+PwDTF5Q7K9XIrihGvbFAVYgqa84w0MK06ZRWxViVKDEQ6VBX8vKD0pHfWtU70giGWGxSpJrLxA800Of2VIBCsZop3X4bTweQzMtaVuEN/WaJExKmT0LZvePiJcQBU0kxNnQ9QX4SmelTEgjlBiEMZqKc4LAwXnkO1Uwe/WeCNBeQPG3xyDpoGeE8LAHQ2wuS5ahldNNq7n0rU/QSzAnt0KE9zFIZldLv0B/WC0jIzrR4JlrVGZKQeqU2UPz/GWiXgPuDhU/4wuDZJLW40iSg== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=jaguarmicro.com; dmarc=pass action=none header.from=jaguarmicro.com; dkim=pass header.d=jaguarmicro.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=jaguarmicro.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=rGXT5dp55x53GCKMub9j80jYSeASZXKO0D8QUsjHpsQ=; b=MC4TNjZ8myO3pJDV3vt1NvI6jCBC9NJgveFBhr+c2dDvYBLM4wkbY+rTxrlEFOo88qoboM9j/EXdoGMGUP8tu19m0+LojW7G4sQQ5L51VQu4X+ar2FFs5p4z4Se6SB8By+lv8oazdTPfK+Gu3urF604KhGU1iHGfG08wermJCNZ3/eDBIoxcB0CXN6cAARy5owFiblR+sRxzu45ruHDxU6AaZ2JoKqTIx6iXcAMxYubPHCHft6ldfAi+Jir/AbTytUBXksQLdqO1jc+l3hByY2tAcP21cJWqwF17bsLR0htbF8asQ5+z3pJsrcXN+Rv7woLj4XVvCWP+98IP2dhvtA== Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=jaguarmicro.com; Received: from SI2PR06MB4752.apcprd06.prod.outlook.com (2603:1096:4:14c::14) by KL1PR06MB6489.apcprd06.prod.outlook.com (2603:1096:820:f8::5) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6521.24; Fri, 30 Jun 2023 03:07:20 +0000 Received: from SI2PR06MB4752.apcprd06.prod.outlook.com ([fe80::25f2:1ee4:9de:7b00]) by SI2PR06MB4752.apcprd06.prod.outlook.com ([fe80::25f2:1ee4:9de:7b00%3]) with mapi id 15.20.6521.026; Fri, 30 Jun 2023 03:07:20 +0000 From: Rma Ma To: dpdk-dev Cc: Maxime Coquelin , Chenbo Xia , Rma Ma Subject: [PATCH] vhost: add notify reply ops to avoid message deadlock Date: Fri, 30 Jun 2023 11:07:05 +0800 Message-Id: <20230630030705.1696-1-rma.ma@jaguarmicro.com> X-Mailer: git-send-email 2.17.1 Content-Type: text/plain X-ClientProxiedBy: TYCP286CA0166.JPNP286.PROD.OUTLOOK.COM (2603:1096:400:3c6::17) To SI2PR06MB4752.apcprd06.prod.outlook.com (2603:1096:4:14c::14) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: SI2PR06MB4752:EE_|KL1PR06MB6489:EE_ X-MS-Office365-Filtering-Correlation-Id: c2b8db0c-639c-4671-9fa7-08db79171e42 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: aCG2ASBDOWJ4enIwbVBZ53m2tsjhXuJy/K7ichMzM/Bnbs2cbXe5fBrD+p85twJcjeWX8REcJjD+0rS/CwMFxeTVOhUzVnlrAhdrL2u2lzNfsl9qfZvilAbhM0JBKheutkRVKQTn3SYQnpvqBm9TnKTC0cHsjxwKl1bx5dvX8vWzF4uodVO7K/H3o5S+eH5kytPfTsV+tjtWulcU53TIEfTfwOnKuO3GDlbw7XC4Rtl2j6elmRFTvE0iwvjLPViBW61MiB+rhV1ySPgQUSuE1lR09wzaWvZYpDumqEkBb94R+mBndfbL1jmJFFq9Bg2Eqvi2QoawIe8Ueya9B08bIXe6+ACSvEgTJZgWpjrEe2DlpIQtKdQKtk01wtqCO0eKFqDQNvf9y2o52coYkPxdsaxSmU5yCw4ffWp5op2OwQ62ccYZNz57UM4uuGWk8Dk7tdWd4vcOT65k7WgNdUdwB4CuWFeS4Z7tzM1tmjpSVPBKc5l5IduIKR7KUWNweRCJsIvRtNXoCCCDYE+1h6sPKBA6fjOHdeG941A3RLkoGu2BDuS1kfw8Kp7oSPeC9OZi+FboLJoaLi9MJyPr2b6ntQYRniVMVWv9JCYq3a4ngGvkFdwZub9NSqENfrZeky2a X-Forefront-Antispam-Report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:SI2PR06MB4752.apcprd06.prod.outlook.com; PTR:; CAT:NONE; SFS:(13230028)(366004)(376002)(39840400004)(396003)(136003)(346002)(451199021)(6666004)(1076003)(2906002)(52116002)(6486002)(107886003)(83380400001)(2616005)(38100700002)(26005)(186003)(6512007)(38350700002)(6506007)(41300700001)(86362001)(54906003)(478600001)(4326008)(66946007)(316002)(66476007)(66556008)(6916009)(44832011)(36756003)(8936002)(5660300002)(8676002)(15650500001); DIR:OUT; SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?us-ascii?Q?fC/L8F9WPzsWIII4ePe0+qwvnbJqArMM0HCt0Tdnyneq+tfhzoQSLHPVqqsQ?= =?us-ascii?Q?IQNHy/6QpWmJ2G8AAUNZUWEX0gKQ6U9H5NHI5ggCfnCv98/gQ4Ymzyk4Vsop?= =?us-ascii?Q?j31vLp9d06gP3AkTlEFF3ICvxw0RMq8tKz4ycv2Z0Ip34bJhpKqvrxBKoSrM?= =?us-ascii?Q?mgfrcRpK7YRmAN/YShHdM3nIIHRkqNiQtQH2Xsgii+EFERCbNPs2tEntgTLl?= =?us-ascii?Q?fXSGDHpLO3HYMwVxfhQtS8EvQCMwd1MyTLSlWtcd00kVeCSq89kQZdkxhYzk?= =?us-ascii?Q?8wHeDkfGR+0v7shC2nyHrEt16s5AYOiw+FdQPwjNk7xHqrVOsmHu9HEUkCMX?= =?us-ascii?Q?BMZMmQvuvuvBMvp39yzkVYUHPiL+7e14Hc9MkwjddzHQtMxNZg/bmbpeLvln?= =?us-ascii?Q?f5ceZuTpYhfGytVK7sGR1PAykcRtFK4nF4IwmymtR1/RBXRs+jIi2SW+Cie+?= =?us-ascii?Q?gf2m/BSaa/62rDG/yiayMuwv+WFxLA1sdlzgcpuPpTNnVOxgSu4f9zlM1+b+?= =?us-ascii?Q?uYiKHoH6Qdte2CT58bUMEprMpjlPwqJcIG+neVvMe2qD8Zz6IcsdJmoZJB1o?= =?us-ascii?Q?n01ljfaYYaoMy2Sy817l3b68Rmbkbhv+JFOscftcDuJ9PjWXqF6TKnhOYJSh?= =?us-ascii?Q?UX2WpkXxA3Pr55KmBO7zTpvQj5N/RCtahT/Uw3DKQryVbDLSstQPSK8g2/gX?= =?us-ascii?Q?sz5cI+y1vNWhPe6OPSYc0jFLtr+fHgQQ4EsT4kKGgBJaALLYjpE2IHW0sncU?= =?us-ascii?Q?Tzp9bsv/qADj0+Vsv4bq9Asr0+fsr1Ckl+pBzsSNV4MxCGcnJ86XLl7O5e6l?= =?us-ascii?Q?1NDuySPWPVf0NHwxllPR349cm71W0VBwDfWLDQB9G3Cs+manNoJUivfJUQpG?= =?us-ascii?Q?pRN+kY+gu7t9jHJiJhLMCQAajPWb+Jal9gqKWcyXDCA9ojOnL6W3oqNN8pnu?= =?us-ascii?Q?D4ohjKa9Wi5BGSaTVjhoQC5Mc+lA9lrpnyCTW8CmijsMPRUZpR82OHWlU0MW?= =?us-ascii?Q?YGn2U6hYbAgmhOfaEnvDWpBqIyJzA0+5AofKY+7s2l53YgKdBiqFi78+5Sll?= =?us-ascii?Q?bIrAvezUEOffW1AKrxWh+Akn08tv06UDWuemqeApe75Rc5R2JhDlj8v9atlW?= =?us-ascii?Q?6cjI2RYWcy7/skiuVpVdWb2owoQrE7l5g2uvWvykWR+vgps15cFN70XIq102?= =?us-ascii?Q?EFXOdG5lrTlB1MyObPCgdjHmsCVMr4sLikMb1sNwz2WyJNAocP8ahnQJTcTe?= =?us-ascii?Q?ezCmHpm9khLqOEqSYTt4h33E1YgMr+Da8NP7a9PGyy4qRYfXMNAAJDTHx4kk?= =?us-ascii?Q?++NN/ohMiakWhcrgzYHXWOXPINs8eIfXRduE1jPi3Mv6+D5lwoXxm8cuitf0?= =?us-ascii?Q?SEYZ/cfCxMILRyhGqDLWBiH/BZBWd/bZRIbekfzD0m+yPmczqW+kxCCNZTz4?= =?us-ascii?Q?8j5/4juCORrVe2SHeEgbVpjUvanjUILwhq0Lr4YfYuSG6KxtjtM+RRWw4omd?= =?us-ascii?Q?LkuJKf0VGQvuWUngb1/ywEiwAnvN9SDIzarsYp2sK78k+N/lx27j3lufir09?= =?us-ascii?Q?oexoy6lRXP4Lzc0mFqWotiswJBHSbiqhkSuNHclr?= X-OriginatorOrg: jaguarmicro.com X-MS-Exchange-CrossTenant-Network-Message-Id: c2b8db0c-639c-4671-9fa7-08db79171e42 X-MS-Exchange-CrossTenant-AuthSource: SI2PR06MB4752.apcprd06.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 30 Jun 2023 03:07:20.4627 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 1e45a5c2-d3e1-46b3-a0e6-c5ebf6d8ba7b X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: Z68uqo+ZHgysr8lqvGTZJKjluGVi6vanPdUbfpzRNNYyvW1QbWznPMGElguy/S9pM4aEkTHG35jsCCy+B346bw== X-MS-Exchange-Transport-CrossTenantHeadersStamped: KL1PR06MB6489 X-Mailman-Approved-At: Fri, 30 Jun 2023 09:15:24 +0200 X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Since backend and frontend message are synchronous in the same thread, there will be a probability of message deadlock. Consider each driver to determine whether to wait for response. Signed-off-by: Rma Ma --- lib/vhost/vdpa_driver.h | 3 +++ lib/vhost/vhost_user.c | 23 ++++++++++++++++++----- 2 files changed, 21 insertions(+), 5 deletions(-) diff --git a/lib/vhost/vdpa_driver.h b/lib/vhost/vdpa_driver.h index 8db4ab9f4d..3d2ea3c90e 100644 --- a/lib/vhost/vdpa_driver.h +++ b/lib/vhost/vdpa_driver.h @@ -81,6 +81,9 @@ struct rte_vdpa_dev_ops { /** get device type: net device, blk device... */ int (*get_dev_type)(struct rte_vdpa_device *dev, uint32_t *type); + + /** Get the notify reply flag */ + int (*get_notify_reply_flag)(int vid, bool *need_reply); }; /** diff --git a/lib/vhost/vhost_user.c b/lib/vhost/vhost_user.c index 901a80bbaa..aa61992939 100644 --- a/lib/vhost/vhost_user.c +++ b/lib/vhost/vhost_user.c @@ -3365,13 +3365,14 @@ rte_vhost_backend_config_change(int vid, bool need_reply) static int vhost_user_backend_set_vring_host_notifier(struct virtio_net *dev, int index, int fd, uint64_t offset, - uint64_t size) + uint64_t size, + bool need_reply) { int ret; struct vhu_msg_context ctx = { .msg = { .request.backend = VHOST_USER_BACKEND_VRING_HOST_NOTIFIER_MSG, - .flags = VHOST_USER_VERSION | VHOST_USER_NEED_REPLY, + .flags = VHOST_USER_VERSION, .size = sizeof(ctx.msg.payload.area), .payload.area = { .u64 = index & VHOST_USER_VRING_IDX_MASK, @@ -3388,7 +3389,13 @@ static int vhost_user_backend_set_vring_host_notifier(struct virtio_net *dev, ctx.fd_num = 1; } - ret = send_vhost_backend_message_process_reply(dev, &ctx); + if (!need_reply) + ret = send_vhost_backend_message(dev, &ctx); + else { + ctx.msg.flags |= VHOST_USER_NEED_REPLY; + ret = send_vhost_backend_message_process_reply(dev, &ctx); + } + if (ret < 0) VHOST_LOG_CONFIG(dev->ifname, ERR, "failed to set host notifier (%d)\n", ret); @@ -3402,6 +3409,7 @@ int rte_vhost_host_notifier_ctrl(int vid, uint16_t qid, bool enable) int vfio_device_fd, ret = 0; uint64_t offset, size; unsigned int i, q_start, q_last; + bool need_reply; dev = get_device(vid); if (!dev) @@ -3440,6 +3448,11 @@ int rte_vhost_host_notifier_ctrl(int vid, uint16_t qid, bool enable) if (vfio_device_fd < 0) return -ENOTSUP; + if (vdpa_dev->ops->get_notify_reply_flag == NULL) + need_reply = true; + else + vdpa_dev->ops->get_notify_reply_flag(vid, &need_reply); + if (enable) { for (i = q_start; i <= q_last; i++) { if (vdpa_dev->ops->get_notify_area(vid, i, &offset, @@ -3449,7 +3462,7 @@ int rte_vhost_host_notifier_ctrl(int vid, uint16_t qid, bool enable) } if (vhost_user_backend_set_vring_host_notifier(dev, i, - vfio_device_fd, offset, size) < 0) { + vfio_device_fd, offset, size, need_reply) < 0) { ret = -EFAULT; goto disable; } @@ -3458,7 +3471,7 @@ int rte_vhost_host_notifier_ctrl(int vid, uint16_t qid, bool enable) disable: for (i = q_start; i <= q_last; i++) { vhost_user_backend_set_vring_host_notifier(dev, i, -1, - 0, 0); + 0, 0, need_reply); } } -- 2.17.1