From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id BB3DB45459; Fri, 14 Jun 2024 09:03:37 +0200 (CEST) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 0547D427E0; Fri, 14 Jun 2024 09:02:25 +0200 (CEST) Received: from NAM12-MW2-obe.outbound.protection.outlook.com (mail-mw2nam12on2128.outbound.protection.outlook.com [40.107.244.128]) by mails.dpdk.org (Postfix) with ESMTP id 353B0402C6 for ; Fri, 14 Jun 2024 09:02:02 +0200 (CEST) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=dqkCF6kKwDptjudxzKfwxQr/m5w1ua5pcfBczixwdmcXU5TwcIeyLMUIey6zNlVK+3dwjJtywWroNRzDwuqJIIfQ7bZY5asRClf8vSlGy1ujETaQFYZrxLDkJy7JA15IubAlePWrzxQ/LCcbdHVnQfSU6wjmjDlCu7omI6GoBFJDRmn2kIYjdqkWcjd+rZKB73hW6gN/sSeYyObR8HpAO5AmPqyLiiyVn1m/+1TqEWY04nbBrsTtOl07nsJs8Qq9CFdRHmt2+T1K4pVudIIBGSZDtWXXbdet7/+47E18HSujllCb+DC0RQl95r3oc2YhxgJiLmLOqJKsfrkb9vXzpg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=XG6vRSkER8ac7CIjvfpuuQM57snZ+XRNK7vObV5TNH8=; b=oSTFkQE8d4GLvSpeJkb723+qHfwBSyOFKWN+/BSEeX0UdNxJNIW5rPtlfBSER8mtgW5nuNm8rBw6cEqqw2URgj0xaXf7b3I97PuUY24yx8VEwG3vvgdU+CyBOd7vp31GTQ0jtVMjuzPuILY7nHeE3TjHMCaYSTn9Wva7vcWAy78why/ANZcjH+HYvaJWOVB42+gz8BPgLvKir7AXJY8VjnD6ejKvoOKYXV380ZGbWPFXuQmhjWh2JljFxMd1dw2hEEdFX9AGAtzs4fteJblGigttFnIFwQVmmRMC7kchk7XKn7LAEiCinFntg1GIWBhetU/6pIl4HrV0pJp9Yz0goA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=corigine.com; dmarc=pass action=none header.from=corigine.com; dkim=pass header.d=corigine.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=corigine.onmicrosoft.com; s=selector2-corigine-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=XG6vRSkER8ac7CIjvfpuuQM57snZ+XRNK7vObV5TNH8=; b=McRctxu1FatCxdTuBPt2jcw4Ic9pBkV7wH5HEIWjKpmbP76lOxOgezhuaNiyxCuubDpgDJ/6NvZmRjYOzNSaOX7ukMYha9iqAJ6m25RjPwmVIpMB+UIUsmKK22tKble9ebTw63pFA+dHotjx6duqbmOqAx54tckVRDRYS701ybo= Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=corigine.com; Received: from SJ0PR13MB5545.namprd13.prod.outlook.com (2603:10b6:a03:424::5) by CH2PR13MB3831.namprd13.prod.outlook.com (2603:10b6:610:a3::14) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7677.25; Fri, 14 Jun 2024 07:02:00 +0000 Received: from SJ0PR13MB5545.namprd13.prod.outlook.com ([fe80::b900:5f05:766f:833]) by SJ0PR13MB5545.namprd13.prod.outlook.com ([fe80::b900:5f05:766f:833%4]) with mapi id 15.20.7677.024; Fri, 14 Jun 2024 07:02:00 +0000 From: Chaoyong He To: dev@dpdk.org Cc: oss-drivers@corigine.com, Xinying Yu , Chaoyong He , Long Wu , Peng Zhang Subject: [PATCH v2 09/11] vdpa/nfp: setup vring relay thread Date: Fri, 14 Jun 2024 15:01:18 +0800 Message-Id: <20240614070120.2663160-10-chaoyong.he@corigine.com> X-Mailer: git-send-email 2.39.1 In-Reply-To: <20240614070120.2663160-1-chaoyong.he@corigine.com> References: <20240426074831.1729792-1-chaoyong.he@corigine.com> <20240614070120.2663160-1-chaoyong.he@corigine.com> Content-Transfer-Encoding: 8bit Content-Type: text/plain X-ClientProxiedBy: BYAPR06CA0025.namprd06.prod.outlook.com (2603:10b6:a03:d4::38) To SJ0PR13MB5545.namprd13.prod.outlook.com (2603:10b6:a03:424::5) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: SJ0PR13MB5545:EE_|CH2PR13MB3831:EE_ X-MS-Office365-Filtering-Correlation-Id: d7fd1649-2d79-4335-d16e-08dc8c3fe2e6 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; ARA:13230035|376009|366011|1800799019|52116009|38350700009; X-Microsoft-Antispam-Message-Info: =?us-ascii?Q?NW9ZNS0ZpfyrHauCFwqoCvA16Zbooq42nxlnNtLQgYaM4YRjhZq4sAPa5al8?= =?us-ascii?Q?ORYXOkel6Xb6izI6KnPVJ06k3Y+oZ8J1ryS0BG85kc9qtClKoMmlfL+sXrur?= =?us-ascii?Q?qI3fM2S+fRIRrIriT2Xru55NRVXq5/4cxja8UaxuPjt9kXG4YJqrtiB5FYHb?= =?us-ascii?Q?ej6WnxaQUheciBiJ5aqbW13cHZ1wm/cqLYzG8U3OroDKR4ug0BzRG5jaEoM4?= =?us-ascii?Q?A4Z38yUhwaNeu6phG8rMKtSZWEemRcx3iFZH5kJCLNAkTcpyTT5ZPbAQWbj8?= =?us-ascii?Q?kDhjO2fCzkXpSIgK3ZN/Wa+iOtdK83gdXam7l8NAsL1BqPGfmHitvwUKXyJ6?= =?us-ascii?Q?S6Dh4N9HMWD+fYGfPk17jcPmEmkb+n6AUmg/eQiBNEy1GFjxpcaf4Lt6lbL/?= =?us-ascii?Q?xufbxu1pHGrJMFp2vP7BJNKXfAsvmbvmuFk042EnKCAEqXGjTMmKU9b9MIw9?= =?us-ascii?Q?ho1b+h7TQBNq/hBxeKNcct+NC+xycAh//Wm5vsS3XkfNwtPtdR/6dEXNbGgU?= =?us-ascii?Q?9JxI5rM33FJx2T820nnjUJjbZU7z1PP9vgB8yCO+uPtx+oGlJVlu4kJYLa3v?= =?us-ascii?Q?ASj3jIqtEeWqSHLzxv40C7uqK3vMyMluKRUu7KrUkH6ofW2/+oOT4Ezl6Gin?= =?us-ascii?Q?ITEKpUDz2l3ch3N4KxPhnBZCJ5DHuoe1257jwUiVsuBizW2Q80yK1qqWqSkG?= =?us-ascii?Q?PSfzNKFibKCX5tqPP2bGgjlC9C7HUKR7lrRsEICXRtXybWZTcWBMOAA5Urm7?= =?us-ascii?Q?P6k3Syzj1eAMO9dZPyYzVVTyQLJrfMZtj9yMhukTnx6UKZi7Ni/BDb2djPX4?= =?us-ascii?Q?7Hov05WMVaW/+ftGbD2fy/G0/LYXcjnjWr9lrdfuCOnVlLulTgLdMVDMTlVx?= =?us-ascii?Q?nGL8J4cpDNXMK0m4e8sQGOLJG44AJMVlNkdY/xoqODVfiwENw0lpzWX0h5Lg?= =?us-ascii?Q?eaTnolgvYWcfXhFiKYOFykRJcnQEWLt51XHZhHjX2qOHXVmLo+TdL/d1Ie1k?= =?us-ascii?Q?PS22fhKJCcXFMjGhQXos09poSM4e6XOTRL0h2hrqdg9gm06gQyB+Fg8RuWKS?= =?us-ascii?Q?ABW+t68qmPn3HUngZuvyqakLIgB/WD76A8eprf2FFSr0imuRx1DqMGh1JdjV?= =?us-ascii?Q?eCaHyHVk45jNYY4cmy7oAjjtzr3FxzgJSTXVbjwRCPrCd6Bm8cufp7tLurX0?= =?us-ascii?Q?3MVYk2gP5i3d6rIYH9hsav6EMqoBN3TfdIQ/EaiERyC7+4Gb0u1Ge9jNuU6W?= =?us-ascii?Q?9unsUBat+cP7IKIP7kTeERazQ1XzyzGFKzOcbRz1nNlU4FYHhO2PkIrP1RM6?= =?us-ascii?Q?mczsYMisBK0uG6BJRd+KVu3gBvPrnP/kb7Q/4ZborwszeQ3MDoMlQ/OTYzmi?= =?us-ascii?Q?3GKqkAI=3D?= X-Forefront-Antispam-Report: CIP:255.255.255.255; CTRY:; LANG:; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:SJ0PR13MB5545.namprd13.prod.outlook.com; PTR:; CAT:NONE; SFS:(13230035)(376009)(366011)(1800799019)(52116009)(38350700009); DIR:OUT; SFP:1102; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?us-ascii?Q?FSkEVEq2pPvPFyFRwJLNIitB/2l8kuHn22TR6lzo9wL/mjO63F1tiVn0xkYv?= =?us-ascii?Q?4Pl1hHVTamTKP9sBktAZ4PhsJ59+F31Qr1bkYObAYzR7L1ppEEKzhvi6BqjY?= =?us-ascii?Q?iogxA8hJP0GwndJ4ZkzLL8mJdTEiza56bA9HxUfXYJ2iAape5n+27EgrXuze?= =?us-ascii?Q?ptZ8+A4Xgz5jL8/Nqn4mxYT1LbIIlCmuR1b4wbukxF+lHt6ZTajriPgGktOC?= =?us-ascii?Q?u5Ln+idlQ79s+NzsYoX5yhbCkTfxxhA7xKQKsviuKLeAh/udhSVKA2jlYuet?= =?us-ascii?Q?OBsP4ZMoIsi/dF3i+pqtNuNCKYomOKcRmKxCZSb69Y20Jh507tdfQQn1gUwe?= =?us-ascii?Q?iastxQ13R86mgtuZvrMwUc764cXI3gsBKj/QDZ8XXt+PUHTEZ+XJnSNAUar0?= =?us-ascii?Q?7LVEkTb5NdrEDG6fivcBTv6hXovh0rVZzd10lP8w4fhjrFCwzhxVh8+C8m6q?= =?us-ascii?Q?Homec3jPfVlR1qWHIrPt+Fxs+g/eP3BiIhWk5EfQE5QMkPxubFR7NNt4gaY8?= =?us-ascii?Q?SkvlWrf6qWM57g/oMB7nmHLBn616YGitDwQbVYPQDqEY3p2m20FO33FxYwjv?= =?us-ascii?Q?HFFTj6l90rgTOXShzASX2Y1AT1pklajRdjR1NhQMvem7tPittfk/nyReoU8j?= =?us-ascii?Q?yTgyCRvbdZTMpx119FPVeKLDsh4u6ssqXmVxYUqlN3xWZybk4/87Rmav5xKd?= =?us-ascii?Q?QFCgNZjuoj+7PDXVcAc/SwmQURLcFBAj7rDuZdh1bWbFtNWhiGIbL1CdQids?= =?us-ascii?Q?JPU04xWjLuRt6sPAPIvGGA6yMXYUD3sib+Zh6YL0weFCF4W5LPdrllMVMXt7?= =?us-ascii?Q?ZlxdUdTDzw2N3fXs+IXe9kRkm0fDGhgR++3gWr8a4xCYYOC0EG46ZCpFuSLn?= =?us-ascii?Q?YcwVa6LfabhmSQviU6VpIKcJbPWqFcTcY5mFU8d45d5EaL5711gLhC4UTDnc?= =?us-ascii?Q?o/r1Zze/DP5Fx4Vm4aLu2KJ+EDdCiFor4u+DF+J1lggOSRGJP/CEvHIFdj9Q?= =?us-ascii?Q?Y7AILA1X1b6IFLzTYg5Si6gSSzaZ6bzrjZtAq5GJ3f8CJCvB3X5uFy+sKpg3?= =?us-ascii?Q?KwmZq4bYW3CrVoGndDCHPw4Jw3mLI0SF1AtFgynSuRU+LVDSKv1sBXw/c/01?= =?us-ascii?Q?GRJoB40kR3IQOZufP8c67gb3iav8L4lgd8Wbw/tZPyE2jarwzoYdvOMvxIGo?= =?us-ascii?Q?zE+pAKUqtEzWS8Xu2YErwBhbHUvFPEzSbSJbj6opC8Efa3aFFzeHaXW+TZU7?= =?us-ascii?Q?SWOaOAmT28fCgll49VE0pB1wMBfCLFZmXn/jhyXrQM4hQvVfZPmWrMqf8kVZ?= =?us-ascii?Q?bYkMmm/fYA/s20yfK00wYFRB3c7xsawVIeAW52B7ES7qUrHdW4VuvQvMcnUE?= =?us-ascii?Q?CjMApFskzwd+pcDHhaWKPXuXA8+4RG7GkbuXEUiXcMMttbJbJ4ZLR4CnTs5Q?= =?us-ascii?Q?/slKGDaJ217XV4Odj1sOehHOVyFdQqBu1eBKQ4NTNnZfoVxrT7F/pe44R0tJ?= =?us-ascii?Q?fte/8QdOhGFoFicsLMNYbT2E9ngUsEPpDQISguonwC7SAlBaJ5XnhWehgN9E?= =?us-ascii?Q?7c6KLADdK27LKw1WkfTiDvKC7/Znh8QcHYpWyada8I3w+QHX9KTAGdiAKeGo?= =?us-ascii?Q?FA=3D=3D?= X-OriginatorOrg: corigine.com X-MS-Exchange-CrossTenant-Network-Message-Id: d7fd1649-2d79-4335-d16e-08dc8c3fe2e6 X-MS-Exchange-CrossTenant-AuthSource: SJ0PR13MB5545.namprd13.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 14 Jun 2024 07:01:59.9979 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: fe128f2c-073b-4c20-818e-7246a585940c X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: i+wDVmMHLdqbeuqddNpK/FsxkgWqRWZm1TdQtsg2KKlhwZemJ4R3CFGT8RJ6Q/ZwFsRMrK+9Gv2g7i5FKB3UlsaBsfNwqwJnmhYVF3ekq8M= X-MS-Exchange-Transport-CrossTenantHeadersStamped: CH2PR13MB3831 X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org From: Xinying Yu Setup the vring relay thread to monitor the interruption from device. And do the dirty page logging or notify device according to event data. Signed-off-by: Xinying Yu Reviewed-by: Chaoyong He Reviewed-by: Long Wu Reviewed-by: Peng Zhang --- drivers/vdpa/nfp/nfp_vdpa.c | 148 +++++++++++++++++++++++++++++++ drivers/vdpa/nfp/nfp_vdpa_core.c | 9 ++ drivers/vdpa/nfp/nfp_vdpa_core.h | 2 + 3 files changed, 159 insertions(+) diff --git a/drivers/vdpa/nfp/nfp_vdpa.c b/drivers/vdpa/nfp/nfp_vdpa.c index 65f7144671..e57765eb1a 100644 --- a/drivers/vdpa/nfp/nfp_vdpa.c +++ b/drivers/vdpa/nfp/nfp_vdpa.c @@ -26,6 +26,8 @@ #define NFP_VDPA_USED_RING_LEN(size) \ ((size) * sizeof(struct vring_used_elem) + sizeof(struct vring_used)) +#define EPOLL_DATA_INTR 1 + struct nfp_vdpa_dev { struct rte_pci_device *pci_dev; struct rte_vdpa_device *vdev; @@ -776,6 +778,139 @@ update_datapath(struct nfp_vdpa_dev *device) return ret; } +static int +nfp_vdpa_vring_epoll_ctl(uint32_t queue_num, + struct nfp_vdpa_dev *device) +{ + int ret; + uint32_t qid; + struct epoll_event ev; + struct rte_vhost_vring vring; + + for (qid = 0; qid < queue_num; qid++) { + ev.events = EPOLLIN | EPOLLPRI; + rte_vhost_get_vhost_vring(device->vid, qid, &vring); + ev.data.u64 = qid << 1 | (uint64_t)vring.kickfd << 32; + ret = epoll_ctl(device->epoll_fd, EPOLL_CTL_ADD, vring.kickfd, &ev); + if (ret < 0) { + DRV_VDPA_LOG(ERR, "Epoll add error for queue %u", qid); + return ret; + } + } + + /* vDPA driver interrupt */ + for (qid = 0; qid < queue_num; qid += 2) { + ev.events = EPOLLIN | EPOLLPRI; + /* Leave a flag to mark it's for interrupt */ + ev.data.u64 = EPOLL_DATA_INTR | qid << 1 | + (uint64_t)device->intr_fd[qid] << 32; + ret = epoll_ctl(device->epoll_fd, EPOLL_CTL_ADD, + device->intr_fd[qid], &ev); + if (ret < 0) { + DRV_VDPA_LOG(ERR, "Epoll add error for queue %u", qid); + return ret; + } + + nfp_vdpa_update_used_ring(device, qid); + } + + return 0; +} + +static int +nfp_vdpa_vring_epoll_wait(uint32_t queue_num, + struct nfp_vdpa_dev *device) +{ + int i; + int fds; + int kickfd; + uint32_t qid; + struct epoll_event events[NFP_VDPA_MAX_QUEUES * 2]; + + for (;;) { + fds = epoll_wait(device->epoll_fd, events, queue_num * 2, -1); + if (fds < 0) { + if (errno == EINTR) + continue; + + DRV_VDPA_LOG(ERR, "Epoll wait fail"); + return -EACCES; + } + + for (i = 0; i < fds; i++) { + qid = events[i].data.u32 >> 1; + kickfd = (uint32_t)(events[i].data.u64 >> 32); + + nfp_vdpa_read_kickfd(kickfd); + if ((events[i].data.u32 & EPOLL_DATA_INTR) != 0) { + nfp_vdpa_update_used_ring(device, qid); + nfp_vdpa_irq_unmask(&device->hw); + } else { + nfp_vdpa_notify_queue(&device->hw, qid); + } + } + } + + return 0; +} + +static uint32_t +nfp_vdpa_vring_relay(void *arg) +{ + int ret; + int epoll_fd; + uint16_t queue_id; + uint32_t queue_num; + struct nfp_vdpa_dev *device = arg; + + epoll_fd = epoll_create(NFP_VDPA_MAX_QUEUES * 2); + if (epoll_fd < 0) { + DRV_VDPA_LOG(ERR, "failed to create epoll instance."); + return 1; + } + + device->epoll_fd = epoll_fd; + + queue_num = rte_vhost_get_vring_num(device->vid); + + ret = nfp_vdpa_vring_epoll_ctl(queue_num, device); + if (ret != 0) + goto notify_exit; + + /* Start relay with a first kick */ + for (queue_id = 0; queue_id < queue_num; queue_id++) + nfp_vdpa_notify_queue(&device->hw, queue_id); + + ret = nfp_vdpa_vring_epoll_wait(queue_num, device); + if (ret != 0) + goto notify_exit; + + return 0; + +notify_exit: + close(device->epoll_fd); + device->epoll_fd = -1; + + return 1; +} + +static int +nfp_vdpa_setup_vring_relay(struct nfp_vdpa_dev *device) +{ + int ret; + char name[RTE_THREAD_INTERNAL_NAME_SIZE]; + + snprintf(name, sizeof(name), "nfp_vring%d", device->vid); + ret = rte_thread_create_internal_control(&device->tid, name, + nfp_vdpa_vring_relay, (void *)device); + if (ret != 0) { + DRV_VDPA_LOG(ERR, "Failed to create vring relay pthread."); + return -EPERM; + } + + return 0; +} + static int nfp_vdpa_sw_fallback(struct nfp_vdpa_dev *device) { @@ -802,10 +937,17 @@ nfp_vdpa_sw_fallback(struct nfp_vdpa_dev *device) if (ret != 0) goto unset_intr; + /* Setup vring relay thread */ + ret = nfp_vdpa_setup_vring_relay(device); + if (ret != 0) + goto stop_vf; + device->hw.sw_fallback_running = true; return 0; +stop_vf: + nfp_vdpa_stop(device, true); unset_intr: nfp_vdpa_disable_vfio_intr(device); error: @@ -859,6 +1001,12 @@ nfp_vdpa_dev_close(int vid) /* Reset VF */ nfp_vdpa_stop(device, true); + /* Remove interrupt setting */ + nfp_vdpa_disable_vfio_intr(device); + + /* Unset DMA map for guest memory */ + nfp_vdpa_dma_map(device, false); + device->hw.sw_fallback_running = false; rte_atomic_store_explicit(&device->dev_attached, 0, diff --git a/drivers/vdpa/nfp/nfp_vdpa_core.c b/drivers/vdpa/nfp/nfp_vdpa_core.c index d7c48e2490..3b3481a99c 100644 --- a/drivers/vdpa/nfp/nfp_vdpa_core.c +++ b/drivers/vdpa/nfp/nfp_vdpa_core.c @@ -270,3 +270,12 @@ nfp_vdpa_notify_queue(struct nfp_vdpa_hw *vdpa_hw, nfp_qcp_notify_ptr_add(vdpa_hw->notify_addr[qid], NFP_QCP_NOTIFY_WRITE_PTR, qid); } + +void nfp_vdpa_irq_unmask(struct nfp_vdpa_hw *vdpa_hw) +{ + struct nfp_hw *hw = &vdpa_hw->super; + + /* Make sure all updates are written before un-masking */ + rte_wmb(); + nn_cfg_writeb(hw, NFP_NET_CFG_ICR(1), NFP_NET_CFG_ICR_UNMASKED); +} diff --git a/drivers/vdpa/nfp/nfp_vdpa_core.h b/drivers/vdpa/nfp/nfp_vdpa_core.h index a339ace601..bc4db556a2 100644 --- a/drivers/vdpa/nfp/nfp_vdpa_core.h +++ b/drivers/vdpa/nfp/nfp_vdpa_core.h @@ -60,4 +60,6 @@ void nfp_vdpa_notify_queue(struct nfp_vdpa_hw *vdpa_hw, uint16_t qid); uint64_t nfp_vdpa_get_queue_notify_offset(struct nfp_vdpa_hw *vdpa_hw, int qid); +void nfp_vdpa_irq_unmask(struct nfp_vdpa_hw *vdpa_hw); + #endif /* __NFP_VDPA_CORE_H__ */ -- 2.39.1