From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id EF89EA0543; Mon, 6 Jun 2022 13:25:45 +0200 (CEST) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 8075542BC3; Mon, 6 Jun 2022 13:23:17 +0200 (CEST) Received: from NAM12-MW2-obe.outbound.protection.outlook.com (mail-mw2nam12on2079.outbound.protection.outlook.com [40.107.244.79]) by mails.dpdk.org (Postfix) with ESMTP id 1254C42BBB for ; Mon, 6 Jun 2022 13:23:16 +0200 (CEST) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=O8xpzz+pRV0g5+cgeQaglTUhYNS2taOydEJ/5GKUG36snonwZJYepKbNCyN+I9rqxLTTHE+/FE/nw1OGzgeWpWSOg73ZwLZXpn7riZnfpufcGDC2K7+wPjPwcNUDtFrz4sZwoEhJ9Kyjr6cunNKw2XT4PVOCZ56VH0ouh+5bcs1cgtlC0DiFNAHx5HdYgJbRFOfBf1QV7r/utQZof/+4fVvK7pfcli30kAIa1ftxBISRP1oQc8L/e6NbRY0PeAP6W5bSERKX75IrwRj9nwL3jvmBPSsaOLsI6cAKnU9OtI2zKIG5eZhBaD9x4R3vP1l4xUBCDFHI90+iXMF/hjyXWA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=XsUnvxy0pfveSZZ6jgiaSH5PTTAq7dr+MbccXrR8wgI=; b=fE6G7lLx1Xd/ATO8lA1rgw2rVAygV7yt01/QUytTq+daRi7owg1+mNVheIAwY5xKv594KgI8CMSpm/kAhJvzy+TfRjxsv+bMxk1mEA+3Sb7womXIKHZdfViJk1DAQrFW0VZ4K2R/49uisJrjJEB0zTsFRIwsnPOb6qelgC/r3KynfDDHWHOr3HDJqXMbDFxaLuNCX8VSY8LVuVBZJB1qXYpCB0+Q3GoldRhXZK+07sNZx/LlKh1VY8Lhuqr3XSKtbZz1SLu4rHG5OhnPLc577LU8nYVeNl9FS3ARZbG8k/HmBVOvtDFjPJRQkwl47nOTk/U1vZL7PLsmpq8I5Kg4Mw== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 12.22.5.236) smtp.rcpttodomain=monjalon.net smtp.mailfrom=nvidia.com; dmarc=pass (p=reject sp=reject pct=100) action=none header.from=nvidia.com; dkim=none (message not signed); arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=XsUnvxy0pfveSZZ6jgiaSH5PTTAq7dr+MbccXrR8wgI=; b=LkppW1JQ0YNvAAs7YhnwZ+L6aPot0gtb6qr7VIH4glEooJfxqqqBKuKhsEI8Z3o+8Ie0vkilt4oMA4orVVT5SG1rAEE7rOMuIPpgDd7m3rNarpevg8pyEn1CCqtxFVR8CVSKzD1cvKndoIU0QrRmLHkZQw4R+FHy5q0D/HsKMrXKarfvN9yhghmWQZxg1TNjY2+02uzHLVjh0VtTfvwKauP0V3TavLuDOTUnHuXg68ny8isw6smcPdO4O0lzmDz6ljxWvOpw2/jqQ2gRFcj2vpIUJEMsHujZJsyeFC5LPQQEU1oBELfaFurvgZp/VN9G5R7HZr96FX1grT/jB1BF5A== Received: from BN0PR03CA0056.namprd03.prod.outlook.com (2603:10b6:408:e7::31) by DS0PR12MB6632.namprd12.prod.outlook.com (2603:10b6:8:d0::9) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5314.19; Mon, 6 Jun 2022 11:23:10 +0000 Received: from BN8NAM11FT046.eop-nam11.prod.protection.outlook.com (2603:10b6:408:e7:cafe::8c) by BN0PR03CA0056.outlook.office365.com (2603:10b6:408:e7::31) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5314.13 via Frontend Transport; Mon, 6 Jun 2022 11:23:10 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 12.22.5.236) smtp.mailfrom=nvidia.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=nvidia.com; Received-SPF: Pass (protection.outlook.com: domain of nvidia.com designates 12.22.5.236 as permitted sender) receiver=protection.outlook.com; client-ip=12.22.5.236; helo=mail.nvidia.com; pr=C Received: from mail.nvidia.com (12.22.5.236) by BN8NAM11FT046.mail.protection.outlook.com (10.13.177.127) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384) id 15.20.5314.12 via Frontend Transport; Mon, 6 Jun 2022 11:23:10 +0000 Received: from rnnvmail201.nvidia.com (10.129.68.8) by DRHQMAIL109.nvidia.com (10.27.9.19) with Microsoft SMTP Server (TLS) id 15.0.1497.32; Mon, 6 Jun 2022 11:23:09 +0000 Received: from nvidia.com (10.126.231.35) by rnnvmail201.nvidia.com (10.129.68.8) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.986.22; Mon, 6 Jun 2022 04:23:06 -0700 From: Li Zhang To: , , , CC: , , , Subject: [PATCH v1 17/17] vdpa/mlx5: prepare virtqueue resource creation Date: Mon, 6 Jun 2022 14:21:09 +0300 Message-ID: <20220606112109.208873-33-lizh@nvidia.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20220606112109.208873-1-lizh@nvidia.com> References: <20220408075606.33056-1-lizh@nvidia.com> <20220606112109.208873-1-lizh@nvidia.com> MIME-Version: 1.0 Content-Transfer-Encoding: 8bit Content-Type: text/plain X-Originating-IP: [10.126.231.35] X-ClientProxiedBy: rnnvmail202.nvidia.com (10.129.68.7) To rnnvmail201.nvidia.com (10.129.68.8) X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: 071330bf-dd87-4811-6cac-08da47aeeffb X-MS-TrafficTypeDiagnostic: DS0PR12MB6632:EE_ X-LD-Processed: 43083d15-7273-40c1-b7db-39efd9ccc17a,ExtAddr X-Microsoft-Antispam-PRVS: X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: pyDrL+z2mKvEPNbu3GFPQnNFogVS0Sbmk5iRqdd1TXfk6Gq47NfIm9on5sHw/kzY4V9DSJmmz+Nez/PTCqTN6Rc4vIeynHhUthxkxFxMoa5Tise6JKXHAXupaCPTokShPz4Wpucsf/3uwjo3ABP9xOiRN9tLymBmJNr46HPHdK8rYMqclzqQm7u5vB7/UfEN7wXXduXK6ElgwrAWqNnkPqVThglMC07/onrCI7aY/KIHlBV41sqrr0tkSap7hAOt8Hl7UdXlUP+ZOAi0ZopW/02MbdPcTdUIGTe7xHn0uMN9Io37ZOWENHHrj6S3/Svyns+PfcoQJPwIxH8NGg/ywi//uQLAyC3MDDhl2ELhdwCwBCTUu7T1TPRTX41X1foX6KYe5qd6FuFAbwKiE7qEgAxhfCHU3rMPrDjozS1rjlctQMsaGQTEv/VXfU5E+NhPCrkyGM6qN6gtal6bT2HXL4utrzqTQyaQr3qyVv5emmKLGP7VDqK9KY58WWwYIFVXRCD6n2CC3YDB2O5bMtfwEyViubAK7PPivlOpyuLTtf881mO5SQY3qXU0CAaDHWoXn0XSXDZ7z7xzUCwy61rmxzKOdfbKMrNp4QMmVqNIsGybx8Fank1aXhXwM1HpgNMy6utT1gFnqC8lqOklRLgdayFOBOHgD5EndLKL4rmtIuewdxZV+9tANDVT5/D+ta6s4ju0fUs10I8I7v+zieNO5A== X-Forefront-Antispam-Report: CIP:12.22.5.236; CTRY:US; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:mail.nvidia.com; PTR:InfoNoRecords; CAT:NONE; SFS:(13230001)(4636009)(40470700004)(36840700001)(46966006)(30864003)(81166007)(6286002)(5660300002)(356005)(82310400005)(26005)(2616005)(36860700001)(107886003)(1076003)(86362001)(55016003)(8936002)(40460700003)(508600001)(54906003)(2906002)(6636002)(110136005)(70586007)(7696005)(316002)(70206006)(47076005)(426003)(83380400001)(8676002)(336012)(4326008)(36756003)(16526019)(186003)(36900700001); DIR:OUT; SFP:1101; X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 06 Jun 2022 11:23:10.0741 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 071330bf-dd87-4811-6cac-08da47aeeffb X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=43083d15-7273-40c1-b7db-39efd9ccc17a; Ip=[12.22.5.236]; Helo=[mail.nvidia.com] X-MS-Exchange-CrossTenant-AuthSource: BN8NAM11FT046.eop-nam11.prod.protection.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: DS0PR12MB6632 X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Split the virtqs virt-queue resource between the configuration threads. Also need pre-created virt-queue resource after virtq destruction. This accelerates the LM process and reduces its time by 30%. Signed-off-by: Li Zhang --- drivers/vdpa/mlx5/mlx5_vdpa.c | 115 ++++++++++++++++++++------ drivers/vdpa/mlx5/mlx5_vdpa.h | 12 ++- drivers/vdpa/mlx5/mlx5_vdpa_cthread.c | 15 +++- drivers/vdpa/mlx5/mlx5_vdpa_virtq.c | 111 +++++++++++++++++++++---- 4 files changed, 208 insertions(+), 45 deletions(-) diff --git a/drivers/vdpa/mlx5/mlx5_vdpa.c b/drivers/vdpa/mlx5/mlx5_vdpa.c index f006a9cd3f..c5d82872c7 100644 --- a/drivers/vdpa/mlx5/mlx5_vdpa.c +++ b/drivers/vdpa/mlx5/mlx5_vdpa.c @@ -275,23 +275,18 @@ mlx5_vdpa_wait_dev_close_tasks_done(struct mlx5_vdpa_priv *priv) } static int -mlx5_vdpa_dev_close(int vid) +_internal_mlx5_vdpa_dev_close(struct mlx5_vdpa_priv *priv, + bool release_resource) { - struct rte_vdpa_device *vdev = rte_vhost_get_vdpa_device(vid); - struct mlx5_vdpa_priv *priv = - mlx5_vdpa_find_priv_resource_by_vdev(vdev); int ret = 0; + int vid = priv->vid; - if (priv == NULL) { - DRV_LOG(ERR, "Invalid vDPA device: %s.", vdev->device->name); - return -1; - } mlx5_vdpa_cqe_event_unset(priv); if (priv->state == MLX5_VDPA_STATE_CONFIGURED) { ret |= mlx5_vdpa_lm_log(priv); priv->state = MLX5_VDPA_STATE_IN_PROGRESS; } - if (priv->use_c_thread) { + if (priv->use_c_thread && !release_resource) { if (priv->last_c_thrd_idx >= (conf_thread_mng.max_thrds - 1)) priv->last_c_thrd_idx = 0; @@ -315,7 +310,7 @@ mlx5_vdpa_dev_close(int vid) pthread_mutex_lock(&priv->steer_update_lock); mlx5_vdpa_steer_unset(priv); pthread_mutex_unlock(&priv->steer_update_lock); - mlx5_vdpa_virtqs_release(priv); + mlx5_vdpa_virtqs_release(priv, release_resource); mlx5_vdpa_drain_cq(priv); if (priv->lm_mr.addr) mlx5_os_wrapped_mkey_destroy(&priv->lm_mr); @@ -329,6 +324,24 @@ mlx5_vdpa_dev_close(int vid) return ret; } +static int +mlx5_vdpa_dev_close(int vid) +{ + struct rte_vdpa_device *vdev = rte_vhost_get_vdpa_device(vid); + struct mlx5_vdpa_priv *priv; + + if (!vdev) { + DRV_LOG(ERR, "Invalid vDPA device."); + return -1; + } + priv = mlx5_vdpa_find_priv_resource_by_vdev(vdev); + if (priv == NULL) { + DRV_LOG(ERR, "Invalid vDPA device: %s.", vdev->device->name); + return -1; + } + return _internal_mlx5_vdpa_dev_close(priv, false); +} + static int mlx5_vdpa_dev_config(int vid) { @@ -624,11 +637,33 @@ mlx5_vdpa_config_get(struct mlx5_kvargs_ctrl *mkvlist, priv->queue_size); } +void +mlx5_vdpa_prepare_virtq_destroy(struct mlx5_vdpa_priv *priv) +{ + uint32_t max_queues, index; + struct mlx5_vdpa_virtq *virtq; + + if (!priv->queues || !priv->queue_size) + return; + max_queues = ((priv->queues * 2) < priv->caps.max_num_virtio_queues) ? + (priv->queues * 2) : (priv->caps.max_num_virtio_queues); + if (mlx5_vdpa_is_modify_virtq_supported(priv)) + mlx5_vdpa_steer_unset(priv); + for (index = 0; index < max_queues; ++index) { + virtq = &priv->virtqs[index]; + if (virtq->virtq) { + pthread_mutex_lock(&virtq->virtq_lock); + mlx5_vdpa_virtq_unset(virtq); + pthread_mutex_unlock(&virtq->virtq_lock); + } + } +} + static int mlx5_vdpa_virtq_resource_prepare(struct mlx5_vdpa_priv *priv) { - uint32_t max_queues; - uint32_t index; + uint32_t remaining_cnt = 0, err_cnt = 0, task_num = 0; + uint32_t max_queues, index, thrd_idx, data[1]; struct mlx5_vdpa_virtq *virtq; for (index = 0; index < priv->caps.max_num_virtio_queues; @@ -640,25 +675,53 @@ mlx5_vdpa_virtq_resource_prepare(struct mlx5_vdpa_priv *priv) return 0; max_queues = (priv->queues < priv->caps.max_num_virtio_queues) ? (priv->queues * 2) : (priv->caps.max_num_virtio_queues); - for (index = 0; index < max_queues; ++index) - if (mlx5_vdpa_virtq_single_resource_prepare(priv, - index)) + if (priv->use_c_thread) { + uint32_t main_task_idx[max_queues]; + + for (index = 0; index < max_queues; ++index) { + thrd_idx = index % (conf_thread_mng.max_thrds + 1); + if (!thrd_idx) { + main_task_idx[task_num] = index; + task_num++; + continue; + } + thrd_idx = priv->last_c_thrd_idx + 1; + if (thrd_idx >= conf_thread_mng.max_thrds) + thrd_idx = 0; + priv->last_c_thrd_idx = thrd_idx; + data[0] = index; + if (mlx5_vdpa_task_add(priv, thrd_idx, + MLX5_VDPA_TASK_PREPARE_VIRTQ, + &remaining_cnt, &err_cnt, + (void **)&data, 1)) { + DRV_LOG(ERR, "Fail to add " + "task prepare virtq (%d).", index); + main_task_idx[task_num] = index; + task_num++; + } + } + for (index = 0; index < task_num; ++index) + if (mlx5_vdpa_virtq_single_resource_prepare(priv, + main_task_idx[index])) + goto error; + if (mlx5_vdpa_c_thread_wait_bulk_tasks_done(&remaining_cnt, + &err_cnt, 2000)) { + DRV_LOG(ERR, + "Failed to wait virt-queue prepare tasks ready."); goto error; + } + } else { + for (index = 0; index < max_queues; ++index) + if (mlx5_vdpa_virtq_single_resource_prepare(priv, + index)) + goto error; + } if (mlx5_vdpa_is_modify_virtq_supported(priv)) if (mlx5_vdpa_steer_update(priv, true)) goto error; return 0; error: - for (index = 0; index < max_queues; ++index) { - virtq = &priv->virtqs[index]; - if (virtq->virtq) { - pthread_mutex_lock(&virtq->virtq_lock); - mlx5_vdpa_virtq_unset(virtq); - pthread_mutex_unlock(&virtq->virtq_lock); - } - } - if (mlx5_vdpa_is_modify_virtq_supported(priv)) - mlx5_vdpa_steer_unset(priv); + mlx5_vdpa_prepare_virtq_destroy(priv); return -1; } @@ -860,7 +923,7 @@ static void mlx5_vdpa_dev_release(struct mlx5_vdpa_priv *priv) { if (priv->state == MLX5_VDPA_STATE_CONFIGURED) - mlx5_vdpa_dev_close(priv->vid); + _internal_mlx5_vdpa_dev_close(priv, true); if (priv->use_c_thread) mlx5_vdpa_wait_dev_close_tasks_done(priv); mlx5_vdpa_release_dev_resources(priv); diff --git a/drivers/vdpa/mlx5/mlx5_vdpa.h b/drivers/vdpa/mlx5/mlx5_vdpa.h index f353db62ac..dc4dfba5ed 100644 --- a/drivers/vdpa/mlx5/mlx5_vdpa.h +++ b/drivers/vdpa/mlx5/mlx5_vdpa.h @@ -85,6 +85,7 @@ enum mlx5_vdpa_task_type { MLX5_VDPA_TASK_SETUP_VIRTQ, MLX5_VDPA_TASK_STOP_VIRTQ, MLX5_VDPA_TASK_DEV_CLOSE_NOWAIT, + MLX5_VDPA_TASK_PREPARE_VIRTQ, }; /* Generic task information and size must be multiple of 4B. */ @@ -128,6 +129,9 @@ struct mlx5_vdpa_virtq { uint32_t configured:1; uint32_t enable:1; uint32_t stopped:1; + uint32_t rx_csum:1; + uint32_t virtio_version_1_0:1; + uint32_t event_mode:3; uint32_t version; pthread_mutex_t virtq_lock; struct mlx5_vdpa_priv *priv; @@ -355,8 +359,12 @@ void mlx5_vdpa_err_event_unset(struct mlx5_vdpa_priv *priv); * * @param[in] priv * The vdpa driver private structure. + * @param[in] release_resource + * The vdpa driver release resource without prepare resource. */ -void mlx5_vdpa_virtqs_release(struct mlx5_vdpa_priv *priv); +void +mlx5_vdpa_virtqs_release(struct mlx5_vdpa_priv *priv, + bool release_resource); /** * Cleanup cached resources of all virtqs. @@ -595,4 +603,6 @@ int mlx5_vdpa_qps2rst2rts(struct mlx5_vdpa_event_qp *eqp); void mlx5_vdpa_virtq_unset(struct mlx5_vdpa_virtq *virtq); +void +mlx5_vdpa_prepare_virtq_destroy(struct mlx5_vdpa_priv *priv); #endif /* RTE_PMD_MLX5_VDPA_H_ */ diff --git a/drivers/vdpa/mlx5/mlx5_vdpa_cthread.c b/drivers/vdpa/mlx5/mlx5_vdpa_cthread.c index bb2279440b..6e6624e5a3 100644 --- a/drivers/vdpa/mlx5/mlx5_vdpa_cthread.c +++ b/drivers/vdpa/mlx5/mlx5_vdpa_cthread.c @@ -153,6 +153,7 @@ mlx5_vdpa_c_thread_handle(void *arg) __atomic_fetch_add( task.err_cnt, 1, __ATOMIC_RELAXED); } + virtq->enable = 1; pthread_mutex_unlock(&virtq->virtq_lock); break; case MLX5_VDPA_TASK_STOP_VIRTQ: @@ -193,7 +194,7 @@ mlx5_vdpa_c_thread_handle(void *arg) pthread_mutex_lock(&priv->steer_update_lock); mlx5_vdpa_steer_unset(priv); pthread_mutex_unlock(&priv->steer_update_lock); - mlx5_vdpa_virtqs_release(priv); + mlx5_vdpa_virtqs_release(priv, false); mlx5_vdpa_drain_cq(priv); if (priv->lm_mr.addr) mlx5_os_wrapped_mkey_destroy( @@ -205,6 +206,18 @@ mlx5_vdpa_c_thread_handle(void *arg) &priv->dev_close_progress, 0, __ATOMIC_RELAXED); break; + case MLX5_VDPA_TASK_PREPARE_VIRTQ: + ret = mlx5_vdpa_virtq_single_resource_prepare( + priv, task.idx); + if (ret) { + DRV_LOG(ERR, + "Failed to prepare virtq %d.", + task.idx); + __atomic_fetch_add( + task.err_cnt, 1, + __ATOMIC_RELAXED); + } + break; default: DRV_LOG(ERR, "Invalid vdpa task type %d.", task.type); diff --git a/drivers/vdpa/mlx5/mlx5_vdpa_virtq.c b/drivers/vdpa/mlx5/mlx5_vdpa_virtq.c index 20ce382487..d4dd73f861 100644 --- a/drivers/vdpa/mlx5/mlx5_vdpa_virtq.c +++ b/drivers/vdpa/mlx5/mlx5_vdpa_virtq.c @@ -116,18 +116,29 @@ mlx5_vdpa_virtq_unreg_intr_handle_all(struct mlx5_vdpa_priv *priv) } } +static void +mlx5_vdpa_vq_destroy(struct mlx5_vdpa_virtq *virtq) +{ + /* Clean pre-created resource in dev removal only */ + claim_zero(mlx5_devx_cmd_destroy(virtq->virtq)); + virtq->index = 0; + virtq->virtq = NULL; + virtq->configured = 0; +} + /* Release cached VQ resources. */ void mlx5_vdpa_virtqs_cleanup(struct mlx5_vdpa_priv *priv) { unsigned int i, j; + mlx5_vdpa_steer_unset(priv); for (i = 0; i < priv->caps.max_num_virtio_queues; i++) { struct mlx5_vdpa_virtq *virtq = &priv->virtqs[i]; - if (virtq->index != i) - continue; pthread_mutex_lock(&virtq->virtq_lock); + if (virtq->virtq) + mlx5_vdpa_vq_destroy(virtq); for (j = 0; j < RTE_DIM(virtq->umems); ++j) { if (virtq->umems[j].obj) { claim_zero(mlx5_glue->devx_umem_dereg @@ -157,29 +168,37 @@ mlx5_vdpa_virtq_unset(struct mlx5_vdpa_virtq *virtq) if (ret) DRV_LOG(WARNING, "Failed to stop virtq %d.", virtq->index); - claim_zero(mlx5_devx_cmd_destroy(virtq->virtq)); - virtq->index = 0; - virtq->virtq = NULL; - virtq->configured = 0; } + mlx5_vdpa_vq_destroy(virtq); virtq->notifier_state = MLX5_VDPA_NOTIFIER_STATE_DISABLED; } void -mlx5_vdpa_virtqs_release(struct mlx5_vdpa_priv *priv) +mlx5_vdpa_virtqs_release(struct mlx5_vdpa_priv *priv, + bool release_resource) { struct mlx5_vdpa_virtq *virtq; - int i; - - for (i = 0; i < priv->nr_virtqs; i++) { + uint32_t i, max_virtq, valid_vq_num; + + valid_vq_num = ((priv->queues * 2) < priv->caps.max_num_virtio_queues) ? + (priv->queues * 2) : priv->caps.max_num_virtio_queues; + max_virtq = (release_resource && + (valid_vq_num) > priv->nr_virtqs) ? + (valid_vq_num) : priv->nr_virtqs; + for (i = 0; i < max_virtq; i++) { virtq = &priv->virtqs[i]; pthread_mutex_lock(&virtq->virtq_lock); mlx5_vdpa_virtq_unset(virtq); - if (i < (priv->queues * 2)) + virtq->enable = 0; + if (!release_resource && i < valid_vq_num) mlx5_vdpa_virtq_single_resource_prepare( priv, i); pthread_mutex_unlock(&virtq->virtq_lock); } + if (!release_resource && priv->queues && + mlx5_vdpa_is_modify_virtq_supported(priv)) + if (mlx5_vdpa_steer_update(priv, true)) + mlx5_vdpa_steer_unset(priv); priv->features = 0; priv->nr_virtqs = 0; } @@ -455,6 +474,9 @@ mlx5_vdpa_virtq_single_resource_prepare(struct mlx5_vdpa_priv *priv, virtq->priv = priv; if (!virtq->virtq) return true; + virtq->rx_csum = attr.rx_csum; + virtq->virtio_version_1_0 = attr.virtio_version_1_0; + virtq->event_mode = attr.event_mode; } return false; } @@ -538,6 +560,9 @@ mlx5_vdpa_virtq_setup(struct mlx5_vdpa_priv *priv, int index, bool reg_kick) goto error; } claim_zero(rte_vhost_enable_guest_notification(priv->vid, index, 1)); + virtq->rx_csum = attr.rx_csum; + virtq->virtio_version_1_0 = attr.virtio_version_1_0; + virtq->event_mode = attr.event_mode; virtq->configured = 1; rte_spinlock_lock(&priv->db_lock); rte_write32(virtq->index, priv->virtq_db_addr); @@ -629,6 +654,31 @@ mlx5_vdpa_features_validate(struct mlx5_vdpa_priv *priv) return 0; } +static bool +mlx5_vdpa_is_pre_created_vq_mismatch(struct mlx5_vdpa_priv *priv, + struct mlx5_vdpa_virtq *virtq) +{ + struct rte_vhost_vring vq; + uint32_t event_mode; + + if (virtq->rx_csum != + !!(priv->features & (1ULL << VIRTIO_NET_F_GUEST_CSUM))) + return true; + if (virtq->virtio_version_1_0 != + !!(priv->features & (1ULL << VIRTIO_F_VERSION_1))) + return true; + if (rte_vhost_get_vhost_vring(priv->vid, virtq->index, &vq)) + return true; + if (vq.size != virtq->vq_size) + return true; + event_mode = vq.callfd != -1 || !(priv->caps.event_mode & + (1 << MLX5_VIRTQ_EVENT_MODE_NO_MSIX)) ? + MLX5_VIRTQ_EVENT_MODE_QP : MLX5_VIRTQ_EVENT_MODE_NO_MSIX; + if (virtq->event_mode != event_mode) + return true; + return false; +} + int mlx5_vdpa_virtqs_prepare(struct mlx5_vdpa_priv *priv) { @@ -664,6 +714,15 @@ mlx5_vdpa_virtqs_prepare(struct mlx5_vdpa_priv *priv) virtq = &priv->virtqs[i]; if (!virtq->enable) continue; + if (priv->queues && virtq->virtq) { + if (mlx5_vdpa_is_pre_created_vq_mismatch(priv, virtq)) { + mlx5_vdpa_prepare_virtq_destroy(priv); + i = 0; + virtq = &priv->virtqs[i]; + if (!virtq->enable) + continue; + } + } thrd_idx = i % (conf_thread_mng.max_thrds + 1); if (!thrd_idx) { main_task_idx[task_num] = i; @@ -693,6 +752,7 @@ mlx5_vdpa_virtqs_prepare(struct mlx5_vdpa_priv *priv) pthread_mutex_unlock(&virtq->virtq_lock); goto error; } + virtq->enable = 1; pthread_mutex_unlock(&virtq->virtq_lock); } if (mlx5_vdpa_c_thread_wait_bulk_tasks_done(&remaining_cnt, @@ -724,20 +784,32 @@ mlx5_vdpa_virtqs_prepare(struct mlx5_vdpa_priv *priv) } else { for (i = 0; i < nr_vring; i++) { virtq = &priv->virtqs[i]; + if (!virtq->enable) + continue; + if (priv->queues && virtq->virtq) { + if (mlx5_vdpa_is_pre_created_vq_mismatch(priv, + virtq)) { + mlx5_vdpa_prepare_virtq_destroy( + priv); + i = 0; + virtq = &priv->virtqs[i]; + if (!virtq->enable) + continue; + } + } pthread_mutex_lock(&virtq->virtq_lock); - if (virtq->enable) { - if (mlx5_vdpa_virtq_setup(priv, i, true)) { - pthread_mutex_unlock( + if (mlx5_vdpa_virtq_setup(priv, i, true)) { + pthread_mutex_unlock( &virtq->virtq_lock); - goto error; - } + goto error; } + virtq->enable = 1; pthread_mutex_unlock(&virtq->virtq_lock); } } return 0; error: - mlx5_vdpa_virtqs_release(priv); + mlx5_vdpa_virtqs_release(priv, true); return -1; } @@ -795,6 +867,11 @@ mlx5_vdpa_virtq_enable(struct mlx5_vdpa_priv *priv, int index, int enable) "for virtq %d.", index); } mlx5_vdpa_virtq_unset(virtq); + } else { + if (virtq->virtq && + mlx5_vdpa_is_pre_created_vq_mismatch(priv, virtq)) + DRV_LOG(WARNING, + "Configuration mismatch dummy virtq %d.", index); } if (enable) { ret = mlx5_vdpa_virtq_setup(priv, index, true); -- 2.31.1