From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id AB320A0542; Mon, 6 Jun 2022 13:49:40 +0200 (CEST) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 3734F42BC6; Mon, 6 Jun 2022 13:48:09 +0200 (CEST) Received: from NAM12-DM6-obe.outbound.protection.outlook.com (mail-dm6nam12on2082.outbound.protection.outlook.com [40.107.243.82]) by mails.dpdk.org (Postfix) with ESMTP id 036B842BC6 for ; Mon, 6 Jun 2022 13:48:07 +0200 (CEST) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=X/FZFNvIIZJOGUaRyl1WQDCYYy9EaZC32+dWfEKSLzPOkT++fGINbMJGx2kqhJGjrIoZV7RKQ2R03vSPDTlATcjNLGPaiRI9CP3CO6TVWuTrLVq1/rKcTHKAxERw5zbIv3Rv6BxDcV9/KLRxhJWwHKJPRso9iR0Q2ejydHNkiDtrxqNsUF2B5Wl940fut5V4DblKpQ9VPATdr3NZTZ648BSwanreqwo/1wCLNXtcSg6BmI3o+XmNrsVBUUhGedlvHrOkcxGXnWztY/xeNojva/PLi4+1loUz+Slke4psEJJdPbdlUm0DxEPY4oOu+WdP0ZElmt/Vi3hjqnqPSYYsCg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=XsUnvxy0pfveSZZ6jgiaSH5PTTAq7dr+MbccXrR8wgI=; b=JJvJqCu/d36rIANLPrsbPwC1W+eLQIQ7/w+nbzfaeeaai//+kA5FW3bhjhi/gb7WF944jGAuQ7bf2R/izeVRh9j1U/lT/lnMm/69gu7ymyAXpZlj05luqn2ryItKjI5P9SpGp/6YR11Ij29fjcKNwF2MY4FEqwk82IM+LY0T4skANBqlhXpjV/f7RuKeTJuRlX3kLhplI+JtPTm1Q1dvaq9sURts9KtIk2CEWnvTkEoOk1Z19i5CA6GirlltwO5psUb7HJ+pTUUy6cjmNL2aqJquWtpDBU76EJVOJfgvAcG9kLUjlcEiUXsfwWtXOys8nFC2FFN8fL1T+MaGYGLwoA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 12.22.5.235) smtp.rcpttodomain=monjalon.net smtp.mailfrom=nvidia.com; dmarc=pass (p=reject sp=reject pct=100) action=none header.from=nvidia.com; dkim=none (message not signed); arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=XsUnvxy0pfveSZZ6jgiaSH5PTTAq7dr+MbccXrR8wgI=; b=dDfe1v6QMyTPfE0hoC2p9FOn8JKceMR7PAd1VVjkkHQ63LSIKcUxW6u9s8D34pNhj29iMEsWwHWgwgrz3+wp19i+tOEby7YSGbaXkQiuLdMorvkH2+/nlTp3df5kzsRawKSMdEjkI8zptD9y+oVT9/y7ppQFC0xiEg2QiXfRbA/wuHmQl4fMiytuRcU1/hF5bczhX30xJVJ5Tg7lg73VLrnLlb7g5Rdp/kndrcTXxJvu8KersK/e8m9qFh5ZjvmwIhE5N7HsbjAbMDV2j5zl5r7OASTPKIrVj5xaSNs4YJ5Emu48MnQYzrLbMWM0jS/hK+4XZSmsUZ7tTpOaYhFi9A== Received: from DM6PR13CA0011.namprd13.prod.outlook.com (2603:10b6:5:bc::24) by MN2PR12MB4223.namprd12.prod.outlook.com (2603:10b6:208:1d3::18) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5314.13; Mon, 6 Jun 2022 11:48:05 +0000 Received: from DM6NAM11FT028.eop-nam11.prod.protection.outlook.com (2603:10b6:5:bc:cafe::6f) by DM6PR13CA0011.outlook.office365.com (2603:10b6:5:bc::24) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5332.7 via Frontend Transport; Mon, 6 Jun 2022 11:48:05 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 12.22.5.235) smtp.mailfrom=nvidia.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=nvidia.com; Received-SPF: Pass (protection.outlook.com: domain of nvidia.com designates 12.22.5.235 as permitted sender) receiver=protection.outlook.com; client-ip=12.22.5.235; helo=mail.nvidia.com; pr=C Received: from mail.nvidia.com (12.22.5.235) by DM6NAM11FT028.mail.protection.outlook.com (10.13.173.140) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384) id 15.20.5314.12 via Frontend Transport; Mon, 6 Jun 2022 11:48:05 +0000 Received: from rnnvmail201.nvidia.com (10.129.68.8) by DRHQMAIL107.nvidia.com (10.27.9.16) with Microsoft SMTP Server (TLS) id 15.0.1497.32; Mon, 6 Jun 2022 11:48:04 +0000 Received: from nvidia.com (10.126.231.35) by rnnvmail201.nvidia.com (10.129.68.8) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.986.22; Mon, 6 Jun 2022 04:48:01 -0700 From: Li Zhang To: , , , CC: , , , Subject: [PATCH v1 17/17] vdpa/mlx5: prepare virtqueue resource creation Date: Mon, 6 Jun 2022 14:46:50 +0300 Message-ID: <20220606114650.209612-18-lizh@nvidia.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20220606114650.209612-1-lizh@nvidia.com> References: <20220408075606.33056-1-lizh@nvidia.com> <20220606114650.209612-1-lizh@nvidia.com> MIME-Version: 1.0 Content-Transfer-Encoding: 8bit Content-Type: text/plain X-Originating-IP: [10.126.231.35] X-ClientProxiedBy: rnnvmail202.nvidia.com (10.129.68.7) To rnnvmail201.nvidia.com (10.129.68.8) X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: ebb4d759-cea9-4ab9-1dfa-08da47b26b09 X-MS-TrafficTypeDiagnostic: MN2PR12MB4223:EE_ X-LD-Processed: 43083d15-7273-40c1-b7db-39efd9ccc17a,ExtAddr X-Microsoft-Antispam-PRVS: X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: JGp1gGKA7fckxnMpydZH8P8rG1YoLUsMhJMVvcgu8F1ZGtz6VeZaCAImcSM/+lTpBD+jBcY+mXAyN9n7L1CZTqZVHZip0pbe9KZGD6PgE/VWzacTtT9p+25wPifv1FIJVCbCDLTEWgvVHqhKfuW4meHLmKwp9/4vwSFvxpqqndrZnOW5UI3T8zPFAYDRAY/jW5+AO3RiKh9kDykQOResX3yhBhN61KqBoSj1x+x3LrTjnu4CV+svMA0ZEqujBrSf2IMTGnBAtDO/vNpeqf33wjrgOuNTtM+nL8urKBChW3SRKSPkQX3GsG8wtExYvXSXo9OstshwOLE4IgOSXV0/w3tMWQpJ+nmiF7yX/CKkr5+HQ/fbcV3zc7/zViMZMuUxpTB9kVOTWVGesReQuGjGR/MS7jLWRbajQdv1awlNa6adqrD47OzpvNXwCHbhES5W6Ooab9uzaV/9xfZn7zxzZSADUwzDsbhXtFgi1y+uEMZDsMDUDR6Ayf9MZ/OkrxSlp8KPzoQXqYfq6kQesSd2z+SsGWM4t6IqzfYQzn9hvX1foH/40KTPUqb/zMWFM3pIvBvvWNJlqCKdBADebvueqsE14sQ5v3lq+EERuMlt/vL+M0sJRnLh8jMWQKgsON3NGv5VOP1V5Epl5fOJ7S1qesO+vEMNCPeejr6+k9CDfoAyqY02BTHvQdDWvcS0ZF+upyom2wOJs1iSXJecbIhVRQ== X-Forefront-Antispam-Report: CIP:12.22.5.235; CTRY:US; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:mail.nvidia.com; PTR:InfoNoRecords; CAT:NONE; SFS:(13230001)(4636009)(40470700004)(46966006)(36840700001)(70586007)(6666004)(36756003)(70206006)(107886003)(8936002)(4326008)(8676002)(7696005)(81166007)(6286002)(83380400001)(26005)(110136005)(36860700001)(508600001)(54906003)(40460700003)(6636002)(47076005)(426003)(336012)(82310400005)(55016003)(316002)(186003)(2906002)(16526019)(1076003)(356005)(2616005)(86362001)(30864003)(5660300002)(36900700001); DIR:OUT; SFP:1101; X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 06 Jun 2022 11:48:05.0606 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: ebb4d759-cea9-4ab9-1dfa-08da47b26b09 X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=43083d15-7273-40c1-b7db-39efd9ccc17a; Ip=[12.22.5.235]; Helo=[mail.nvidia.com] X-MS-Exchange-CrossTenant-AuthSource: DM6NAM11FT028.eop-nam11.prod.protection.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: MN2PR12MB4223 X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Split the virtqs virt-queue resource between the configuration threads. Also need pre-created virt-queue resource after virtq destruction. This accelerates the LM process and reduces its time by 30%. Signed-off-by: Li Zhang --- drivers/vdpa/mlx5/mlx5_vdpa.c | 115 ++++++++++++++++++++------ drivers/vdpa/mlx5/mlx5_vdpa.h | 12 ++- drivers/vdpa/mlx5/mlx5_vdpa_cthread.c | 15 +++- drivers/vdpa/mlx5/mlx5_vdpa_virtq.c | 111 +++++++++++++++++++++---- 4 files changed, 208 insertions(+), 45 deletions(-) diff --git a/drivers/vdpa/mlx5/mlx5_vdpa.c b/drivers/vdpa/mlx5/mlx5_vdpa.c index f006a9cd3f..c5d82872c7 100644 --- a/drivers/vdpa/mlx5/mlx5_vdpa.c +++ b/drivers/vdpa/mlx5/mlx5_vdpa.c @@ -275,23 +275,18 @@ mlx5_vdpa_wait_dev_close_tasks_done(struct mlx5_vdpa_priv *priv) } static int -mlx5_vdpa_dev_close(int vid) +_internal_mlx5_vdpa_dev_close(struct mlx5_vdpa_priv *priv, + bool release_resource) { - struct rte_vdpa_device *vdev = rte_vhost_get_vdpa_device(vid); - struct mlx5_vdpa_priv *priv = - mlx5_vdpa_find_priv_resource_by_vdev(vdev); int ret = 0; + int vid = priv->vid; - if (priv == NULL) { - DRV_LOG(ERR, "Invalid vDPA device: %s.", vdev->device->name); - return -1; - } mlx5_vdpa_cqe_event_unset(priv); if (priv->state == MLX5_VDPA_STATE_CONFIGURED) { ret |= mlx5_vdpa_lm_log(priv); priv->state = MLX5_VDPA_STATE_IN_PROGRESS; } - if (priv->use_c_thread) { + if (priv->use_c_thread && !release_resource) { if (priv->last_c_thrd_idx >= (conf_thread_mng.max_thrds - 1)) priv->last_c_thrd_idx = 0; @@ -315,7 +310,7 @@ mlx5_vdpa_dev_close(int vid) pthread_mutex_lock(&priv->steer_update_lock); mlx5_vdpa_steer_unset(priv); pthread_mutex_unlock(&priv->steer_update_lock); - mlx5_vdpa_virtqs_release(priv); + mlx5_vdpa_virtqs_release(priv, release_resource); mlx5_vdpa_drain_cq(priv); if (priv->lm_mr.addr) mlx5_os_wrapped_mkey_destroy(&priv->lm_mr); @@ -329,6 +324,24 @@ mlx5_vdpa_dev_close(int vid) return ret; } +static int +mlx5_vdpa_dev_close(int vid) +{ + struct rte_vdpa_device *vdev = rte_vhost_get_vdpa_device(vid); + struct mlx5_vdpa_priv *priv; + + if (!vdev) { + DRV_LOG(ERR, "Invalid vDPA device."); + return -1; + } + priv = mlx5_vdpa_find_priv_resource_by_vdev(vdev); + if (priv == NULL) { + DRV_LOG(ERR, "Invalid vDPA device: %s.", vdev->device->name); + return -1; + } + return _internal_mlx5_vdpa_dev_close(priv, false); +} + static int mlx5_vdpa_dev_config(int vid) { @@ -624,11 +637,33 @@ mlx5_vdpa_config_get(struct mlx5_kvargs_ctrl *mkvlist, priv->queue_size); } +void +mlx5_vdpa_prepare_virtq_destroy(struct mlx5_vdpa_priv *priv) +{ + uint32_t max_queues, index; + struct mlx5_vdpa_virtq *virtq; + + if (!priv->queues || !priv->queue_size) + return; + max_queues = ((priv->queues * 2) < priv->caps.max_num_virtio_queues) ? + (priv->queues * 2) : (priv->caps.max_num_virtio_queues); + if (mlx5_vdpa_is_modify_virtq_supported(priv)) + mlx5_vdpa_steer_unset(priv); + for (index = 0; index < max_queues; ++index) { + virtq = &priv->virtqs[index]; + if (virtq->virtq) { + pthread_mutex_lock(&virtq->virtq_lock); + mlx5_vdpa_virtq_unset(virtq); + pthread_mutex_unlock(&virtq->virtq_lock); + } + } +} + static int mlx5_vdpa_virtq_resource_prepare(struct mlx5_vdpa_priv *priv) { - uint32_t max_queues; - uint32_t index; + uint32_t remaining_cnt = 0, err_cnt = 0, task_num = 0; + uint32_t max_queues, index, thrd_idx, data[1]; struct mlx5_vdpa_virtq *virtq; for (index = 0; index < priv->caps.max_num_virtio_queues; @@ -640,25 +675,53 @@ mlx5_vdpa_virtq_resource_prepare(struct mlx5_vdpa_priv *priv) return 0; max_queues = (priv->queues < priv->caps.max_num_virtio_queues) ? (priv->queues * 2) : (priv->caps.max_num_virtio_queues); - for (index = 0; index < max_queues; ++index) - if (mlx5_vdpa_virtq_single_resource_prepare(priv, - index)) + if (priv->use_c_thread) { + uint32_t main_task_idx[max_queues]; + + for (index = 0; index < max_queues; ++index) { + thrd_idx = index % (conf_thread_mng.max_thrds + 1); + if (!thrd_idx) { + main_task_idx[task_num] = index; + task_num++; + continue; + } + thrd_idx = priv->last_c_thrd_idx + 1; + if (thrd_idx >= conf_thread_mng.max_thrds) + thrd_idx = 0; + priv->last_c_thrd_idx = thrd_idx; + data[0] = index; + if (mlx5_vdpa_task_add(priv, thrd_idx, + MLX5_VDPA_TASK_PREPARE_VIRTQ, + &remaining_cnt, &err_cnt, + (void **)&data, 1)) { + DRV_LOG(ERR, "Fail to add " + "task prepare virtq (%d).", index); + main_task_idx[task_num] = index; + task_num++; + } + } + for (index = 0; index < task_num; ++index) + if (mlx5_vdpa_virtq_single_resource_prepare(priv, + main_task_idx[index])) + goto error; + if (mlx5_vdpa_c_thread_wait_bulk_tasks_done(&remaining_cnt, + &err_cnt, 2000)) { + DRV_LOG(ERR, + "Failed to wait virt-queue prepare tasks ready."); goto error; + } + } else { + for (index = 0; index < max_queues; ++index) + if (mlx5_vdpa_virtq_single_resource_prepare(priv, + index)) + goto error; + } if (mlx5_vdpa_is_modify_virtq_supported(priv)) if (mlx5_vdpa_steer_update(priv, true)) goto error; return 0; error: - for (index = 0; index < max_queues; ++index) { - virtq = &priv->virtqs[index]; - if (virtq->virtq) { - pthread_mutex_lock(&virtq->virtq_lock); - mlx5_vdpa_virtq_unset(virtq); - pthread_mutex_unlock(&virtq->virtq_lock); - } - } - if (mlx5_vdpa_is_modify_virtq_supported(priv)) - mlx5_vdpa_steer_unset(priv); + mlx5_vdpa_prepare_virtq_destroy(priv); return -1; } @@ -860,7 +923,7 @@ static void mlx5_vdpa_dev_release(struct mlx5_vdpa_priv *priv) { if (priv->state == MLX5_VDPA_STATE_CONFIGURED) - mlx5_vdpa_dev_close(priv->vid); + _internal_mlx5_vdpa_dev_close(priv, true); if (priv->use_c_thread) mlx5_vdpa_wait_dev_close_tasks_done(priv); mlx5_vdpa_release_dev_resources(priv); diff --git a/drivers/vdpa/mlx5/mlx5_vdpa.h b/drivers/vdpa/mlx5/mlx5_vdpa.h index f353db62ac..dc4dfba5ed 100644 --- a/drivers/vdpa/mlx5/mlx5_vdpa.h +++ b/drivers/vdpa/mlx5/mlx5_vdpa.h @@ -85,6 +85,7 @@ enum mlx5_vdpa_task_type { MLX5_VDPA_TASK_SETUP_VIRTQ, MLX5_VDPA_TASK_STOP_VIRTQ, MLX5_VDPA_TASK_DEV_CLOSE_NOWAIT, + MLX5_VDPA_TASK_PREPARE_VIRTQ, }; /* Generic task information and size must be multiple of 4B. */ @@ -128,6 +129,9 @@ struct mlx5_vdpa_virtq { uint32_t configured:1; uint32_t enable:1; uint32_t stopped:1; + uint32_t rx_csum:1; + uint32_t virtio_version_1_0:1; + uint32_t event_mode:3; uint32_t version; pthread_mutex_t virtq_lock; struct mlx5_vdpa_priv *priv; @@ -355,8 +359,12 @@ void mlx5_vdpa_err_event_unset(struct mlx5_vdpa_priv *priv); * * @param[in] priv * The vdpa driver private structure. + * @param[in] release_resource + * The vdpa driver release resource without prepare resource. */ -void mlx5_vdpa_virtqs_release(struct mlx5_vdpa_priv *priv); +void +mlx5_vdpa_virtqs_release(struct mlx5_vdpa_priv *priv, + bool release_resource); /** * Cleanup cached resources of all virtqs. @@ -595,4 +603,6 @@ int mlx5_vdpa_qps2rst2rts(struct mlx5_vdpa_event_qp *eqp); void mlx5_vdpa_virtq_unset(struct mlx5_vdpa_virtq *virtq); +void +mlx5_vdpa_prepare_virtq_destroy(struct mlx5_vdpa_priv *priv); #endif /* RTE_PMD_MLX5_VDPA_H_ */ diff --git a/drivers/vdpa/mlx5/mlx5_vdpa_cthread.c b/drivers/vdpa/mlx5/mlx5_vdpa_cthread.c index bb2279440b..6e6624e5a3 100644 --- a/drivers/vdpa/mlx5/mlx5_vdpa_cthread.c +++ b/drivers/vdpa/mlx5/mlx5_vdpa_cthread.c @@ -153,6 +153,7 @@ mlx5_vdpa_c_thread_handle(void *arg) __atomic_fetch_add( task.err_cnt, 1, __ATOMIC_RELAXED); } + virtq->enable = 1; pthread_mutex_unlock(&virtq->virtq_lock); break; case MLX5_VDPA_TASK_STOP_VIRTQ: @@ -193,7 +194,7 @@ mlx5_vdpa_c_thread_handle(void *arg) pthread_mutex_lock(&priv->steer_update_lock); mlx5_vdpa_steer_unset(priv); pthread_mutex_unlock(&priv->steer_update_lock); - mlx5_vdpa_virtqs_release(priv); + mlx5_vdpa_virtqs_release(priv, false); mlx5_vdpa_drain_cq(priv); if (priv->lm_mr.addr) mlx5_os_wrapped_mkey_destroy( @@ -205,6 +206,18 @@ mlx5_vdpa_c_thread_handle(void *arg) &priv->dev_close_progress, 0, __ATOMIC_RELAXED); break; + case MLX5_VDPA_TASK_PREPARE_VIRTQ: + ret = mlx5_vdpa_virtq_single_resource_prepare( + priv, task.idx); + if (ret) { + DRV_LOG(ERR, + "Failed to prepare virtq %d.", + task.idx); + __atomic_fetch_add( + task.err_cnt, 1, + __ATOMIC_RELAXED); + } + break; default: DRV_LOG(ERR, "Invalid vdpa task type %d.", task.type); diff --git a/drivers/vdpa/mlx5/mlx5_vdpa_virtq.c b/drivers/vdpa/mlx5/mlx5_vdpa_virtq.c index 20ce382487..d4dd73f861 100644 --- a/drivers/vdpa/mlx5/mlx5_vdpa_virtq.c +++ b/drivers/vdpa/mlx5/mlx5_vdpa_virtq.c @@ -116,18 +116,29 @@ mlx5_vdpa_virtq_unreg_intr_handle_all(struct mlx5_vdpa_priv *priv) } } +static void +mlx5_vdpa_vq_destroy(struct mlx5_vdpa_virtq *virtq) +{ + /* Clean pre-created resource in dev removal only */ + claim_zero(mlx5_devx_cmd_destroy(virtq->virtq)); + virtq->index = 0; + virtq->virtq = NULL; + virtq->configured = 0; +} + /* Release cached VQ resources. */ void mlx5_vdpa_virtqs_cleanup(struct mlx5_vdpa_priv *priv) { unsigned int i, j; + mlx5_vdpa_steer_unset(priv); for (i = 0; i < priv->caps.max_num_virtio_queues; i++) { struct mlx5_vdpa_virtq *virtq = &priv->virtqs[i]; - if (virtq->index != i) - continue; pthread_mutex_lock(&virtq->virtq_lock); + if (virtq->virtq) + mlx5_vdpa_vq_destroy(virtq); for (j = 0; j < RTE_DIM(virtq->umems); ++j) { if (virtq->umems[j].obj) { claim_zero(mlx5_glue->devx_umem_dereg @@ -157,29 +168,37 @@ mlx5_vdpa_virtq_unset(struct mlx5_vdpa_virtq *virtq) if (ret) DRV_LOG(WARNING, "Failed to stop virtq %d.", virtq->index); - claim_zero(mlx5_devx_cmd_destroy(virtq->virtq)); - virtq->index = 0; - virtq->virtq = NULL; - virtq->configured = 0; } + mlx5_vdpa_vq_destroy(virtq); virtq->notifier_state = MLX5_VDPA_NOTIFIER_STATE_DISABLED; } void -mlx5_vdpa_virtqs_release(struct mlx5_vdpa_priv *priv) +mlx5_vdpa_virtqs_release(struct mlx5_vdpa_priv *priv, + bool release_resource) { struct mlx5_vdpa_virtq *virtq; - int i; - - for (i = 0; i < priv->nr_virtqs; i++) { + uint32_t i, max_virtq, valid_vq_num; + + valid_vq_num = ((priv->queues * 2) < priv->caps.max_num_virtio_queues) ? + (priv->queues * 2) : priv->caps.max_num_virtio_queues; + max_virtq = (release_resource && + (valid_vq_num) > priv->nr_virtqs) ? + (valid_vq_num) : priv->nr_virtqs; + for (i = 0; i < max_virtq; i++) { virtq = &priv->virtqs[i]; pthread_mutex_lock(&virtq->virtq_lock); mlx5_vdpa_virtq_unset(virtq); - if (i < (priv->queues * 2)) + virtq->enable = 0; + if (!release_resource && i < valid_vq_num) mlx5_vdpa_virtq_single_resource_prepare( priv, i); pthread_mutex_unlock(&virtq->virtq_lock); } + if (!release_resource && priv->queues && + mlx5_vdpa_is_modify_virtq_supported(priv)) + if (mlx5_vdpa_steer_update(priv, true)) + mlx5_vdpa_steer_unset(priv); priv->features = 0; priv->nr_virtqs = 0; } @@ -455,6 +474,9 @@ mlx5_vdpa_virtq_single_resource_prepare(struct mlx5_vdpa_priv *priv, virtq->priv = priv; if (!virtq->virtq) return true; + virtq->rx_csum = attr.rx_csum; + virtq->virtio_version_1_0 = attr.virtio_version_1_0; + virtq->event_mode = attr.event_mode; } return false; } @@ -538,6 +560,9 @@ mlx5_vdpa_virtq_setup(struct mlx5_vdpa_priv *priv, int index, bool reg_kick) goto error; } claim_zero(rte_vhost_enable_guest_notification(priv->vid, index, 1)); + virtq->rx_csum = attr.rx_csum; + virtq->virtio_version_1_0 = attr.virtio_version_1_0; + virtq->event_mode = attr.event_mode; virtq->configured = 1; rte_spinlock_lock(&priv->db_lock); rte_write32(virtq->index, priv->virtq_db_addr); @@ -629,6 +654,31 @@ mlx5_vdpa_features_validate(struct mlx5_vdpa_priv *priv) return 0; } +static bool +mlx5_vdpa_is_pre_created_vq_mismatch(struct mlx5_vdpa_priv *priv, + struct mlx5_vdpa_virtq *virtq) +{ + struct rte_vhost_vring vq; + uint32_t event_mode; + + if (virtq->rx_csum != + !!(priv->features & (1ULL << VIRTIO_NET_F_GUEST_CSUM))) + return true; + if (virtq->virtio_version_1_0 != + !!(priv->features & (1ULL << VIRTIO_F_VERSION_1))) + return true; + if (rte_vhost_get_vhost_vring(priv->vid, virtq->index, &vq)) + return true; + if (vq.size != virtq->vq_size) + return true; + event_mode = vq.callfd != -1 || !(priv->caps.event_mode & + (1 << MLX5_VIRTQ_EVENT_MODE_NO_MSIX)) ? + MLX5_VIRTQ_EVENT_MODE_QP : MLX5_VIRTQ_EVENT_MODE_NO_MSIX; + if (virtq->event_mode != event_mode) + return true; + return false; +} + int mlx5_vdpa_virtqs_prepare(struct mlx5_vdpa_priv *priv) { @@ -664,6 +714,15 @@ mlx5_vdpa_virtqs_prepare(struct mlx5_vdpa_priv *priv) virtq = &priv->virtqs[i]; if (!virtq->enable) continue; + if (priv->queues && virtq->virtq) { + if (mlx5_vdpa_is_pre_created_vq_mismatch(priv, virtq)) { + mlx5_vdpa_prepare_virtq_destroy(priv); + i = 0; + virtq = &priv->virtqs[i]; + if (!virtq->enable) + continue; + } + } thrd_idx = i % (conf_thread_mng.max_thrds + 1); if (!thrd_idx) { main_task_idx[task_num] = i; @@ -693,6 +752,7 @@ mlx5_vdpa_virtqs_prepare(struct mlx5_vdpa_priv *priv) pthread_mutex_unlock(&virtq->virtq_lock); goto error; } + virtq->enable = 1; pthread_mutex_unlock(&virtq->virtq_lock); } if (mlx5_vdpa_c_thread_wait_bulk_tasks_done(&remaining_cnt, @@ -724,20 +784,32 @@ mlx5_vdpa_virtqs_prepare(struct mlx5_vdpa_priv *priv) } else { for (i = 0; i < nr_vring; i++) { virtq = &priv->virtqs[i]; + if (!virtq->enable) + continue; + if (priv->queues && virtq->virtq) { + if (mlx5_vdpa_is_pre_created_vq_mismatch(priv, + virtq)) { + mlx5_vdpa_prepare_virtq_destroy( + priv); + i = 0; + virtq = &priv->virtqs[i]; + if (!virtq->enable) + continue; + } + } pthread_mutex_lock(&virtq->virtq_lock); - if (virtq->enable) { - if (mlx5_vdpa_virtq_setup(priv, i, true)) { - pthread_mutex_unlock( + if (mlx5_vdpa_virtq_setup(priv, i, true)) { + pthread_mutex_unlock( &virtq->virtq_lock); - goto error; - } + goto error; } + virtq->enable = 1; pthread_mutex_unlock(&virtq->virtq_lock); } } return 0; error: - mlx5_vdpa_virtqs_release(priv); + mlx5_vdpa_virtqs_release(priv, true); return -1; } @@ -795,6 +867,11 @@ mlx5_vdpa_virtq_enable(struct mlx5_vdpa_priv *priv, int index, int enable) "for virtq %d.", index); } mlx5_vdpa_virtq_unset(virtq); + } else { + if (virtq->virtq && + mlx5_vdpa_is_pre_created_vq_mismatch(priv, virtq)) + DRV_LOG(WARNING, + "Configuration mismatch dummy virtq %d.", index); } if (enable) { ret = mlx5_vdpa_virtq_setup(priv, index, true); -- 2.31.1