From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 06310A00BE; Fri, 8 Apr 2022 09:57:10 +0200 (CEST) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id ECB1A4069F; Fri, 8 Apr 2022 09:57:09 +0200 (CEST) Received: from NAM10-MW2-obe.outbound.protection.outlook.com (mail-mw2nam10on2053.outbound.protection.outlook.com [40.107.94.53]) by mails.dpdk.org (Postfix) with ESMTP id 7768B4003F for ; Fri, 8 Apr 2022 09:57:08 +0200 (CEST) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=l9XfmOXI5J6y5gJKRATiE+REm/N3L9DdEAUOBI4IBeBCcfbRe3lFUNWQyvja7bVO0szB4ANoRyFEKAtQhzSecKykZwEqx+9PL/hY0MLES1Q4HQd0tAD+pA1sOmvhYFoUEa1LJHSKpfwQUAnVn7FSTgVC+vn6i86sQCPCvq2Y6wJdfrw00GeW1mPCx1uN715oqz6Q1fiEJI9VlwWMfu45P7f+oeEZ26R7+hSXczmFnnMGXSxE1S8zN5ptg7XVvYygSz7JLhGGh5y4ivhmFAAmQ6v8OW0oMdueMdLlNBjgeOJu1QpD+PMKOPx/JIRIiL9Vafohhbx4ak64IpAjwyPM0w== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=5llDo/VEnhj/LFT+BCQtiTmTHeZopi7Sdt7SvKuZetw=; b=YrWXYaiAB1hFIonoSWPaiKBpOki/B3/0U24a17n+89TTgiCcgMud0FCGTmmtuLcishgh5C/cvJ2kR0sV5ITMmaXC5laHfn5Mse29GfgInOxa/5Ib8+cca3rdFWHuc+kfUfKfT6OVauwvTA4sHZgPlaeIxlJ0Xm6Z7dJeGTc31oGNrWER5gbrX0wP08kL9A4kmeIUYcXTEJaRd1o/pttdC4VGiOM4DdPMZnudq7NuzdvkXE9GbA8nfeuQFwxL2dIZP3U0MLZenwBUQtWkxgJ4/eij+3stH1lSw3nqd/W8dG9FvRpsmJdcIheFQrU43LyYKmmuvCzX/0IgJ8zAXqkqfw== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 12.22.5.236) smtp.rcpttodomain=dpdk.org smtp.mailfrom=nvidia.com; dmarc=pass (p=reject sp=reject pct=100) action=none header.from=nvidia.com; dkim=none (message not signed); arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=5llDo/VEnhj/LFT+BCQtiTmTHeZopi7Sdt7SvKuZetw=; b=q2fJI+LZ/1Ycy1FYTt90MlJnLypkDAdk0xmaArnD+R9iE4Gi/unmQhslb9HA0J9+O87YMkQT2WTd9ZqxZosq/PVqIm8x0s+lo+dgyczHCQZ4MrZuHCDEzwPb65id1QSnjpYnhvOzhBHicMdqdCxOwLb52WA23A9F3QX4Rpc2k5tA5wpLTDH0NTsiYLUBxj30IuYKZzxr3lUSEBddQhg4B2i2UUDEBhk45qkuDPW9ZN51BrNBUzCBqYGiDwW3vs5rl733OZ9k0lpn9CLen0LdrRFrTJDw+gx+B1zxp5C/1qsf5d02i6hclMhYbTnwoC/dI2kjCG8lqJL5v8rrYohLCA== Received: from BN9PR03CA0416.namprd03.prod.outlook.com (2603:10b6:408:111::31) by BYAPR12MB2936.namprd12.prod.outlook.com (2603:10b6:a03:12f::17) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5123.31; Fri, 8 Apr 2022 07:57:06 +0000 Received: from BN8NAM11FT043.eop-nam11.prod.protection.outlook.com (2603:10b6:408:111:cafe::ec) by BN9PR03CA0416.outlook.office365.com (2603:10b6:408:111::31) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5144.25 via Frontend Transport; Fri, 8 Apr 2022 07:57:05 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 12.22.5.236) smtp.mailfrom=nvidia.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=nvidia.com; Received-SPF: Pass (protection.outlook.com: domain of nvidia.com designates 12.22.5.236 as permitted sender) receiver=protection.outlook.com; client-ip=12.22.5.236; helo=mail.nvidia.com; Received: from mail.nvidia.com (12.22.5.236) by BN8NAM11FT043.mail.protection.outlook.com (10.13.177.218) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384) id 15.20.5144.20 via Frontend Transport; Fri, 8 Apr 2022 07:57:05 +0000 Received: from rnnvmail201.nvidia.com (10.129.68.8) by DRHQMAIL109.nvidia.com (10.27.9.19) with Microsoft SMTP Server (TLS) id 15.0.1497.32; Fri, 8 Apr 2022 07:57:04 +0000 Received: from nvidia.com (10.126.231.35) by rnnvmail201.nvidia.com (10.129.68.8) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.986.22; Fri, 8 Apr 2022 00:57:02 -0700 From: Li Zhang To: , , , CC: , , Yajun Wu Subject: [RFC 04/15] vdpa/mlx5: support event qp reuse Date: Fri, 8 Apr 2022 10:55:54 +0300 Message-ID: <20220408075606.33056-5-lizh@nvidia.com> X-Mailer: git-send-email 2.27.0 In-Reply-To: <20220408075606.33056-1-lizh@nvidia.com> References: <20220408075606.33056-1-lizh@nvidia.com> MIME-Version: 1.0 Content-Transfer-Encoding: 8bit Content-Type: text/plain X-Originating-IP: [10.126.231.35] X-ClientProxiedBy: rnnvmail203.nvidia.com (10.129.68.9) To rnnvmail201.nvidia.com (10.129.68.8) X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: dc7651d8-fe49-4af3-85c8-08da19355fe5 X-MS-TrafficTypeDiagnostic: BYAPR12MB2936:EE_ X-LD-Processed: 43083d15-7273-40c1-b7db-39efd9ccc17a,ExtAddr X-Microsoft-Antispam-PRVS: X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: Iphc43C1cRTChwErwkMsNqMI1TowcryqJQQdGK6W/9c6KIo/a6jmxqxUzWp2W0FyVaz/ZLOqjmtfH4Kx/JDc0Lke3senw0XcW2nXyKKePlgcYBtivay0WT+B5+STR6rALVT5qg6HG8zYbeLLrtkqWmkDXmrdhvKEoxFwpdmi1YHtT7AsenOwBvSCAeL1C23Ybnp/T7CfYzTPbcbmoE8MuNCmZvrV7mFs9vqzISyoIdnJi7Fo5K6uZMtidwa9eovusWgqic6r30G1z+RdsINooar1ZbnSw2LBnWrkWBMGV/4EaPu8W6KtAAylMYxh/LbMDQj2w9xXLT5Lyp0TH2sTFp/aCEa4gq/W4JQBeJCzZIqpHBNiNweaqcqanM7lOIk2feiGOf+9aa6Mg+B1ijBfwDX9FgptY3chhMsxcUoUDEuesBTu8mUioC8Qyi6tIP7K++g8olwUx8BVvtTWGV0YQ7EFqRfSLiOSsAmPbKvsUbk2VvkFyG3T1QEeEu9Qz3Dyf/TJR4AoBy0gXRTfPvGCthKOKtDQIqUhO9oAmXqJiKAfPgzkXrZn0WH+vH0Rrp8RhmPp1bwO9DZu+ptjo1NK3dNajxBSMstTvfBNaA+xNbkr0RkQzjb2oM/4ZVfNQKwpEF97yps/3on9DW/RAFcdgNdiXzG+xaqsQrxy4UnvxLtWMo430xdwrzsAKia1mK72O4e3IADvZ2Tru9B9lIm3yA== X-Forefront-Antispam-Report: CIP:12.22.5.236; CTRY:US; LANG:en; SCL:1; SRV:; IPV:CAL; SFV:NSPM; H:mail.nvidia.com; PTR:InfoNoRecords; CAT:NONE; SFS:(13230001)(4636009)(40470700004)(36840700001)(46966006)(6286002)(54906003)(107886003)(5660300002)(336012)(508600001)(2906002)(7696005)(8936002)(110136005)(6666004)(1076003)(356005)(2616005)(316002)(82310400005)(426003)(83380400001)(36860700001)(8676002)(55016003)(70206006)(4326008)(26005)(81166007)(70586007)(186003)(47076005)(86362001)(36756003)(16526019)(40460700003)(36900700001); DIR:OUT; SFP:1101; X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 08 Apr 2022 07:57:05.7466 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: dc7651d8-fe49-4af3-85c8-08da19355fe5 X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=43083d15-7273-40c1-b7db-39efd9ccc17a; Ip=[12.22.5.236]; Helo=[mail.nvidia.com] X-MS-Exchange-CrossTenant-AuthSource: BN8NAM11FT043.eop-nam11.prod.protection.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: BYAPR12MB2936 X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org From: Yajun Wu To speed up queue create time, event qp and cq will create only once. Each virtq creation will reuse same event qp and cq. Because FW will set event qp to error state during virtq destory, need modify event qp to RESET state, then modify qp to RTS state as usual. This can save about 1.5ms for each virtq creation. After SW qp reset, qp pi/ci all become 0 while cq pi/ci keep as previous. Add new variable qp_ci to save SW qp ci. Move qp pi independently with cq ci. Add new function mlx5_vdpa_drain_cq to drain cq CQE after virtq release. Signed-off-by: Yajun Wu --- drivers/vdpa/mlx5/mlx5_vdpa.c | 8 ++++ drivers/vdpa/mlx5/mlx5_vdpa.h | 12 +++++- drivers/vdpa/mlx5/mlx5_vdpa_event.c | 60 +++++++++++++++++++++++++++-- drivers/vdpa/mlx5/mlx5_vdpa_virtq.c | 6 +-- 4 files changed, 78 insertions(+), 8 deletions(-) diff --git a/drivers/vdpa/mlx5/mlx5_vdpa.c b/drivers/vdpa/mlx5/mlx5_vdpa.c index 57f9b05e35..03ad01c156 100644 --- a/drivers/vdpa/mlx5/mlx5_vdpa.c +++ b/drivers/vdpa/mlx5/mlx5_vdpa.c @@ -269,6 +269,7 @@ mlx5_vdpa_dev_close(int vid) } mlx5_vdpa_steer_unset(priv); mlx5_vdpa_virtqs_release(priv); + mlx5_vdpa_drain_cq(priv); if (priv->lm_mr.addr) mlx5_os_wrapped_mkey_destroy(&priv->lm_mr); priv->state = MLX5_VDPA_STATE_PROBED; @@ -555,7 +556,14 @@ mlx5_vdpa_virtq_resource_prepare(struct mlx5_vdpa_priv *priv) return 0; for (index = 0; index < (priv->queues * 2); ++index) { struct mlx5_vdpa_virtq *virtq = &priv->virtqs[index]; + int ret = mlx5_vdpa_event_qp_prepare(priv, priv->queue_size, + -1, &virtq->eqp); + if (ret) { + DRV_LOG(ERR, "Failed to create event QPs for virtq %d.", + index); + return -1; + } if (priv->caps.queue_counters_valid) { if (!virtq->counters) virtq->counters = diff --git a/drivers/vdpa/mlx5/mlx5_vdpa.h b/drivers/vdpa/mlx5/mlx5_vdpa.h index f6719a3c60..bf82026e37 100644 --- a/drivers/vdpa/mlx5/mlx5_vdpa.h +++ b/drivers/vdpa/mlx5/mlx5_vdpa.h @@ -55,6 +55,7 @@ struct mlx5_vdpa_event_qp { struct mlx5_vdpa_cq cq; struct mlx5_devx_obj *fw_qp; struct mlx5_devx_qp sw_qp; + uint16_t qp_pi; }; struct mlx5_vdpa_query_mr { @@ -226,7 +227,7 @@ int mlx5_vdpa_mem_register(struct mlx5_vdpa_priv *priv); * @return * 0 on success, -1 otherwise and rte_errno is set. */ -int mlx5_vdpa_event_qp_create(struct mlx5_vdpa_priv *priv, uint16_t desc_n, +int mlx5_vdpa_event_qp_prepare(struct mlx5_vdpa_priv *priv, uint16_t desc_n, int callfd, struct mlx5_vdpa_event_qp *eqp); /** @@ -479,4 +480,13 @@ mlx5_vdpa_virtq_stats_get(struct mlx5_vdpa_priv *priv, int qid, */ int mlx5_vdpa_virtq_stats_reset(struct mlx5_vdpa_priv *priv, int qid); + +/** + * Drain virtq CQ CQE. + * + * @param[in] priv + * The vdpa driver private structure. + */ +void +mlx5_vdpa_drain_cq(struct mlx5_vdpa_priv *priv); #endif /* RTE_PMD_MLX5_VDPA_H_ */ diff --git a/drivers/vdpa/mlx5/mlx5_vdpa_event.c b/drivers/vdpa/mlx5/mlx5_vdpa_event.c index 7167a98db0..b43dca9255 100644 --- a/drivers/vdpa/mlx5/mlx5_vdpa_event.c +++ b/drivers/vdpa/mlx5/mlx5_vdpa_event.c @@ -137,7 +137,7 @@ mlx5_vdpa_cq_poll(struct mlx5_vdpa_cq *cq) }; uint32_t word; } last_word; - uint16_t next_wqe_counter = cq->cq_ci; + uint16_t next_wqe_counter = eqp->qp_pi; uint16_t cur_wqe_counter; uint16_t comp; @@ -156,9 +156,10 @@ mlx5_vdpa_cq_poll(struct mlx5_vdpa_cq *cq) rte_io_wmb(); /* Ring CQ doorbell record. */ cq->cq_obj.db_rec[0] = rte_cpu_to_be_32(cq->cq_ci); + eqp->qp_pi += comp; rte_io_wmb(); /* Ring SW QP doorbell record. */ - eqp->sw_qp.db_rec[0] = rte_cpu_to_be_32(cq->cq_ci + cq_size); + eqp->sw_qp.db_rec[0] = rte_cpu_to_be_32(eqp->qp_pi + cq_size); } return comp; } @@ -232,6 +233,25 @@ mlx5_vdpa_queues_complete(struct mlx5_vdpa_priv *priv) return max; } +void +mlx5_vdpa_drain_cq(struct mlx5_vdpa_priv *priv) +{ + unsigned int i; + + for (i = 0; i < priv->caps.max_num_virtio_queues * 2; i++) { + struct mlx5_vdpa_cq *cq = &priv->virtqs[i].eqp.cq; + + mlx5_vdpa_queue_complete(cq); + if (cq->cq_obj.cq) { + cq->cq_obj.cqes[0].wqe_counter = + rte_cpu_to_be_16(UINT16_MAX); + priv->virtqs[i].eqp.qp_pi = 0; + if (!cq->armed) + mlx5_vdpa_cq_arm(priv, cq); + } + } +} + /* Wait on all CQs channel for completion event. */ static struct mlx5_vdpa_cq * mlx5_vdpa_event_wait(struct mlx5_vdpa_priv *priv __rte_unused) @@ -574,14 +594,44 @@ mlx5_vdpa_qps2rts(struct mlx5_vdpa_event_qp *eqp) return 0; } +static int +mlx5_vdpa_qps2rst2rts(struct mlx5_vdpa_event_qp *eqp) +{ + if (mlx5_devx_cmd_modify_qp_state(eqp->fw_qp, MLX5_CMD_OP_QP_2RST, + eqp->sw_qp.qp->id)) { + DRV_LOG(ERR, "Failed to modify FW QP to RST state(%u).", + rte_errno); + return -1; + } + if (mlx5_devx_cmd_modify_qp_state(eqp->sw_qp.qp, + MLX5_CMD_OP_QP_2RST, eqp->fw_qp->id)) { + DRV_LOG(ERR, "Failed to modify SW QP to RST state(%u).", + rte_errno); + return -1; + } + return mlx5_vdpa_qps2rts(eqp); +} + int -mlx5_vdpa_event_qp_create(struct mlx5_vdpa_priv *priv, uint16_t desc_n, +mlx5_vdpa_event_qp_prepare(struct mlx5_vdpa_priv *priv, uint16_t desc_n, int callfd, struct mlx5_vdpa_event_qp *eqp) { struct mlx5_devx_qp_attr attr = {0}; uint16_t log_desc_n = rte_log2_u32(desc_n); uint32_t ret; + if (eqp->cq.cq_obj.cq != NULL && log_desc_n == eqp->cq.log_desc_n) { + /* Reuse existing resources. */ + eqp->cq.callfd = callfd; + /* FW will set event qp to error state in q destroy. */ + if (!mlx5_vdpa_qps2rst2rts(eqp)) { + rte_write32(rte_cpu_to_be_32(RTE_BIT32(log_desc_n)), + &eqp->sw_qp.db_rec[0]); + return 0; + } + } + if (eqp->fw_qp) + mlx5_vdpa_event_qp_destroy(eqp); if (mlx5_vdpa_cq_create(priv, log_desc_n, callfd, &eqp->cq)) return -1; attr.pd = priv->cdev->pdn; @@ -608,8 +658,10 @@ mlx5_vdpa_event_qp_create(struct mlx5_vdpa_priv *priv, uint16_t desc_n, } if (mlx5_vdpa_qps2rts(eqp)) goto error; + eqp->qp_pi = 0; /* First ringing. */ - rte_write32(rte_cpu_to_be_32(RTE_BIT32(log_desc_n)), + if (eqp->sw_qp.db_rec) + rte_write32(rte_cpu_to_be_32(RTE_BIT32(log_desc_n)), &eqp->sw_qp.db_rec[0]); return 0; error: diff --git a/drivers/vdpa/mlx5/mlx5_vdpa_virtq.c b/drivers/vdpa/mlx5/mlx5_vdpa_virtq.c index e025be47d2..28cef69a58 100644 --- a/drivers/vdpa/mlx5/mlx5_vdpa_virtq.c +++ b/drivers/vdpa/mlx5/mlx5_vdpa_virtq.c @@ -87,6 +87,8 @@ mlx5_vdpa_virtqs_cleanup(struct mlx5_vdpa_priv *priv) } virtq->umems[j].size = 0; } + if (virtq->eqp.fw_qp) + mlx5_vdpa_event_qp_destroy(&virtq->eqp); } } @@ -117,8 +119,6 @@ mlx5_vdpa_virtq_unset(struct mlx5_vdpa_virtq *virtq) claim_zero(mlx5_devx_cmd_destroy(virtq->virtq)); } virtq->virtq = NULL; - if (virtq->eqp.fw_qp) - mlx5_vdpa_event_qp_destroy(&virtq->eqp); virtq->notifier_state = MLX5_VDPA_NOTIFIER_STATE_DISABLED; return 0; } @@ -246,7 +246,7 @@ mlx5_vdpa_virtq_setup(struct mlx5_vdpa_priv *priv, int index) MLX5_VIRTQ_EVENT_MODE_QP : MLX5_VIRTQ_EVENT_MODE_NO_MSIX; if (attr.event_mode == MLX5_VIRTQ_EVENT_MODE_QP) { - ret = mlx5_vdpa_event_qp_create(priv, vq.size, vq.callfd, + ret = mlx5_vdpa_event_qp_prepare(priv, vq.size, vq.callfd, &virtq->eqp); if (ret) { DRV_LOG(ERR, "Failed to create event QPs for virtq %d.", -- 2.27.0