From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 6539FA0543; Mon, 6 Jun 2022 13:23:03 +0200 (CEST) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 9838242B9B; Mon, 6 Jun 2022 13:22:14 +0200 (CEST) Received: from NAM12-MW2-obe.outbound.protection.outlook.com (mail-mw2nam12on2045.outbound.protection.outlook.com [40.107.244.45]) by mails.dpdk.org (Postfix) with ESMTP id 138BC42B99 for ; Mon, 6 Jun 2022 13:22:13 +0200 (CEST) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=CCzxuv7GcNaIw/7OF1/L5OjUOxpGYeW4kpBqeTg7DW3d5rmQZBOkLXdu4g6XDuN+xYx4Anzrg2fNYhBQKIdstEn2KFhr56a8F38R1G80MNuiUnjc8ubLZW4eUmbSmcAFAKDEz8Z/50Nsct8XAceTDr4JlfMtadxEIo9hskLI689mJbnb3WC1nvjyHhwtV5p4LCiOESKB6Ml449YPbfDMQoQyRi9gWANGlkVNY3joQzghMvww3wIg5f5an3qfASHWcdFOU4gMeMxbyih3ycOwQuO/JaR7Aguj+pkrahosh2ytVdn3WvZInGsMmiXmP+35M9kk5HCmytqrf23DrB1ofg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=2zp3UNiQdPV3kY553RSNWiyXlxSBg95xFLWHTx0l1LI=; b=BuAW6ycYCYSakQNfSRyHDWyZZrEnCuZj9DQQgr3fIvBkRgGVD8jyeqOJP7YHhxdq4wy6IGMjJJA9LjOD6z8v5yPAEwrmCXiw65e8Nca526VuiT5u9WckRn8B7atWCgKh7HT/wOWI0igjJjajz0271c0K2KhKHDQRU5URifQdxl6V3zsn6gehDHJg/gVg6RU4GlOT7vgRXGua3qYOXhWvoyaNxXsjGySNOQSBIv9UMR9LAfMydt7uIvBgwZyIESfjCuN13e3aCJB2uu+74yi3b306MW+yqF47sKzL+nOPnPqjFRQntGBfYKWp3nXD3uLNjMEK/tWW5pFBtH5p9avKVQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 12.22.5.236) smtp.rcpttodomain=monjalon.net smtp.mailfrom=nvidia.com; dmarc=pass (p=reject sp=reject pct=100) action=none header.from=nvidia.com; dkim=none (message not signed); arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=2zp3UNiQdPV3kY553RSNWiyXlxSBg95xFLWHTx0l1LI=; b=byoa3eZGv07qO53eftgwJrCTyKOFPpUf5WS1PogTvWWxgKqPmiiVDfBgHhv03w4/6KmhjmCbDiRsG+XVnero1vQ+UfpkxdqizCxgeRyxZYmxkKXSvPuMiy/2skrwcgYzUqGoJIOWm4+tvRNeKKd5zxdAwi90E/CK5COuOJrzgAZq+NKRBwPjZBtKeaFQ4CuPazF0A4RXaRfPq/BL1U8U/PKO0gYOrEk8Rny9xIzo9+KUT2dwqd+7vsMTMTFqMNLoOKHszUAu2reIAXR1MHZWvUB0QXAbXy4Gyue0EEf56ZV2sqfEUpoconPoJlCMNIXg141HdGFdD7OXyUg69dkTdQ== Received: from BN6PR16CA0034.namprd16.prod.outlook.com (2603:10b6:405:14::20) by MWHPR1201MB0272.namprd12.prod.outlook.com (2603:10b6:301:52::13) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5314.15; Mon, 6 Jun 2022 11:22:11 +0000 Received: from BN8NAM11FT061.eop-nam11.prod.protection.outlook.com (2603:10b6:405:14:cafe::2f) by BN6PR16CA0034.outlook.office365.com (2603:10b6:405:14::20) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5314.19 via Frontend Transport; Mon, 6 Jun 2022 11:22:10 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 12.22.5.236) smtp.mailfrom=nvidia.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=nvidia.com; Received-SPF: Pass (protection.outlook.com: domain of nvidia.com designates 12.22.5.236 as permitted sender) receiver=protection.outlook.com; client-ip=12.22.5.236; helo=mail.nvidia.com; pr=C Received: from mail.nvidia.com (12.22.5.236) by BN8NAM11FT061.mail.protection.outlook.com (10.13.177.144) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384) id 15.20.5314.12 via Frontend Transport; Mon, 6 Jun 2022 11:22:10 +0000 Received: from rnnvmail201.nvidia.com (10.129.68.8) by DRHQMAIL109.nvidia.com (10.27.9.19) with Microsoft SMTP Server (TLS) id 15.0.1497.32; Mon, 6 Jun 2022 11:22:09 +0000 Received: from nvidia.com (10.126.231.35) by rnnvmail201.nvidia.com (10.129.68.8) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.986.22; Mon, 6 Jun 2022 04:22:06 -0700 From: Li Zhang To: , , , CC: , , , , Yajun Wu Subject: [PATCH v1 06/17] vdpa/mlx5: support event qp reuse Date: Mon, 6 Jun 2022 14:20:48 +0300 Message-ID: <20220606112109.208873-12-lizh@nvidia.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20220606112109.208873-1-lizh@nvidia.com> References: <20220408075606.33056-1-lizh@nvidia.com> <20220606112109.208873-1-lizh@nvidia.com> MIME-Version: 1.0 Content-Transfer-Encoding: 8bit Content-Type: text/plain X-Originating-IP: [10.126.231.35] X-ClientProxiedBy: rnnvmail202.nvidia.com (10.129.68.7) To rnnvmail201.nvidia.com (10.129.68.8) X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: 2307824a-a604-421d-b255-08da47aecc61 X-MS-TrafficTypeDiagnostic: MWHPR1201MB0272:EE_ X-LD-Processed: 43083d15-7273-40c1-b7db-39efd9ccc17a,ExtAddr X-Microsoft-Antispam-PRVS: X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: 4xBASMewzYlhAzWvdmFaHMgGX+DLBGBAqTkq0O5KdRsPwQ6OFILiskATEOvay9LjnI5orN/h9jB5kvQkgYkWKxx/sbPgk9vdJwwEl8G8kUe0T8SIdoeSMQwABX5uN1QXCozXPnsVlQsToGhsPSloSnZmEyhVzlWAKe29vxtv+ZliBsiLtRAD9YqIUND+N+Pilb5UIXKsOPW50qpJmb0qz3RLO8G2PUdtU3RcNZZarW9Aq2Y+y4Nl9+uS8/qyKxttO6tNUrCFzBbq0MBmc1a/mNXNO3dP/7o9lSnxBDclPSNbRgCIJ0yn1iXK/VzwdIporUqabDg3cdupR+HSPyxeueUe0SSFPnu/2DvAPT3jt65Z7XqCYS9lX+ydLi4Oa/Bp5Jr7tKp7lWN3tprCVwbUoB+bdk1qNap8iaNhu6G+W8h2Y2Afy4T9FflAI9N3KjjPiMicZzaS6FBCko/Ys1bRk/bwxq8fxHjClbLVHfHo1pwHiEGSN1ZwyUS/11i+jDGgdyrnnTqnpfnj5NXKCmsUB2BuP9XjWSnvcA387QEFSyWVoEMTrEE5vSPp7DPwjSQvaUbWWNygEoGWdc2hv7MOfbiU2ob3YgZzZVXNF0888qo/ruaqRndXoJ162QCLuMCev/n5krd+rZn5ADK+JC7LN5MhRnK3clJ0u7tP89L2A4Y8SHV1f09++sdqqiQfI3HCj8G+C1+cC2j9Yp3lsesGSA== X-Forefront-Antispam-Report: CIP:12.22.5.236; CTRY:US; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:mail.nvidia.com; PTR:InfoNoRecords; CAT:NONE; SFS:(13230001)(4636009)(46966006)(40470700004)(36840700001)(110136005)(36860700001)(54906003)(316002)(6286002)(82310400005)(6666004)(8676002)(4326008)(7696005)(70586007)(2906002)(70206006)(6636002)(86362001)(8936002)(81166007)(36756003)(107886003)(40460700003)(83380400001)(26005)(55016003)(5660300002)(2616005)(1076003)(336012)(186003)(426003)(508600001)(356005)(47076005)(16526019)(36900700001); DIR:OUT; SFP:1101; X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 06 Jun 2022 11:22:10.3464 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 2307824a-a604-421d-b255-08da47aecc61 X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=43083d15-7273-40c1-b7db-39efd9ccc17a; Ip=[12.22.5.236]; Helo=[mail.nvidia.com] X-MS-Exchange-CrossTenant-AuthSource: BN8NAM11FT061.eop-nam11.prod.protection.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: MWHPR1201MB0272 X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org From: Yajun Wu To speed up queue create time, event qp and cq will create only once. Each virtq creation will reuse same event qp and cq. Because FW will set event qp to error state during virtq destroy, need modify event qp to RESET state, then modify qp to RTS state as usual. This can save about 1.5ms for each virtq creation. After SW qp reset, qp pi/ci all become 0 while cq pi/ci keep as previous. Add new variable qp_ci to save SW qp ci. Move qp pi independently with cq ci. Add new function mlx5_vdpa_drain_cq to drain cq CQE after virtq release. Signed-off-by: Yajun Wu --- drivers/vdpa/mlx5/mlx5_vdpa.c | 8 ++++ drivers/vdpa/mlx5/mlx5_vdpa.h | 12 +++++- drivers/vdpa/mlx5/mlx5_vdpa_event.c | 60 +++++++++++++++++++++++++++-- drivers/vdpa/mlx5/mlx5_vdpa_virtq.c | 6 +-- 4 files changed, 78 insertions(+), 8 deletions(-) diff --git a/drivers/vdpa/mlx5/mlx5_vdpa.c b/drivers/vdpa/mlx5/mlx5_vdpa.c index faf833ee2f..ee99952e11 100644 --- a/drivers/vdpa/mlx5/mlx5_vdpa.c +++ b/drivers/vdpa/mlx5/mlx5_vdpa.c @@ -269,6 +269,7 @@ mlx5_vdpa_dev_close(int vid) } mlx5_vdpa_steer_unset(priv); mlx5_vdpa_virtqs_release(priv); + mlx5_vdpa_drain_cq(priv); if (priv->lm_mr.addr) mlx5_os_wrapped_mkey_destroy(&priv->lm_mr); priv->state = MLX5_VDPA_STATE_PROBED; @@ -555,7 +556,14 @@ mlx5_vdpa_virtq_resource_prepare(struct mlx5_vdpa_priv *priv) return 0; for (index = 0; index < (priv->queues * 2); ++index) { struct mlx5_vdpa_virtq *virtq = &priv->virtqs[index]; + int ret = mlx5_vdpa_event_qp_prepare(priv, priv->queue_size, + -1, &virtq->eqp); + if (ret) { + DRV_LOG(ERR, "Failed to create event QPs for virtq %d.", + index); + return -1; + } if (priv->caps.queue_counters_valid) { if (!virtq->counters) virtq->counters = diff --git a/drivers/vdpa/mlx5/mlx5_vdpa.h b/drivers/vdpa/mlx5/mlx5_vdpa.h index f6719a3c60..bf82026e37 100644 --- a/drivers/vdpa/mlx5/mlx5_vdpa.h +++ b/drivers/vdpa/mlx5/mlx5_vdpa.h @@ -55,6 +55,7 @@ struct mlx5_vdpa_event_qp { struct mlx5_vdpa_cq cq; struct mlx5_devx_obj *fw_qp; struct mlx5_devx_qp sw_qp; + uint16_t qp_pi; }; struct mlx5_vdpa_query_mr { @@ -226,7 +227,7 @@ int mlx5_vdpa_mem_register(struct mlx5_vdpa_priv *priv); * @return * 0 on success, -1 otherwise and rte_errno is set. */ -int mlx5_vdpa_event_qp_create(struct mlx5_vdpa_priv *priv, uint16_t desc_n, +int mlx5_vdpa_event_qp_prepare(struct mlx5_vdpa_priv *priv, uint16_t desc_n, int callfd, struct mlx5_vdpa_event_qp *eqp); /** @@ -479,4 +480,13 @@ mlx5_vdpa_virtq_stats_get(struct mlx5_vdpa_priv *priv, int qid, */ int mlx5_vdpa_virtq_stats_reset(struct mlx5_vdpa_priv *priv, int qid); + +/** + * Drain virtq CQ CQE. + * + * @param[in] priv + * The vdpa driver private structure. + */ +void +mlx5_vdpa_drain_cq(struct mlx5_vdpa_priv *priv); #endif /* RTE_PMD_MLX5_VDPA_H_ */ diff --git a/drivers/vdpa/mlx5/mlx5_vdpa_event.c b/drivers/vdpa/mlx5/mlx5_vdpa_event.c index 7167a98db0..b43dca9255 100644 --- a/drivers/vdpa/mlx5/mlx5_vdpa_event.c +++ b/drivers/vdpa/mlx5/mlx5_vdpa_event.c @@ -137,7 +137,7 @@ mlx5_vdpa_cq_poll(struct mlx5_vdpa_cq *cq) }; uint32_t word; } last_word; - uint16_t next_wqe_counter = cq->cq_ci; + uint16_t next_wqe_counter = eqp->qp_pi; uint16_t cur_wqe_counter; uint16_t comp; @@ -156,9 +156,10 @@ mlx5_vdpa_cq_poll(struct mlx5_vdpa_cq *cq) rte_io_wmb(); /* Ring CQ doorbell record. */ cq->cq_obj.db_rec[0] = rte_cpu_to_be_32(cq->cq_ci); + eqp->qp_pi += comp; rte_io_wmb(); /* Ring SW QP doorbell record. */ - eqp->sw_qp.db_rec[0] = rte_cpu_to_be_32(cq->cq_ci + cq_size); + eqp->sw_qp.db_rec[0] = rte_cpu_to_be_32(eqp->qp_pi + cq_size); } return comp; } @@ -232,6 +233,25 @@ mlx5_vdpa_queues_complete(struct mlx5_vdpa_priv *priv) return max; } +void +mlx5_vdpa_drain_cq(struct mlx5_vdpa_priv *priv) +{ + unsigned int i; + + for (i = 0; i < priv->caps.max_num_virtio_queues * 2; i++) { + struct mlx5_vdpa_cq *cq = &priv->virtqs[i].eqp.cq; + + mlx5_vdpa_queue_complete(cq); + if (cq->cq_obj.cq) { + cq->cq_obj.cqes[0].wqe_counter = + rte_cpu_to_be_16(UINT16_MAX); + priv->virtqs[i].eqp.qp_pi = 0; + if (!cq->armed) + mlx5_vdpa_cq_arm(priv, cq); + } + } +} + /* Wait on all CQs channel for completion event. */ static struct mlx5_vdpa_cq * mlx5_vdpa_event_wait(struct mlx5_vdpa_priv *priv __rte_unused) @@ -574,14 +594,44 @@ mlx5_vdpa_qps2rts(struct mlx5_vdpa_event_qp *eqp) return 0; } +static int +mlx5_vdpa_qps2rst2rts(struct mlx5_vdpa_event_qp *eqp) +{ + if (mlx5_devx_cmd_modify_qp_state(eqp->fw_qp, MLX5_CMD_OP_QP_2RST, + eqp->sw_qp.qp->id)) { + DRV_LOG(ERR, "Failed to modify FW QP to RST state(%u).", + rte_errno); + return -1; + } + if (mlx5_devx_cmd_modify_qp_state(eqp->sw_qp.qp, + MLX5_CMD_OP_QP_2RST, eqp->fw_qp->id)) { + DRV_LOG(ERR, "Failed to modify SW QP to RST state(%u).", + rte_errno); + return -1; + } + return mlx5_vdpa_qps2rts(eqp); +} + int -mlx5_vdpa_event_qp_create(struct mlx5_vdpa_priv *priv, uint16_t desc_n, +mlx5_vdpa_event_qp_prepare(struct mlx5_vdpa_priv *priv, uint16_t desc_n, int callfd, struct mlx5_vdpa_event_qp *eqp) { struct mlx5_devx_qp_attr attr = {0}; uint16_t log_desc_n = rte_log2_u32(desc_n); uint32_t ret; + if (eqp->cq.cq_obj.cq != NULL && log_desc_n == eqp->cq.log_desc_n) { + /* Reuse existing resources. */ + eqp->cq.callfd = callfd; + /* FW will set event qp to error state in q destroy. */ + if (!mlx5_vdpa_qps2rst2rts(eqp)) { + rte_write32(rte_cpu_to_be_32(RTE_BIT32(log_desc_n)), + &eqp->sw_qp.db_rec[0]); + return 0; + } + } + if (eqp->fw_qp) + mlx5_vdpa_event_qp_destroy(eqp); if (mlx5_vdpa_cq_create(priv, log_desc_n, callfd, &eqp->cq)) return -1; attr.pd = priv->cdev->pdn; @@ -608,8 +658,10 @@ mlx5_vdpa_event_qp_create(struct mlx5_vdpa_priv *priv, uint16_t desc_n, } if (mlx5_vdpa_qps2rts(eqp)) goto error; + eqp->qp_pi = 0; /* First ringing. */ - rte_write32(rte_cpu_to_be_32(RTE_BIT32(log_desc_n)), + if (eqp->sw_qp.db_rec) + rte_write32(rte_cpu_to_be_32(RTE_BIT32(log_desc_n)), &eqp->sw_qp.db_rec[0]); return 0; error: diff --git a/drivers/vdpa/mlx5/mlx5_vdpa_virtq.c b/drivers/vdpa/mlx5/mlx5_vdpa_virtq.c index c258eb3024..6637ba1503 100644 --- a/drivers/vdpa/mlx5/mlx5_vdpa_virtq.c +++ b/drivers/vdpa/mlx5/mlx5_vdpa_virtq.c @@ -87,6 +87,8 @@ mlx5_vdpa_virtqs_cleanup(struct mlx5_vdpa_priv *priv) } virtq->umems[j].size = 0; } + if (virtq->eqp.fw_qp) + mlx5_vdpa_event_qp_destroy(&virtq->eqp); } } @@ -117,8 +119,6 @@ mlx5_vdpa_virtq_unset(struct mlx5_vdpa_virtq *virtq) claim_zero(mlx5_devx_cmd_destroy(virtq->virtq)); } virtq->virtq = NULL; - if (virtq->eqp.fw_qp) - mlx5_vdpa_event_qp_destroy(&virtq->eqp); virtq->notifier_state = MLX5_VDPA_NOTIFIER_STATE_DISABLED; return 0; } @@ -246,7 +246,7 @@ mlx5_vdpa_virtq_setup(struct mlx5_vdpa_priv *priv, int index) MLX5_VIRTQ_EVENT_MODE_QP : MLX5_VIRTQ_EVENT_MODE_NO_MSIX; if (attr.event_mode == MLX5_VIRTQ_EVENT_MODE_QP) { - ret = mlx5_vdpa_event_qp_create(priv, vq.size, vq.callfd, + ret = mlx5_vdpa_event_qp_prepare(priv, vq.size, vq.callfd, &virtq->eqp); if (ret) { DRV_LOG(ERR, "Failed to create event QPs for virtq %d.", -- 2.31.1