From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id D1260A0C41; Thu, 15 Jul 2021 18:42:39 +0200 (CEST) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 18ADF412DF; Thu, 15 Jul 2021 18:42:16 +0200 (CEST) Received: from NAM12-DM6-obe.outbound.protection.outlook.com (mail-dm6nam12on2077.outbound.protection.outlook.com [40.107.243.77]) by mails.dpdk.org (Postfix) with ESMTP id BECE1412BD for ; Thu, 15 Jul 2021 18:42:11 +0200 (CEST) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=I1H/Iw6vEW5i+UGjhS/g601POnd4j0Fm2+J7cKn58gwcIhY25BZysDaS/rG06ABdIZX34OjMg1KlJCCHOm/x9F4yimySRGXgdd/5cRqesFTHzZvGr0bWR9o4C1AgytClxSOYRNTDz8OChEQ6qpzz+N+AWsf8CceAeRW7e7aIVcvesIwnRfYJybtl9qnaNcoTHQqmvUhcEBxGnNXU6IYr3v7FC0fCt1dzFpk/IvgDROqIcrN7fPuhQbpss70zEe9fir1eDVm32iGgH92YiItWtICZcQ39CHumaPZ1q1Jfuraccn/wDOCgRKjfOIrH2v5qa5iBTO7S9p37Br7hwsX3uA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=93LjWRSjUeNmDXYn/ad/Rnaw5v8pzsLWQWfAJwC89a8=; b=BUoXXRbEb6gvgsrV+WruXwc6XhZOFSC2DISS8DSjK3UWCxM+T828EHFNcej57NtQxvtcoHFHGbe1yJjiIcICmJJTwccWxjompzABe+8UasodSxBr7mwBeqYz960XhR7x35Iy2/MpU7aASSbz4rTxZy2vHuM/D1MtApDQT2YUrxpfedW//+mXqLyuHdBNV19RTZpFzf/2iHxlybF6gCwM8c9voxZ4Yn0galeiE/OZfyeDVaTUVQlkGCuJoQhd4ShApy03QiYN2w86DGlILReSIhI+tY6i50uQBRYHXC2dZIlI5pIaeCamJ6Aj5zPHT5rMUsbZVvp0N56I9lIq8eau7w== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 216.228.112.34) smtp.rcpttodomain=redhat.com smtp.mailfrom=nvidia.com; dmarc=pass (p=quarantine sp=none pct=100) action=none header.from=nvidia.com; dkim=none (message not signed); arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=93LjWRSjUeNmDXYn/ad/Rnaw5v8pzsLWQWfAJwC89a8=; b=Teq3d7qvrjJEtpAYw/WcjfnOvIBLE6eIOmFvmuvJinY8Qf0OjN31ens9qcgEZmc+5NZf030uzCwZxv3XDbJwBWQSR0+qETaon8avhA6TPuaQNWHphzn77vWttHyYga9mT6P6p4y7kfnOYK0Mh1C1OdGriYC27SPoglCwdSLuAa9yP0zduhi3EpoGvlkr/KN56BVyKaTcLR279W2k2n8eTTjUe9NCTeE55ly2Tc+Ak0Np2Dw+cJzV2MUwzrHH/2838P6i6ZNzYzq4zgbsJwzygja8dUg4TTNp8Z83gtdcSpo+ErJgjjfpVWkLLb2EnSYsYkClH5tmqivM21k/3hb/pA== Received: from MW4PR04CA0023.namprd04.prod.outlook.com (2603:10b6:303:69::28) by DM5PR1201MB0169.namprd12.prod.outlook.com (2603:10b6:4:55::22) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4331.21; Thu, 15 Jul 2021 16:42:10 +0000 Received: from CO1NAM11FT066.eop-nam11.prod.protection.outlook.com (2603:10b6:303:69:cafe::f5) by MW4PR04CA0023.outlook.office365.com (2603:10b6:303:69::28) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4331.21 via Frontend Transport; Thu, 15 Jul 2021 16:42:09 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 216.228.112.34) smtp.mailfrom=nvidia.com; redhat.com; dkim=none (message not signed) header.d=none;redhat.com; dmarc=pass action=none header.from=nvidia.com; Received-SPF: Pass (protection.outlook.com: domain of nvidia.com designates 216.228.112.34 as permitted sender) receiver=protection.outlook.com; client-ip=216.228.112.34; helo=mail.nvidia.com; Received: from mail.nvidia.com (216.228.112.34) by CO1NAM11FT066.mail.protection.outlook.com (10.13.175.18) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384) id 15.20.4331.21 via Frontend Transport; Thu, 15 Jul 2021 16:42:09 +0000 Received: from nvidia.com (172.20.187.6) by HQMAIL107.nvidia.com (172.20.187.13) with Microsoft SMTP Server (TLS) id 15.0.1497.2; Thu, 15 Jul 2021 16:42:07 +0000 From: Shiri Kuzin To: CC: , , , Date: Thu, 15 Jul 2021 19:41:15 +0300 Message-ID: <20210715164126.54073-6-shirik@nvidia.com> X-Mailer: git-send-email 2.27.0 In-Reply-To: <20210715164126.54073-1-shirik@nvidia.com> References: <20210715150817.51485-1-shirik@nvidia.com> <20210715164126.54073-1-shirik@nvidia.com> MIME-Version: 1.0 Content-Transfer-Encoding: 8bit Content-Type: text/plain X-Originating-IP: [172.20.187.6] X-ClientProxiedBy: HQMAIL105.nvidia.com (172.20.187.12) To HQMAIL107.nvidia.com (172.20.187.13) X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: aac4000f-14d7-42a5-6a59-08d947af7d5f X-MS-TrafficTypeDiagnostic: DM5PR1201MB0169: X-Microsoft-Antispam-PRVS: X-MS-Oob-TLC-OOBClassifiers: OLM:2512; X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: D1/49Xn4EHrPfURjhSv6eIRW2zct/gjuN+LeYZOjNOfabA4JcBuwlXEFXGkJcsY2jCjXlbAWA44L6zK04GX5eNN0lLzYnuLMxMUatHhDqHbPkXr//hrs3OMC+hjAVCDjD+mL8lGGmzF50Ic+mkp/HP6b0TKFe1i9ruv5IdT91flwDIl7AhhVrZQZXcyv1hmm2kRauqtIZG0ztVPs6mc+BVBqrfiE3AVfMLjLEU2POY3kVE4vpAMT82OZXV8AOCbmxFEn6g/p+Spf50RzLPZhprV7jDFkrQQE1NH5fZCGJPFhJ5GXB/79vD6oVvDibOUQFa5o8A8I4s43d/SE9XaC/p1Ia4DX+Zgj9v51JTkhcVnKqizCbO9kFLZBgnHmuFLpPH0oj1eU/KiOUT5TEHMyaR/U1yL0o11nP311AXSV8qaypG5/K2276vsETSUnLCuIQQv7MuXsH/obxEbGlyLldWFU77PjcHRVky2tH5ksiNtQZGw2KTNl11b5sIytnvKV/8c2AQsi49GkjFy0xYANenetBbtYIxQQTJ27NQtqEXtT3fj7D8bFgdyTXf7MDBkxApYxgyjGmNA9W2oBt7OkJPnsTZfLzHbCqbcVjzuCW+Xm45w6hqAcZM2oyHxIXgTEYdeDHHW2kESATfBL95qdYzuZt7mHxm18iIXdelz8rQ9K1Ew0tnFYQNqQvtt0A3vOajj6mJuV7eJSM2sd3qgNkH8/PWIGkPu37U9XQg87Ri8= X-Forefront-Antispam-Report: CIP:216.228.112.34; CTRY:US; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:mail.nvidia.com; PTR:schybrid03.nvidia.com; CAT:NONE; SFS:(4636009)(39860400002)(346002)(136003)(396003)(376002)(36840700001)(46966006)(82310400003)(55016002)(8676002)(36860700001)(70586007)(8936002)(47076005)(316002)(4326008)(5660300002)(426003)(82740400003)(336012)(36906005)(86362001)(356005)(34020700004)(2616005)(54906003)(6916009)(186003)(2906002)(7636003)(83380400001)(478600001)(16526019)(36756003)(6286002)(26005)(6666004)(7696005)(70206006)(1076003); DIR:OUT; SFP:1101; X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 15 Jul 2021 16:42:09.6940 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: aac4000f-14d7-42a5-6a59-08d947af7d5f X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=43083d15-7273-40c1-b7db-39efd9ccc17a; Ip=[216.228.112.34]; Helo=[mail.nvidia.com] X-MS-Exchange-CrossTenant-AuthSource: CO1NAM11FT066.eop-nam11.prod.protection.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: DM5PR1201MB0169 Subject: [dpdk-dev] [PATCH v8 05/16] crypto/mlx5: add queue pairs operations X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" The HW queue pairs are a pair of send queue and receive queue of independent work queues packed together in one object for the purpose of transferring data between nodes of a network. Completion Queue is a FIFO queue of completed work requests. In crypto driver we use one QP in loopback in order to encrypt and decrypt data locally without sending it to the wire. In the configured QP we only use the SQ to perform the encryption and decryption operations. Added implementation for the QP setup function which creates the CQ, creates the QP and changes its state to RTS (ready to send). Added implementation for the release QP function to release all the QP resources. Added the ops structure that contains any operation which is supported by the cryptodev. Signed-off-by: Shiri Kuzin Acked-by: Matan Azrad --- drivers/crypto/mlx5/mlx5_crypto.c | 129 +++++++++++++++++++++++++++++- drivers/crypto/mlx5/mlx5_crypto.h | 11 +++ 2 files changed, 137 insertions(+), 3 deletions(-) diff --git a/drivers/crypto/mlx5/mlx5_crypto.c b/drivers/crypto/mlx5/mlx5_crypto.c index a7e44deb9e..ebaa65c7a9 100644 --- a/drivers/crypto/mlx5/mlx5_crypto.c +++ b/drivers/crypto/mlx5/mlx5_crypto.c @@ -7,6 +7,7 @@ #include #include #include +#include #include #include @@ -19,6 +20,7 @@ #define MLX5_CRYPTO_DRIVER_NAME mlx5_crypto #define MLX5_CRYPTO_LOG_NAME pmd.crypto.mlx5 +#define MLX5_CRYPTO_MAX_QPS 1024 #define MLX5_CRYPTO_FEATURE_FLAGS \ (RTE_CRYPTODEV_FF_SYMMETRIC_CRYPTO | RTE_CRYPTODEV_FF_HW_ACCELERATED | \ @@ -94,7 +96,7 @@ mlx5_crypto_dev_infos_get(struct rte_cryptodev *dev, dev_info->driver_id = mlx5_crypto_driver_id; dev_info->feature_flags = MLX5_CRYPTO_FEATURE_FLAGS; dev_info->capabilities = mlx5_crypto_caps; - dev_info->max_nb_queue_pairs = 0; + dev_info->max_nb_queue_pairs = MLX5_CRYPTO_MAX_QPS; dev_info->min_mbuf_headroom_req = 0; dev_info->min_mbuf_tailroom_req = 0; dev_info->sym.max_nb_sessions = 0; @@ -235,6 +237,127 @@ mlx5_crypto_sym_session_clear(struct rte_cryptodev *dev, DRV_LOG(DEBUG, "Session %p was cleared.", spriv); } +static int +mlx5_crypto_queue_pair_release(struct rte_cryptodev *dev, uint16_t qp_id) +{ + struct mlx5_crypto_qp *qp = dev->data->queue_pairs[qp_id]; + + if (qp->qp_obj != NULL) + claim_zero(mlx5_devx_cmd_destroy(qp->qp_obj)); + if (qp->umem_obj != NULL) + claim_zero(mlx5_glue->devx_umem_dereg(qp->umem_obj)); + if (qp->umem_buf != NULL) + rte_free(qp->umem_buf); + mlx5_devx_cq_destroy(&qp->cq_obj); + rte_free(qp); + dev->data->queue_pairs[qp_id] = NULL; + return 0; +} + +static int +mlx5_crypto_qp2rts(struct mlx5_crypto_qp *qp) +{ + /* + * In Order to configure self loopback, when calling these functions the + * remote QP id that is used is the id of the same QP. + */ + if (mlx5_devx_cmd_modify_qp_state(qp->qp_obj, MLX5_CMD_OP_RST2INIT_QP, + qp->qp_obj->id)) { + DRV_LOG(ERR, "Failed to modify QP to INIT state(%u).", + rte_errno); + return -1; + } + if (mlx5_devx_cmd_modify_qp_state(qp->qp_obj, MLX5_CMD_OP_INIT2RTR_QP, + qp->qp_obj->id)) { + DRV_LOG(ERR, "Failed to modify QP to RTR state(%u).", + rte_errno); + return -1; + } + if (mlx5_devx_cmd_modify_qp_state(qp->qp_obj, MLX5_CMD_OP_RTR2RTS_QP, + qp->qp_obj->id)) { + DRV_LOG(ERR, "Failed to modify QP to RTS state(%u).", + rte_errno); + return -1; + } + return 0; +} + +static int +mlx5_crypto_queue_pair_setup(struct rte_cryptodev *dev, uint16_t qp_id, + const struct rte_cryptodev_qp_conf *qp_conf, + int socket_id) +{ + struct mlx5_crypto_priv *priv = dev->data->dev_private; + struct mlx5_devx_qp_attr attr = {0}; + struct mlx5_crypto_qp *qp; + uint16_t log_nb_desc = rte_log2_u32(qp_conf->nb_descriptors); + uint32_t umem_size = RTE_BIT32(log_nb_desc) * + MLX5_CRYPTO_WQE_SET_SIZE + + sizeof(*qp->db_rec) * 2; + uint32_t alloc_size = sizeof(*qp); + struct mlx5_devx_cq_attr cq_attr = { + .uar_page_id = mlx5_os_get_devx_uar_page_id(priv->uar), + }; + + if (dev->data->queue_pairs[qp_id] != NULL) + mlx5_crypto_queue_pair_release(dev, qp_id); + alloc_size = RTE_ALIGN(alloc_size, RTE_CACHE_LINE_SIZE); + alloc_size += sizeof(struct rte_crypto_op *) * RTE_BIT32(log_nb_desc); + qp = rte_zmalloc_socket(__func__, alloc_size, RTE_CACHE_LINE_SIZE, + socket_id); + if (qp == NULL) { + DRV_LOG(ERR, "Failed to allocate QP memory."); + rte_errno = ENOMEM; + return -rte_errno; + } + if (mlx5_devx_cq_create(priv->ctx, &qp->cq_obj, log_nb_desc, + &cq_attr, socket_id) != 0) { + DRV_LOG(ERR, "Failed to create CQ."); + goto error; + } + qp->umem_buf = rte_zmalloc_socket(__func__, umem_size, 4096, socket_id); + if (qp->umem_buf == NULL) { + DRV_LOG(ERR, "Failed to allocate QP umem."); + rte_errno = ENOMEM; + goto error; + } + qp->umem_obj = mlx5_glue->devx_umem_reg(priv->ctx, + (void *)(uintptr_t)qp->umem_buf, + umem_size, + IBV_ACCESS_LOCAL_WRITE); + if (qp->umem_obj == NULL) { + DRV_LOG(ERR, "Failed to register QP umem."); + goto error; + } + attr.pd = priv->pdn; + attr.uar_index = mlx5_os_get_devx_uar_page_id(priv->uar); + attr.cqn = qp->cq_obj.cq->id; + attr.log_page_size = rte_log2_u32(sysconf(_SC_PAGESIZE)); + attr.rq_size = 0; + attr.sq_size = RTE_BIT32(log_nb_desc); + attr.dbr_umem_valid = 1; + attr.wq_umem_id = qp->umem_obj->umem_id; + attr.wq_umem_offset = 0; + attr.dbr_umem_id = qp->umem_obj->umem_id; + attr.dbr_address = RTE_BIT64(log_nb_desc) * + MLX5_CRYPTO_WQE_SET_SIZE; + qp->qp_obj = mlx5_devx_cmd_create_qp(priv->ctx, &attr); + if (qp->qp_obj == NULL) { + DRV_LOG(ERR, "Failed to create QP(%u).", rte_errno); + goto error; + } + qp->db_rec = RTE_PTR_ADD(qp->umem_buf, (uintptr_t)attr.dbr_address); + if (mlx5_crypto_qp2rts(qp)) + goto error; + qp->ops = (struct rte_crypto_op **)RTE_ALIGN((uintptr_t)(qp + 1), + RTE_CACHE_LINE_SIZE); + dev->data->queue_pairs[qp_id] = qp; + return 0; +error: + mlx5_crypto_queue_pair_release(dev, qp_id); + return -1; +} + static struct rte_cryptodev_ops mlx5_crypto_ops = { .dev_configure = mlx5_crypto_dev_configure, .dev_start = NULL, @@ -243,8 +366,8 @@ static struct rte_cryptodev_ops mlx5_crypto_ops = { .dev_infos_get = mlx5_crypto_dev_infos_get, .stats_get = NULL, .stats_reset = NULL, - .queue_pair_setup = NULL, - .queue_pair_release = NULL, + .queue_pair_setup = mlx5_crypto_queue_pair_setup, + .queue_pair_release = mlx5_crypto_queue_pair_release, .sym_session_get_size = mlx5_crypto_sym_session_get_size, .sym_session_configure = mlx5_crypto_sym_session_configure, .sym_session_clear = mlx5_crypto_sym_session_clear, diff --git a/drivers/crypto/mlx5/mlx5_crypto.h b/drivers/crypto/mlx5/mlx5_crypto.h index a0df775407..1c3c4855ec 100644 --- a/drivers/crypto/mlx5/mlx5_crypto.h +++ b/drivers/crypto/mlx5/mlx5_crypto.h @@ -11,9 +11,11 @@ #include #include +#include #define MLX5_CRYPTO_DEK_HTABLE_SZ (1 << 11) #define MLX5_CRYPTO_KEY_LENGTH 80 +#define MLX5_CRYPTO_WQE_SET_SIZE 1024 struct mlx5_crypto_priv { TAILQ_ENTRY(mlx5_crypto_priv) next; @@ -27,6 +29,15 @@ struct mlx5_crypto_priv { struct rte_cryptodev_config dev_config; }; +struct mlx5_crypto_qp { + struct mlx5_devx_cq cq_obj; + struct mlx5_devx_obj *qp_obj; + struct mlx5dv_devx_umem *umem_obj; + void *umem_buf; + volatile uint32_t *db_rec; + struct rte_crypto_op **ops; +}; + struct mlx5_crypto_dek { struct mlx5_list_entry entry; /* Pointer to DEK hash list entry. */ struct mlx5_devx_obj *obj; /* Pointer to DEK DevX object. */ -- 2.27.0