From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id C59D0A0548; Sun, 9 May 2021 18:06:26 +0200 (CEST) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 8A69241141; Sun, 9 May 2021 18:05:45 +0200 (CEST) Received: from NAM10-DM6-obe.outbound.protection.outlook.com (mail-dm6nam10on2044.outbound.protection.outlook.com [40.107.93.44]) by mails.dpdk.org (Postfix) with ESMTP id 0F22641136 for ; Sun, 9 May 2021 18:05:43 +0200 (CEST) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=b0V64lpui6djBlMn8VZQdNEt7cseYgvHpVV/7Sexc3cyLJ4jjaTkySADrH5OigBT5Qy6eakYCQf5vxdZ6HWNJPeeKGyP5dtV5Ah/uIKhtR7IDKzypANFuHbLO7G140saARzNqOHT2uIms+YI5yCN8YjRaB2VIdlLMAFXOYnhKk7FgGDrtVu1l3Utn9iocsBao6twOXzcjSP65S39WK8V3CjtImFIAwff5STj9ta7n0udfD3sWhVmQer5l44kZMm0AGfc/lNcsKiv43pFDOLekWXuyHNHxIekcmu7+uFytHyW6dEuwAfECnMnOCoU4DeO1jVDWLu43udsIa9LqblBIw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=zxP8n+NPzkiS6yIVmVRouNF8g1dbKcgKQgi7BYcCKXM=; b=ihbSK6U+CMHn8CgzhJzlYoEemHw7mSPRIAzjw2RGAqn7pzO3NUjliOOm/RmWO5ORFvfnf5Y3XcWWXrjhtLHUTCXQd0Myy8HOOgAoTxCYr3uHQ7yAtM2xsXYYHqIo16LP2cKyUiLDycXhBe8h2DrN4M7RNRdM0swdA9flMB6woKK4Lw62RBA/AH/8pwGQ7PCnBwWk0KqvCqEeKeVVA2hL1PSC/v2QCNJTn8FF6hdDJ8MacYHKShmdBKQ6voIuXAFoUNh7lqubmZM9E+8c9bCr4qc9YiRYfdl6+82zkVoCBDxxBw829fNKW7I9jqsBhsX6HJFkYLT89yKLEC/Z1JnLZQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 216.228.112.34) smtp.rcpttodomain=monjalon.net smtp.mailfrom=nvidia.com; dmarc=pass (p=none sp=none pct=100) action=none header.from=nvidia.com; dkim=none (message not signed); arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=zxP8n+NPzkiS6yIVmVRouNF8g1dbKcgKQgi7BYcCKXM=; b=KdQ9yw7Px1BFnTmLMsKjqG8sHMd7gwtV9c7h8i8S8r/OhPtA5d/B3O81Op5hGgiNl3ez25jJ6z0f6490NrUWbiRYH2lmKg+oi4fHkoojVxPr/yegsUtHcKXTK2apDdIkJvE6TU6Qiq/lnbG5m7Hxs6/yoXAnq3tU/DfxESoWVf3G9f3vHPwy6nL5PJc+lZ/Ld1D5TisBBsE48jtIXHVpVjzE6tXxJxM3lPoW2JXxwjrCspo3Z35ZJXTANhAPnI98AbmHwOud9QRtokeMqJxC+5YSpUSDT2ZZCRXZTeRMWKBvlX/gKxBdigQg59L+ZYGnwDudGRwq6iy56ohuRQr5Xg== Received: from MWHPR07CA0024.namprd07.prod.outlook.com (2603:10b6:300:116::34) by DM5PR12MB1611.namprd12.prod.outlook.com (2603:10b6:4:c::8) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4108.30; Sun, 9 May 2021 16:05:39 +0000 Received: from CO1NAM11FT042.eop-nam11.prod.protection.outlook.com (2603:10b6:300:116:cafe::5f) by MWHPR07CA0024.outlook.office365.com (2603:10b6:300:116::34) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4108.24 via Frontend Transport; Sun, 9 May 2021 16:05:39 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 216.228.112.34) smtp.mailfrom=nvidia.com; monjalon.net; dkim=none (message not signed) header.d=none;monjalon.net; dmarc=pass action=none header.from=nvidia.com; Received-SPF: Pass (protection.outlook.com: domain of nvidia.com designates 216.228.112.34 as permitted sender) receiver=protection.outlook.com; client-ip=216.228.112.34; helo=mail.nvidia.com; Received: from mail.nvidia.com (216.228.112.34) by CO1NAM11FT042.mail.protection.outlook.com (10.13.174.250) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384) id 15.20.4108.25 via Frontend Transport; Sun, 9 May 2021 16:05:39 +0000 Received: from nvidia.com (172.20.145.6) by HQMAIL107.nvidia.com (172.20.187.13) with Microsoft SMTP Server (TLS) id 15.0.1497.2; Sun, 9 May 2021 16:05:34 +0000 From: Matan Azrad To: CC: , , , "Thomas Monjalon" , Shiri Kuzin Date: Sun, 9 May 2021 19:05:00 +0300 Message-ID: <20210509160507.224644-8-matan@nvidia.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210509160507.224644-1-matan@nvidia.com> References: <20210504210857.3398397-1-matan@nvidia.com> <20210509160507.224644-1-matan@nvidia.com> MIME-Version: 1.0 Content-Transfer-Encoding: 8bit Content-Type: text/plain X-Originating-IP: [172.20.145.6] X-ClientProxiedBy: HQMAIL105.nvidia.com (172.20.187.12) To HQMAIL107.nvidia.com (172.20.187.13) X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: 19632b69-f1a6-4333-6a87-08d913044a33 X-MS-TrafficTypeDiagnostic: DM5PR12MB1611: X-LD-Processed: 43083d15-7273-40c1-b7db-39efd9ccc17a,ExtAddr X-Microsoft-Antispam-PRVS: X-MS-Oob-TLC-OOBClassifiers: OLM:216; X-MS-Exchange-SenderADCheck: 1 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: 5RZRDy1lq7zgGkTOU7h1kJMITbHtPalTRG53yRQQTMx2+kS/WciOFTUEehlxiQmb7pg/OmFoqdkfKpAgYEYbFvJ3JMYrFaQ/5AGfMIPJ3AMmHeEBMHS1JyfwV2NBLx1PCXbj71OvxZap3xE382BVWa/lAuaK7+Xmv01MTaq6Glz8s0Dzo+Md7VYGKltjVJeTAzFt5KMcsVmF0I5c4UQnM8JZPY9NX7axI9v1hANg6/ZHouv1Gle+q3wuZaeSzn2eM2FLsR0ideDMdpnoQH8vLArzeM86Dl8miRFQZXKIf3YhGu3RBW6a4mzP9xIemiIQYIEeN91Rbdh7/mUkbUojs4GcJi+4oGBy2pdkzUIf7BfWgLE+N9goBjNniusssjrWnGwIIDyDENdloFofo0XrTs651C1jYgX/Tl8SHCMNHVmCJ6cwnD6R+406038G5lu8CPTe0+2bfIKKMCQ4dPTO9U/n6PsEe6tD47q7QiyBkUF2lta/85KkE0w16mhKYkZIZUKtxGHajxb65fytOfDyAls2CgbprBHj4N1Oi+VMeBsHBGcP/owhTJ5qu5tf23rLVPkAvcbciR13fvioyRkylz7YcfrTRgn1oievxLGgbEMc9mww5MOTHZtaSRXRxQJjtvb7l6idZUS+T57zubrpxw== X-Forefront-Antispam-Report: CIP:216.228.112.34; CTRY:US; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:mail.nvidia.com; PTR:schybrid03.nvidia.com; CAT:NONE; SFS:(4636009)(346002)(136003)(376002)(39860400002)(396003)(36840700001)(46966006)(86362001)(54906003)(186003)(6286002)(7696005)(107886003)(356005)(7636003)(316002)(2906002)(1076003)(8676002)(26005)(336012)(6666004)(8936002)(70586007)(478600001)(6916009)(2616005)(83380400001)(47076005)(426003)(55016002)(16526019)(36906005)(70206006)(5660300002)(36756003)(36860700001)(82310400003)(4326008)(82740400003); DIR:OUT; SFP:1101; X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 09 May 2021 16:05:39.4346 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 19632b69-f1a6-4333-6a87-08d913044a33 X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=43083d15-7273-40c1-b7db-39efd9ccc17a; Ip=[216.228.112.34]; Helo=[mail.nvidia.com] X-MS-Exchange-CrossTenant-AuthSource: CO1NAM11FT042.eop-nam11.prod.protection.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: DM5PR12MB1611 Subject: [dpdk-dev] [PATCH v4 07/14] crypto/mlx5: add memory region management X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" From: Shiri Kuzin Mellanox user space drivers don't deal with physical addresses as part of a memory protection mechanism. The device translates the given virtual address to a physical address using the given memory key as an address space identifier. That's why any mbuf virtual address is moved directly to the HW descriptor(WQE). The mapping between the virtual address to the physical address is saved in MR configured by the kernel to the HW. Each MR has a key that should also be moved to the WQE by the SW. When the SW sees an unmapped address, it extends the address range and creates a MR using a system call. Add memory region cache management: - 2 level cache per queue-pair - no locks. - 1 shared cache between all the queues using a lock. Using this way, the MR key search per data-path address is optimized. Signed-off-by: Shiri Kuzin Acked-by: Matan Azrad --- drivers/crypto/mlx5/mlx5_crypto.c | 20 ++++++++++++++++++++ drivers/crypto/mlx5/mlx5_crypto.h | 3 +++ 2 files changed, 23 insertions(+) diff --git a/drivers/crypto/mlx5/mlx5_crypto.c b/drivers/crypto/mlx5/mlx5_crypto.c index e3e2401a25..d063c36e9b 100644 --- a/drivers/crypto/mlx5/mlx5_crypto.c +++ b/drivers/crypto/mlx5/mlx5_crypto.c @@ -233,6 +233,7 @@ mlx5_crypto_queue_pair_release(struct rte_cryptodev *dev, uint16_t qp_id) claim_zero(mlx5_glue->devx_umem_dereg(qp->umem_obj)); if (qp->umem_buf != NULL) rte_free(qp->umem_buf); + mlx5_mr_btree_free(&qp->mr_ctrl.cache_bh); mlx5_devx_cq_destroy(&qp->cq_obj); rte_free(qp); dev->data->queue_pairs[qp_id] = NULL; @@ -312,6 +313,13 @@ mlx5_crypto_queue_pair_setup(struct rte_cryptodev *dev, uint16_t qp_id, DRV_LOG(ERR, "Failed to register QP umem."); goto error; } + if (mlx5_mr_btree_init(&qp->mr_ctrl.cache_bh, MLX5_MR_BTREE_CACHE_N, + priv->dev_config.socket_id) != 0) { + DRV_LOG(ERR, "Cannot allocate MR Btree for qp %u.", + (uint32_t)qp_id); + rte_errno = ENOMEM; + goto error; + } attr.pd = priv->pdn; attr.uar_index = mlx5_os_get_devx_uar_page_id(priv->uar); attr.cqn = qp->cq_obj.cq->id; @@ -500,6 +508,17 @@ mlx5_crypto_pci_probe(struct rte_pci_driver *pci_drv, claim_zero(mlx5_glue->close_device(priv->ctx)); return -1; } + if (mlx5_mr_btree_init(&priv->mr_scache.cache, + MLX5_MR_BTREE_CACHE_N * 2, rte_socket_id()) != 0) { + DRV_LOG(ERR, "Failed to allocate shared cache MR memory."); + mlx5_crypto_hw_global_release(priv); + rte_cryptodev_pmd_destroy(priv->crypto_dev); + claim_zero(mlx5_glue->close_device(priv->ctx)); + rte_errno = ENOMEM; + return -rte_errno; + } + priv->mr_scache.reg_mr_cb = mlx5_common_verbs_reg_mr; + priv->mr_scache.dereg_mr_cb = mlx5_common_verbs_dereg_mr; pthread_mutex_lock(&priv_list_lock); TAILQ_INSERT_TAIL(&mlx5_crypto_priv_list, priv, next); pthread_mutex_unlock(&priv_list_lock); @@ -519,6 +538,7 @@ mlx5_crypto_pci_remove(struct rte_pci_device *pdev) TAILQ_REMOVE(&mlx5_crypto_priv_list, priv, next); pthread_mutex_unlock(&priv_list_lock); if (priv) { + mlx5_mr_release_cache(&priv->mr_scache); mlx5_crypto_hw_global_release(priv); rte_cryptodev_pmd_destroy(priv->crypto_dev); claim_zero(mlx5_glue->close_device(priv->ctx)); diff --git a/drivers/crypto/mlx5/mlx5_crypto.h b/drivers/crypto/mlx5/mlx5_crypto.h index f5313b89f2..397267d249 100644 --- a/drivers/crypto/mlx5/mlx5_crypto.h +++ b/drivers/crypto/mlx5/mlx5_crypto.h @@ -12,6 +12,7 @@ #include #include +#include #define MLX5_CRYPTO_DEK_HTABLE_SZ (1 << 11) #define MLX5_CRYPTO_KEY_LENGTH 80 @@ -27,6 +28,7 @@ struct mlx5_crypto_priv { struct ibv_pd *pd; struct mlx5_hlist *dek_hlist; /* Dek hash list. */ struct rte_cryptodev_config dev_config; + struct mlx5_mr_share_cache mr_scache; /* Global shared MR cache. */ }; struct mlx5_crypto_qp { @@ -36,6 +38,7 @@ struct mlx5_crypto_qp { void *umem_buf; volatile uint32_t *db_rec; struct rte_crypto_op **ops; + struct mlx5_mr_ctrl mr_ctrl; }; struct mlx5_crypto_dek { -- 2.25.1