From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id ED4A9A0C48; Thu, 8 Jul 2021 17:26:17 +0200 (CEST) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 7E7A341618; Thu, 8 Jul 2021 17:26:12 +0200 (CEST) Received: from NAM10-BN7-obe.outbound.protection.outlook.com (mail-bn7nam10on2074.outbound.protection.outlook.com [40.107.92.74]) by mails.dpdk.org (Postfix) with ESMTP id 2F85C4014F for ; Thu, 8 Jul 2021 17:26:10 +0200 (CEST) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=eWWQuaBW3GHUHTw49vBPw4+aRBcAU0ay6XYOLVkuB0OdSDDQ/3bWFeAUquIZq8uJZ9CE5vyh+b9N2sv/zfAfWcA+og2IVjyM9OcnFVXya1I9gz9DttS1rGW6BQvqjQ2LcYXQZi99OX9TzQqqa/79Imd/r7cn9WGrLbZwy1PDbOWxbF2yHdSTendKLMlV7IveBS2J8UJo1rpZefVdjGt7zXg3qLgomj92Wme5v5bsOSRVSiPLqkwas5nP+BaeAAhTuq/5OErC8rW9HJJeOvBGSOFMhWvoGY+TDGFmeoCP8VqmsDNwhXXDA3R8q+x3PQ6ERu7zFovOcPtrO2lIAgsTjA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=i/iDr1oRsIqjsrxJ8C6ofoUzbSgmJPKNKfCXYy120MY=; b=SsEAX6d3nI1LSuXMUyIuWAz/ZVQJmZZKXd+mATBrhvcRhPLxeu6EtreexIm1HayZZ+eG1fE7LscPFLijy9WXlzotjLSLi+AKRYGAtqoFvAj+Qe+jlgiG7YTNeR5e2cVB/ziZV7powxVrTnl+MsJ2wg0ffRfN6JmzYn3O6QLIMs8WxCZkgPCSJYuEGQ6oRZhPDc0+P894KdaFMoPmlW0f4ENLAYgPu0SenhQ6uULha1cpHDv1M4Djr1DLamufz6g5kkO6N7WkEXQcWzf0Awn3z1aK38uku9VX4RjnoqA/Re3QQb2ahDAtqvdTksl86uMDtgKEg/ZQznTI82iAkF5l2A== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 216.228.112.34) smtp.rcpttodomain=redhat.com smtp.mailfrom=nvidia.com; dmarc=pass (p=none sp=none pct=100) action=none header.from=nvidia.com; dkim=none (message not signed); arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=i/iDr1oRsIqjsrxJ8C6ofoUzbSgmJPKNKfCXYy120MY=; b=OvgWhiIo/sOkKYdO7nA32h5RcVYGuXCGU7nTtukfqavl1w3SOt30x0CnA1Q/ldkRSoL+mtGcC7kuuVuJU3qVqEOtWzHz0ArWGO6ZSGUHSic/IxHddxsLu35aBCoSZ2L41Feo1oIy2p6vVzuXbnYsKv2x6+TOcHsD5T8zKJsZ2LIRwU4IXsImp+W3oQNTB2hTKQ3LEJ1l2BFNTbbwcILr8ebJLWMvpKMOJ38k02eQMHiqnJ9chIHFhwOh4SA85MubseGfl4u4IGd3DVMi+eXnTE9yjWxBHxio0ZOEWuDrOYYwnSWd+8NmrUlIDub4h1Se2YsrAJPpl4/EAQv5K9J5ZA== Received: from CO2PR04CA0200.namprd04.prod.outlook.com (2603:10b6:104:5::30) by DM8PR12MB5496.namprd12.prod.outlook.com (2603:10b6:8:38::11) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4287.31; Thu, 8 Jul 2021 15:26:08 +0000 Received: from CO1NAM11FT008.eop-nam11.prod.protection.outlook.com (2603:10b6:104:5:cafe::ab) by CO2PR04CA0200.outlook.office365.com (2603:10b6:104:5::30) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4308.21 via Frontend Transport; Thu, 8 Jul 2021 15:26:08 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 216.228.112.34) smtp.mailfrom=nvidia.com; redhat.com; dkim=none (message not signed) header.d=none;redhat.com; dmarc=pass action=none header.from=nvidia.com; Received-SPF: Pass (protection.outlook.com: domain of nvidia.com designates 216.228.112.34 as permitted sender) receiver=protection.outlook.com; client-ip=216.228.112.34; helo=mail.nvidia.com; Received: from mail.nvidia.com (216.228.112.34) by CO1NAM11FT008.mail.protection.outlook.com (10.13.175.191) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384) id 15.20.4308.20 via Frontend Transport; Thu, 8 Jul 2021 15:26:08 +0000 Received: from nvidia.com (172.20.187.6) by HQMAIL107.nvidia.com (172.20.187.13) with Microsoft SMTP Server (TLS) id 15.0.1497.2; Thu, 8 Jul 2021 15:26:05 +0000 From: Shiri Kuzin To: CC: , , , Date: Thu, 8 Jul 2021 18:25:16 +0300 Message-ID: <20210708152530.25835-2-shirik@nvidia.com> X-Mailer: git-send-email 2.27.0 In-Reply-To: <20210708152530.25835-1-shirik@nvidia.com> References: <20210701132609.53727-1-shirik@nvidia.com> <20210708152530.25835-1-shirik@nvidia.com> MIME-Version: 1.0 Content-Transfer-Encoding: 8bit Content-Type: text/plain X-Originating-IP: [172.20.187.6] X-ClientProxiedBy: HQMAIL105.nvidia.com (172.20.187.12) To HQMAIL107.nvidia.com (172.20.187.13) X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: 4f8a6852-9784-4164-0d69-08d94224b5bb X-MS-TrafficTypeDiagnostic: DM8PR12MB5496: X-Microsoft-Antispam-PRVS: X-MS-Oob-TLC-OOBClassifiers: OLM:121; X-MS-Exchange-SenderADCheck: 1 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: uhzf4TQGRprQIfvwCqMEXWznyoalHdk7v8zMrVwlj7/CI8xNypw/+fbK4IJ3HR20RV3L/wESkU4aMdevsFPdbuuX4U5EWHhRrLW521UpMM/a1XZ3YP5Fx6Mn1lc/HCw0V6ZpSCvohr+sUvmiRx00vADJCuxubq0VYKxBr7iOf/g+bcvuZvFes3+TBjrhedopasDMEZAMDbK5bxzL3gz64rv/55KcaIp94kEb45M6qJevXZu5asgkfn40VnHRgl2HJjcJykfCSUFO7My0NwkusVNW2LBL4sgjRWQnFJUeFkBjyQZJgwx36LhNZaARi4QxnA/8fJ6TLWIXtL5jngAKbYIvyAXDEtxvh6j1MfahXTiFl8eRzS5KJS2oJcWHOkZOqCXa93A4JK9mzJl+TDdajp5C1GiNKR5PsfHHqUTN1qFOBngF5co8uJCD12sh7ZMASx0kS27ugXfXPh7Ka7jhc9SOy85UPN7Bfq6hAq+RCrKmPJBmwdQi8NAJfrhrv9rawmruc+6ItWaSeKqwXPrxrPsXJcqS7wCCFFDIg5yGbcIIFzKyxw20aLXDq5qSgXitrQddGy7B7TgBD8eeRWUBRKg10b86dhRRXIN/bSlKv9pLJxEzbsjgGxKalFHdPEaQcxeAlH9HJ7s5+cTq7r+9Wd9OHdsH0dwudFtoxMS+skO85GitkbUuoAYRKmWKjBFNtzdDcLHHcgBn9lkkwZYEBA== X-Forefront-Antispam-Report: CIP:216.228.112.34; CTRY:US; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:mail.nvidia.com; PTR:schybrid03.nvidia.com; CAT:NONE; SFS:(4636009)(396003)(39860400002)(346002)(136003)(376002)(36840700001)(46966006)(6286002)(54906003)(336012)(8936002)(70206006)(55016002)(82310400003)(86362001)(6916009)(7696005)(5660300002)(8676002)(36860700001)(16526019)(2616005)(6666004)(30864003)(36756003)(2906002)(36906005)(316002)(82740400003)(83380400001)(47076005)(7636003)(426003)(1076003)(186003)(70586007)(356005)(26005)(4326008)(478600001); DIR:OUT; SFP:1101; X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 08 Jul 2021 15:26:08.3777 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 4f8a6852-9784-4164-0d69-08d94224b5bb X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=43083d15-7273-40c1-b7db-39efd9ccc17a; Ip=[216.228.112.34]; Helo=[mail.nvidia.com] X-MS-Exchange-CrossTenant-AuthSource: CO1NAM11FT008.eop-nam11.prod.protection.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: DM8PR12MB5496 Subject: [dpdk-dev] [PATCH v6 01/15] drivers: introduce mlx5 crypto PMD X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" Add a new PMD for Mellanox devices- crypto PMD. The crypto PMD will be supported starting Nvidia ConnectX6 and BlueField2. The crypto PMD will add the support of encryption and decryption using the AES-XTS symmetric algorithm. The crypto PMD requires rdma-core and uses mlx5 DevX. This patch adds the PCI probing, basic functions, build files and log utility. Signed-off-by: Shiri Kuzin Acked-by: Matan Azrad --- MAINTAINERS | 4 + doc/guides/cryptodevs/features/mlx5.ini | 27 +++ doc/guides/cryptodevs/index.rst | 1 + doc/guides/cryptodevs/mlx5.rst | 63 ++++++ doc/guides/rel_notes/release_21_08.rst | 5 + drivers/common/mlx5/mlx5_common.h | 1 + drivers/common/mlx5/mlx5_common_pci.c | 14 ++ drivers/common/mlx5/mlx5_common_pci.h | 21 +- drivers/crypto/meson.build | 1 + drivers/crypto/mlx5/meson.build | 26 +++ drivers/crypto/mlx5/mlx5_crypto.c | 275 ++++++++++++++++++++++++ drivers/crypto/mlx5/mlx5_crypto_utils.h | 19 ++ drivers/crypto/mlx5/version.map | 3 + 13 files changed, 450 insertions(+), 10 deletions(-) create mode 100644 doc/guides/cryptodevs/features/mlx5.ini create mode 100644 doc/guides/cryptodevs/mlx5.rst create mode 100644 drivers/crypto/mlx5/meson.build create mode 100644 drivers/crypto/mlx5/mlx5_crypto.c create mode 100644 drivers/crypto/mlx5/mlx5_crypto_utils.h create mode 100644 drivers/crypto/mlx5/version.map diff --git a/MAINTAINERS b/MAINTAINERS index f64fce84ab..0cf9e91330 100644 --- a/MAINTAINERS +++ b/MAINTAINERS @@ -1088,6 +1088,10 @@ F: drivers/crypto/octeontx2/ F: doc/guides/cryptodevs/octeontx2.rst F: doc/guides/cryptodevs/features/octeontx2.ini +Mellanox mlx5 +M: Matan Azrad +F: drivers/crypto/mlx5/ + Null Crypto M: Declan Doherty F: drivers/crypto/null/ diff --git a/doc/guides/cryptodevs/features/mlx5.ini b/doc/guides/cryptodevs/features/mlx5.ini new file mode 100644 index 0000000000..ceadd967b6 --- /dev/null +++ b/doc/guides/cryptodevs/features/mlx5.ini @@ -0,0 +1,27 @@ +; +; Features of a mlx5 crypto driver. +; +; Refer to default.ini for the full list of available PMD features. +; +[Features] +HW Accelerated = Y + +; +; Supported crypto algorithms of a mlx5 crypto driver. +; +[Cipher] + +; +; Supported authentication algorithms of a mlx5 crypto driver. +; +[Auth] + +; +; Supported AEAD algorithms of a mlx5 crypto driver. +; +[AEAD] + +; +; Supported Asymmetric algorithms of a mlx5 crypto driver. +; +[Asymmetric] diff --git a/doc/guides/cryptodevs/index.rst b/doc/guides/cryptodevs/index.rst index 279f56a002..747409c441 100644 --- a/doc/guides/cryptodevs/index.rst +++ b/doc/guides/cryptodevs/index.rst @@ -22,6 +22,7 @@ Crypto Device Drivers octeontx octeontx2 openssl + mlx5 mvsam nitrox null diff --git a/doc/guides/cryptodevs/mlx5.rst b/doc/guides/cryptodevs/mlx5.rst new file mode 100644 index 0000000000..05a0a449e2 --- /dev/null +++ b/doc/guides/cryptodevs/mlx5.rst @@ -0,0 +1,63 @@ +.. SPDX-License-Identifier: BSD-3-Clause + Copyright (c) 2021 NVIDIA Corporation & Affiliates + +.. include:: + +MLX5 Crypto Driver +================== + +The MLX5 crypto driver library +(**librte_crypto_mlx5**) provides support for **Mellanox ConnectX-6** +family adapters. + +Overview +-------- + +The device can provide disk encryption services, allowing data encryption +and decryption towards a disk. Having all encryption/decryption +operations done in a single device can reduce cost and overheads of the related +FIPS certification, as ConnectX-6 is FIPS 140-2 level-2 ready. +The encryption cipher is AES-XTS of 256/512 bit key size. + +MKEY is a memory region object in the hardware, that holds address translation information and +attributes per memory area. Its ID must be tied to addresses provided to the hardware. +The encryption operations are performed with MKEY read/write transactions, when +the MKEY is configured to perform crypto operations. + +The encryption does not require text to be aligned to the AES block size (128b). + +The PMD uses libibverbs and libmlx5 to access the device firmware or to +access the hardware components directly. +There are different levels of objects and bypassing abilities. +To get the best performances: + +- Verbs is a complete high-level generic API. +- Direct Verbs is a device-specific API. +- DevX allows to access firmware objects. + +Enabling librte_crypto_mlx5 causes DPDK applications to be linked against +libibverbs. + + +Driver options +-------------- + +- ``class`` parameter [string] + + Select the class of the driver that should probe the device. + `crypto` for the mlx5 crypto driver. + + +Supported NICs +-------------- + +* Mellanox\ |reg| ConnectX\ |reg|-6 200G MCX654106A-HCAT (2x200G) + +Prerequisites +------------- + +- Mellanox OFED version: **5.3** + see :doc:`../../nics/mlx5` guide for more Mellanox OFED details. + +- Compilation can be done also with rdma-core v15+. + see :doc:`../../nics/mlx5` guide for more rdma-core details. diff --git a/doc/guides/rel_notes/release_21_08.rst b/doc/guides/rel_notes/release_21_08.rst index aca34105c2..d6f98811b5 100644 --- a/doc/guides/rel_notes/release_21_08.rst +++ b/doc/guides/rel_notes/release_21_08.rst @@ -98,6 +98,11 @@ New Features * Added support for Meter hierarchy. * Added devargs options ``allow_duplicate_pattern``. +* **Added support for Nvidia crypto device driver.** + + * Added mlx5 crypto driver to support AES-XTS cipher operations. + The first device to support it is ConnectX-6. + Removed Items ------------- diff --git a/drivers/common/mlx5/mlx5_common.h b/drivers/common/mlx5/mlx5_common.h index 306f2f1ab7..790b45f8ea 100644 --- a/drivers/common/mlx5/mlx5_common.h +++ b/drivers/common/mlx5/mlx5_common.h @@ -214,6 +214,7 @@ enum mlx5_class { MLX5_CLASS_VDPA = RTE_BIT64(1), MLX5_CLASS_REGEX = RTE_BIT64(2), MLX5_CLASS_COMPRESS = RTE_BIT64(3), + MLX5_CLASS_CRYPTO = RTE_BIT64(4), }; #define MLX5_DBR_SIZE RTE_CACHE_LINE_SIZE diff --git a/drivers/common/mlx5/mlx5_common_pci.c b/drivers/common/mlx5/mlx5_common_pci.c index 34747c4e07..5547e62d6b 100644 --- a/drivers/common/mlx5/mlx5_common_pci.c +++ b/drivers/common/mlx5/mlx5_common_pci.c @@ -31,6 +31,7 @@ static const struct { { .name = "net", .driver_class = MLX5_CLASS_NET }, { .name = "regex", .driver_class = MLX5_CLASS_REGEX }, { .name = "compress", .driver_class = MLX5_CLASS_COMPRESS }, + { .name = "crypto", .driver_class = MLX5_CLASS_CRYPTO }, }; static const unsigned int mlx5_class_combinations[] = { @@ -38,13 +39,26 @@ static const unsigned int mlx5_class_combinations[] = { MLX5_CLASS_VDPA, MLX5_CLASS_REGEX, MLX5_CLASS_COMPRESS, + MLX5_CLASS_CRYPTO, MLX5_CLASS_NET | MLX5_CLASS_REGEX, MLX5_CLASS_VDPA | MLX5_CLASS_REGEX, MLX5_CLASS_NET | MLX5_CLASS_COMPRESS, MLX5_CLASS_VDPA | MLX5_CLASS_COMPRESS, MLX5_CLASS_REGEX | MLX5_CLASS_COMPRESS, + MLX5_CLASS_NET | MLX5_CLASS_CRYPTO, + MLX5_CLASS_VDPA | MLX5_CLASS_CRYPTO, + MLX5_CLASS_REGEX | MLX5_CLASS_CRYPTO, + MLX5_CLASS_COMPRESS | MLX5_CLASS_CRYPTO, MLX5_CLASS_NET | MLX5_CLASS_REGEX | MLX5_CLASS_COMPRESS, MLX5_CLASS_VDPA | MLX5_CLASS_REGEX | MLX5_CLASS_COMPRESS, + MLX5_CLASS_NET | MLX5_CLASS_REGEX | MLX5_CLASS_CRYPTO, + MLX5_CLASS_VDPA | MLX5_CLASS_REGEX | MLX5_CLASS_CRYPTO, + MLX5_CLASS_NET | MLX5_CLASS_COMPRESS | MLX5_CLASS_CRYPTO, + MLX5_CLASS_VDPA | MLX5_CLASS_COMPRESS | MLX5_CLASS_CRYPTO, + MLX5_CLASS_NET | MLX5_CLASS_REGEX | MLX5_CLASS_COMPRESS | + MLX5_CLASS_CRYPTO, + MLX5_CLASS_VDPA | MLX5_CLASS_REGEX | MLX5_CLASS_COMPRESS | + MLX5_CLASS_CRYPTO, /* New class combination should be added here. */ }; diff --git a/drivers/common/mlx5/mlx5_common_pci.h b/drivers/common/mlx5/mlx5_common_pci.h index de89bb98bc..cb8d2f5f87 100644 --- a/drivers/common/mlx5/mlx5_common_pci.h +++ b/drivers/common/mlx5/mlx5_common_pci.h @@ -9,17 +9,18 @@ * @file * * RTE Mellanox PCI Driver Interface - * Mellanox ConnectX PCI device supports multiple class: net,vdpa,regex and - * compress devices. This layer enables creating such multiple class of devices - * on a single PCI device by allowing to bind multiple class specific device - * driver to attach to mlx5_pci driver. + * Mellanox ConnectX PCI device supports multiple class: net,vdpa,regex,compress + * and crypto devices. This layer enables creating such multiple class of + * devices on a single PCI device by allowing to bind multiple class specific + * device driver to attach to mlx5_pci driver. * - * ----------- ------------ ------------- ---------------- - * | mlx5 | | mlx5 | | mlx5 | | mlx5 | - * | net pmd | | vdpa pmd | | regex pmd | | compress pmd | - * ----------- ------------ ------------- ---------------- - * \ \ / / - * \ \ / / + * -------- -------- --------- ------------ ---------- + * | mlx5 | | mlx5 | | mlx5 | | mlx5 | | mlx5 | + * | net | | vdpa | | regex | | compress | | crypto | + * | pmd | | pmd | | pmd | | pmd | | pmd | + * -------- -------- --------- ------------ ---------- + * \ \ | / / + * \ \ | / / * \ \_--------------_/ / * \_______________| mlx5 |_______________/ * | pci common | diff --git a/drivers/crypto/meson.build b/drivers/crypto/meson.build index b9fdf9392f..6951607def 100644 --- a/drivers/crypto/meson.build +++ b/drivers/crypto/meson.build @@ -15,6 +15,7 @@ drivers = [ 'dpaa_sec', 'dpaa2_sec', 'kasumi', + 'mlx5', 'mvsam', 'nitrox', 'null', diff --git a/drivers/crypto/mlx5/meson.build b/drivers/crypto/mlx5/meson.build new file mode 100644 index 0000000000..6fd70bc477 --- /dev/null +++ b/drivers/crypto/mlx5/meson.build @@ -0,0 +1,26 @@ +# SPDX-License-Identifier: BSD-3-Clause +# Copyright (c) 2021 NVIDIA Corporation & Affiliates + +if not is_linux + build = false + reason = 'only supported on Linux' + subdir_done() +endif + +fmt_name = 'mlx5_crypto' +deps += ['common_mlx5', 'eal', 'cryptodev'] +sources = files( + 'mlx5_crypto.c', +) +cflags_options = [ + '-std=c11', + '-Wno-strict-prototypes', + '-D_BSD_SOURCE', + '-D_DEFAULT_SOURCE', + '-D_XOPEN_SOURCE=600', +] +foreach option:cflags_options + if cc.has_argument(option) + cflags += option + endif +endforeach diff --git a/drivers/crypto/mlx5/mlx5_crypto.c b/drivers/crypto/mlx5/mlx5_crypto.c new file mode 100644 index 0000000000..fbe3c21aae --- /dev/null +++ b/drivers/crypto/mlx5/mlx5_crypto.c @@ -0,0 +1,275 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright (c) 2021 NVIDIA Corporation & Affiliates + */ + +#include +#include +#include +#include +#include +#include +#include + +#include +#include +#include +#include +#include + +#include "mlx5_crypto_utils.h" + +#define MLX5_CRYPTO_DRIVER_NAME mlx5_crypto +#define MLX5_CRYPTO_LOG_NAME pmd.crypto.mlx5 + +#define MLX5_CRYPTO_FEATURE_FLAGS \ + RTE_CRYPTODEV_FF_HW_ACCELERATED + +struct mlx5_crypto_priv { + TAILQ_ENTRY(mlx5_crypto_priv) next; + struct ibv_context *ctx; /* Device context. */ + struct rte_pci_device *pci_dev; + struct rte_cryptodev *crypto_dev; + void *uar; /* User Access Region. */ + uint32_t pdn; /* Protection Domain number. */ + struct ibv_pd *pd; +}; + +TAILQ_HEAD(mlx5_crypto_privs, mlx5_crypto_priv) mlx5_crypto_priv_list = + TAILQ_HEAD_INITIALIZER(mlx5_crypto_priv_list); +static pthread_mutex_t priv_list_lock = PTHREAD_MUTEX_INITIALIZER; + +int mlx5_crypto_logtype; + +uint8_t mlx5_crypto_driver_id; + +static const char mlx5_crypto_drv_name[] = RTE_STR(MLX5_CRYPTO_DRIVER_NAME); + +static const struct rte_driver mlx5_drv = { + .name = mlx5_crypto_drv_name, + .alias = mlx5_crypto_drv_name +}; + +static struct cryptodev_driver mlx5_cryptodev_driver; + +static struct rte_cryptodev_ops mlx5_crypto_ops = { + .dev_configure = NULL, + .dev_start = NULL, + .dev_stop = NULL, + .dev_close = NULL, + .dev_infos_get = NULL, + .stats_get = NULL, + .stats_reset = NULL, + .queue_pair_setup = NULL, + .queue_pair_release = NULL, + .sym_session_get_size = NULL, + .sym_session_configure = NULL, + .sym_session_clear = NULL, + .sym_get_raw_dp_ctx_size = NULL, + .sym_configure_raw_dp_ctx = NULL, +}; + +static void +mlx5_crypto_hw_global_release(struct mlx5_crypto_priv *priv) +{ + if (priv->pd != NULL) { + claim_zero(mlx5_glue->dealloc_pd(priv->pd)); + priv->pd = NULL; + } + if (priv->uar != NULL) { + mlx5_glue->devx_free_uar(priv->uar); + priv->uar = NULL; + } +} + +static int +mlx5_crypto_pd_create(struct mlx5_crypto_priv *priv) +{ +#ifdef HAVE_IBV_FLOW_DV_SUPPORT + struct mlx5dv_obj obj; + struct mlx5dv_pd pd_info; + int ret; + + priv->pd = mlx5_glue->alloc_pd(priv->ctx); + if (priv->pd == NULL) { + DRV_LOG(ERR, "Failed to allocate PD."); + return errno ? -errno : -ENOMEM; + } + obj.pd.in = priv->pd; + obj.pd.out = &pd_info; + ret = mlx5_glue->dv_init_obj(&obj, MLX5DV_OBJ_PD); + if (ret != 0) { + DRV_LOG(ERR, "Fail to get PD object info."); + mlx5_glue->dealloc_pd(priv->pd); + priv->pd = NULL; + return -errno; + } + priv->pdn = pd_info.pdn; + return 0; +#else + (void)priv; + DRV_LOG(ERR, "Cannot get pdn - no DV support."); + return -ENOTSUP; +#endif /* HAVE_IBV_FLOW_DV_SUPPORT */ +} + +static int +mlx5_crypto_hw_global_prepare(struct mlx5_crypto_priv *priv) +{ + if (mlx5_crypto_pd_create(priv) != 0) + return -1; + priv->uar = mlx5_devx_alloc_uar(priv->ctx, -1); + if (priv->uar == NULL || mlx5_os_get_devx_uar_reg_addr(priv->uar) == + NULL) { + rte_errno = errno; + claim_zero(mlx5_glue->dealloc_pd(priv->pd)); + DRV_LOG(ERR, "Failed to allocate UAR."); + return -1; + } + return 0; +} + +/** + * DPDK callback to register a PCI device. + * + * This function spawns crypto device out of a given PCI device. + * + * @param[in] pci_drv + * PCI driver structure (mlx5_crypto_driver). + * @param[in] pci_dev + * PCI device information. + * + * @return + * 0 on success, 1 to skip this driver, a negative errno value otherwise + * and rte_errno is set. + */ +static int +mlx5_crypto_pci_probe(struct rte_pci_driver *pci_drv, + struct rte_pci_device *pci_dev) +{ + struct ibv_device *ibv; + struct rte_cryptodev *crypto_dev; + struct ibv_context *ctx; + struct mlx5_crypto_priv *priv; + struct mlx5_hca_attr attr = { 0 }; + struct rte_cryptodev_pmd_init_params init_params = { + .name = "", + .private_data_size = sizeof(struct mlx5_crypto_priv), + .socket_id = pci_dev->device.numa_node, + .max_nb_queue_pairs = + RTE_CRYPTODEV_PMD_DEFAULT_MAX_NB_QUEUE_PAIRS, + }; + RTE_SET_USED(pci_drv); + if (rte_eal_process_type() != RTE_PROC_PRIMARY) { + DRV_LOG(ERR, "Non-primary process type is not supported."); + rte_errno = ENOTSUP; + return -rte_errno; + } + ibv = mlx5_os_get_ibv_device(&pci_dev->addr); + if (ibv == NULL) { + DRV_LOG(ERR, "No matching IB device for PCI slot " + PCI_PRI_FMT ".", pci_dev->addr.domain, + pci_dev->addr.bus, pci_dev->addr.devid, + pci_dev->addr.function); + return -rte_errno; + } + DRV_LOG(INFO, "PCI information matches for device \"%s\".", ibv->name); + ctx = mlx5_glue->dv_open_device(ibv); + if (ctx == NULL) { + DRV_LOG(ERR, "Failed to open IB device \"%s\".", ibv->name); + rte_errno = ENODEV; + return -rte_errno; + } + if (mlx5_devx_cmd_query_hca_attr(ctx, &attr) != 0 || + attr.crypto == 0 || attr.aes_xts == 0) { + DRV_LOG(ERR, "Not enough capabilities to support crypto " + "operations, maybe old FW/OFED version?"); + claim_zero(mlx5_glue->close_device(ctx)); + rte_errno = ENOTSUP; + return -ENOTSUP; + } + crypto_dev = rte_cryptodev_pmd_create(ibv->name, &pci_dev->device, + &init_params); + if (crypto_dev == NULL) { + DRV_LOG(ERR, "Failed to create device \"%s\".", ibv->name); + claim_zero(mlx5_glue->close_device(ctx)); + return -ENODEV; + } + DRV_LOG(INFO, + "Crypto device %s was created successfully.", ibv->name); + crypto_dev->dev_ops = &mlx5_crypto_ops; + crypto_dev->dequeue_burst = NULL; + crypto_dev->enqueue_burst = NULL; + crypto_dev->feature_flags = MLX5_CRYPTO_FEATURE_FLAGS; + crypto_dev->driver_id = mlx5_crypto_driver_id; + priv = crypto_dev->data->dev_private; + priv->ctx = ctx; + priv->pci_dev = pci_dev; + priv->crypto_dev = crypto_dev; + if (mlx5_crypto_hw_global_prepare(priv) != 0) { + rte_cryptodev_pmd_destroy(priv->crypto_dev); + claim_zero(mlx5_glue->close_device(priv->ctx)); + return -1; + } + pthread_mutex_lock(&priv_list_lock); + TAILQ_INSERT_TAIL(&mlx5_crypto_priv_list, priv, next); + pthread_mutex_unlock(&priv_list_lock); + return 0; +} + +static int +mlx5_crypto_pci_remove(struct rte_pci_device *pdev) +{ + struct mlx5_crypto_priv *priv = NULL; + + pthread_mutex_lock(&priv_list_lock); + TAILQ_FOREACH(priv, &mlx5_crypto_priv_list, next) + if (rte_pci_addr_cmp(&priv->pci_dev->addr, &pdev->addr) != 0) + break; + if (priv) + TAILQ_REMOVE(&mlx5_crypto_priv_list, priv, next); + pthread_mutex_unlock(&priv_list_lock); + if (priv) { + mlx5_crypto_hw_global_release(priv); + rte_cryptodev_pmd_destroy(priv->crypto_dev); + claim_zero(mlx5_glue->close_device(priv->ctx)); + } + return 0; +} + +static const struct rte_pci_id mlx5_crypto_pci_id_map[] = { + { + RTE_PCI_DEVICE(PCI_VENDOR_ID_MELLANOX, + PCI_DEVICE_ID_MELLANOX_CONNECTX6) + }, + { + .vendor_id = 0 + } + }; + +static struct mlx5_pci_driver mlx5_crypto_driver = { + .driver_class = MLX5_CLASS_CRYPTO, + .pci_driver = { + .driver = { + .name = RTE_STR(MLX5_CRYPTO_DRIVER_NAME), + }, + .id_table = mlx5_crypto_pci_id_map, + .probe = mlx5_crypto_pci_probe, + .remove = mlx5_crypto_pci_remove, + .drv_flags = 0, + }, +}; + +RTE_INIT(rte_mlx5_crypto_init) +{ + mlx5_common_init(); + if (mlx5_glue != NULL) + mlx5_pci_driver_register(&mlx5_crypto_driver); +} + +RTE_PMD_REGISTER_CRYPTO_DRIVER(mlx5_cryptodev_driver, mlx5_drv, + mlx5_crypto_driver_id); + +RTE_LOG_REGISTER_DEFAULT(mlx5_crypto_logtype, NOTICE) +RTE_PMD_EXPORT_NAME(MLX5_CRYPTO_DRIVER_NAME, __COUNTER__); +RTE_PMD_REGISTER_PCI_TABLE(MLX5_CRYPTO_DRIVER_NAME, mlx5_crypto_pci_id_map); +RTE_PMD_REGISTER_KMOD_DEP(MLX5_CRYPTO_DRIVER_NAME, "* ib_uverbs & mlx5_core & mlx5_ib"); diff --git a/drivers/crypto/mlx5/mlx5_crypto_utils.h b/drivers/crypto/mlx5/mlx5_crypto_utils.h new file mode 100644 index 0000000000..b6c60ca782 --- /dev/null +++ b/drivers/crypto/mlx5/mlx5_crypto_utils.h @@ -0,0 +1,19 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright (c) 2021 NVIDIA Corporation & Affiliates + */ + +#ifndef RTE_PMD_MLX5_CRYPTO_UTILS_H_ +#define RTE_PMD_MLX5_CRYPTO_UTILS_H_ + +#include + +extern int mlx5_crypto_logtype; + +#define MLX5_CRYPTO_LOG_PREFIX "mlx5_crypto" +/* Generic printf()-like logging macro with automatic line feed. */ +#define DRV_LOG(level, ...) \ + PMD_DRV_LOG_(level, mlx5_crypto_logtype, MLX5_CRYPTO_LOG_PREFIX, \ + __VA_ARGS__ PMD_DRV_LOG_STRIP PMD_DRV_LOG_OPAREN, \ + PMD_DRV_LOG_CPAREN) + +#endif /* RTE_PMD_MLX5_CRYPTO_UTILS_H_ */ diff --git a/drivers/crypto/mlx5/version.map b/drivers/crypto/mlx5/version.map new file mode 100644 index 0000000000..4a76d1d52d --- /dev/null +++ b/drivers/crypto/mlx5/version.map @@ -0,0 +1,3 @@ +DPDK_21 { + local: *; +}; -- 2.27.0