From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id AC75C45455; Fri, 14 Jun 2024 02:31:16 +0200 (CEST) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id E64604275D; Fri, 14 Jun 2024 02:31:09 +0200 (CEST) Received: from NAM10-MW2-obe.outbound.protection.outlook.com (mail-mw2nam10on2059.outbound.protection.outlook.com [40.107.94.59]) by mails.dpdk.org (Postfix) with ESMTP id DB52A410E4 for ; Fri, 14 Jun 2024 02:31:07 +0200 (CEST) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=BM0Ygzh6lhIYPXTIBnHO6aTxp7dQip1UhB3d4MXXNFQFoW6tSXjR1p2BfI9wVuBoJ0OusDw+EXK9BnBfWxcynMVU54HMFKaadBMBb9UYuDf8fo22NMGlMLqedPKbqVtGMrA3FIfcJhky51u1RQ+3EIkgL1+fBZrRY3bdQTvJ9gYGm9HiLyeszXu848L06/EcReBRMKGXU5BDSNfCBMIdCd92SZ2jVCQg1WcymAQIKiZBZYDFTtnoGMioFDprwAhT6aVbz+TuaF1az40+PUi7OLwnOIGqYp5U3Ud82Orftsy/DMWiEIx6yBqr70sTo6D4Eil16s/zYeGQqb994OKtEg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=Wj8VOFMQgfPGGSygXYa5sXzX7W+tieE2mhrsB/q+Wcs=; b=JmbT33DhNT83vWaFsIAOu9lVjCjMFFEvLStzNDKZXmdr8gKVSe2zxZiddn/3TYKc/H3CoNRZKaqXpNstBb62Lgj8JZAkvZnVNPbcrT/kbN3iNBJNpmCxI2JBXQ5qKxQXGWs4Vxolc2unrhBLuUMQqRl0PNP/9toarwY1XkGojyri+ypJLran98y9tDfHvSiW+DJoANA876sCe0Fxax3Vgcwrp9XhdYZPE99NapZPPTt/7NlaOtQVNomYkzG88zB9N9aU0lXb7KVep+6s0pv5FzoLUueA7kODc0kj7qJIZKKCe3Wwg+IQdhFl3su+PAdxP22HtQh56VjaXEnvDhiHNg== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 216.228.117.160) smtp.rcpttodomain=dpdk.org smtp.mailfrom=nvidia.com; dmarc=pass (p=reject sp=reject pct=100) action=none header.from=nvidia.com; dkim=none (message not signed); arc=none (0) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=Wj8VOFMQgfPGGSygXYa5sXzX7W+tieE2mhrsB/q+Wcs=; b=J8ZfEIUj2bPzB6O6h2W6cf3vTuiBHjScnxOeTlTOASeEVpwrnKJP2hEpcEpK//7zbF9M7LiReQB4YZ1PShDzFqDjTXd8JP/GzWS09WJAjdBVpCpBGhH8l7hMdly6zE9GdAnuk6DgiXSmJUOJgxeVbnP73tOO2xLxgMfTzwREPTKehN2Nd1HpsQJV/v95DfvgNFaJ2F7c6MUJ7Ye8q6MwecH5kGSQOF/oebMQN63czswYZRf2ADpNabNZl1j1IbnzipRe1q9MtiwSp8E3yC1s8ENUx8S+NxbV7tp0x135itLDFQwPn1e/2dG2SbGAaoTelYAsWqh/cfQMCkNolnYAtw== Received: from DM6PR08CA0021.namprd08.prod.outlook.com (2603:10b6:5:80::34) by PH7PR12MB7162.namprd12.prod.outlook.com (2603:10b6:510:201::22) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7633.37; Fri, 14 Jun 2024 00:31:04 +0000 Received: from DS3PEPF0000C37F.namprd04.prod.outlook.com (2603:10b6:5:80:cafe::5) by DM6PR08CA0021.outlook.office365.com (2603:10b6:5:80::34) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7677.25 via Frontend Transport; Fri, 14 Jun 2024 00:31:03 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 216.228.117.160) smtp.mailfrom=nvidia.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=nvidia.com; Received-SPF: Pass (protection.outlook.com: domain of nvidia.com designates 216.228.117.160 as permitted sender) receiver=protection.outlook.com; client-ip=216.228.117.160; helo=mail.nvidia.com; pr=C Received: from mail.nvidia.com (216.228.117.160) by DS3PEPF0000C37F.mail.protection.outlook.com (10.167.23.9) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7677.15 via Frontend Transport; Fri, 14 Jun 2024 00:31:03 +0000 Received: from rnnvmail201.nvidia.com (10.129.68.8) by mail.nvidia.com (10.129.200.66) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.1544.4; Thu, 13 Jun 2024 17:30:48 -0700 Received: from nvidia.com (10.126.230.35) by rnnvmail201.nvidia.com (10.129.68.8) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.1544.4; Thu, 13 Jun 2024 17:30:46 -0700 From: Suanming Mou To: Matan Azrad CC: , Subject: [PATCH v2 2/2] crypto/mlx5: add out of place mode for IPsec operation Date: Fri, 14 Jun 2024 08:30:31 +0800 Message-ID: <20240614003031.2006131-3-suanmingm@nvidia.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20240614003031.2006131-1-suanmingm@nvidia.com> References: <20240530072413.1602343-1-suanmingm@nvidia.com> <20240614003031.2006131-1-suanmingm@nvidia.com> MIME-Version: 1.0 Content-Transfer-Encoding: 8bit Content-Type: text/plain X-Originating-IP: [10.126.230.35] X-ClientProxiedBy: rnnvmail203.nvidia.com (10.129.68.9) To rnnvmail201.nvidia.com (10.129.68.8) X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: DS3PEPF0000C37F:EE_|PH7PR12MB7162:EE_ X-MS-Office365-Filtering-Correlation-Id: 363745e3-f4e5-41c9-26a7-08dc8c094617 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; ARA:13230035|36860700008|1800799019|376009|82310400021; X-Microsoft-Antispam-Message-Info: =?us-ascii?Q?A/kZVvdPq+Oz0BBbI0xieESq8wAPZ6z3jNMdPhSYrf0EQjTqhphz5fm2itbW?= =?us-ascii?Q?7yIUutz7sMvNU3QUQIHqweVEINcWidxPweE92HEokZ8o0y19F2XVnNQ3saj+?= =?us-ascii?Q?ayGB6xq+d73f9gdzAJKAu9TvMRC7PUoXh68wnenXhF/NBbQgSYCte+pSIT3e?= =?us-ascii?Q?sPBpO9gGCQ9E5FEuKKvjPAMe5r3MRUpanKP9Vsv60TxNLG2J3jgLVCtMVbky?= =?us-ascii?Q?vvHOuMBSK6c6P43OogqDK5PVrr+1i9XFhocgnv4WzoSdE4eLdNadNkg2MfND?= =?us-ascii?Q?hqs1RCRsoxXiit1+d967tKSAAEANURw7m152/1fi2mSOAMLGPkMvKoZ4BS0k?= =?us-ascii?Q?B3jS7GYsGC+CkqR0POtz4zqaVEsyNFlvwyYZcogO7Ie++onQcfKKDrb62oXW?= =?us-ascii?Q?ZPW9EbpcbPpJ5AbyJUDI9yOl6siAUw3RGlaMo3Zz1ln+uu2nZ02+2e3D3FvO?= =?us-ascii?Q?V+HqcerUGqL/j2w02rQN6axFUjuuDZwfl4mdwlz98sQwLNTMv3A57b2wrIxM?= =?us-ascii?Q?duZQVqSXPM+kSKN1M/E9iWnPD4yGtNASCxjjuJqpon2J/bSoS9651kOjJ4vf?= =?us-ascii?Q?JVBpa04nZDXwKL99chx4RwYZpFmmfn0UcUPsfc3A6NjmMpoTZQZwtLDDo8qY?= =?us-ascii?Q?e/LXBJFh7yBkBVkHjXdz0bO/uVq+YCyXxzSa3mPghS8zx2zSrnNeHiWyLbTX?= =?us-ascii?Q?FCA+8P7sjfHSFJhJkGrjI36SeEP34NkzB/ycL2AFJFBzbuqEtk5vX5ps78ly?= =?us-ascii?Q?o3AR+riHWxCOspOS8GaTRy48iMXTzf6pe0LzXCqqb4cK0P7GbjBJobzUYmAt?= =?us-ascii?Q?WWPvcBqAo/rSCnoTBtsQETjyDeHhmkrf018gh4/XfwwzwmOzEwqcYX1+b6Kg?= =?us-ascii?Q?nAzP/aoWzf+A86YVhJ0uGpRFtmmtAfS1+xsr1lEsW1L4HwpPeCSiZjDQTQ+7?= =?us-ascii?Q?cvze9d3cWX9Xkm/rUjXghJUL9i9XuDWDs1SCwMJXGxcG1PE7x9fDwgQ7u6uq?= =?us-ascii?Q?JuLpgoOI4+fYSxDo8pEDXrQm6F5qEMxD/ID/6ObsoTk45LzmT24RtPpODJqt?= =?us-ascii?Q?1iCAwfER3na8ftIcAy955XhecxW+YHldapVrEsnM1qdYq+ov1SAlkWswW/8j?= =?us-ascii?Q?huLOoND1h5NMb2fHixfz5NZ58VxS8VVTme4bV2TM3xyi9Q4UNW1uBrTudsKS?= =?us-ascii?Q?utEHzB5dQ7gqpjDgnQ0I+g9i7Tr24iZwkQ+8f9aTr+BMLR4z/FYgAhx9cMQp?= =?us-ascii?Q?TRPSi9QyxU76HQXna/O1EzX2mb4D67JMkDVBJ4iTXXKSXwE4sU2iJZIrq1LK?= =?us-ascii?Q?p7lGbsTgzHwylRgwJwLa41iqW5d8qg0Ma7qJa6yWxxHeyXuXsVJh99WYDCWb?= =?us-ascii?Q?MugFmDAMQtC1ervDDAVAHGBazwDw6uyLdrJJhTuXpMqTmXT9Xw=3D=3D?= X-Forefront-Antispam-Report: CIP:216.228.117.160; CTRY:US; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:mail.nvidia.com; PTR:dc6edge1.nvidia.com; CAT:NONE; SFS:(13230035)(36860700008)(1800799019)(376009)(82310400021); DIR:OUT; SFP:1101; X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 14 Jun 2024 00:31:03.6878 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 363745e3-f4e5-41c9-26a7-08dc8c094617 X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=43083d15-7273-40c1-b7db-39efd9ccc17a; Ip=[216.228.117.160]; Helo=[mail.nvidia.com] X-MS-Exchange-CrossTenant-AuthSource: DS3PEPF0000C37F.namprd04.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: PH7PR12MB7162 X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org The IPsec operation shrinks AAD directly before payload in enqueue burst and restores the memory in dequeue burst. This commit adds the support of OOP mode follows the similar strategy. Signed-off-by: Suanming Mou Acked-by: Matan Azrad --- doc/guides/cryptodevs/mlx5.rst | 3 ++ drivers/crypto/mlx5/mlx5_crypto.c | 2 +- drivers/crypto/mlx5/mlx5_crypto_gcm.c | 43 +++++++++++++++++++++------ 3 files changed, 38 insertions(+), 10 deletions(-) diff --git a/doc/guides/cryptodevs/mlx5.rst b/doc/guides/cryptodevs/mlx5.rst index 320f57bb02..a88d0e07b6 100644 --- a/doc/guides/cryptodevs/mlx5.rst +++ b/doc/guides/cryptodevs/mlx5.rst @@ -201,6 +201,9 @@ for an additional list of options shared with other mlx5 drivers. AAD (ESP SPI and SN) to the payload during enqueue OP. It then restores the original memory layout in the decrypt OP. ESP.IV size supported range is [0,16] bytes. + For OOP case, PMD will replace the bytes preceding the OP destination + address to match the information found between the AAD pointer and the + OP source address. User should prepare this headroom in this case. Set to ``full_capable`` by default. diff --git a/drivers/crypto/mlx5/mlx5_crypto.c b/drivers/crypto/mlx5/mlx5_crypto.c index d49a375dcb..bf9cbd4a6a 100644 --- a/drivers/crypto/mlx5/mlx5_crypto.c +++ b/drivers/crypto/mlx5/mlx5_crypto.c @@ -25,6 +25,7 @@ #define MLX5_CRYPTO_FEATURE_FLAGS(wrapped_mode) \ (RTE_CRYPTODEV_FF_SYMMETRIC_CRYPTO | RTE_CRYPTODEV_FF_HW_ACCELERATED | \ + RTE_CRYPTODEV_FF_OOP_LB_IN_LB_OUT | \ (wrapped_mode ? RTE_CRYPTODEV_FF_CIPHER_WRAPPED_KEY : 0) | \ RTE_CRYPTODEV_FF_CIPHER_MULTIPLE_DATA_UNITS) @@ -61,7 +62,6 @@ mlx5_crypto_dev_infos_get(struct rte_cryptodev *dev, RTE_CRYPTODEV_FF_IN_PLACE_SGL | RTE_CRYPTODEV_FF_OOP_SGL_IN_SGL_OUT | RTE_CRYPTODEV_FF_OOP_SGL_IN_LB_OUT | - RTE_CRYPTODEV_FF_OOP_LB_IN_LB_OUT | RTE_CRYPTODEV_FF_OOP_LB_IN_SGL_OUT; dev_info->capabilities = priv->caps; diff --git a/drivers/crypto/mlx5/mlx5_crypto_gcm.c b/drivers/crypto/mlx5/mlx5_crypto_gcm.c index 189e798d1d..f598273873 100644 --- a/drivers/crypto/mlx5/mlx5_crypto_gcm.c +++ b/drivers/crypto/mlx5/mlx5_crypto_gcm.c @@ -1000,6 +1000,7 @@ mlx5_crypto_gcm_ipsec_enqueue_burst(void *queue_pair, struct mlx5_crypto_gcm_data gcm_data; struct rte_crypto_op *op; struct rte_mbuf *m_src; + struct rte_mbuf *m_dst; uint16_t mask = qp->entries_n - 1; uint16_t remain = qp->entries_n - (qp->pi - qp->qp_ci); uint32_t idx; @@ -1029,19 +1030,32 @@ mlx5_crypto_gcm_ipsec_enqueue_burst(void *queue_pair, MLX5_ASSERT(pkt_iv_len <= MLX5_CRYPTO_GCM_IPSEC_IV_SIZE); gcm_data.src_bytes = op->sym->aead.data.length + sess->aad_len; gcm_data.src_mkey = mlx5_mr_mb2mr(&qp->mr_ctrl, op->sym->m_src); - /* OOP mode is not supported. */ - MLX5_ASSERT(!op->sym->m_dst || op->sym->m_dst == m_src); - gcm_data.dst_addr = gcm_data.src_addr; - gcm_data.dst_mkey = gcm_data.src_mkey; + m_dst = op->sym->m_dst; + if (m_dst && m_dst != m_src) { + MLX5_ASSERT(m_dst->nb_segs == 1 && + (rte_pktmbuf_headroom(m_dst) + op->sym->aead.data.offset) + >= sess->aad_len + pkt_iv_len); + gcm_data.dst_addr = RTE_PTR_SUB + (rte_pktmbuf_mtod_offset(m_dst, + void *, op->sym->aead.data.offset), sess->aad_len); + gcm_data.dst_mkey = mlx5_mr_mb2mr(&qp->mr_ctrl, m_dst); + } else { + gcm_data.dst_addr = gcm_data.src_addr; + gcm_data.dst_mkey = gcm_data.src_mkey; + } gcm_data.dst_bytes = gcm_data.src_bytes; /* Digest should follow payload. */ - MLX5_ASSERT(RTE_PTR_ADD - (gcm_data.src_addr, sess->aad_len + op->sym->aead.data.length) == - op->sym->aead.digest.data); - if (sess->op_type == MLX5_CRYPTO_OP_TYPE_ENCRYPTION) + if (sess->op_type == MLX5_CRYPTO_OP_TYPE_ENCRYPTION) { + MLX5_ASSERT(RTE_PTR_ADD(gcm_data.dst_addr, + sess->aad_len + op->sym->aead.data.length) == + op->sym->aead.digest.data); gcm_data.dst_bytes += sess->tag_len; - else + } else { + MLX5_ASSERT(RTE_PTR_ADD(gcm_data.src_addr, + sess->aad_len + op->sym->aead.data.length) == + op->sym->aead.digest.data); gcm_data.src_bytes += sess->tag_len; + } mlx5_crypto_gcm_wqe_set(qp, op, idx, &gcm_data); /* * All the data such as IV have been copied above, @@ -1080,6 +1094,7 @@ mlx5_crypto_gcm_restore_ipsec_mem(struct mlx5_crypto_qp *qp, struct mlx5_crypto_session *sess; struct rte_crypto_op *op; struct rte_mbuf *m_src; + struct rte_mbuf *m_dst; uint8_t *payload; while (orci != rci) { @@ -1095,6 +1110,16 @@ mlx5_crypto_gcm_restore_ipsec_mem(struct mlx5_crypto_qp *qp, RTE_PTR_SUB(payload, sess->aad_len), sess->aad_len); rte_memcpy(RTE_PTR_SUB(payload, MLX5_CRYPTO_GCM_IPSEC_IV_SIZE), &qp->ipsec_mem[idx], MLX5_CRYPTO_GCM_IPSEC_IV_SIZE); + m_dst = op->sym->m_dst; + if (m_dst && m_dst != m_src) { + uint32_t bytes_to_copy; + + bytes_to_copy = RTE_PTR_DIFF(payload, op->sym->aead.aad.data); + rte_memcpy(RTE_PTR_SUB(rte_pktmbuf_mtod_offset(m_dst, void *, + op->sym->aead.data.offset), bytes_to_copy), + op->sym->aead.aad.data, + bytes_to_copy); + } orci++; } } -- 2.34.1