From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id E8875A0C55; Wed, 13 Oct 2021 20:32:27 +0200 (CEST) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id B051B411A6; Wed, 13 Oct 2021 20:32:02 +0200 (CEST) Received: from EUR03-DB5-obe.outbound.protection.outlook.com (mail-eopbgr40056.outbound.protection.outlook.com [40.107.4.56]) by mails.dpdk.org (Postfix) with ESMTP id 43E66411DB for ; Wed, 13 Oct 2021 20:32:01 +0200 (CEST) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=OO3telnEyWY2sAs40w3IqFPYzLo6g18SSM0HkJtlGiwMAyRQcBCZgJHBo8UgEJlaWpbrx1mgBEqA7fj5jUKmmGZ8US7kq/Z1NXrwOSEeQ83wub7QlS8WXJ9Ichsh+RGZTPK0CGlLrRY6Rw2rTDKr/QEGLpbqVx5TKUIycAckQLlc/LSjS0tcVd6W43sOS/F8HB0dTb3tyrA5cwhWgUo8lwfuAMYRs1DKfL71hsV1ZAvbAEX8iBKfrW13O+fBtA4xz6a/FEWdXLp3u/vTLE/yT8vp98oDLMu2yUy6gDUjVsIJpQfR4g/I8AnOhtp+WD/H7wh30SIUNtsENrq53rnIrA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=rapCdkjhzgAgbmaJQ/UcFEIO7eCvfLiseZlb/up/OI8=; b=J8afWgazakzyrwcn+SZN6aFS1BHTZYXG5HMIoYsZSKasvMCGrRbZvMQpfAHTKkqnzYR2zJflv2bNfWh4J0tHi1ooR6TVy3TXhIoi5XSO3rn3aLK3cBpeNmzwn1d1OkxTQoMjqEB3zjSH1aIZPma9xIqQE7RYCkZQwf9MY83/iWzHCn4VZ/+HG2ZHZhINMzPxOToFku/LebgJmC9Nw/SHykkvsoOq0ueGwjGgAohsSOVHi5wpHfB/cNR0Bf21MFiCi/V4v+dK3CSpwLnLxnoyjXq3o+ECKetK4dDxdgxCAhkKmKv44RVHoxmOqVuBgMr7RlXrCOboMG5O6tvy1mykBg== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=nxp.com; dmarc=pass action=none header.from=nxp.com; dkim=pass header.d=nxp.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=nxp.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=rapCdkjhzgAgbmaJQ/UcFEIO7eCvfLiseZlb/up/OI8=; b=KSP230jBp0TvVetkRGogt9/zw4Wkj90pQwmUwKhagPtv9SfNjE6UPk6uZiN8Y+ORiqyNT2ahMmlKuIe0jA9JQauJMVpD5NpMg1kVwg8UT4Ne6Swp97w9GsW+lrz1fG64UsGFlxL45zPG4+58JSPJ7rJhTm8XcSHTc7RrkYNVjm0= Authentication-Results: dpdk.org; dkim=none (message not signed) header.d=none;dpdk.org; dmarc=none action=none header.from=nxp.com; Received: from DU2PR04MB8630.eurprd04.prod.outlook.com (2603:10a6:10:2dd::15) by DU2PR04MB8597.eurprd04.prod.outlook.com (2603:10a6:10:2d8::8) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4608.16; Wed, 13 Oct 2021 18:32:00 +0000 Received: from DU2PR04MB8630.eurprd04.prod.outlook.com ([fe80::945d:e362:712d:1b80]) by DU2PR04MB8630.eurprd04.prod.outlook.com ([fe80::945d:e362:712d:1b80%3]) with mapi id 15.20.4587.026; Wed, 13 Oct 2021 18:32:00 +0000 From: Hemant Agrawal To: dev@dpdk.org, gakhil@marvell.com Cc: roy.fan.zhang@intel.com, konstantin.ananyev@intel.com, Gagandeep Singh Date: Wed, 13 Oct 2021 23:57:10 +0530 Message-Id: <20211013182720.32486-6-hemant.agrawal@nxp.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20211013182720.32486-1-hemant.agrawal@nxp.com> References: <20211013182720.32486-1-hemant.agrawal@nxp.com> Content-Type: text/plain X-ClientProxiedBy: SG3P274CA0016.SGPP274.PROD.OUTLOOK.COM (2603:1096:4:be::28) To DU2PR04MB8630.eurprd04.prod.outlook.com (2603:10a6:10:2dd::15) MIME-Version: 1.0 Received: from dpdk-xeon.ap.freescale.net (92.120.0.67) by SG3P274CA0016.SGPP274.PROD.OUTLOOK.COM (2603:1096:4:be::28) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4608.15 via Frontend Transport; Wed, 13 Oct 2021 18:31:58 +0000 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: 4f3d48d0-fd57-45ac-d0db-08d98e77be78 X-MS-TrafficTypeDiagnostic: DU2PR04MB8597: X-MS-Exchange-Transport-Forked: True X-Microsoft-Antispam-PRVS: X-MS-Oob-TLC-OOBClassifiers: OLM:4941; X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: 44/2dEANIbUxAfLz6RT8pNwiqScxB8/6E9V+8giK2igjbgvl4uxrQewCOIqZ914bjxUduidA1mEUEp+x6sV3+76OuQS/kCd9ciCuKXpiXZFR6xCkZSYE1gryHRw+ELwrbHxpUbFMjDi0RAbR56xg/bU/aRMAIFW/XL6T2e5AA4zRMkVM3E8jKWvAhKKEuuQ7Lo39W1RnEU7hr0WK22UU1Ogz4LpHl0zIdpL79M2pZFARTzZapBiMHSJnNqR14vIRLXW2z+VnoPhmFL6RvTqb08on28JAtZACZDhvVzj5guh+FH9cAABmEP6FP+uyWKQ3iWkwtFewz82MSGu9MDtiCVi0iGW+nhjnAQvIcgGBjdXgdzUGCaFc9RdVYsyF6EXGK9/s9jvxgyvzIUdue9qHgwi5TjGwCZ8FnwfubNfmhCL/f5nmELkXW0GBp7VgtqcABYM+iplTNDw4XSD0n68uBE3fAsPUnDAVs4rk+tv8+kOxVNrjr7cJdwMk7VldSfWgrx3dLjw1S3+m0OmMpP56UKKDS/sJ1DSzawCBVFlME0+QY/N190id7ozI35VXhP0m41DtLSQqOJiCN9OD77FZoBYTZFcVKZcEmT5hrK0BF2bSGdjAOyhEGWUPZu3uT2cM7PHALM67tlFwsTXKunL0YYKH8rQ8P9GlgjAmttHX3ChrarOcrJwBkWErIvfeQEixS8iK06Syd4j4WkPQVXEJ1g== X-Forefront-Antispam-Report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:DU2PR04MB8630.eurprd04.prod.outlook.com; PTR:; CAT:NONE; SFS:(4636009)(366004)(1076003)(30864003)(2616005)(956004)(66556008)(6512007)(2906002)(83380400001)(5660300002)(508600001)(6666004)(44832011)(36756003)(4326008)(86362001)(6486002)(38350700002)(186003)(6506007)(52116002)(8936002)(38100700002)(26005)(66946007)(8676002)(316002)(66476007); DIR:OUT; SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?us-ascii?Q?KcV2i19PVlctvW7T3z8QMh6jxA02M6ib9/juANqggSuwd0gSVqpt0Hu7G2B+?= =?us-ascii?Q?KhMV65rZk0V14TPN3NVjAnYH0OSrpmAS7u/Ba3RL1PTYSzhbcCHlb2YZtgvw?= =?us-ascii?Q?Z3ENuLrsha76+qT9sQ8P8dumYcoybf085PNznEuPfUVM8Bpqa97ahsdIrNsW?= =?us-ascii?Q?s+O+AssnSADCtaH19saN+MxwoEtdQB3lX93m11GV7ghJ/moWgphFiheJK4M8?= =?us-ascii?Q?Vg7AR9Z0TXKWVByNAeaxWU5q6EXMjofbtu8eZRojUUkNOaWXpeHbRPf5d1iv?= =?us-ascii?Q?KWMYmtiLv/6TjoD0fCCCSMuRpPGFAXGa7/Oa9fijdMHsx3xuZdH3D8aDiVaP?= =?us-ascii?Q?jz4G4To88xXMLs7J/A9RtsuO/wqnL8BxMw88nW0ZLZvDremKGIzmEhKUXAvG?= =?us-ascii?Q?kmZGzGa5kqMlExH/1DoW83jDi9PTqbkzR7+BevIwkcgv7++YW+gjnkohH0ok?= =?us-ascii?Q?hrgjK1Tn90tmiAOpSZOcEYBID2qEh4AnYAQE5787CbJcdL2Q4OUav1pN/DK+?= =?us-ascii?Q?fkhm0KwWE6z+ddLCCNvQ/+J+BN2ErX0GEpLFvPXB8NhUVRaLvtObpCKctgdG?= =?us-ascii?Q?x1tKmII5mTXSzGNMSxwcZCVv5DxUlQ7Cv+vIfQO3T3Jxy1PIinBtcV7Ph9kw?= =?us-ascii?Q?a4PRTCvM3tqK02AizXATQCdwSrmBOyRzpGKt2wqolcLBPrW6YoYNHvRhzJjx?= =?us-ascii?Q?kBT5RJOSGOxxfhC0/qDW6/X9i8kDl5Oj0253vhMhJOQcghjxHpETduFxXv7X?= =?us-ascii?Q?MUIK36hv43kRuZwNPDPe0J1JgU24yFHddjEHs9NDqZ/iE4P+FtnGS8W8ePiW?= =?us-ascii?Q?WYxH+9/MppQkEQrKgt24WHQcX6gB9biv9TxtfrXDlED5FpiNoeAaJxmG7j33?= =?us-ascii?Q?audl78upzPqhgwa2nwOCPSDjQNWR3h4UuBprGdA5sm71dzz0GHDzbzW/zRpd?= =?us-ascii?Q?g3iNugTlI63EUDG6Z6SZTGmOvyUawvp7OYTmnrVRgD/gWwVh3UNXnMCP3xu6?= =?us-ascii?Q?12uXwi694at1zs0ic4Jlfcbz8fb1TOyGE/OE8fmKEN43zBpEYUUpVLVZnb7g?= =?us-ascii?Q?QiYKmRDX9K1Cdyxl8n3+lxTq47TxN7w8YoQ2psudodLm2Ni41qqrW9Fu37wV?= =?us-ascii?Q?X0bfD4KBi+D2LMkh85wYmNSclcbw5UtNzApV2yghcV4+06AGYqSCR8U/6oh8?= =?us-ascii?Q?liebaxPceseYiBGsLzJaoqwzXshAYAqRZazsULd2Tz8ARZdne9gh27a2p/+u?= =?us-ascii?Q?Jp5QPHmx1ErOhslRZo2tVeyCY9meALBz0ezQ4MqSnzMpKTvwaW3AEQH5qzfX?= =?us-ascii?Q?phmlED1ddbOB6kEwoLX4afx9?= X-OriginatorOrg: nxp.com X-MS-Exchange-CrossTenant-Network-Message-Id: 4f3d48d0-fd57-45ac-d0db-08d98e77be78 X-MS-Exchange-CrossTenant-AuthSource: DU2PR04MB8630.eurprd04.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 13 Oct 2021 18:32:00.0593 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 686ea1d3-bc2b-4c6f-a92c-d99c5c301635 X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: BhTIvHCRnobzHEWe7z7YCQ6nQWPFayLM/k5I7L/dUp02co8tEojGXE9v3mP7CqX6HQ2aTNkl3EfKjt8p9QbgAA== X-MS-Exchange-Transport-CrossTenantHeadersStamped: DU2PR04MB8597 Subject: [dpdk-dev] [PATCH v3 05/15] crypto/dpaa2_sec: support raw datapath APIs X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" From: Gagandeep Singh This path add framework for raw API support. The initial patch only test cipher only part. Signed-off-by: Hemant Agrawal Signed-off-by: Gagandeep Singh --- doc/guides/rel_notes/release_21_11.rst | 1 + drivers/crypto/dpaa2_sec/dpaa2_sec_dpseci.c | 13 +- drivers/crypto/dpaa2_sec/dpaa2_sec_priv.h | 60 +- drivers/crypto/dpaa2_sec/dpaa2_sec_raw_dp.c | 595 ++++++++++++++++++++ drivers/crypto/dpaa2_sec/meson.build | 3 +- 5 files changed, 643 insertions(+), 29 deletions(-) create mode 100644 drivers/crypto/dpaa2_sec/dpaa2_sec_raw_dp.c diff --git a/doc/guides/rel_notes/release_21_11.rst b/doc/guides/rel_notes/release_21_11.rst index a8900a3079..b1049a92e3 100644 --- a/doc/guides/rel_notes/release_21_11.rst +++ b/doc/guides/rel_notes/release_21_11.rst @@ -106,6 +106,7 @@ New Features * **Updated NXP dpaa2_sec crypto PMD.** * Added PDCP short MAC-I support. + * Added raw vector datapath API support * **Updated the turbo_sw bbdev PMD.** diff --git a/drivers/crypto/dpaa2_sec/dpaa2_sec_dpseci.c b/drivers/crypto/dpaa2_sec/dpaa2_sec_dpseci.c index dfa72f3f93..4eb3615250 100644 --- a/drivers/crypto/dpaa2_sec/dpaa2_sec_dpseci.c +++ b/drivers/crypto/dpaa2_sec/dpaa2_sec_dpseci.c @@ -49,15 +49,8 @@ #define FSL_MC_DPSECI_DEVID 3 #define NO_PREFETCH 0 -/* FLE_POOL_NUM_BUFS is set as per the ipsec-secgw application */ -#define FLE_POOL_NUM_BUFS 32000 -#define FLE_POOL_BUF_SIZE 256 -#define FLE_POOL_CACHE_SIZE 512 -#define FLE_SG_MEM_SIZE(num) (FLE_POOL_BUF_SIZE + ((num) * 32)) -#define SEC_FLC_DHR_OUTBOUND -114 -#define SEC_FLC_DHR_INBOUND 0 -static uint8_t cryptodev_driver_id; +uint8_t cryptodev_driver_id; #ifdef RTE_LIB_SECURITY static inline int @@ -3828,6 +3821,9 @@ static struct rte_cryptodev_ops crypto_ops = { .sym_session_get_size = dpaa2_sec_sym_session_get_size, .sym_session_configure = dpaa2_sec_sym_session_configure, .sym_session_clear = dpaa2_sec_sym_session_clear, + /* Raw data-path API related operations */ + .sym_get_raw_dp_ctx_size = dpaa2_sec_get_dp_ctx_size, + .sym_configure_raw_dp_ctx = dpaa2_sec_configure_raw_dp_ctx, }; #ifdef RTE_LIB_SECURITY @@ -3910,6 +3906,7 @@ dpaa2_sec_dev_init(struct rte_cryptodev *cryptodev) RTE_CRYPTODEV_FF_HW_ACCELERATED | RTE_CRYPTODEV_FF_SYM_OPERATION_CHAINING | RTE_CRYPTODEV_FF_SECURITY | + RTE_CRYPTODEV_FF_SYM_RAW_DP | RTE_CRYPTODEV_FF_IN_PLACE_SGL | RTE_CRYPTODEV_FF_OOP_SGL_IN_SGL_OUT | RTE_CRYPTODEV_FF_OOP_SGL_IN_LB_OUT | diff --git a/drivers/crypto/dpaa2_sec/dpaa2_sec_priv.h b/drivers/crypto/dpaa2_sec/dpaa2_sec_priv.h index 8dee0a4bda..e9b888186e 100644 --- a/drivers/crypto/dpaa2_sec/dpaa2_sec_priv.h +++ b/drivers/crypto/dpaa2_sec/dpaa2_sec_priv.h @@ -15,6 +15,16 @@ #define CRYPTODEV_NAME_DPAA2_SEC_PMD crypto_dpaa2_sec /**< NXP DPAA2 - SEC PMD device name */ +extern uint8_t cryptodev_driver_id; + +/* FLE_POOL_NUM_BUFS is set as per the ipsec-secgw application */ +#define FLE_POOL_NUM_BUFS 32000 +#define FLE_POOL_BUF_SIZE 256 +#define FLE_POOL_CACHE_SIZE 512 +#define FLE_SG_MEM_SIZE(num) (FLE_POOL_BUF_SIZE + ((num) * 32)) +#define SEC_FLC_DHR_OUTBOUND -114 +#define SEC_FLC_DHR_INBOUND 0 + #define MAX_QUEUES 64 #define MAX_DESC_SIZE 64 /** private data structure for each DPAA2_SEC device */ @@ -158,6 +168,24 @@ struct dpaa2_pdcp_ctxt { uint32_t hfn_threshold; /*!< HFN Threashold for key renegotiation */ }; #endif + +typedef int (*dpaa2_sec_build_fd_t)( + void *qp, uint8_t *drv_ctx, struct rte_crypto_vec *data_vec, + uint16_t n_data_vecs, union rte_crypto_sym_ofs ofs, + struct rte_crypto_va_iova_ptr *iv, + struct rte_crypto_va_iova_ptr *digest, + struct rte_crypto_va_iova_ptr *aad_or_auth_iv, + void *user_data); + +typedef int (*dpaa2_sec_build_raw_dp_fd_t)(uint8_t *drv_ctx, + struct rte_crypto_sgl *sgl, + struct rte_crypto_va_iova_ptr *iv, + struct rte_crypto_va_iova_ptr *digest, + struct rte_crypto_va_iova_ptr *auth_iv, + union rte_crypto_sym_ofs ofs, + void *userdata, + struct qbman_fd *fd); + typedef struct dpaa2_sec_session_entry { void *ctxt; uint8_t ctxt_type; @@ -165,6 +193,8 @@ typedef struct dpaa2_sec_session_entry { enum rte_crypto_cipher_algorithm cipher_alg; /*!< Cipher Algorithm*/ enum rte_crypto_auth_algorithm auth_alg; /*!< Authentication Algorithm*/ enum rte_crypto_aead_algorithm aead_alg; /*!< AEAD Algorithm*/ + dpaa2_sec_build_fd_t build_fd; + dpaa2_sec_build_raw_dp_fd_t build_raw_dp_fd; union { struct { uint8_t *data; /**< pointer to key data */ @@ -547,26 +577,6 @@ static const struct rte_cryptodev_capabilities dpaa2_sec_capabilities[] = { }, } }, } }, - { /* NULL (CIPHER) */ - .op = RTE_CRYPTO_OP_TYPE_SYMMETRIC, - {.sym = { - .xform_type = RTE_CRYPTO_SYM_XFORM_CIPHER, - {.cipher = { - .algo = RTE_CRYPTO_CIPHER_NULL, - .block_size = 1, - .key_size = { - .min = 0, - .max = 0, - .increment = 0 - }, - .iv_size = { - .min = 0, - .max = 0, - .increment = 0 - } - }, }, - }, } - }, { /* AES CBC */ .op = RTE_CRYPTO_OP_TYPE_SYMMETRIC, {.sym = { @@ -983,4 +993,14 @@ calc_chksum(void *buffer, int len) return result; } +int +dpaa2_sec_configure_raw_dp_ctx(struct rte_cryptodev *dev, uint16_t qp_id, + struct rte_crypto_raw_dp_ctx *raw_dp_ctx, + enum rte_crypto_op_sess_type sess_type, + union rte_cryptodev_session_ctx session_ctx, uint8_t is_update); + +int +dpaa2_sec_get_dp_ctx_size(struct rte_cryptodev *dev); + + #endif /* _DPAA2_SEC_PMD_PRIVATE_H_ */ diff --git a/drivers/crypto/dpaa2_sec/dpaa2_sec_raw_dp.c b/drivers/crypto/dpaa2_sec/dpaa2_sec_raw_dp.c new file mode 100644 index 0000000000..8925c8e938 --- /dev/null +++ b/drivers/crypto/dpaa2_sec/dpaa2_sec_raw_dp.c @@ -0,0 +1,595 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright 2021 NXP + */ + +#include +#include +#include +#include +#include + +#include "dpaa2_sec_priv.h" +#include "dpaa2_sec_logs.h" + +struct dpaa2_sec_raw_dp_ctx { + dpaa2_sec_session *session; + uint32_t tail; + uint32_t head; + uint16_t cached_enqueue; + uint16_t cached_dequeue; +}; + +static int +build_raw_dp_chain_fd(uint8_t *drv_ctx, + struct rte_crypto_sgl *sgl, + struct rte_crypto_va_iova_ptr *iv, + struct rte_crypto_va_iova_ptr *digest, + struct rte_crypto_va_iova_ptr *auth_iv, + union rte_crypto_sym_ofs ofs, + void *userdata, + struct qbman_fd *fd) +{ + RTE_SET_USED(drv_ctx); + RTE_SET_USED(sgl); + RTE_SET_USED(iv); + RTE_SET_USED(digest); + RTE_SET_USED(auth_iv); + RTE_SET_USED(ofs); + RTE_SET_USED(userdata); + RTE_SET_USED(fd); + + return 0; +} + +static int +build_raw_dp_aead_fd(uint8_t *drv_ctx, + struct rte_crypto_sgl *sgl, + struct rte_crypto_va_iova_ptr *iv, + struct rte_crypto_va_iova_ptr *digest, + struct rte_crypto_va_iova_ptr *auth_iv, + union rte_crypto_sym_ofs ofs, + void *userdata, + struct qbman_fd *fd) +{ + RTE_SET_USED(drv_ctx); + RTE_SET_USED(sgl); + RTE_SET_USED(iv); + RTE_SET_USED(digest); + RTE_SET_USED(auth_iv); + RTE_SET_USED(ofs); + RTE_SET_USED(userdata); + RTE_SET_USED(fd); + + return 0; +} + +static int +build_raw_dp_auth_fd(uint8_t *drv_ctx, + struct rte_crypto_sgl *sgl, + struct rte_crypto_va_iova_ptr *iv, + struct rte_crypto_va_iova_ptr *digest, + struct rte_crypto_va_iova_ptr *auth_iv, + union rte_crypto_sym_ofs ofs, + void *userdata, + struct qbman_fd *fd) +{ + RTE_SET_USED(drv_ctx); + RTE_SET_USED(sgl); + RTE_SET_USED(iv); + RTE_SET_USED(digest); + RTE_SET_USED(auth_iv); + RTE_SET_USED(ofs); + RTE_SET_USED(userdata); + RTE_SET_USED(fd); + + return 0; +} + +static int +build_raw_dp_proto_fd(uint8_t *drv_ctx, + struct rte_crypto_sgl *sgl, + struct rte_crypto_va_iova_ptr *iv, + struct rte_crypto_va_iova_ptr *digest, + struct rte_crypto_va_iova_ptr *auth_iv, + union rte_crypto_sym_ofs ofs, + void *userdata, + struct qbman_fd *fd) +{ + RTE_SET_USED(drv_ctx); + RTE_SET_USED(sgl); + RTE_SET_USED(iv); + RTE_SET_USED(digest); + RTE_SET_USED(auth_iv); + RTE_SET_USED(ofs); + RTE_SET_USED(userdata); + RTE_SET_USED(fd); + + return 0; +} + +static int +build_raw_dp_proto_compound_fd(uint8_t *drv_ctx, + struct rte_crypto_sgl *sgl, + struct rte_crypto_va_iova_ptr *iv, + struct rte_crypto_va_iova_ptr *digest, + struct rte_crypto_va_iova_ptr *auth_iv, + union rte_crypto_sym_ofs ofs, + void *userdata, + struct qbman_fd *fd) +{ + RTE_SET_USED(drv_ctx); + RTE_SET_USED(sgl); + RTE_SET_USED(iv); + RTE_SET_USED(digest); + RTE_SET_USED(auth_iv); + RTE_SET_USED(ofs); + RTE_SET_USED(userdata); + RTE_SET_USED(fd); + + return 0; +} + +static int +build_raw_dp_cipher_fd(uint8_t *drv_ctx, + struct rte_crypto_sgl *sgl, + struct rte_crypto_va_iova_ptr *iv, + struct rte_crypto_va_iova_ptr *digest, + struct rte_crypto_va_iova_ptr *auth_iv, + union rte_crypto_sym_ofs ofs, + void *userdata, + struct qbman_fd *fd) +{ + RTE_SET_USED(digest); + RTE_SET_USED(auth_iv); + + dpaa2_sec_session *sess = + ((struct dpaa2_sec_raw_dp_ctx *)drv_ctx)->session; + struct qbman_fle *ip_fle, *op_fle, *sge, *fle; + int total_len = 0, data_len = 0, data_offset; + struct sec_flow_context *flc; + struct ctxt_priv *priv = sess->ctxt; + unsigned int i; + + for (i = 0; i < sgl->num; i++) + total_len += sgl->vec[i].len; + + data_len = total_len - ofs.ofs.cipher.head - ofs.ofs.cipher.tail; + data_offset = ofs.ofs.cipher.head; + + if (sess->cipher_alg == RTE_CRYPTO_CIPHER_SNOW3G_UEA2 || + sess->cipher_alg == RTE_CRYPTO_CIPHER_ZUC_EEA3) { + if ((data_len & 7) || (data_offset & 7)) { + DPAA2_SEC_ERR("CIPHER: len/offset must be full bytes"); + return -ENOTSUP; + } + + data_len = data_len >> 3; + data_offset = data_offset >> 3; + } + + /* first FLE entry used to store mbuf and session ctxt */ + fle = (struct qbman_fle *)rte_malloc(NULL, + FLE_SG_MEM_SIZE(2*sgl->num), + RTE_CACHE_LINE_SIZE); + if (!fle) { + DPAA2_SEC_ERR("RAW CIPHER SG: Memory alloc failed for SGE"); + return -ENOMEM; + } + memset(fle, 0, FLE_SG_MEM_SIZE(2*sgl->num)); + /* first FLE entry used to store userdata and session ctxt */ + DPAA2_SET_FLE_ADDR(fle, (size_t)userdata); + DPAA2_FLE_SAVE_CTXT(fle, (ptrdiff_t)priv); + + op_fle = fle + 1; + ip_fle = fle + 2; + sge = fle + 3; + + flc = &priv->flc_desc[0].flc; + + DPAA2_SEC_DP_DEBUG( + "RAW CIPHER SG: cipher_off: 0x%x/length %d, ivlen=%d\n", + data_offset, + data_len, + sess->iv.length); + + /* o/p fle */ + DPAA2_SET_FLE_ADDR(op_fle, DPAA2_VADDR_TO_IOVA(sge)); + op_fle->length = data_len; + DPAA2_SET_FLE_SG_EXT(op_fle); + + /* o/p 1st seg */ + DPAA2_SET_FLE_ADDR(sge, sgl->vec[0].iova); + DPAA2_SET_FLE_OFFSET(sge, data_offset); + sge->length = sgl->vec[0].len - data_offset; + + /* o/p segs */ + for (i = 1; i < sgl->num; i++) { + sge++; + DPAA2_SET_FLE_ADDR(sge, sgl->vec[i].iova); + DPAA2_SET_FLE_OFFSET(sge, 0); + sge->length = sgl->vec[i].len; + } + DPAA2_SET_FLE_FIN(sge); + + DPAA2_SEC_DP_DEBUG( + "RAW CIPHER SG: 1 - flc = %p, fle = %p FLEaddr = %x-%x, len %d\n", + flc, fle, fle->addr_hi, fle->addr_lo, + fle->length); + + /* i/p fle */ + sge++; + DPAA2_SET_FLE_ADDR(ip_fle, DPAA2_VADDR_TO_IOVA(sge)); + ip_fle->length = sess->iv.length + data_len; + DPAA2_SET_FLE_SG_EXT(ip_fle); + + /* i/p IV */ + DPAA2_SET_FLE_ADDR(sge, iv->iova); + DPAA2_SET_FLE_OFFSET(sge, 0); + sge->length = sess->iv.length; + + sge++; + + /* i/p 1st seg */ + DPAA2_SET_FLE_ADDR(sge, sgl->vec[0].iova); + DPAA2_SET_FLE_OFFSET(sge, data_offset); + sge->length = sgl->vec[0].len - data_offset; + + /* i/p segs */ + for (i = 1; i < sgl->num; i++) { + sge++; + DPAA2_SET_FLE_ADDR(sge, sgl->vec[i].iova); + DPAA2_SET_FLE_OFFSET(sge, 0); + sge->length = sgl->vec[i].len; + } + DPAA2_SET_FLE_FIN(sge); + DPAA2_SET_FLE_FIN(ip_fle); + + /* sg fd */ + DPAA2_SET_FD_ADDR(fd, DPAA2_VADDR_TO_IOVA(op_fle)); + DPAA2_SET_FD_LEN(fd, ip_fle->length); + DPAA2_SET_FD_COMPOUND_FMT(fd); + DPAA2_SET_FD_FLC(fd, DPAA2_VADDR_TO_IOVA(flc)); + + DPAA2_SEC_DP_DEBUG( + "RAW CIPHER SG: fdaddr =%" PRIx64 " off =%d, len =%d\n", + DPAA2_GET_FD_ADDR(fd), + DPAA2_GET_FD_OFFSET(fd), + DPAA2_GET_FD_LEN(fd)); + + return 0; +} + +static __rte_always_inline uint32_t +dpaa2_sec_raw_enqueue_burst(void *qp_data, uint8_t *drv_ctx, + struct rte_crypto_sym_vec *vec, union rte_crypto_sym_ofs ofs, + void *user_data[], int *status) +{ + RTE_SET_USED(user_data); + uint32_t loop; + int32_t ret; + struct qbman_fd fd_arr[MAX_TX_RING_SLOTS]; + uint32_t frames_to_send, retry_count; + struct qbman_eq_desc eqdesc; + struct dpaa2_sec_qp *dpaa2_qp = (struct dpaa2_sec_qp *)qp_data; + dpaa2_sec_session *sess = + ((struct dpaa2_sec_raw_dp_ctx *)drv_ctx)->session; + struct qbman_swp *swp; + uint16_t num_tx = 0; + uint32_t flags[MAX_TX_RING_SLOTS] = {0}; + + if (unlikely(vec->num == 0)) + return 0; + + if (sess == NULL) { + DPAA2_SEC_ERR("sessionless raw crypto not supported"); + return 0; + } + /*Prepare enqueue descriptor*/ + qbman_eq_desc_clear(&eqdesc); + qbman_eq_desc_set_no_orp(&eqdesc, DPAA2_EQ_RESP_ERR_FQ); + qbman_eq_desc_set_response(&eqdesc, 0, 0); + qbman_eq_desc_set_fq(&eqdesc, dpaa2_qp->tx_vq.fqid); + + if (!DPAA2_PER_LCORE_DPIO) { + ret = dpaa2_affine_qbman_swp(); + if (ret) { + DPAA2_SEC_ERR( + "Failed to allocate IO portal, tid: %d\n", + rte_gettid()); + return 0; + } + } + swp = DPAA2_PER_LCORE_PORTAL; + + while (vec->num) { + frames_to_send = (vec->num > dpaa2_eqcr_size) ? + dpaa2_eqcr_size : vec->num; + + for (loop = 0; loop < frames_to_send; loop++) { + /*Clear the unused FD fields before sending*/ + memset(&fd_arr[loop], 0, sizeof(struct qbman_fd)); + ret = sess->build_raw_dp_fd(drv_ctx, + &vec->src_sgl[loop], + &vec->iv[loop], + &vec->digest[loop], + &vec->auth_iv[loop], + ofs, + user_data[loop], + &fd_arr[loop]); + if (ret) { + DPAA2_SEC_ERR("error: Improper packet contents" + " for crypto operation"); + goto skip_tx; + } + status[loop] = 1; + } + + loop = 0; + retry_count = 0; + while (loop < frames_to_send) { + ret = qbman_swp_enqueue_multiple(swp, &eqdesc, + &fd_arr[loop], + &flags[loop], + frames_to_send - loop); + if (unlikely(ret < 0)) { + retry_count++; + if (retry_count > DPAA2_MAX_TX_RETRY_COUNT) { + num_tx += loop; + vec->num -= loop; + goto skip_tx; + } + } else { + loop += ret; + retry_count = 0; + } + } + + num_tx += loop; + vec->num -= loop; + } +skip_tx: + dpaa2_qp->tx_vq.tx_pkts += num_tx; + dpaa2_qp->tx_vq.err_pkts += vec->num; + + return num_tx; +} + +static __rte_always_inline int +dpaa2_sec_raw_enqueue(void *qp_data, uint8_t *drv_ctx, + struct rte_crypto_vec *data_vec, + uint16_t n_data_vecs, union rte_crypto_sym_ofs ofs, + struct rte_crypto_va_iova_ptr *iv, + struct rte_crypto_va_iova_ptr *digest, + struct rte_crypto_va_iova_ptr *aad_or_auth_iv, + void *user_data) +{ + RTE_SET_USED(qp_data); + RTE_SET_USED(drv_ctx); + RTE_SET_USED(data_vec); + RTE_SET_USED(n_data_vecs); + RTE_SET_USED(ofs); + RTE_SET_USED(iv); + RTE_SET_USED(digest); + RTE_SET_USED(aad_or_auth_iv); + RTE_SET_USED(user_data); + + return 0; +} + +static inline void * +sec_fd_to_userdata(const struct qbman_fd *fd) +{ + struct qbman_fle *fle; + void *userdata; + fle = (struct qbman_fle *)DPAA2_IOVA_TO_VADDR(DPAA2_GET_FD_ADDR(fd)); + + DPAA2_SEC_DP_DEBUG("FLE addr = %x - %x, offset = %x\n", + fle->addr_hi, fle->addr_lo, fle->fin_bpid_offset); + userdata = (struct rte_crypto_op *)DPAA2_GET_FLE_ADDR((fle - 1)); + /* free the fle memory */ + rte_free((void *)(fle-1)); + + return userdata; +} + +static __rte_always_inline uint32_t +dpaa2_sec_raw_dequeue_burst(void *qp_data, uint8_t *drv_ctx, + rte_cryptodev_raw_get_dequeue_count_t get_dequeue_count, + uint32_t max_nb_to_dequeue, + rte_cryptodev_raw_post_dequeue_t post_dequeue, + void **out_user_data, uint8_t is_user_data_array, + uint32_t *n_success, int *dequeue_status) +{ + RTE_SET_USED(drv_ctx); + RTE_SET_USED(get_dequeue_count); + + /* Function is responsible to receive frames for a given device and VQ*/ + struct dpaa2_sec_qp *dpaa2_qp = (struct dpaa2_sec_qp *)qp_data; + struct qbman_result *dq_storage; + uint32_t fqid = dpaa2_qp->rx_vq.fqid; + int ret, num_rx = 0; + uint8_t is_last = 0, status; + struct qbman_swp *swp; + const struct qbman_fd *fd; + struct qbman_pull_desc pulldesc; + void *user_data; + uint32_t nb_ops = max_nb_to_dequeue; + + if (!DPAA2_PER_LCORE_DPIO) { + ret = dpaa2_affine_qbman_swp(); + if (ret) { + DPAA2_SEC_ERR( + "Failed to allocate IO portal, tid: %d\n", + rte_gettid()); + return 0; + } + } + swp = DPAA2_PER_LCORE_PORTAL; + dq_storage = dpaa2_qp->rx_vq.q_storage->dq_storage[0]; + + qbman_pull_desc_clear(&pulldesc); + qbman_pull_desc_set_numframes(&pulldesc, + (nb_ops > dpaa2_dqrr_size) ? + dpaa2_dqrr_size : nb_ops); + qbman_pull_desc_set_fq(&pulldesc, fqid); + qbman_pull_desc_set_storage(&pulldesc, dq_storage, + (uint64_t)DPAA2_VADDR_TO_IOVA(dq_storage), + 1); + + /*Issue a volatile dequeue command. */ + while (1) { + if (qbman_swp_pull(swp, &pulldesc)) { + DPAA2_SEC_WARN( + "SEC VDQ command is not issued : QBMAN busy"); + /* Portal was busy, try again */ + continue; + } + break; + }; + + /* Receive the packets till Last Dequeue entry is found with + * respect to the above issues PULL command. + */ + while (!is_last) { + /* Check if the previous issued command is completed. + * Also seems like the SWP is shared between the Ethernet Driver + * and the SEC driver. + */ + while (!qbman_check_command_complete(dq_storage)) + ; + + /* Loop until the dq_storage is updated with + * new token by QBMAN + */ + while (!qbman_check_new_result(dq_storage)) + ; + /* Check whether Last Pull command is Expired and + * setting Condition for Loop termination + */ + if (qbman_result_DQ_is_pull_complete(dq_storage)) { + is_last = 1; + /* Check for valid frame. */ + status = (uint8_t)qbman_result_DQ_flags(dq_storage); + if (unlikely( + (status & QBMAN_DQ_STAT_VALIDFRAME) == 0)) { + DPAA2_SEC_DP_DEBUG("No frame is delivered\n"); + continue; + } + } + + fd = qbman_result_DQ_fd(dq_storage); + user_data = sec_fd_to_userdata(fd); + if (is_user_data_array) + out_user_data[num_rx] = user_data; + else + out_user_data[0] = user_data; + if (unlikely(fd->simple.frc)) { + /* TODO Parse SEC errors */ + DPAA2_SEC_ERR("SEC returned Error - %x", + fd->simple.frc); + status = RTE_CRYPTO_OP_STATUS_ERROR; + } else { + status = RTE_CRYPTO_OP_STATUS_SUCCESS; + } + post_dequeue(user_data, num_rx, status); + + num_rx++; + dq_storage++; + } /* End of Packet Rx loop */ + + dpaa2_qp->rx_vq.rx_pkts += num_rx; + *dequeue_status = 1; + *n_success = num_rx; + + DPAA2_SEC_DP_DEBUG("SEC Received %d Packets\n", num_rx); + /*Return the total number of packets received to DPAA2 app*/ + return num_rx; +} + +static __rte_always_inline void * +dpaa2_sec_raw_dequeue(void *qp_data, uint8_t *drv_ctx, int *dequeue_status, + enum rte_crypto_op_status *op_status) +{ + RTE_SET_USED(qp_data); + RTE_SET_USED(drv_ctx); + RTE_SET_USED(dequeue_status); + RTE_SET_USED(op_status); + + return NULL; +} + +static __rte_always_inline int +dpaa2_sec_raw_enqueue_done(void *qp_data, uint8_t *drv_ctx, uint32_t n) +{ + RTE_SET_USED(qp_data); + RTE_SET_USED(drv_ctx); + RTE_SET_USED(n); + + return 0; +} + +static __rte_always_inline int +dpaa2_sec_raw_dequeue_done(void *qp_data, uint8_t *drv_ctx, uint32_t n) +{ + RTE_SET_USED(qp_data); + RTE_SET_USED(drv_ctx); + RTE_SET_USED(n); + + return 0; +} + +int +dpaa2_sec_configure_raw_dp_ctx(struct rte_cryptodev *dev, uint16_t qp_id, + struct rte_crypto_raw_dp_ctx *raw_dp_ctx, + enum rte_crypto_op_sess_type sess_type, + union rte_cryptodev_session_ctx session_ctx, uint8_t is_update) +{ + dpaa2_sec_session *sess; + struct dpaa2_sec_raw_dp_ctx *dp_ctx; + RTE_SET_USED(qp_id); + + if (!is_update) { + memset(raw_dp_ctx, 0, sizeof(*raw_dp_ctx)); + raw_dp_ctx->qp_data = dev->data->queue_pairs[qp_id]; + } + + if (sess_type == RTE_CRYPTO_OP_SECURITY_SESSION) + sess = (dpaa2_sec_session *)get_sec_session_private_data( + session_ctx.sec_sess); + else if (sess_type == RTE_CRYPTO_OP_WITH_SESSION) + sess = (dpaa2_sec_session *)get_sym_session_private_data( + session_ctx.crypto_sess, cryptodev_driver_id); + else + return -ENOTSUP; + raw_dp_ctx->dequeue_burst = dpaa2_sec_raw_dequeue_burst; + raw_dp_ctx->dequeue = dpaa2_sec_raw_dequeue; + raw_dp_ctx->dequeue_done = dpaa2_sec_raw_dequeue_done; + raw_dp_ctx->enqueue_burst = dpaa2_sec_raw_enqueue_burst; + raw_dp_ctx->enqueue = dpaa2_sec_raw_enqueue; + raw_dp_ctx->enqueue_done = dpaa2_sec_raw_enqueue_done; + + if (sess->ctxt_type == DPAA2_SEC_CIPHER_HASH) + sess->build_raw_dp_fd = build_raw_dp_chain_fd; + else if (sess->ctxt_type == DPAA2_SEC_AEAD) + sess->build_raw_dp_fd = build_raw_dp_aead_fd; + else if (sess->ctxt_type == DPAA2_SEC_AUTH) + sess->build_raw_dp_fd = build_raw_dp_auth_fd; + else if (sess->ctxt_type == DPAA2_SEC_CIPHER) + sess->build_raw_dp_fd = build_raw_dp_cipher_fd; + else if (sess->ctxt_type == DPAA2_SEC_IPSEC) + sess->build_raw_dp_fd = build_raw_dp_proto_fd; + else if (sess->ctxt_type == DPAA2_SEC_PDCP) + sess->build_raw_dp_fd = build_raw_dp_proto_compound_fd; + else + return -ENOTSUP; + dp_ctx = (struct dpaa2_sec_raw_dp_ctx *)raw_dp_ctx->drv_ctx_data; + dp_ctx->session = sess; + + return 0; +} + +int +dpaa2_sec_get_dp_ctx_size(__rte_unused struct rte_cryptodev *dev) +{ + return sizeof(struct dpaa2_sec_raw_dp_ctx); +} diff --git a/drivers/crypto/dpaa2_sec/meson.build b/drivers/crypto/dpaa2_sec/meson.build index ea1d73a13d..e6e5abb3c1 100644 --- a/drivers/crypto/dpaa2_sec/meson.build +++ b/drivers/crypto/dpaa2_sec/meson.build @@ -1,5 +1,5 @@ # SPDX-License-Identifier: BSD-3-Clause -# Copyright 2018 NXP +# Copyright 2018,2021 NXP if not is_linux build = false @@ -9,6 +9,7 @@ endif deps += ['security', 'mempool_dpaa2'] sources = files( 'dpaa2_sec_dpseci.c', + 'dpaa2_sec_raw_dp.c', 'mc/dpseci.c', ) -- 2.17.1