From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 60240A0C52; Thu, 12 Aug 2021 09:16:44 +0200 (CEST) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 22FB44121C; Thu, 12 Aug 2021 09:16:17 +0200 (CEST) Received: from EUR02-VE1-obe.outbound.protection.outlook.com (mail-eopbgr20062.outbound.protection.outlook.com [40.107.2.62]) by mails.dpdk.org (Postfix) with ESMTP id 64FBD4121B for ; Thu, 12 Aug 2021 09:16:14 +0200 (CEST) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=QU34dua90PsOwIyyEkJl+tm5g7omzz3dRJqTK5bqECRJz9aRUlbuI6w0Jug3INU0a18TUa/+k9WV3K2Nf5Jk5VStmYTLsUsz0UPCfYtemLglK78CJVkIC7rwrdmxxAXyElCQzAtAlW+cJQL6FgMFl7hUh7nIA9rd3+NTv67ub/vpXlxTzOgp5PlGwUbBAvtTRjeOXoBCgReiEXBcyQcwnI3K1KhpFaTJ0iTpp3t0JivrnlWpwWXEoUiSsG5cAXaUPag161kivAoFKGC6AgmKBLYXDKv93jP5tEYNnrQGfmic5jllrYL4R9aa/B3ZasuJWEIwr+v3Jsymjq5/ILarzg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=c7Up2SY2H/NTgGiTSWGqlhXAI89ELAWr4jP+np6sgRI=; b=e2DpMeQM3NZe3mQe9eVaxsZuMUUrgFRhe26gzVs80R4nvGD/Qs+GI9XWi3p/V01q8ZBVEU0+OLp6oMNK+XGyYuFxFXBsFvzEKaFj8FhR1ZQ0J8/2/XtKfiwQp1K6eYQyw6FsmkXI0Espkn4KjnRT2ctJCGOtQtZbYviBfSGcGJ2U53cJUrVlejkyUjmA9P7w4qZxeKKiUQ40JibVBP4NHoj07AaBIXDm87gU/kDg2pbVOnFL8vahxavmshwijTcdfwxPMqZMSAf1WxpbzzRufeLOcOXoYaku9UNVav5hRHll2DO7R7h+yhYR/XihnHi8gQTSToTpxAPMunFTKMMJSA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=nxp.com; dmarc=pass action=none header.from=nxp.com; dkim=pass header.d=nxp.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=nxp.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=c7Up2SY2H/NTgGiTSWGqlhXAI89ELAWr4jP+np6sgRI=; b=dUxSYUvkRzBD28lhxycjGrNnGXDs2rjKwB9MjC8gEy7wLxy066OEswVi+3c8WeNWVFkM4MMLd1YTU61lKN8r59zofMm7Q8+6QLdvJ7/0Nss/xyrBQPCPV/A0it1q1u4Kj606omuIr3T0SX+swDs6NgDSH0v/EkEBCHAG5igfu5k= Authentication-Results: dpdk.org; dkim=none (message not signed) header.d=none;dpdk.org; dmarc=none action=none header.from=nxp.com; Received: from DU2PR04MB8630.eurprd04.prod.outlook.com (2603:10a6:10:2dd::15) by DB9PR04MB8461.eurprd04.prod.outlook.com (2603:10a6:10:2bc::7) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4415.13; Thu, 12 Aug 2021 07:16:13 +0000 Received: from DU2PR04MB8630.eurprd04.prod.outlook.com ([fe80::2590:514a:df1f:a793]) by DU2PR04MB8630.eurprd04.prod.outlook.com ([fe80::2590:514a:df1f:a793%9]) with mapi id 15.20.4415.016; Thu, 12 Aug 2021 07:16:12 +0000 From: Hemant Agrawal To: dev@dpdk.org, gakhil@marvell.com Cc: Gagandeep Singh Date: Thu, 12 Aug 2021 12:42:33 +0530 Message-Id: <20210812071244.28799-6-hemant.agrawal@nxp.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20210812071244.28799-1-hemant.agrawal@nxp.com> References: <20210812071244.28799-1-hemant.agrawal@nxp.com> Content-Type: text/plain X-ClientProxiedBy: SG2PR02CA0105.apcprd02.prod.outlook.com (2603:1096:4:92::21) To DU2PR04MB8630.eurprd04.prod.outlook.com (2603:10a6:10:2dd::15) MIME-Version: 1.0 X-MS-Exchange-MessageSentRepresentingType: 1 Received: from dpdk-xeon.ap.freescale.net (92.120.0.67) by SG2PR02CA0105.apcprd02.prod.outlook.com (2603:1096:4:92::21) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4415.14 via Frontend Transport; Thu, 12 Aug 2021 07:16:11 +0000 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: d641d215-c4f3-44f5-ccde-08d95d6110c7 X-MS-TrafficTypeDiagnostic: DB9PR04MB8461: X-MS-Exchange-Transport-Forked: True X-Microsoft-Antispam-PRVS: X-MS-Oob-TLC-OOBClassifiers: OLM:4714; X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: n3z19iowR9IVE9cOgB6l81IJd2u3CINOSsvPTxxKPwWiNC1DwZPyip4Fou77DnvmfJZoWt969uM4cBhOmh3ImXr7iebjBAzA3lmptj01ZMsX77YYL0suA7FSOuMCtyR3xwez7+kB70H+SUc9s7UzsMQ9FJOLf5Eye8aForEgska75Xie2kEo3/lIbAnhaHNn3peWoka5FuVkn/j1Iuc0RM2ddOyRzqjen5YA3J5wxqjsTtKcnorwjBS1crWh97uGL9art8lrPWcPZvPmmQ7U9gu+82jbJx7QY5efb6Ql9/XYtb2Z4FBMBSSXuHTjOgEI0Wmsc8fOWvhEDY9p4+jPZ5fkcCPArA4QHAjP0qvxkY/qC2AurV1YqiGFse6PB4dicRqkALQA9BureJ/wif6p5wNMtAr4jVGLg79UgwzDC9xDlrCz2GjRlKfcY3qA3Ise9PGaoyax/X74GPSebcfsihjbshTEqKm1UaBZKAK/tTaxVrTFf43ZTmTCwuon6ybka5QK/O+vGCyS2BGK7lrJeBPmnkrXHWnU2wrKKWRi/eiyK4fgd/eo+9Ae/vJiwaa7w51LAQAV+Ps9L37YIY77rxKlr6Ya91s9B9sK+OT7H0Q0eu7dTnAhs8juMvrsEqgiuZke1O2mb0QEe35HtgaNwcxwJHp9xSCGGzM6SsJi0Dv30+2xSBNg4Q1gQFByfPIUMrywaSowKekd3UGyuzuORA== X-Forefront-Antispam-Report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:DU2PR04MB8630.eurprd04.prod.outlook.com; PTR:; CAT:NONE; SFS:(4636009)(376002)(346002)(39860400002)(396003)(136003)(366004)(30864003)(8936002)(956004)(6512007)(2616005)(2906002)(6666004)(36756003)(86362001)(4326008)(83380400001)(38350700002)(44832011)(478600001)(26005)(6506007)(66476007)(186003)(52116002)(5660300002)(1076003)(66556008)(6486002)(38100700002)(8676002)(66946007)(316002); DIR:OUT; SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?us-ascii?Q?Q98UK/WcZm5TKJq5EuEPDzflPcjIxp36LXGmK+iz/3Je95ZOjyg4fL5UvjVS?= =?us-ascii?Q?oAUQjyGUay+sKLEG1M+bhBNKLQs1rlK/Z0t71Md0OdPX6P83ZFgoO6r0QBAi?= =?us-ascii?Q?QgsusbEAwXYBp4ant5yKzCJJTEFmZx1RA7WVmBts/jnVyDgtgc3gN0KvtX8T?= =?us-ascii?Q?awmWe5YwtkITfVu9z4/sn5NZJet8Nu4hs74wczHJrjf+C3j1YUa6O19oSW+9?= =?us-ascii?Q?OJgLMIpoBpiw1hRuUjg1nIJbOSxw9MfO7NoFDtxz8zSATk4vB7mHl0B15mBv?= =?us-ascii?Q?IAcUhmQrYZjKOg/UXvXfEpoCajzB2y8TC59RIuLhXEdDGqVjDqtdoSfZTTmX?= =?us-ascii?Q?yT5p6Re+Vove2DVlYXdqFIXvFxCPr0aqgjcgeAblRvo3aHsRve5FSLTHL/lX?= =?us-ascii?Q?0n3ohxzzaPTS0iOwLnMWu0VwZK7SDR0R6JeOZ5WPa3nCtOrFAfpc+rj4SsEp?= =?us-ascii?Q?wh2n6BEB4+lSyn/BMZ4FqTNvvp4IZPWonNOslGqHS5OF1MDijRiKFHV7BTvR?= =?us-ascii?Q?ZKYlTB4qwvoKQYhVUIYkNKtLlA8Qw1taKjQe/KbBExG1+qqddR+29sSPCkqv?= =?us-ascii?Q?6Es1XYHLV2CNruemSzMG63vgRYurLUq0Meei+w9MRK4ePLiTzg3S/3oME7cS?= =?us-ascii?Q?JYPSdEUfwAWiDTwB5ahbbrMNyfz5drMT1s6LO9m15ISf2eenzDjLZ17y13HS?= =?us-ascii?Q?Vb5IOaS7fRQsts2gtry319THXzSiLs0D6OXsXuu5eK2kIRapN3uRuKOaDTZa?= =?us-ascii?Q?wj+uPI0KF/8dOO7UT0Rmm861dHJZjqFsfCQgZbNkp4993jFrcCBQFsMTzrEr?= =?us-ascii?Q?JQHTWcOTnkzyps/i9trDzwF5vlIX5dvks+XJ5SG3Kh41E+C8WnAfoY5DoYwa?= =?us-ascii?Q?WFSn+/8vXPNMjQAbXc12Y4+3H4oXZDfygjqXfzVcFATkKMd1axPkvrjpDf8E?= =?us-ascii?Q?Tc4wE2XKF+cGKvvnWHO9zBdCPZ1X7d2i7bYfJPz9L+Rio7aiZL+e+T1uiI5/?= =?us-ascii?Q?E5F/hjV2e1jXf29HSaVpOyMIenMQxDtnbWL4W75kZAnX9c9xpaJYrhXU/l8T?= =?us-ascii?Q?SfzknxdFbz+ocr6kE9jlMwnVJLlCzjkaDnEYNuXEAo5KjgTpw3w24QLui4BE?= =?us-ascii?Q?xotxajst0Q43UJ8Fbt8n9H5MB58hpSuW6VBz2pUWk0MNhPqjQdbN6GhEFWGi?= =?us-ascii?Q?PQYXz1BajC9aOLF8FWpMHoqQ0uiOctck11l+b52pH2+c8DObCKU0AOCPG1Ho?= =?us-ascii?Q?HCH3sr8aOBOkyXyazYUBppnrLOi07eXl6fnwCkeJnaZLiIieOHiVDN5ePXwm?= =?us-ascii?Q?xN2uFWShSRcftCT8my8U/aLK?= X-OriginatorOrg: nxp.com X-MS-Exchange-CrossTenant-Network-Message-Id: d641d215-c4f3-44f5-ccde-08d95d6110c7 X-MS-Exchange-CrossTenant-AuthSource: DU2PR04MB8630.eurprd04.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 12 Aug 2021 07:16:12.8398 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 686ea1d3-bc2b-4c6f-a92c-d99c5c301635 X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: 19gMIcNNdi4VosE473M5l9PU+WevITeuzFkj416aZCCbivkT/fBbMBx4wS+xVIGdchcfb6vt7YLZSA/iAnMV3A== X-MS-Exchange-Transport-CrossTenantHeadersStamped: DB9PR04MB8461 Subject: [dpdk-dev] [RFC 05/16] crypto/dpaa2_sec: support raw datapath APIs X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" From: Gagandeep Singh This path add framework for raw API support. The initial patch only test cipher only part. Signed-off-by: Hemant Agrawal Signed-off-by: Gagandeep Singh --- drivers/crypto/dpaa2_sec/dpaa2_sec_dpseci.c | 13 +- drivers/crypto/dpaa2_sec/dpaa2_sec_priv.h | 60 +- drivers/crypto/dpaa2_sec/dpaa2_sec_raw_dp.c | 596 ++++++++++++++++++++ drivers/crypto/dpaa2_sec/meson.build | 3 +- 4 files changed, 643 insertions(+), 29 deletions(-) create mode 100644 drivers/crypto/dpaa2_sec/dpaa2_sec_raw_dp.c diff --git a/drivers/crypto/dpaa2_sec/dpaa2_sec_dpseci.c b/drivers/crypto/dpaa2_sec/dpaa2_sec_dpseci.c index 1ccead3641..fe90d9d2d8 100644 --- a/drivers/crypto/dpaa2_sec/dpaa2_sec_dpseci.c +++ b/drivers/crypto/dpaa2_sec/dpaa2_sec_dpseci.c @@ -49,15 +49,8 @@ #define FSL_MC_DPSECI_DEVID 3 #define NO_PREFETCH 0 -/* FLE_POOL_NUM_BUFS is set as per the ipsec-secgw application */ -#define FLE_POOL_NUM_BUFS 32000 -#define FLE_POOL_BUF_SIZE 256 -#define FLE_POOL_CACHE_SIZE 512 -#define FLE_SG_MEM_SIZE(num) (FLE_POOL_BUF_SIZE + ((num) * 32)) -#define SEC_FLC_DHR_OUTBOUND -114 -#define SEC_FLC_DHR_INBOUND 0 -static uint8_t cryptodev_driver_id; +uint8_t cryptodev_driver_id; #ifdef RTE_LIB_SECURITY static inline int @@ -3805,6 +3798,9 @@ static struct rte_cryptodev_ops crypto_ops = { .sym_session_get_size = dpaa2_sec_sym_session_get_size, .sym_session_configure = dpaa2_sec_sym_session_configure, .sym_session_clear = dpaa2_sec_sym_session_clear, + /* Raw data-path API related operations */ + .sym_get_raw_dp_ctx_size = dpaa2_sec_get_dp_ctx_size, + .sym_configure_raw_dp_ctx = dpaa2_sec_configure_raw_dp_ctx, }; #ifdef RTE_LIB_SECURITY @@ -3887,6 +3883,7 @@ dpaa2_sec_dev_init(struct rte_cryptodev *cryptodev) RTE_CRYPTODEV_FF_HW_ACCELERATED | RTE_CRYPTODEV_FF_SYM_OPERATION_CHAINING | RTE_CRYPTODEV_FF_SECURITY | + RTE_CRYPTODEV_FF_SYM_RAW_DP | RTE_CRYPTODEV_FF_IN_PLACE_SGL | RTE_CRYPTODEV_FF_OOP_SGL_IN_SGL_OUT | RTE_CRYPTODEV_FF_OOP_SGL_IN_LB_OUT | diff --git a/drivers/crypto/dpaa2_sec/dpaa2_sec_priv.h b/drivers/crypto/dpaa2_sec/dpaa2_sec_priv.h index 7dbc69f6cb..860c9b6520 100644 --- a/drivers/crypto/dpaa2_sec/dpaa2_sec_priv.h +++ b/drivers/crypto/dpaa2_sec/dpaa2_sec_priv.h @@ -15,6 +15,16 @@ #define CRYPTODEV_NAME_DPAA2_SEC_PMD crypto_dpaa2_sec /**< NXP DPAA2 - SEC PMD device name */ +extern uint8_t cryptodev_driver_id; + +/* FLE_POOL_NUM_BUFS is set as per the ipsec-secgw application */ +#define FLE_POOL_NUM_BUFS 32000 +#define FLE_POOL_BUF_SIZE 256 +#define FLE_POOL_CACHE_SIZE 512 +#define FLE_SG_MEM_SIZE(num) (FLE_POOL_BUF_SIZE + ((num) * 32)) +#define SEC_FLC_DHR_OUTBOUND -114 +#define SEC_FLC_DHR_INBOUND 0 + #define MAX_QUEUES 64 #define MAX_DESC_SIZE 64 /** private data structure for each DPAA2_SEC device */ @@ -158,6 +168,24 @@ struct dpaa2_pdcp_ctxt { uint32_t hfn_threshold; /*!< HFN Threashold for key renegotiation */ }; #endif + +typedef int (*dpaa2_sec_build_fd_t)( + void *qp, uint8_t *drv_ctx, struct rte_crypto_vec *data_vec, + uint16_t n_data_vecs, union rte_crypto_sym_ofs ofs, + struct rte_crypto_va_iova_ptr *iv, + struct rte_crypto_va_iova_ptr *digest, + struct rte_crypto_va_iova_ptr *aad_or_auth_iv, + void *user_data); + +typedef int (*dpaa2_sec_build_raw_dp_fd_t)(uint8_t *drv_ctx, + struct rte_crypto_sgl *sgl, + struct rte_crypto_va_iova_ptr *iv, + struct rte_crypto_va_iova_ptr *digest, + struct rte_crypto_va_iova_ptr *auth_iv, + union rte_crypto_sym_ofs ofs, + void *userdata, + struct qbman_fd *fd); + typedef struct dpaa2_sec_session_entry { void *ctxt; uint8_t ctxt_type; @@ -165,6 +193,8 @@ typedef struct dpaa2_sec_session_entry { enum rte_crypto_cipher_algorithm cipher_alg; /*!< Cipher Algorithm*/ enum rte_crypto_auth_algorithm auth_alg; /*!< Authentication Algorithm*/ enum rte_crypto_aead_algorithm aead_alg; /*!< AEAD Algorithm*/ + dpaa2_sec_build_fd_t build_fd; + dpaa2_sec_build_raw_dp_fd_t build_raw_dp_fd; union { struct { uint8_t *data; /**< pointer to key data */ @@ -547,26 +577,6 @@ static const struct rte_cryptodev_capabilities dpaa2_sec_capabilities[] = { }, } }, } }, - { /* NULL (CIPHER) */ - .op = RTE_CRYPTO_OP_TYPE_SYMMETRIC, - {.sym = { - .xform_type = RTE_CRYPTO_SYM_XFORM_CIPHER, - {.cipher = { - .algo = RTE_CRYPTO_CIPHER_NULL, - .block_size = 1, - .key_size = { - .min = 0, - .max = 0, - .increment = 0 - }, - .iv_size = { - .min = 0, - .max = 0, - .increment = 0 - } - }, }, - }, } - }, { /* AES CBC */ .op = RTE_CRYPTO_OP_TYPE_SYMMETRIC, {.sym = { @@ -974,4 +984,14 @@ calc_chksum(void *buffer, int len) return result; } +int +dpaa2_sec_configure_raw_dp_ctx(struct rte_cryptodev *dev, uint16_t qp_id, + struct rte_crypto_raw_dp_ctx *raw_dp_ctx, + enum rte_crypto_op_sess_type sess_type, + union rte_cryptodev_session_ctx session_ctx, uint8_t is_update); + +int +dpaa2_sec_get_dp_ctx_size(struct rte_cryptodev *dev); + + #endif /* _DPAA2_SEC_PMD_PRIVATE_H_ */ diff --git a/drivers/crypto/dpaa2_sec/dpaa2_sec_raw_dp.c b/drivers/crypto/dpaa2_sec/dpaa2_sec_raw_dp.c new file mode 100644 index 0000000000..10cb4247b0 --- /dev/null +++ b/drivers/crypto/dpaa2_sec/dpaa2_sec_raw_dp.c @@ -0,0 +1,596 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright 2021 NXP + */ + +#include +#include +#include +#include +#include + +#include "dpaa2_sec_priv.h" +#include "dpaa2_sec_logs.h" + +struct dpaa2_sec_raw_dp_ctx { + dpaa2_sec_session *session; + uint32_t tail; + uint32_t head; + uint16_t cached_enqueue; + uint16_t cached_dequeue; +}; + +static int +build_raw_dp_chain_fd(uint8_t *drv_ctx, + struct rte_crypto_sgl *sgl, + struct rte_crypto_va_iova_ptr *iv, + struct rte_crypto_va_iova_ptr *digest, + struct rte_crypto_va_iova_ptr *auth_iv, + union rte_crypto_sym_ofs ofs, + void *userdata, + struct qbman_fd *fd) +{ + RTE_SET_USED(drv_ctx); + RTE_SET_USED(sgl); + RTE_SET_USED(iv); + RTE_SET_USED(digest); + RTE_SET_USED(auth_iv); + RTE_SET_USED(ofs); + RTE_SET_USED(userdata); + RTE_SET_USED(fd); + + return 0; +} + +static int +build_raw_dp_aead_fd(uint8_t *drv_ctx, + struct rte_crypto_sgl *sgl, + struct rte_crypto_va_iova_ptr *iv, + struct rte_crypto_va_iova_ptr *digest, + struct rte_crypto_va_iova_ptr *auth_iv, + union rte_crypto_sym_ofs ofs, + void *userdata, + struct qbman_fd *fd) +{ + RTE_SET_USED(drv_ctx); + RTE_SET_USED(sgl); + RTE_SET_USED(iv); + RTE_SET_USED(digest); + RTE_SET_USED(auth_iv); + RTE_SET_USED(ofs); + RTE_SET_USED(userdata); + RTE_SET_USED(fd); + + return 0; +} + +static int +build_raw_dp_auth_fd(uint8_t *drv_ctx, + struct rte_crypto_sgl *sgl, + struct rte_crypto_va_iova_ptr *iv, + struct rte_crypto_va_iova_ptr *digest, + struct rte_crypto_va_iova_ptr *auth_iv, + union rte_crypto_sym_ofs ofs, + void *userdata, + struct qbman_fd *fd) +{ + RTE_SET_USED(drv_ctx); + RTE_SET_USED(sgl); + RTE_SET_USED(iv); + RTE_SET_USED(digest); + RTE_SET_USED(auth_iv); + RTE_SET_USED(ofs); + RTE_SET_USED(userdata); + RTE_SET_USED(fd); + + return 0; +} + +static int +build_raw_dp_proto_fd(uint8_t *drv_ctx, + struct rte_crypto_sgl *sgl, + struct rte_crypto_va_iova_ptr *iv, + struct rte_crypto_va_iova_ptr *digest, + struct rte_crypto_va_iova_ptr *auth_iv, + union rte_crypto_sym_ofs ofs, + void *userdata, + struct qbman_fd *fd) +{ + RTE_SET_USED(drv_ctx); + RTE_SET_USED(sgl); + RTE_SET_USED(iv); + RTE_SET_USED(digest); + RTE_SET_USED(auth_iv); + RTE_SET_USED(ofs); + RTE_SET_USED(userdata); + RTE_SET_USED(fd); + + return 0; +} + +static int +build_raw_dp_proto_compound_fd(uint8_t *drv_ctx, + struct rte_crypto_sgl *sgl, + struct rte_crypto_va_iova_ptr *iv, + struct rte_crypto_va_iova_ptr *digest, + struct rte_crypto_va_iova_ptr *auth_iv, + union rte_crypto_sym_ofs ofs, + void *userdata, + struct qbman_fd *fd) +{ + RTE_SET_USED(drv_ctx); + RTE_SET_USED(sgl); + RTE_SET_USED(iv); + RTE_SET_USED(digest); + RTE_SET_USED(auth_iv); + RTE_SET_USED(ofs); + RTE_SET_USED(userdata); + RTE_SET_USED(fd); + + return 0; +} + +static int +build_raw_dp_cipher_fd(uint8_t *drv_ctx, + struct rte_crypto_sgl *sgl, + struct rte_crypto_va_iova_ptr *iv, + struct rte_crypto_va_iova_ptr *digest, + struct rte_crypto_va_iova_ptr *auth_iv, + union rte_crypto_sym_ofs ofs, + void *userdata, + struct qbman_fd *fd) +{ + RTE_SET_USED(digest); + RTE_SET_USED(auth_iv); + + dpaa2_sec_session *sess = + ((struct dpaa2_sec_raw_dp_ctx *)drv_ctx)->session; + struct qbman_fle *ip_fle, *op_fle, *sge, *fle; + int total_len = 0, data_len = 0, data_offset; + struct sec_flow_context *flc; + struct ctxt_priv *priv = sess->ctxt; + unsigned int i; + + for (i = 0; i < sgl->num; i++) + total_len += sgl->vec[i].len; + + data_len = total_len - ofs.ofs.cipher.head - ofs.ofs.cipher.tail; + data_offset = ofs.ofs.cipher.head; + + if (sess->cipher_alg == RTE_CRYPTO_CIPHER_SNOW3G_UEA2 || + sess->cipher_alg == RTE_CRYPTO_CIPHER_ZUC_EEA3) { + if ((data_len & 7) || (data_offset & 7)) { + DPAA2_SEC_ERR("CIPHER: len/offset must be full bytes"); + return -ENOTSUP; + } + + data_len = data_len >> 3; + data_offset = data_offset >> 3; + } + + /* first FLE entry used to store mbuf and session ctxt */ + fle = (struct qbman_fle *)rte_malloc(NULL, + FLE_SG_MEM_SIZE(2*sgl->num), + RTE_CACHE_LINE_SIZE); + if (!fle) { + DPAA2_SEC_ERR("RAW CIPHER SG: Memory alloc failed for SGE"); + return -ENOMEM; + } + memset(fle, 0, FLE_SG_MEM_SIZE(2*sgl->num)); + /* first FLE entry used to store userdata and session ctxt */ + DPAA2_SET_FLE_ADDR(fle, (size_t)userdata); + DPAA2_FLE_SAVE_CTXT(fle, (ptrdiff_t)priv); + + op_fle = fle + 1; + ip_fle = fle + 2; + sge = fle + 3; + + flc = &priv->flc_desc[0].flc; + + DPAA2_SEC_DP_DEBUG( + "RAW CIPHER SG: cipher_off: 0x%x/length %d, ivlen=%d\n", + data_offset, + data_len, + sess->iv.length); + + /* o/p fle */ + DPAA2_SET_FLE_ADDR(op_fle, DPAA2_VADDR_TO_IOVA(sge)); + op_fle->length = data_len; + DPAA2_SET_FLE_SG_EXT(op_fle); + + /* o/p 1st seg */ + DPAA2_SET_FLE_ADDR(sge, sgl->vec[0].iova); + DPAA2_SET_FLE_OFFSET(sge, data_offset); + sge->length = sgl->vec[0].len - data_offset; + + /* o/p segs */ + for (i = 1; i < sgl->num; i++) { + sge++; + DPAA2_SET_FLE_ADDR(sge, sgl->vec[i].iova); + DPAA2_SET_FLE_OFFSET(sge, 0); + sge->length = sgl->vec[i].len; + } + DPAA2_SET_FLE_FIN(sge); + + DPAA2_SEC_DP_DEBUG( + "RAW CIPHER SG: 1 - flc = %p, fle = %p FLEaddr = %x-%x, len %d\n", + flc, fle, fle->addr_hi, fle->addr_lo, + fle->length); + + /* i/p fle */ + sge++; + DPAA2_SET_FLE_ADDR(ip_fle, DPAA2_VADDR_TO_IOVA(sge)); + ip_fle->length = sess->iv.length + data_len; + DPAA2_SET_FLE_SG_EXT(ip_fle); + + /* i/p IV */ + DPAA2_SET_FLE_ADDR(sge, iv->iova); + DPAA2_SET_FLE_OFFSET(sge, 0); + sge->length = sess->iv.length; + + sge++; + + /* i/p 1st seg */ + DPAA2_SET_FLE_ADDR(sge, sgl->vec[0].iova); + DPAA2_SET_FLE_OFFSET(sge, data_offset); + sge->length = sgl->vec[0].len - data_offset; + + /* i/p segs */ + for (i = 1; i < sgl->num; i++) { + sge++; + DPAA2_SET_FLE_ADDR(sge, sgl->vec[i].iova); + DPAA2_SET_FLE_OFFSET(sge, 0); + sge->length = sgl->vec[i].len; + } + DPAA2_SET_FLE_FIN(sge); + DPAA2_SET_FLE_FIN(ip_fle); + + /* sg fd */ + DPAA2_SET_FD_ADDR(fd, DPAA2_VADDR_TO_IOVA(op_fle)); + DPAA2_SET_FD_LEN(fd, ip_fle->length); + DPAA2_SET_FD_COMPOUND_FMT(fd); + DPAA2_SET_FD_FLC(fd, DPAA2_VADDR_TO_IOVA(flc)); + + DPAA2_SEC_DP_DEBUG( + "RAW CIPHER SG: fdaddr =%" PRIx64 " off =%d, len =%d\n", + DPAA2_GET_FD_ADDR(fd), + DPAA2_GET_FD_OFFSET(fd), + DPAA2_GET_FD_LEN(fd)); + + return 0; +} + +static __rte_always_inline uint32_t +dpaa2_sec_raw_enqueue_burst(void *qp_data, uint8_t *drv_ctx, + struct rte_crypto_sym_vec *vec, union rte_crypto_sym_ofs ofs, + void *user_data[], int *status) +{ + RTE_SET_USED(user_data); + uint32_t loop; + int32_t ret; + struct qbman_fd fd_arr[MAX_TX_RING_SLOTS]; + uint32_t frames_to_send, retry_count; + struct qbman_eq_desc eqdesc; + struct dpaa2_sec_qp *dpaa2_qp = (struct dpaa2_sec_qp *)qp_data; + dpaa2_sec_session *sess = + ((struct dpaa2_sec_raw_dp_ctx *)drv_ctx)->session; + struct qbman_swp *swp; + uint16_t num_tx = 0; + uint32_t flags[MAX_TX_RING_SLOTS] = {0}; + + if (unlikely(vec->num == 0)) + return 0; + + if (sess == NULL) { + DPAA2_SEC_ERR("sessionless raw crypto not supported"); + return 0; + } + /*Prepare enqueue descriptor*/ + qbman_eq_desc_clear(&eqdesc); + qbman_eq_desc_set_no_orp(&eqdesc, DPAA2_EQ_RESP_ERR_FQ); + qbman_eq_desc_set_response(&eqdesc, 0, 0); + qbman_eq_desc_set_fq(&eqdesc, dpaa2_qp->tx_vq.fqid); + + if (!DPAA2_PER_LCORE_DPIO) { + ret = dpaa2_affine_qbman_swp(); + if (ret) { + DPAA2_SEC_ERR( + "Failed to allocate IO portal, tid: %d\n", + rte_gettid()); + return 0; + } + } + swp = DPAA2_PER_LCORE_PORTAL; + + while (vec->num) { + frames_to_send = (vec->num > dpaa2_eqcr_size) ? + dpaa2_eqcr_size : vec->num; + + for (loop = 0; loop < frames_to_send; loop++) { + /*Clear the unused FD fields before sending*/ + memset(&fd_arr[loop], 0, sizeof(struct qbman_fd)); + ret = sess->build_raw_dp_fd(drv_ctx, + &vec->src_sgl[loop], + &vec->iv[loop], + &vec->digest[loop], + &vec->auth_iv[loop], + ofs, + user_data[loop], + &fd_arr[loop]); + if (ret) { + DPAA2_SEC_ERR("error: Improper packet contents" + " for crypto operation"); + goto skip_tx; + } + status[loop] = 1; + } + + loop = 0; + retry_count = 0; + while (loop < frames_to_send) { + ret = qbman_swp_enqueue_multiple(swp, &eqdesc, + &fd_arr[loop], + &flags[loop], + frames_to_send - loop); + if (unlikely(ret < 0)) { + retry_count++; + if (retry_count > DPAA2_MAX_TX_RETRY_COUNT) { + num_tx += loop; + vec->num -= loop; + goto skip_tx; + } + } else { + loop += ret; + retry_count = 0; + } + } + + num_tx += loop; + vec->num -= loop; + } +skip_tx: + dpaa2_qp->tx_vq.tx_pkts += num_tx; + dpaa2_qp->tx_vq.err_pkts += vec->num; + + return num_tx; +} + +static __rte_always_inline int +dpaa2_sec_raw_enqueue(void *qp_data, uint8_t *drv_ctx, + struct rte_crypto_vec *data_vec, + uint16_t n_data_vecs, union rte_crypto_sym_ofs ofs, + struct rte_crypto_va_iova_ptr *iv, + struct rte_crypto_va_iova_ptr *digest, + struct rte_crypto_va_iova_ptr *aad_or_auth_iv, + void *user_data) +{ + RTE_SET_USED(qp_data); + RTE_SET_USED(drv_ctx); + RTE_SET_USED(data_vec); + RTE_SET_USED(n_data_vecs); + RTE_SET_USED(ofs); + RTE_SET_USED(iv); + RTE_SET_USED(digest); + RTE_SET_USED(aad_or_auth_iv); + RTE_SET_USED(user_data); + + return 0; +} + +static inline void * +sec_fd_to_userdata(const struct qbman_fd *fd) +{ + struct qbman_fle *fle; + void *userdata; + fle = (struct qbman_fle *)DPAA2_IOVA_TO_VADDR(DPAA2_GET_FD_ADDR(fd)); + + DPAA2_SEC_DP_DEBUG("FLE addr = %x - %x, offset = %x\n", + fle->addr_hi, fle->addr_lo, fle->fin_bpid_offset); + userdata = (struct rte_crypto_op *)DPAA2_GET_FLE_ADDR((fle - 1)); + /* free the fle memory */ + rte_free((void *)(fle-1)); + + return userdata; +} + +static __rte_always_inline uint32_t +dpaa2_sec_raw_dequeue_burst(void *qp_data, uint8_t *drv_ctx, + rte_cryptodev_raw_get_dequeue_count_t get_dequeue_count, + uint32_t max_nb_to_dequeue, + rte_cryptodev_raw_post_dequeue_t post_dequeue, + void **out_user_data, uint8_t is_user_data_array, + uint32_t *n_success, int *dequeue_status) +{ + RTE_SET_USED(drv_ctx); + RTE_SET_USED(get_dequeue_count); + + /* Function is responsible to receive frames for a given device and VQ*/ + struct dpaa2_sec_qp *dpaa2_qp = (struct dpaa2_sec_qp *)qp_data; + struct qbman_result *dq_storage; + uint32_t fqid = dpaa2_qp->rx_vq.fqid; + int ret, num_rx = 0; + uint8_t is_last = 0, status; + struct qbman_swp *swp; + const struct qbman_fd *fd; + struct qbman_pull_desc pulldesc; + void *user_data; + uint32_t nb_ops = max_nb_to_dequeue; + + if (!DPAA2_PER_LCORE_DPIO) { + ret = dpaa2_affine_qbman_swp(); + if (ret) { + DPAA2_SEC_ERR( + "Failed to allocate IO portal, tid: %d\n", + rte_gettid()); + return 0; + } + } + swp = DPAA2_PER_LCORE_PORTAL; + dq_storage = dpaa2_qp->rx_vq.q_storage->dq_storage[0]; + + qbman_pull_desc_clear(&pulldesc); + qbman_pull_desc_set_numframes(&pulldesc, + (nb_ops > dpaa2_dqrr_size) ? + dpaa2_dqrr_size : nb_ops); + qbman_pull_desc_set_fq(&pulldesc, fqid); + qbman_pull_desc_set_storage(&pulldesc, dq_storage, + (uint64_t)DPAA2_VADDR_TO_IOVA(dq_storage), + 1); + + /*Issue a volatile dequeue command. */ + while (1) { + if (qbman_swp_pull(swp, &pulldesc)) { + DPAA2_SEC_WARN( + "SEC VDQ command is not issued : QBMAN busy"); + /* Portal was busy, try again */ + continue; + } + break; + }; + + /* Receive the packets till Last Dequeue entry is found with + * respect to the above issues PULL command. + */ + while (!is_last) { + /* Check if the previous issued command is completed. + * Also seems like the SWP is shared between the Ethernet Driver + * and the SEC driver. + */ + while (!qbman_check_command_complete(dq_storage)) + ; + + /* Loop until the dq_storage is updated with + * new token by QBMAN + */ + while (!qbman_check_new_result(dq_storage)) + ; + /* Check whether Last Pull command is Expired and + * setting Condition for Loop termination + */ + if (qbman_result_DQ_is_pull_complete(dq_storage)) { + is_last = 1; + /* Check for valid frame. */ + status = (uint8_t)qbman_result_DQ_flags(dq_storage); + if (unlikely( + (status & QBMAN_DQ_STAT_VALIDFRAME) == 0)) { + DPAA2_SEC_DP_DEBUG("No frame is delivered\n"); + continue; + } + } + + fd = qbman_result_DQ_fd(dq_storage); + user_data = sec_fd_to_userdata(fd); + if (is_user_data_array) + out_user_data[num_rx] = user_data; + else + out_user_data[0] = user_data; + if (unlikely(fd->simple.frc)) { + /* TODO Parse SEC errors */ + DPAA2_SEC_ERR("SEC returned Error - %x", + fd->simple.frc); + status = RTE_CRYPTO_OP_STATUS_ERROR; + } else { + status = RTE_CRYPTO_OP_STATUS_SUCCESS; + } + post_dequeue(user_data, num_rx, status); + + num_rx++; + dq_storage++; + } /* End of Packet Rx loop */ + + dpaa2_qp->rx_vq.rx_pkts += num_rx; + *dequeue_status = 1; + *n_success = num_rx; + + DPAA2_SEC_DP_DEBUG("SEC Received %d Packets\n", num_rx); + /*Return the total number of packets received to DPAA2 app*/ + return num_rx; +} + +static __rte_always_inline void * +dpaa2_sec_raw_dequeue(void *qp_data, uint8_t *drv_ctx, int *dequeue_status, + enum rte_crypto_op_status *op_status) +{ + RTE_SET_USED(qp_data); + RTE_SET_USED(drv_ctx); + RTE_SET_USED(dequeue_status); + RTE_SET_USED(op_status); + + return NULL; +} + +static __rte_always_inline int +dpaa2_sec_raw_enqueue_done(void *qp_data, uint8_t *drv_ctx, uint32_t n) +{ + RTE_SET_USED(qp_data); + RTE_SET_USED(drv_ctx); + RTE_SET_USED(n); + + return 0; +} + +static __rte_always_inline int +dpaa2_sec_raw_dequeue_done(void *qp_data, uint8_t *drv_ctx, uint32_t n) +{ + RTE_SET_USED(qp_data); + RTE_SET_USED(drv_ctx); + RTE_SET_USED(n); + + return 0; +} + +int +dpaa2_sec_configure_raw_dp_ctx(struct rte_cryptodev *dev, uint16_t qp_id, + struct rte_crypto_raw_dp_ctx *raw_dp_ctx, + enum rte_crypto_op_sess_type sess_type, + union rte_cryptodev_session_ctx session_ctx, uint8_t is_update) +{ + dpaa2_sec_session *sess; + struct dpaa2_sec_raw_dp_ctx *dp_ctx; + RTE_SET_USED(qp_id); + + if (!is_update) { + memset(raw_dp_ctx, 0, sizeof(*raw_dp_ctx) + + sizeof(dpaa2_sec_session)); + raw_dp_ctx->qp_data = dev->data->queue_pairs[qp_id]; + } + + if (sess_type == RTE_CRYPTO_OP_SECURITY_SESSION) + sess = (dpaa2_sec_session *)get_sec_session_private_data( + session_ctx.sec_sess); + else if (sess_type == RTE_CRYPTO_OP_WITH_SESSION) + sess = (dpaa2_sec_session *)get_sym_session_private_data( + session_ctx.crypto_sess, cryptodev_driver_id); + else + return -ENOTSUP; + raw_dp_ctx->dequeue_burst = dpaa2_sec_raw_dequeue_burst; + raw_dp_ctx->dequeue = dpaa2_sec_raw_dequeue; + raw_dp_ctx->dequeue_done = dpaa2_sec_raw_dequeue_done; + raw_dp_ctx->enqueue_burst = dpaa2_sec_raw_enqueue_burst; + raw_dp_ctx->enqueue = dpaa2_sec_raw_enqueue; + raw_dp_ctx->enqueue_done = dpaa2_sec_raw_enqueue_done; + + if (sess->ctxt_type == DPAA2_SEC_CIPHER_HASH) + sess->build_raw_dp_fd = build_raw_dp_chain_fd; + else if (sess->ctxt_type == DPAA2_SEC_AEAD) + sess->build_raw_dp_fd = build_raw_dp_aead_fd; + else if (sess->ctxt_type == DPAA2_SEC_AUTH) + sess->build_raw_dp_fd = build_raw_dp_auth_fd; + else if (sess->ctxt_type == DPAA2_SEC_CIPHER) + sess->build_raw_dp_fd = build_raw_dp_cipher_fd; + else if (sess->ctxt_type == DPAA2_SEC_IPSEC) + sess->build_raw_dp_fd = build_raw_dp_proto_fd; + else if (sess->ctxt_type == DPAA2_SEC_PDCP) + sess->build_raw_dp_fd = build_raw_dp_proto_compound_fd; + else + return -ENOTSUP; + dp_ctx = (struct dpaa2_sec_raw_dp_ctx *)raw_dp_ctx->drv_ctx_data; + dp_ctx->session = sess; + + return 0; +} + +int +dpaa2_sec_get_dp_ctx_size(__rte_unused struct rte_cryptodev *dev) +{ + return sizeof(struct dpaa2_sec_raw_dp_ctx); +} diff --git a/drivers/crypto/dpaa2_sec/meson.build b/drivers/crypto/dpaa2_sec/meson.build index ea1d73a13d..e6e5abb3c1 100644 --- a/drivers/crypto/dpaa2_sec/meson.build +++ b/drivers/crypto/dpaa2_sec/meson.build @@ -1,5 +1,5 @@ # SPDX-License-Identifier: BSD-3-Clause -# Copyright 2018 NXP +# Copyright 2018,2021 NXP if not is_linux build = false @@ -9,6 +9,7 @@ endif deps += ['security', 'mempool_dpaa2'] sources = files( 'dpaa2_sec_dpseci.c', + 'dpaa2_sec_raw_dp.c', 'mc/dpseci.c', ) -- 2.17.1