From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 20D5DA00C2; Thu, 10 Feb 2022 11:59:51 +0100 (CET) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id A3A6B426EE; Thu, 10 Feb 2022 11:59:25 +0100 (CET) Received: from EUR05-AM6-obe.outbound.protection.outlook.com (mail-am6eur05on2054.outbound.protection.outlook.com [40.107.22.54]) by mails.dpdk.org (Postfix) with ESMTP id 221DD426EA for ; Thu, 10 Feb 2022 11:59:24 +0100 (CET) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=BFd1b48oW0YxgUBnBQE5pgZj0FvtXv09U4SFDhLy487zuY9dzE+rEn/4oHH4f5O6K1ChBQX3LBFQR+kyLAfnsIFcFO3L774m+6kXCKNt5qRJ8t4uzQfHUS23l2Dn5/WUn5j2de8++ACd7Ck+ixpQ+C3BQF0ZYxsSTbCKcYBLe1iFwaw2afo+zVKq4Pj5kUdRusaNJMPPMe/e72GhDSJXXDdmGQYWhZlhLDi/vFV8d4nBQUTdvVBF6sO3OELZH1OtrzuadTjiAyfu8BlypuU8+h2gQYTQgIQWaLjoFAAuFyztvyoQ17TeMOYudw2riJUVi+T5zjj6YJ5BT38T2JMhhw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=DnLOhN8zTMf9QWd91HP6lKOboxe5okfCH0+9/SrSDzw=; b=ZuJ3yUe+p8gzlMbthDI48CxGkggh3V+X4CD7gfgN/xdNxuh7bSDewWqYy1OmFpm5CZqgzfMNf5qGEqEsTNPAtPuYDxPeFnnci/53lcP7826o/P5I1uDO/gf97OtPtydNo7n5iZT9FpPmcOqbK4scg7iVH7E1MKV57MSG6Krm2OoP3VHN0nlfmgzAexGHYh+k8ei3uEjIERdnf+XBJ4/9pBFaIPgSvmQh9JC3qeWlbVvf4KIm79kE24Cf3pa8n0Tp+BQGenCU3QoRGdtoIN5GcXChFdtt+wzVS0e+rBoCLowCjZkFNdrWaSmhBKntCtperTx5OUnrzopaWcTG3etmHw== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=none; dmarc=none; dkim=none; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=nxp.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=DnLOhN8zTMf9QWd91HP6lKOboxe5okfCH0+9/SrSDzw=; b=btEWwDXdv6OLoYb+OwiY2GQ2IU6kOxO3mIJ5nv+NJRIXZWs16cXRwyRzgmsr0i4EKPd9+HIsdIv17kVKUAyqRMHiMQIBkR72VnCR9OcysSCwgb4ComdvrrSTbdhBa71gOx6V9xknkX7TBWmgzDlNVUSO035M2iTHimCRtVtNHE8= Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=nxp.com; Received: from AS8PR04MB8198.eurprd04.prod.outlook.com (2603:10a6:20b:3b0::14) by AM6PR04MB4294.eurprd04.prod.outlook.com (2603:10a6:209:4a::13) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4975.11; Thu, 10 Feb 2022 10:59:22 +0000 Received: from AS8PR04MB8198.eurprd04.prod.outlook.com ([fe80::4146:965b:5645:9269]) by AS8PR04MB8198.eurprd04.prod.outlook.com ([fe80::4146:965b:5645:9269%4]) with mapi id 15.20.4975.011; Thu, 10 Feb 2022 10:59:22 +0000 From: Gagandeep Singh To: gakhil@marvell.com, dev@dpdk.org Cc: Nipun Gupta , Gagandeep Singh Subject: [PATCH v4 6/7] crypto/dpaa2_sec: ordered queue support Date: Thu, 10 Feb 2022 16:28:51 +0530 Message-Id: <20220210105852.1268506-7-g.singh@nxp.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20220210105852.1268506-1-g.singh@nxp.com> References: <20220210043136.304845-2-g.singh@nxp.com> <20220210105852.1268506-1-g.singh@nxp.com> Content-Transfer-Encoding: 8bit Content-Type: text/plain X-ClientProxiedBy: SG2PR03CA0092.apcprd03.prod.outlook.com (2603:1096:4:7c::20) To AS8PR04MB8198.eurprd04.prod.outlook.com (2603:10a6:20b:3b0::14) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: e56c5557-79f2-4b6e-d568-08d9ec84650d X-MS-TrafficTypeDiagnostic: AM6PR04MB4294:EE_ X-Microsoft-Antispam-PRVS: X-MS-Oob-TLC-OOBClassifiers: OLM:7219; X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: GN3jkB5ia+WBT4E85e5GeKPdLh4R3940C7PZVHcd291jhqMCOhZjkJdPDxfXHS7OcYfUhiuqtYBuP2jCZAKyINn0VAXN9ClCTaXVXejp66Wp7TFvCp7y5ru+LMTWfxodKgQcgBmj2qFbVhuSLVnPAahqYPRsHvHHLIFh2r1upp4hWzcTpk64+CdWOKCHaMKohrVUFrlMxxPLOTW3lu4gqwd0pgNboSDscZE11fyyPFlbbQWjPOE5R65oPkfGT9hsmtGFaoUpVcuXWhXDLFB9r6fGMW1gWsT05m3EIrBfKOHI9XYWUHEyT03iXWOjYhy0J0zGSPPzVkNtBlRFHm9GYZ5v2tel/YhkyW46QbhP/IgmUux7XHyslNRfPMTzweMM1qTSrhc3UgXllR/vEQGFn1FO0mZWX19k45FpFabhefR7Dx8EluN26VZFil133E3lff/AEd/Q5mRD0Ty4VNumP+nvf7FI0iOBH1SoxmjCAsTvOGaxvEWU/PMUlv5tp4wPgK5+MtGAlRlfC5BAqFRmExvzlGMvI023fCWMDtQy21+nBHG6BLavWUzKqkaPy+3m6Aw3NvVeJL88W1O+zFXf9C1YV/ktSfXqklIj9HBn6Xv5OWrvZbo/0L1Xrq2Qgb+33hcaoh94jX324nHioC6VNmeWdX0paVLhuh9wTcDnAv2vtmYqjsROnsCVv8wKDvbKOeaVCVqhmb9H+XOf3K8SmQ== X-Forefront-Antispam-Report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:AS8PR04MB8198.eurprd04.prod.outlook.com; PTR:; CAT:NONE; SFS:(13230001)(4636009)(366004)(6666004)(1076003)(30864003)(186003)(86362001)(6486002)(26005)(316002)(508600001)(66556008)(4326008)(66476007)(8676002)(54906003)(66946007)(2906002)(5660300002)(2616005)(8936002)(6512007)(55236004)(36756003)(83380400001)(38350700002)(38100700002)(52116002)(6506007); DIR:OUT; SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?us-ascii?Q?cyH9QicZDpxzO3llfIo6J4rGLrlmzUQ46ANEQSXlFfI2pJ4z+aFji6YRkK5n?= =?us-ascii?Q?ydWuZsAPrAKAPUFdVx3mjhYFUKgYA1apU53sM2yAtq92L+a0sf8ej0s9nvba?= =?us-ascii?Q?7Iu+F5IxOH4fbFZ2nGlIwVIBisC8nkayHXKuS/RPTBhx0rypqeDe2q0iAPui?= =?us-ascii?Q?qR+jpTtxdZZ99kiO9D5FT5gOVcK6f6Ird1wecF2jw5nkkyAyH+KFfFcPG9nm?= =?us-ascii?Q?3BgUB7GP3ihOF4XYTpwtGKYTg5XqiDXqwwwusGtzOrmVulGwK8fF3R8nk/jH?= =?us-ascii?Q?e6z2iSa6A7vDod4mcfYQyYY11BXFZrsMnNuKF21SG6JsWreLcYfMsxKJmG+I?= =?us-ascii?Q?qimn47U5uqFEsnf+wiosEYT+YM261iOPkNPNY7k3YDZDa+C2jZHJ9iyYP0xS?= =?us-ascii?Q?bASaGmyKTARcWmgF1Pg6Kq/G8g0PPpNlvekmQLOTDVLLXcCympiI0oAoBwXR?= =?us-ascii?Q?eWy2HIeujewRN1AYS6iRCA/1MmAWNscuyo1TG10IotHgiCSYI8ESiMRFIxME?= =?us-ascii?Q?W3PfrK3Lq3QsWksDCjME8YotFWwOM+4sh4BWI1yBi1Z9UGTgiFyQSF4MWAar?= =?us-ascii?Q?5j/A5Yup0XYeVkJLtG0SxUXeVMvySreZl1b9/0cXaI8VvSVCQcxtDOlmo4HP?= =?us-ascii?Q?jzpZYE5CprJaRKgOuQkpSz3JY9eO9vm6jFoGEL8fzksDttjSXaAba2kI5wjJ?= =?us-ascii?Q?NHoZEj8LZsmSNtxhea71K0Fktel8Ap5sR154IVTvppDMER2mlKXHyG+BtQFT?= =?us-ascii?Q?T2Kqt55jXoQPHfxTiMZV08xEdTOLG0y+zQDAE2m1DFoqrJol897TUmCwYvIb?= =?us-ascii?Q?zxq1gQ7k+d8vjZQ2dA0enc79wG+8+2NLO+EhuFVj7Tjm26atnKFH1oylBm8E?= =?us-ascii?Q?MwBiz9DN2i5t3adcWgRTVBUlbjL9Q2WirIFlPRVbNeX6VbwE1NMdRMJUQj8X?= =?us-ascii?Q?QQ1KPPeKhf3X14624FBxXVCUvizcKrLAGC2vO6YHQOoEcdwdAGmuRXMpeV/B?= =?us-ascii?Q?Wh9HM0qEyw/SgATvxnVZh0ysga6drOcl1MK4tg1BkqWl1/7RQYrvB6D/iVj/?= =?us-ascii?Q?mGgvWLTizrn/7AHhBsOpuMDUMWj7k8njDFKa8NiAdxFYOGSqHWSj82GMxnJJ?= =?us-ascii?Q?aexrM2GaeuVsMyvIdozLUXl2aEcnrPFxT/zrOQShNxz8m8gIc/Rg3mQYG0qh?= =?us-ascii?Q?FPlR/tEZrHjyeRp8juIdgwapcR7TvjWL2f7nExd+SJKp6CHMHWD+w1TRLizs?= =?us-ascii?Q?RzNGda3B/9enTYeNhkvErR1w6DFUEwcAb1c3nqdq2x2SKTf5Ak1cy9amMcHl?= =?us-ascii?Q?+xD7EPT9eassjYdR78w1wXZ3g6Ohavmm+Z77tNqU6FHaotkGLgwRakPvfxjl?= =?us-ascii?Q?dRwE9fDr1XUjg9GeP8TaPUc1wDj4qXQJB81O8jm36zwFB+cJvgKRuFyR3JtN?= =?us-ascii?Q?vcWSBu9bffLLXpMIg1Q/cRV5BWsZPkmHulzyrGzZwszdklyTEsgVhvLiqTWp?= =?us-ascii?Q?yxG2psekYYe/N/nn5/E7r4x1V1/feck+axso1X3x/t3atRLeYPoO4eKbhLB/?= =?us-ascii?Q?a6+UxhV0JZPr855VQU4=3D?= X-OriginatorOrg: nxp.com X-MS-Exchange-CrossTenant-Network-Message-Id: e56c5557-79f2-4b6e-d568-08d9ec84650d X-MS-Exchange-CrossTenant-AuthSource: AS8PR04MB8198.eurprd04.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 10 Feb 2022 10:59:22.8485 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 686ea1d3-bc2b-4c6f-a92c-d99c5c301635 X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: ZrRvTeZ73uUswnkVgsat8o8VVnOzGrnLxR2h0qIOL+rGu1ljjEkKCKtxx3tg9Tc+ X-MS-Exchange-Transport-CrossTenantHeadersStamped: AM6PR04MB4294 X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org From: Nipun Gupta This patch supports ordered queue for DPAA2 platform. Signed-off-by: Nipun Gupta Signed-off-by: Gagandeep Singh --- doc/guides/cryptodevs/dpaa2_sec.rst | 7 + drivers/crypto/dpaa2_sec/dpaa2_sec_dpseci.c | 288 ++++++++++++++++++-- drivers/crypto/dpaa2_sec/dpaa2_sec_priv.h | 2 + drivers/crypto/dpaa2_sec/mc/fsl_dpseci.h | 14 +- 4 files changed, 290 insertions(+), 21 deletions(-) diff --git a/doc/guides/cryptodevs/dpaa2_sec.rst b/doc/guides/cryptodevs/dpaa2_sec.rst index 875d918068..1a590309a0 100644 --- a/doc/guides/cryptodevs/dpaa2_sec.rst +++ b/doc/guides/cryptodevs/dpaa2_sec.rst @@ -185,3 +185,10 @@ on error, mode 1 means dump HW error code and mode 2 means dump HW error code along with other useful debugging information like session, queue, descriptor data. e.g. ``fslmc:dpseci.1,drv_dump_mode=1`` + +Enable strict ordering +---------------------- + +Use dev arg option ``drv_strict_order=1`` to enable strict ordering. +By default, loose ordering is set for ordered schedule type event. +e.g. ``fslmc:dpseci.1,drv_strict_order=1`` diff --git a/drivers/crypto/dpaa2_sec/dpaa2_sec_dpseci.c b/drivers/crypto/dpaa2_sec/dpaa2_sec_dpseci.c index cb8aaf6446..e62d04852b 100644 --- a/drivers/crypto/dpaa2_sec/dpaa2_sec_dpseci.c +++ b/drivers/crypto/dpaa2_sec/dpaa2_sec_dpseci.c @@ -52,6 +52,7 @@ #define NO_PREFETCH 0 #define DRIVER_DUMP_MODE "drv_dump_mode" +#define DRIVER_STRICT_ORDER "drv_strict_order" /* DPAA2_SEC_DP_DUMP levels */ enum dpaa2_sec_dump_levels { @@ -1477,14 +1478,14 @@ dpaa2_sec_enqueue_burst(void *qp, struct rte_crypto_op **ops, for (loop = 0; loop < frames_to_send; loop++) { if (*dpaa2_seqn((*ops)->sym->m_src)) { - uint8_t dqrr_index = - *dpaa2_seqn((*ops)->sym->m_src) - 1; - - flags[loop] = QBMAN_ENQUEUE_FLAG_DCA | dqrr_index; - DPAA2_PER_LCORE_DQRR_SIZE--; - DPAA2_PER_LCORE_DQRR_HELD &= ~(1 << dqrr_index); - *dpaa2_seqn((*ops)->sym->m_src) = - DPAA2_INVALID_MBUF_SEQN; + if (*dpaa2_seqn((*ops)->sym->m_src) & QBMAN_ENQUEUE_FLAG_DCA) { + DPAA2_PER_LCORE_DQRR_SIZE--; + DPAA2_PER_LCORE_DQRR_HELD &= ~(1 << + *dpaa2_seqn((*ops)->sym->m_src) & + QBMAN_EQCR_DCA_IDXMASK); + } + flags[loop] = *dpaa2_seqn((*ops)->sym->m_src); + *dpaa2_seqn((*ops)->sym->m_src) = DPAA2_INVALID_MBUF_SEQN; } /*Clear the unused FD fields before sending*/ @@ -1709,6 +1710,168 @@ dpaa2_sec_dump(struct rte_crypto_op *op) } +static void +dpaa2_sec_free_eqresp_buf(uint16_t eqresp_ci) +{ + struct dpaa2_dpio_dev *dpio_dev = DPAA2_PER_LCORE_DPIO; + struct rte_crypto_op *op; + struct qbman_fd *fd; + + fd = qbman_result_eqresp_fd(&dpio_dev->eqresp[eqresp_ci]); + op = sec_fd_to_mbuf(fd); + /* Instead of freeing, enqueue it to the sec tx queue (sec->core) + * after setting an error in FD. But this will have performance impact. + */ + rte_pktmbuf_free(op->sym->m_src); +} + +static void +dpaa2_sec_set_enqueue_descriptor(struct dpaa2_queue *dpaa2_q, + struct rte_mbuf *m, + struct qbman_eq_desc *eqdesc) +{ + struct dpaa2_dpio_dev *dpio_dev = DPAA2_PER_LCORE_DPIO; + struct eqresp_metadata *eqresp_meta; + struct dpaa2_sec_dev_private *priv = dpaa2_q->crypto_data->dev_private; + uint16_t orpid, seqnum; + uint8_t dq_idx; + + if (*dpaa2_seqn(m) & DPAA2_ENQUEUE_FLAG_ORP) { + orpid = (*dpaa2_seqn(m) & DPAA2_EQCR_OPRID_MASK) >> + DPAA2_EQCR_OPRID_SHIFT; + seqnum = (*dpaa2_seqn(m) & DPAA2_EQCR_SEQNUM_MASK) >> + DPAA2_EQCR_SEQNUM_SHIFT; + + + if (!priv->en_loose_ordered) { + qbman_eq_desc_set_orp(eqdesc, 1, orpid, seqnum, 0); + qbman_eq_desc_set_response(eqdesc, (uint64_t) + DPAA2_VADDR_TO_IOVA(&dpio_dev->eqresp[ + dpio_dev->eqresp_pi]), 1); + qbman_eq_desc_set_token(eqdesc, 1); + + eqresp_meta = &dpio_dev->eqresp_meta[dpio_dev->eqresp_pi]; + eqresp_meta->dpaa2_q = dpaa2_q; + eqresp_meta->mp = m->pool; + + dpio_dev->eqresp_pi + 1 < MAX_EQ_RESP_ENTRIES ? + dpio_dev->eqresp_pi++ : (dpio_dev->eqresp_pi = 0); + } else { + qbman_eq_desc_set_orp(eqdesc, 0, orpid, seqnum, 0); + } + } else { + dq_idx = *dpaa2_seqn(m) - 1; + qbman_eq_desc_set_dca(eqdesc, 1, dq_idx, 0); + DPAA2_PER_LCORE_DQRR_SIZE--; + DPAA2_PER_LCORE_DQRR_HELD &= ~(1 << dq_idx); + } + *dpaa2_seqn(m) = DPAA2_INVALID_MBUF_SEQN; +} + + +static uint16_t +dpaa2_sec_enqueue_burst_ordered(void *qp, struct rte_crypto_op **ops, + uint16_t nb_ops) +{ + /* Function to transmit the frames to given device and VQ*/ + uint32_t loop; + int32_t ret; + struct qbman_fd fd_arr[MAX_TX_RING_SLOTS]; + uint32_t frames_to_send, num_free_eq_desc, retry_count; + struct qbman_eq_desc eqdesc[MAX_TX_RING_SLOTS]; + struct dpaa2_sec_qp *dpaa2_qp = (struct dpaa2_sec_qp *)qp; + struct qbman_swp *swp; + uint16_t num_tx = 0; + uint16_t bpid; + struct rte_mempool *mb_pool; + struct dpaa2_sec_dev_private *priv = + dpaa2_qp->tx_vq.crypto_data->dev_private; + + if (unlikely(nb_ops == 0)) + return 0; + + if (ops[0]->sess_type == RTE_CRYPTO_OP_SESSIONLESS) { + DPAA2_SEC_ERR("sessionless crypto op not supported"); + return 0; + } + + if (!DPAA2_PER_LCORE_DPIO) { + ret = dpaa2_affine_qbman_swp(); + if (ret) { + DPAA2_SEC_ERR("Failure in affining portal"); + return 0; + } + } + swp = DPAA2_PER_LCORE_PORTAL; + + while (nb_ops) { + frames_to_send = (nb_ops > dpaa2_eqcr_size) ? + dpaa2_eqcr_size : nb_ops; + + if (!priv->en_loose_ordered) { + if (*dpaa2_seqn((*ops)->sym->m_src)) { + num_free_eq_desc = dpaa2_free_eq_descriptors(); + if (num_free_eq_desc < frames_to_send) + frames_to_send = num_free_eq_desc; + } + } + + for (loop = 0; loop < frames_to_send; loop++) { + /*Prepare enqueue descriptor*/ + qbman_eq_desc_clear(&eqdesc[loop]); + qbman_eq_desc_set_fq(&eqdesc[loop], dpaa2_qp->tx_vq.fqid); + + if (*dpaa2_seqn((*ops)->sym->m_src)) + dpaa2_sec_set_enqueue_descriptor( + &dpaa2_qp->tx_vq, + (*ops)->sym->m_src, + &eqdesc[loop]); + else + qbman_eq_desc_set_no_orp(&eqdesc[loop], + DPAA2_EQ_RESP_ERR_FQ); + + /*Clear the unused FD fields before sending*/ + memset(&fd_arr[loop], 0, sizeof(struct qbman_fd)); + mb_pool = (*ops)->sym->m_src->pool; + bpid = mempool_to_bpid(mb_pool); + ret = build_sec_fd(*ops, &fd_arr[loop], bpid); + if (ret) { + DPAA2_SEC_ERR("error: Improper packet contents" + " for crypto operation"); + goto skip_tx; + } + ops++; + } + + loop = 0; + retry_count = 0; + while (loop < frames_to_send) { + ret = qbman_swp_enqueue_multiple_desc(swp, + &eqdesc[loop], &fd_arr[loop], + frames_to_send - loop); + if (unlikely(ret < 0)) { + retry_count++; + if (retry_count > DPAA2_MAX_TX_RETRY_COUNT) { + num_tx += loop; + nb_ops -= loop; + goto skip_tx; + } + } else { + loop += ret; + retry_count = 0; + } + } + + num_tx += loop; + nb_ops -= loop; + } + +skip_tx: + dpaa2_qp->tx_vq.tx_pkts += num_tx; + dpaa2_qp->tx_vq.err_pkts += nb_ops; + return num_tx; +} + static uint16_t dpaa2_sec_dequeue_burst(void *qp, struct rte_crypto_op **ops, uint16_t nb_ops) @@ -3622,6 +3785,10 @@ dpaa2_sec_dev_start(struct rte_cryptodev *dev) PMD_INIT_FUNC_TRACE(); + /* Change the tx burst function if ordered queues are used */ + if (priv->en_ordered) + dev->enqueue_burst = dpaa2_sec_enqueue_burst_ordered; + memset(&attr, 0, sizeof(struct dpseci_attr)); ret = dpseci_enable(dpseci, CMD_PRI_LOW, priv->token); @@ -3834,12 +4001,46 @@ dpaa2_sec_process_atomic_event(struct qbman_swp *swp __rte_unused, ev->event_ptr = sec_fd_to_mbuf(fd); dqrr_index = qbman_get_dqrr_idx(dq); - *dpaa2_seqn(crypto_op->sym->m_src) = dqrr_index + 1; + *dpaa2_seqn(crypto_op->sym->m_src) = QBMAN_ENQUEUE_FLAG_DCA | dqrr_index; DPAA2_PER_LCORE_DQRR_SIZE++; DPAA2_PER_LCORE_DQRR_HELD |= 1 << dqrr_index; DPAA2_PER_LCORE_DQRR_MBUF(dqrr_index) = crypto_op->sym->m_src; } +static void __rte_hot +dpaa2_sec_process_ordered_event(struct qbman_swp *swp, + const struct qbman_fd *fd, + const struct qbman_result *dq, + struct dpaa2_queue *rxq, + struct rte_event *ev) +{ + struct rte_crypto_op *crypto_op = (struct rte_crypto_op *)ev->event_ptr; + + /* Prefetching mbuf */ + rte_prefetch0((void *)(size_t)(DPAA2_GET_FD_ADDR(fd)- + rte_dpaa2_bpid_info[DPAA2_GET_FD_BPID(fd)].meta_data_size)); + + /* Prefetching ipsec crypto_op stored in priv data of mbuf */ + rte_prefetch0((void *)(size_t)(DPAA2_GET_FD_ADDR(fd)-64)); + + ev->flow_id = rxq->ev.flow_id; + ev->sub_event_type = rxq->ev.sub_event_type; + ev->event_type = RTE_EVENT_TYPE_CRYPTODEV; + ev->op = RTE_EVENT_OP_NEW; + ev->sched_type = rxq->ev.sched_type; + ev->queue_id = rxq->ev.queue_id; + ev->priority = rxq->ev.priority; + ev->event_ptr = sec_fd_to_mbuf(fd); + + *dpaa2_seqn(crypto_op->sym->m_src) = DPAA2_ENQUEUE_FLAG_ORP; + *dpaa2_seqn(crypto_op->sym->m_src) |= qbman_result_DQ_odpid(dq) << + DPAA2_EQCR_OPRID_SHIFT; + *dpaa2_seqn(crypto_op->sym->m_src) |= qbman_result_DQ_seqnum(dq) << + DPAA2_EQCR_SEQNUM_SHIFT; + + qbman_swp_dqrr_consume(swp, dq); +} + int dpaa2_sec_eventq_attach(const struct rte_cryptodev *dev, int qp_id, @@ -3857,6 +4058,8 @@ dpaa2_sec_eventq_attach(const struct rte_cryptodev *dev, qp->rx_vq.cb = dpaa2_sec_process_parallel_event; else if (event->sched_type == RTE_SCHED_TYPE_ATOMIC) qp->rx_vq.cb = dpaa2_sec_process_atomic_event; + else if (event->sched_type == RTE_SCHED_TYPE_ORDERED) + qp->rx_vq.cb = dpaa2_sec_process_ordered_event; else return -EINVAL; @@ -3875,6 +4078,37 @@ dpaa2_sec_eventq_attach(const struct rte_cryptodev *dev, cfg.options |= DPSECI_QUEUE_OPT_ORDER_PRESERVATION; cfg.order_preservation_en = 1; } + + if (event->sched_type == RTE_SCHED_TYPE_ORDERED) { + struct opr_cfg ocfg; + + /* Restoration window size = 256 frames */ + ocfg.oprrws = 3; + /* Restoration window size = 512 frames for LX2 */ + if (dpaa2_svr_family == SVR_LX2160A) + ocfg.oprrws = 4; + /* Auto advance NESN window enabled */ + ocfg.oa = 1; + /* Late arrival window size disabled */ + ocfg.olws = 0; + /* ORL resource exhaustaion advance NESN disabled */ + ocfg.oeane = 0; + + if (priv->en_loose_ordered) + ocfg.oloe = 1; + else + ocfg.oloe = 0; + + ret = dpseci_set_opr(dpseci, CMD_PRI_LOW, priv->token, + qp_id, OPR_OPT_CREATE, &ocfg); + if (ret) { + RTE_LOG(ERR, PMD, "Error setting opr: ret: %d\n", ret); + return ret; + } + qp->tx_vq.cb_eqresp_free = dpaa2_sec_free_eqresp_buf; + priv->en_ordered = 1; + } + ret = dpseci_set_rx_queue(dpseci, CMD_PRI_LOW, priv->token, qp_id, &cfg); if (ret) { @@ -3979,24 +4213,35 @@ dpaa2_sec_uninit(const struct rte_cryptodev *dev) } static int -check_devargs_handler(__rte_unused const char *key, const char *value, - __rte_unused void *opaque) +check_devargs_handler(const char *key, const char *value, + void *opaque) { - dpaa2_sec_dp_dump = atoi(value); - if (dpaa2_sec_dp_dump > DPAA2_SEC_DP_FULL_DUMP) { - DPAA2_SEC_WARN("WARN: DPAA2_SEC_DP_DUMP_LEVEL is not " - "supported, changing to FULL error prints\n"); - dpaa2_sec_dp_dump = DPAA2_SEC_DP_FULL_DUMP; - } + struct rte_cryptodev *dev = (struct rte_cryptodev *)opaque; + struct dpaa2_sec_dev_private *priv = dev->data->dev_private; + + if (!strcmp(key, "drv_strict_order")) { + priv->en_loose_ordered = false; + } else if (!strcmp(key, "drv_dump_mode")) { + dpaa2_sec_dp_dump = atoi(value); + if (dpaa2_sec_dp_dump > DPAA2_SEC_DP_FULL_DUMP) { + DPAA2_SEC_WARN("WARN: DPAA2_SEC_DP_DUMP_LEVEL is not " + "supported, changing to FULL error" + " prints\n"); + dpaa2_sec_dp_dump = DPAA2_SEC_DP_FULL_DUMP; + } + } else + return -1; return 0; } static void -dpaa2_sec_get_devargs(struct rte_devargs *devargs, const char *key) +dpaa2_sec_get_devargs(struct rte_cryptodev *cryptodev, const char *key) { struct rte_kvargs *kvlist; + struct rte_devargs *devargs; + devargs = cryptodev->device->devargs; if (!devargs) return; @@ -4010,7 +4255,7 @@ dpaa2_sec_get_devargs(struct rte_devargs *devargs, const char *key) } rte_kvargs_process(kvlist, key, - check_devargs_handler, NULL); + check_devargs_handler, (void *)cryptodev); rte_kvargs_free(kvlist); } @@ -4101,6 +4346,7 @@ dpaa2_sec_dev_init(struct rte_cryptodev *cryptodev) cryptodev->data->nb_queue_pairs = internals->max_nb_queue_pairs; internals->hw = dpseci; internals->token = token; + internals->en_loose_ordered = true; snprintf(str, sizeof(str), "sec_fle_pool_p%d_%d", getpid(), cryptodev->data->dev_id); @@ -4115,7 +4361,8 @@ dpaa2_sec_dev_init(struct rte_cryptodev *cryptodev) goto init_error; } - dpaa2_sec_get_devargs(cryptodev->device->devargs, DRIVER_DUMP_MODE); + dpaa2_sec_get_devargs(cryptodev, DRIVER_DUMP_MODE); + dpaa2_sec_get_devargs(cryptodev, DRIVER_STRICT_ORDER); DPAA2_SEC_INFO("driver %s: created", cryptodev->data->name); return 0; @@ -4215,5 +4462,6 @@ RTE_PMD_REGISTER_DPAA2(CRYPTODEV_NAME_DPAA2_SEC_PMD, rte_dpaa2_sec_driver); RTE_PMD_REGISTER_CRYPTO_DRIVER(dpaa2_sec_crypto_drv, rte_dpaa2_sec_driver.driver, cryptodev_driver_id); RTE_PMD_REGISTER_PARAM_STRING(CRYPTODEV_NAME_DPAA2_SEC_PMD, + DRIVER_STRICT_ORDER "=" DRIVER_DUMP_MODE "="); RTE_LOG_REGISTER(dpaa2_logtype_sec, pmd.crypto.dpaa2, NOTICE); diff --git a/drivers/crypto/dpaa2_sec/dpaa2_sec_priv.h b/drivers/crypto/dpaa2_sec/dpaa2_sec_priv.h index a8f9440632..3094778a7a 100644 --- a/drivers/crypto/dpaa2_sec/dpaa2_sec_priv.h +++ b/drivers/crypto/dpaa2_sec/dpaa2_sec_priv.h @@ -37,6 +37,8 @@ struct dpaa2_sec_dev_private { uint16_t token; /**< Token required by DPxxx objects */ unsigned int max_nb_queue_pairs; /**< Max number of queue pairs supported by device */ + uint8_t en_ordered; + uint8_t en_loose_ordered; }; struct dpaa2_sec_qp { diff --git a/drivers/crypto/dpaa2_sec/mc/fsl_dpseci.h b/drivers/crypto/dpaa2_sec/mc/fsl_dpseci.h index 279e8f4d4a..c295c04f24 100644 --- a/drivers/crypto/dpaa2_sec/mc/fsl_dpseci.h +++ b/drivers/crypto/dpaa2_sec/mc/fsl_dpseci.h @@ -1,7 +1,7 @@ /* SPDX-License-Identifier: (BSD-3-Clause OR GPL-2.0) * * Copyright 2013-2016 Freescale Semiconductor Inc. - * Copyright 2016-2017 NXP + * Copyright 2016-2020 NXP * */ #ifndef __FSL_DPSECI_H @@ -11,6 +11,8 @@ * Contains initialization APIs and runtime control APIs for DPSECI */ +#include + struct fsl_mc_io; /** @@ -41,6 +43,16 @@ int dpseci_close(struct fsl_mc_io *mc_io, */ #define DPSECI_OPT_HAS_CG 0x000020 +/** + * Enable the Order Restoration support + */ +#define DPSECI_OPT_HAS_OPR 0x000040 + +/** + * Order Point Records are shared for the entire DPSECI + */ +#define DPSECI_OPT_OPR_SHARED 0x000080 + /** * struct dpseci_cfg - Structure representing DPSECI configuration * @options: Any combination of the following options: -- 2.25.1