From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id D9EA2A00BE; Mon, 20 Dec 2021 11:27:58 +0100 (CET) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 90C9341143; Mon, 20 Dec 2021 11:27:47 +0100 (CET) Received: from EUR02-AM5-obe.outbound.protection.outlook.com (mail-eopbgr00074.outbound.protection.outlook.com [40.107.0.74]) by mails.dpdk.org (Postfix) with ESMTP id 7888E4067C for ; Mon, 20 Dec 2021 11:27:45 +0100 (CET) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=Nz8IM6nSP+3hq4Og/pauP+bnFVa3qqcBd/4X1yAGXzxqYTgfgTmtwiFyP4/cALQ0UHBTwyVOF1dWICoUIA8hXJ3v3Rzlc9YkC036vPJ8D1gAMsnZgPbRmeok5SbNh0NUqTvFHcMlyltxAjo0M/DjoEBQeM0FcMY28lALHIT3fW7ylLcGpAM3+8ijP/6oYENqifgXo2gORC5McDRgoTgM+VPryuNReHfTuE8ZtPsrPlWaccUeW7tS9Yi6DMZFAFLuaVb3+zfZ5f7p3MH+JSHiep3hlTRmWs6ur92gu/61fY6z1mfF4m6wJsJMpCTXb7A4MGLuE2bSha2oDL+pZRUvcQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=lDjWAy47x5cY8icoRld4jLPDQey9Vg9RQMCZ5uSIl+M=; b=LyYMf7LSfHwmrmvexs517kDC1DhSZOB1CYzScePDcOihNj4k5VoRAa6dVkSfQX1521HcrIObzitKzmhNWpM6k13wdY+Hvq+VfuvbcVVw6nDmIhWZuWm8n9qVSC+Nnea4IfLEH8la0uMaON+smwm2UT+Ye1jZZO+M2SH8+ScXX4HZx2jL3xYHX84aGz5PhDgo6qbeB//R799nRv0HGDrBwSIc561IiDWO9hh9p4f/kJ7HhcRYjBvr5pDN1CU1eB4UAvv+8dIHcFybCRYgdwilXlOruKbVNmJ4GnW95KtBN8a0vjZVFVH3oKcUb/Qh8TAzaqbGkWOztNI206f3EII75w== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=nxp.com; dmarc=pass action=none header.from=nxp.com; dkim=pass header.d=nxp.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=nxp.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=lDjWAy47x5cY8icoRld4jLPDQey9Vg9RQMCZ5uSIl+M=; b=DByvM25xeyYicUZFPTQYVaCgBqgBwiwjrqBDSQ/A+iixBqZ4qt00Cg1xs7f3uetNT5I0M0TxmL5omfn/2m8kcsRu5AaqLJH/viWTmVPsTDxVnkrcPaNb7fG0NoWPJqVFWcZBRHAC/9Rhz54BHHT2NTjjBbNSWpjIOEdQ+TipdEY= Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=nxp.com; Received: from AS8PR04MB8198.eurprd04.prod.outlook.com (2603:10a6:20b:3b0::14) by AM6PR0402MB3703.eurprd04.prod.outlook.com (2603:10a6:209:19::30) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4801.20; Mon, 20 Dec 2021 10:27:43 +0000 Received: from AS8PR04MB8198.eurprd04.prod.outlook.com ([fe80::71ab:27ce:856d:5aac]) by AS8PR04MB8198.eurprd04.prod.outlook.com ([fe80::71ab:27ce:856d:5aac%4]) with mapi id 15.20.4801.020; Mon, 20 Dec 2021 10:27:43 +0000 From: Gagandeep Singh To: gakhil@marvell.com, dev@dpdk.org Cc: Nipun Gupta Subject: [PATCH 3/8] crypto/dpaa2_sec: ordered queue support Date: Mon, 20 Dec 2021 15:57:05 +0530 Message-Id: <20211220102710.3083370-3-g.singh@nxp.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20211220102710.3083370-1-g.singh@nxp.com> References: <20211220102710.3083370-1-g.singh@nxp.com> Content-Transfer-Encoding: 8bit Content-Type: text/plain X-ClientProxiedBy: SG2PR06CA0205.apcprd06.prod.outlook.com (2603:1096:4:68::13) To AS8PR04MB8198.eurprd04.prod.outlook.com (2603:10a6:20b:3b0::14) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: 2873c474-b176-4a61-b837-08d9c3a35b5d X-MS-TrafficTypeDiagnostic: AM6PR0402MB3703:EE_ X-Microsoft-Antispam-PRVS: X-MS-Oob-TLC-OOBClassifiers: OLM:7219; X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: Zyhv1CCgCeShfyvpuHajAXTZouToKth7C0iywczgMjQqPCMqYBC0PO7WRS22u4SC8IpCCDrCc1SBa+5JgWn54EVwOZLEtOhXumRYdq9WJxwrol1nj+i+QfxdS0ewEdAbB/EuGD/c6hIqieiTgG/wZw1+equojvBiMU0HsyEm9jGiu+AE9bAzPlvvJL809jHWv3TleFt3Gn+nYRbJnHjv9EhQ6lbQAR0xIMo+5QvHQeCYP1wGYEvogzm5AflBsJQfzBO8ylQ4C0tpMqPLooklz8kNTL3hAAWgeg9pOh0dcT+GcU5l8+KcCvC4WVwbbJdF30q890BjlnjIHDSLK4vJWTq0AFoGhIzoePy5VEihqjdkM1+4etjng1l9zbkd8Zi4EJSS+v/J9bkpSxEaaWMnFltM0BrrOygYoG3dH0fhgcbGGmJDMVlFbEIcTRpstqPB7YAqwN7O9LFQ29bpuzv9Q3QqhJEhEWRNGFwhDhPFxuIduUDKkxtntFW5hxXL0LDePFl7YGk+BdcoJ0fgvCqEc7IbDYdxGGG7oqaDjCG+WviR5onKqj8EyMEuMKSo+U9hDALlmwBu8CA9jIsvo9H/5F6/m485YaVcRH/Q/Egv/HS1f0yyap0WCPwFbo4VqtIFEZQH9WL3CliRD8ar8Dw5vbkNYmRDQFxMPvBI1O/OA/lgMvRfc0XfFcTE2pU/DLP00uLIWYoLrgvkfQaw/Xr0yg== X-Forefront-Antispam-Report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:AS8PR04MB8198.eurprd04.prod.outlook.com; PTR:; CAT:NONE; SFS:(4636009)(366004)(83380400001)(5660300002)(38350700002)(8676002)(6666004)(38100700002)(316002)(55236004)(66556008)(186003)(66476007)(66946007)(30864003)(2906002)(52116002)(26005)(6506007)(8936002)(86362001)(4326008)(6486002)(1076003)(2616005)(6512007)(508600001)(36756003); DIR:OUT; SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?us-ascii?Q?uDjQ3wqj53tQ7VaE5FzEtWaJbyaxANDAA2KhncItkyXzvb+AVvABbWwDD4Af?= =?us-ascii?Q?CmEtlOrjGAt4j99i0x8eBocAhK0w+Ul7dLfhvxfFYqu1qVLJHLOymoU2qyVH?= =?us-ascii?Q?MqiF7O4FgcbmuXXiFYE3c7wbzb/OPD65Hyz5wup6By9ivdaYGumSYdbDF4th?= =?us-ascii?Q?afSKgX5vrZODhuE5xMftMw0FNih1mfTJNcOE4gbya0TXN1aDnlD4vrV7toMj?= =?us-ascii?Q?Gg0xwk8fiM7mD5Pde2cVSCPBeVhIjLGK9FSvfZtwaI+Iwht0E7Jnkwmt4x1+?= =?us-ascii?Q?coMl7miO8CoYfE6g5465RcQ2pygmnU5ciqrIeAULJT0iXPZARZ8eYM82vPv7?= =?us-ascii?Q?x52pjVj+EjK9FIjLZyuS20wdgyGOVChtjdCikVkBYw+W5dc+PQL1DA7kOZRc?= =?us-ascii?Q?+71uGUZk5SxfZySw7GfgoGs4YwQAczmPZr6O2Z0D0biWSFCmCc6J6RRnS5Jo?= =?us-ascii?Q?j+71sPfS1dMcuhCsRD2oHgqxCnGKWN+B1XrlVzlpbnLEv4VlLfBjINu2k8YG?= =?us-ascii?Q?4nS84aDiQ1HL5nOywTwyKbR7ySmLKQQxpCxNlBwJyAS10odpk5TUqz0oD9LU?= =?us-ascii?Q?jWaG/dlZo6jBedGUuMu5AVFLFOaqrvk8o/0lLYeYWOP4SCdDT7T3/kOQMFXI?= =?us-ascii?Q?YygS8hGJaD928p+HiA7MEguooAQ8qzbyG38JC+pNldjQ9IAhB81OnFBWq306?= =?us-ascii?Q?V6Ecv3j87/PrYsNmt5jiD3M9/JxIZrRG6r7nTyT1EGQ1lQJ9ipGdkZLKlcgt?= =?us-ascii?Q?kS0vJ+udEnbo+1EQCeNqXovwAu3ANCWYCH3ldAcWKV4nibt/xJqhYQnQB4Ct?= =?us-ascii?Q?CKPz4e9uYThAxWfjqo+7wZSDGCCTchaQUPXlHcddjq30l3PXUVFWjBIVw8xk?= =?us-ascii?Q?rP6U5SaeOvtPUpD+sglbeRZlaYFUrpARtsofYeePsMe8WJEUIK+O1iUCakdA?= =?us-ascii?Q?Bba1iaqVVLPcoiOIGeTwdNmRpdAKE1gBme5BLGbOZFPpF2v1WClQVcgOZqto?= =?us-ascii?Q?KRXW39Kavwm6EpobLBJLYnjik5bwuwpejrU1WgJTfJWsQEf8fOE5NDC/kN7U?= =?us-ascii?Q?t28QrsOPy0KGk/XF9yWv2iXXAYYtXFNCoJ/fni8S58n3Jkihx/plw7L08D4V?= =?us-ascii?Q?5NR0B2F2SZgaf91LafNbnrm/Zl1I+EXQ+DCwr0ompua6xGzKL8rN6i4gCCzX?= =?us-ascii?Q?6FMlN/7KOncwZjFqSKb2D9Qxq8zjuCyRJnBryft8voAnwMxTPLePZKYH+6xE?= =?us-ascii?Q?g1sqiwhSG2Ky2BQ3NvT8AZ3zVw9fUm4v5KqJI+1igWXTuIakPOUIjJJdf1h6?= =?us-ascii?Q?4JA0UfRf5dso1aCTaTTibW7+iQvQbB8G9t0SPN0y7KI8Yed1U4uLc7l+3ibp?= =?us-ascii?Q?cAuyjNalq7SKF/KpNGfSF+st1LX9/soaLFl4PbgtMpuuE6qoDZZhms0nzW0g?= =?us-ascii?Q?FLDLFCZ1IF15oPyyNTkEYuflziPrDedcVMjtQU+t4fMwLkoJmuoCxYbz23dA?= =?us-ascii?Q?6rzEZCrQo7s2ToVWBl7GjmzcL8YV8Mf1obVkT2FqGqrNGKzKqSIDirA27EN/?= =?us-ascii?Q?zyjftaatmfaEuvHlzSg=3D?= X-OriginatorOrg: nxp.com X-MS-Exchange-CrossTenant-Network-Message-Id: 2873c474-b176-4a61-b837-08d9c3a35b5d X-MS-Exchange-CrossTenant-AuthSource: AS8PR04MB8198.eurprd04.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 20 Dec 2021 10:27:43.3796 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 686ea1d3-bc2b-4c6f-a92c-d99c5c301635 X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: Q2I4F/VI6kpnBAJwPJmVHg4BdwIY4545XIXqgKeKwXB+juaMZ59b3nzprDqCVRe4 X-MS-Exchange-Transport-CrossTenantHeadersStamped: AM6PR0402MB3703 X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org From: Nipun Gupta This patch supports ordered queue for DPAA2 platform. Signed-off-by: Nipun Gupta --- drivers/crypto/dpaa2_sec/dpaa2_sec_dpseci.c | 255 +++++++++++++++++++- drivers/crypto/dpaa2_sec/dpaa2_sec_priv.h | 8 +- drivers/crypto/dpaa2_sec/mc/fsl_dpseci.h | 14 +- 3 files changed, 263 insertions(+), 14 deletions(-) diff --git a/drivers/crypto/dpaa2_sec/dpaa2_sec_dpseci.c b/drivers/crypto/dpaa2_sec/dpaa2_sec_dpseci.c index 1e6b3e548a..a9fda67ac3 100644 --- a/drivers/crypto/dpaa2_sec/dpaa2_sec_dpseci.c +++ b/drivers/crypto/dpaa2_sec/dpaa2_sec_dpseci.c @@ -1466,14 +1466,14 @@ dpaa2_sec_enqueue_burst(void *qp, struct rte_crypto_op **ops, for (loop = 0; loop < frames_to_send; loop++) { if (*dpaa2_seqn((*ops)->sym->m_src)) { - uint8_t dqrr_index = - *dpaa2_seqn((*ops)->sym->m_src) - 1; - - flags[loop] = QBMAN_ENQUEUE_FLAG_DCA | dqrr_index; - DPAA2_PER_LCORE_DQRR_SIZE--; - DPAA2_PER_LCORE_DQRR_HELD &= ~(1 << dqrr_index); - *dpaa2_seqn((*ops)->sym->m_src) = - DPAA2_INVALID_MBUF_SEQN; + if (*dpaa2_seqn((*ops)->sym->m_src) & QBMAN_ENQUEUE_FLAG_DCA) { + DPAA2_PER_LCORE_DQRR_SIZE--; + DPAA2_PER_LCORE_DQRR_HELD &= ~(1 << + *dpaa2_seqn((*ops)->sym->m_src) & + QBMAN_EQCR_DCA_IDXMASK); + } + flags[loop] = *dpaa2_seqn((*ops)->sym->m_src); + *dpaa2_seqn((*ops)->sym->m_src) = DPAA2_INVALID_MBUF_SEQN; } /*Clear the unused FD fields before sending*/ @@ -1621,6 +1621,169 @@ sec_fd_to_mbuf(const struct qbman_fd *fd) return op; } +static void +dpaa2_sec_free_eqresp_buf(uint16_t eqresp_ci) +{ + struct dpaa2_dpio_dev *dpio_dev = DPAA2_PER_LCORE_DPIO; + struct rte_crypto_op *op; + struct qbman_fd *fd; + + fd = qbman_result_eqresp_fd(&dpio_dev->eqresp[eqresp_ci]); + op = sec_fd_to_mbuf(fd); + /* Instead of freeing, enqueue it to the sec tx queue (sec->core) + * after setting an error in FD. But this will have performance impact. + */ + rte_pktmbuf_free(op->sym->m_src); +} + +static void +dpaa2_sec_set_enqueue_descriptor(struct dpaa2_queue *dpaa2_q, + struct rte_mbuf *m, + struct qbman_eq_desc *eqdesc) +{ + struct dpaa2_dpio_dev *dpio_dev = DPAA2_PER_LCORE_DPIO; + struct eqresp_metadata *eqresp_meta; + struct dpaa2_sec_dev_private *priv = dpaa2_q->crypto_data->dev_private; + uint16_t orpid, seqnum; + uint8_t dq_idx; + + if (*dpaa2_seqn(m) & DPAA2_ENQUEUE_FLAG_ORP) { + orpid = (*dpaa2_seqn(m) & DPAA2_EQCR_OPRID_MASK) >> + DPAA2_EQCR_OPRID_SHIFT; + seqnum = (*dpaa2_seqn(m) & DPAA2_EQCR_SEQNUM_MASK) >> + DPAA2_EQCR_SEQNUM_SHIFT; + + + if (!priv->en_loose_ordered) { + qbman_eq_desc_set_orp(eqdesc, 1, orpid, seqnum, 0); + qbman_eq_desc_set_response(eqdesc, (uint64_t) + DPAA2_VADDR_TO_IOVA(&dpio_dev->eqresp[ + dpio_dev->eqresp_pi]), 1); + qbman_eq_desc_set_token(eqdesc, 1); + + eqresp_meta = &dpio_dev->eqresp_meta[dpio_dev->eqresp_pi]; + eqresp_meta->dpaa2_q = dpaa2_q; + eqresp_meta->mp = m->pool; + + dpio_dev->eqresp_pi + 1 < MAX_EQ_RESP_ENTRIES ? + dpio_dev->eqresp_pi++ : (dpio_dev->eqresp_pi = 0); + } else { + qbman_eq_desc_set_orp(eqdesc, 0, orpid, seqnum, 0); + } + } else { + dq_idx = *dpaa2_seqn(m) - 1; + qbman_eq_desc_set_dca(eqdesc, 1, dq_idx, 0); + DPAA2_PER_LCORE_DQRR_SIZE--; + DPAA2_PER_LCORE_DQRR_HELD &= ~(1 << dq_idx); + } + *dpaa2_seqn(m) = DPAA2_INVALID_MBUF_SEQN; +} + + +static uint16_t +dpaa2_sec_enqueue_burst_ordered(void *qp, struct rte_crypto_op **ops, + uint16_t nb_ops) +{ + /* Function to transmit the frames to given device and VQ*/ + uint32_t loop; + int32_t ret; + struct qbman_fd fd_arr[MAX_TX_RING_SLOTS]; + uint32_t frames_to_send, num_free_eq_desc, retry_count; + struct qbman_eq_desc eqdesc[MAX_TX_RING_SLOTS]; + struct dpaa2_sec_qp *dpaa2_qp = (struct dpaa2_sec_qp *)qp; + struct qbman_swp *swp; + uint16_t num_tx = 0; + /*todo - need to support multiple buffer pools */ + uint16_t bpid; + struct rte_mempool *mb_pool; + struct dpaa2_sec_dev_private *priv = + dpaa2_qp->tx_vq.crypto_data->dev_private; + + if (unlikely(nb_ops == 0)) + return 0; + + if (ops[0]->sess_type == RTE_CRYPTO_OP_SESSIONLESS) { + DPAA2_SEC_ERR("sessionless crypto op not supported"); + return 0; + } + + if (!DPAA2_PER_LCORE_DPIO) { + ret = dpaa2_affine_qbman_swp(); + if (ret) { + DPAA2_SEC_ERR("Failure in affining portal"); + return 0; + } + } + swp = DPAA2_PER_LCORE_PORTAL; + + while (nb_ops) { + frames_to_send = (nb_ops > dpaa2_eqcr_size) ? + dpaa2_eqcr_size : nb_ops; + + if (!priv->en_loose_ordered) { + if (*dpaa2_seqn((*ops)->sym->m_src)) { + num_free_eq_desc = dpaa2_free_eq_descriptors(); + if (num_free_eq_desc < frames_to_send) + frames_to_send = num_free_eq_desc; + } + } + + for (loop = 0; loop < frames_to_send; loop++) { + /*Prepare enqueue descriptor*/ + qbman_eq_desc_clear(&eqdesc[loop]); + qbman_eq_desc_set_fq(&eqdesc[loop], dpaa2_qp->tx_vq.fqid); + + if (*dpaa2_seqn((*ops)->sym->m_src)) + dpaa2_sec_set_enqueue_descriptor( + &dpaa2_qp->tx_vq, + (*ops)->sym->m_src, + &eqdesc[loop]); + else + qbman_eq_desc_set_no_orp(&eqdesc[loop], + DPAA2_EQ_RESP_ERR_FQ); + + /*Clear the unused FD fields before sending*/ + memset(&fd_arr[loop], 0, sizeof(struct qbman_fd)); + mb_pool = (*ops)->sym->m_src->pool; + bpid = mempool_to_bpid(mb_pool); + ret = build_sec_fd(*ops, &fd_arr[loop], bpid); + if (ret) { + DPAA2_SEC_ERR("error: Improper packet contents" + " for crypto operation"); + goto skip_tx; + } + ops++; + } + + loop = 0; + retry_count = 0; + while (loop < frames_to_send) { + ret = qbman_swp_enqueue_multiple_desc(swp, + &eqdesc[loop], &fd_arr[loop], + frames_to_send - loop); + if (unlikely(ret < 0)) { + retry_count++; + if (retry_count > DPAA2_MAX_TX_RETRY_COUNT) { + num_tx += loop; + nb_ops -= loop; + goto skip_tx; + } + } else { + loop += ret; + retry_count = 0; + } + } + + num_tx += loop; + nb_ops -= loop; + } + +skip_tx: + dpaa2_qp->tx_vq.tx_pkts += num_tx; + dpaa2_qp->tx_vq.err_pkts += nb_ops; + return num_tx; +} + static uint16_t dpaa2_sec_dequeue_burst(void *qp, struct rte_crypto_op **ops, uint16_t nb_ops) @@ -3527,6 +3690,10 @@ dpaa2_sec_dev_start(struct rte_cryptodev *dev) PMD_INIT_FUNC_TRACE(); + /* Change the tx burst function if ordered queues are used */ + if (priv->en_ordered) + dev->enqueue_burst = dpaa2_sec_enqueue_burst_ordered; + memset(&attr, 0, sizeof(struct dpseci_attr)); ret = dpseci_enable(dpseci, CMD_PRI_LOW, priv->token); @@ -3739,12 +3906,46 @@ dpaa2_sec_process_atomic_event(struct qbman_swp *swp __rte_unused, ev->event_ptr = sec_fd_to_mbuf(fd); dqrr_index = qbman_get_dqrr_idx(dq); - *dpaa2_seqn(crypto_op->sym->m_src) = dqrr_index + 1; + *dpaa2_seqn(crypto_op->sym->m_src) = QBMAN_ENQUEUE_FLAG_DCA | dqrr_index; DPAA2_PER_LCORE_DQRR_SIZE++; DPAA2_PER_LCORE_DQRR_HELD |= 1 << dqrr_index; DPAA2_PER_LCORE_DQRR_MBUF(dqrr_index) = crypto_op->sym->m_src; } +static void __attribute__((hot)) +dpaa2_sec_process_ordered_event(struct qbman_swp *swp, + const struct qbman_fd *fd, + const struct qbman_result *dq, + struct dpaa2_queue *rxq, + struct rte_event *ev) +{ + struct rte_crypto_op *crypto_op = (struct rte_crypto_op *)ev->event_ptr; + + /* Prefetching mbuf */ + rte_prefetch0((void *)(size_t)(DPAA2_GET_FD_ADDR(fd)- + rte_dpaa2_bpid_info[DPAA2_GET_FD_BPID(fd)].meta_data_size)); + + /* Prefetching ipsec crypto_op stored in priv data of mbuf */ + rte_prefetch0((void *)(size_t)(DPAA2_GET_FD_ADDR(fd)-64)); + + ev->flow_id = rxq->ev.flow_id; + ev->sub_event_type = rxq->ev.sub_event_type; + ev->event_type = RTE_EVENT_TYPE_CRYPTODEV; + ev->op = RTE_EVENT_OP_NEW; + ev->sched_type = rxq->ev.sched_type; + ev->queue_id = rxq->ev.queue_id; + ev->priority = rxq->ev.priority; + ev->event_ptr = sec_fd_to_mbuf(fd); + + *dpaa2_seqn(crypto_op->sym->m_src) = DPAA2_ENQUEUE_FLAG_ORP; + *dpaa2_seqn(crypto_op->sym->m_src) |= qbman_result_DQ_odpid(dq) << + DPAA2_EQCR_OPRID_SHIFT; + *dpaa2_seqn(crypto_op->sym->m_src) |= qbman_result_DQ_seqnum(dq) << + DPAA2_EQCR_SEQNUM_SHIFT; + + qbman_swp_dqrr_consume(swp, dq); +} + int dpaa2_sec_eventq_attach(const struct rte_cryptodev *dev, int qp_id, @@ -3762,6 +3963,8 @@ dpaa2_sec_eventq_attach(const struct rte_cryptodev *dev, qp->rx_vq.cb = dpaa2_sec_process_parallel_event; else if (event->sched_type == RTE_SCHED_TYPE_ATOMIC) qp->rx_vq.cb = dpaa2_sec_process_atomic_event; + else if (event->sched_type == RTE_SCHED_TYPE_ORDERED) + qp->rx_vq.cb = dpaa2_sec_process_ordered_event; else return -EINVAL; @@ -3780,6 +3983,40 @@ dpaa2_sec_eventq_attach(const struct rte_cryptodev *dev, cfg.options |= DPSECI_QUEUE_OPT_ORDER_PRESERVATION; cfg.order_preservation_en = 1; } + + if (event->sched_type == RTE_SCHED_TYPE_ORDERED) { + struct opr_cfg ocfg; + + /* Restoration window size = 256 frames */ + ocfg.oprrws = 3; + /* Restoration window size = 512 frames for LX2 */ + if (dpaa2_svr_family == SVR_LX2160A) + ocfg.oprrws = 4; + /* Auto advance NESN window enabled */ + ocfg.oa = 1; + /* Late arrival window size disabled */ + ocfg.olws = 0; + /* ORL resource exhaustaion advance NESN disabled */ + ocfg.oeane = 0; + /* Loose ordering enabled */ + ocfg.oloe = 1; + priv->en_loose_ordered = 1; + /* Strict ordering enabled if explicitly set */ + if (getenv("DPAA2_STRICT_ORDERING_ENABLE")) { + ocfg.oloe = 0; + priv->en_loose_ordered = 0; + } + + ret = dpseci_set_opr(dpseci, CMD_PRI_LOW, priv->token, + qp_id, OPR_OPT_CREATE, &ocfg); + if (ret) { + RTE_LOG(ERR, PMD, "Error setting opr: ret: %d\n", ret); + return ret; + } + qp->tx_vq.cb_eqresp_free = dpaa2_sec_free_eqresp_buf; + priv->en_ordered = 1; + } + ret = dpseci_set_rx_queue(dpseci, CMD_PRI_LOW, priv->token, qp_id, &cfg); if (ret) { diff --git a/drivers/crypto/dpaa2_sec/dpaa2_sec_priv.h b/drivers/crypto/dpaa2_sec/dpaa2_sec_priv.h index 05bd7c0736..1756d917dd 100644 --- a/drivers/crypto/dpaa2_sec/dpaa2_sec_priv.h +++ b/drivers/crypto/dpaa2_sec/dpaa2_sec_priv.h @@ -1,8 +1,6 @@ /* SPDX-License-Identifier: BSD-3-Clause - * - * Copyright (c) 2016 Freescale Semiconductor, Inc. All rights reserved. - * Copyright 2016,2020-2021 NXP - * + * Copyright (c) 2016 Freescale Semiconductor, Inc. All rights reserved. + * Copyright 2016,2019-2021 NXP */ #ifndef _DPAA2_SEC_PMD_PRIVATE_H_ @@ -37,6 +35,8 @@ struct dpaa2_sec_dev_private { uint16_t token; /**< Token required by DPxxx objects */ unsigned int max_nb_queue_pairs; /**< Max number of queue pairs supported by device */ + uint8_t en_ordered; + uint8_t en_loose_ordered; }; struct dpaa2_sec_qp { diff --git a/drivers/crypto/dpaa2_sec/mc/fsl_dpseci.h b/drivers/crypto/dpaa2_sec/mc/fsl_dpseci.h index 279e8f4d4a..c295c04f24 100644 --- a/drivers/crypto/dpaa2_sec/mc/fsl_dpseci.h +++ b/drivers/crypto/dpaa2_sec/mc/fsl_dpseci.h @@ -1,7 +1,7 @@ /* SPDX-License-Identifier: (BSD-3-Clause OR GPL-2.0) * * Copyright 2013-2016 Freescale Semiconductor Inc. - * Copyright 2016-2017 NXP + * Copyright 2016-2020 NXP * */ #ifndef __FSL_DPSECI_H @@ -11,6 +11,8 @@ * Contains initialization APIs and runtime control APIs for DPSECI */ +#include + struct fsl_mc_io; /** @@ -41,6 +43,16 @@ int dpseci_close(struct fsl_mc_io *mc_io, */ #define DPSECI_OPT_HAS_CG 0x000020 +/** + * Enable the Order Restoration support + */ +#define DPSECI_OPT_HAS_OPR 0x000040 + +/** + * Order Point Records are shared for the entire DPSECI + */ +#define DPSECI_OPT_OPR_SHARED 0x000080 + /** * struct dpseci_cfg - Structure representing DPSECI configuration * @options: Any combination of the following options: -- 2.25.1