From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id C7420A0C46; Fri, 18 Jun 2021 14:19:51 +0200 (CEST) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id BD21A4111C; Fri, 18 Jun 2021 14:19:40 +0200 (CEST) Received: from mga07.intel.com (mga07.intel.com [134.134.136.100]) by mails.dpdk.org (Postfix) with ESMTP id DAED6410F6 for ; Fri, 18 Jun 2021 14:19:38 +0200 (CEST) IronPort-SDR: YLDUyPdxd1d64D+2hOobqbUUFL7MsjFbGHf7bRt6eT6kUjPJWaatmnXh/fVbek3STLpBf/GamQ NLQtzVysOuVw== X-IronPort-AV: E=McAfee;i="6200,9189,10018"; a="270392078" X-IronPort-AV: E=Sophos;i="5.83,283,1616482800"; d="scan'208";a="270392078" Received: from fmsmga001.fm.intel.com ([10.253.24.23]) by orsmga105.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 18 Jun 2021 05:19:38 -0700 IronPort-SDR: uwheBzVNE/aXByWsOYNfYSw1aWCP5EBvGJCFD4cpLah/HZSZ6p1yJZCDdW8bKnFk02HwQBgNgi HpJ3QxiDlFqA== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.83,283,1616482800"; d="scan'208";a="555560532" Received: from silpixa00400320.ir.intel.com ([10.237.214.214]) by fmsmga001.fm.intel.com with ESMTP; 18 Jun 2021 05:19:36 -0700 From: pbronowx To: dev@dpdk.org Cc: roy.fan.zhang@intel.com, thomas@monjalon.net, gakhil@marvell.com, ferruh.yigit@intel.com, declan.doherty@intel.com, pbronowx Date: Fri, 18 Jun 2021 12:18:00 +0000 Message-Id: <20210618121803.1189857-5-piotrx.bronowski@intel.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210618121803.1189857-1-piotrx.bronowski@intel.com> References: <20210618121803.1189857-1-piotrx.bronowski@intel.com> MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Subject: [dpdk-dev] [RFC 4/7] crypto/ipsec_mb: move kasumi PMD to ipsec_mb framework X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" This patch removes crypto/kasumi folder and gathers all kasumi PMD implementation specific details into single file pmd_kasumi.c in crypto/ipsec_mb folder. Signed-off-by: pbronowx --- drivers/crypto/ipsec_mb/meson.build | 3 +- drivers/crypto/ipsec_mb/pmd_kasumi.c | 557 +++++++++++++++ .../ipsec_mb/rte_ipsec_mb_pmd_private.h | 7 + drivers/crypto/kasumi/kasumi_pmd_private.h | 81 --- drivers/crypto/kasumi/meson.build | 24 - drivers/crypto/kasumi/rte_kasumi_pmd.c | 642 ------------------ drivers/crypto/kasumi/rte_kasumi_pmd_ops.c | 316 --------- drivers/crypto/kasumi/version.map | 3 - drivers/crypto/meson.build | 1 - 9 files changed, 566 insertions(+), 1068 deletions(-) create mode 100644 drivers/crypto/ipsec_mb/pmd_kasumi.c delete mode 100644 drivers/crypto/kasumi/kasumi_pmd_private.h delete mode 100644 drivers/crypto/kasumi/meson.build delete mode 100644 drivers/crypto/kasumi/rte_kasumi_pmd.c delete mode 100644 drivers/crypto/kasumi/rte_kasumi_pmd_ops.c delete mode 100644 drivers/crypto/kasumi/version.map diff --git a/drivers/crypto/ipsec_mb/meson.build b/drivers/crypto/ipsec_mb/meson.build index 2f3a170fde..039725ce7d 100644 --- a/drivers/crypto/ipsec_mb/meson.build +++ b/drivers/crypto/ipsec_mb/meson.build @@ -24,6 +24,7 @@ endif sources = files('rte_ipsec_mb_pmd.c', 'rte_ipsec_mb_pmd_ops.c', 'pmd_aesni_mb.c', - 'pmd_aesni_gcm.c' + 'pmd_aesni_gcm.c', + 'pmd_kasumi.c' ) deps += ['bus_vdev', 'net', 'security'] diff --git a/drivers/crypto/ipsec_mb/pmd_kasumi.c b/drivers/crypto/ipsec_mb/pmd_kasumi.c new file mode 100644 index 0000000000..ab0e7f8052 --- /dev/null +++ b/drivers/crypto/ipsec_mb/pmd_kasumi.c @@ -0,0 +1,557 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(c) 2016-2021 Intel Corporation + */ + +#include +#include +#include +#include +#include +#include +#include + +#include "rte_ipsec_mb_pmd_private.h" + +#define KASUMI_KEY_LENGTH 16 +#define KASUMI_IV_LENGTH 8 +#define KASUMI_MAX_BURST 4 +#define BYTE_LEN 8 +#define KASUMI_DIGEST_LENGTH 4 + +uint8_t pmd_driver_id_kasumi; + +static const struct rte_cryptodev_capabilities kasumi_capabilities[] = { + { /* KASUMI (F9) */ + .op = RTE_CRYPTO_OP_TYPE_SYMMETRIC, + {.sym = { + .xform_type = RTE_CRYPTO_SYM_XFORM_AUTH, + {.auth = { + .algo = RTE_CRYPTO_AUTH_KASUMI_F9, + .block_size = 8, + .key_size = { + .min = 16, + .max = 16, + .increment = 0 + }, + .digest_size = { + .min = 4, + .max = 4, + .increment = 0 + }, + .iv_size = { 0 } + }, } + }, } + }, + { /* KASUMI (F8) */ + .op = RTE_CRYPTO_OP_TYPE_SYMMETRIC, + {.sym = { + .xform_type = RTE_CRYPTO_SYM_XFORM_CIPHER, + {.cipher = { + .algo = RTE_CRYPTO_CIPHER_KASUMI_F8, + .block_size = 8, + .key_size = { + .min = 16, + .max = 16, + .increment = 0 + }, + .iv_size = { + .min = 8, + .max = 8, + .increment = 0 + } + }, } + }, } + }, + RTE_CRYPTODEV_END_OF_CAPABILITIES_LIST() +}; + +/** KASUMI private session structure */ +struct kasumi_session { + /* Keys have to be 16-byte aligned */ + kasumi_key_sched_t pKeySched_cipher; + kasumi_key_sched_t pKeySched_hash; + enum ipsec_mb_operation op; + enum rte_crypto_auth_operation auth_op; + uint16_t cipher_iv_offset; +} __rte_cache_aligned; + +struct kasumi_qp_data { + uint8_t temp_digest[KASUMI_DIGEST_LENGTH]; + /* *< Buffers used to store the digest generated + * by the driver when verifying a digest provided + * by the user (using authentication verify operation) + */ +}; + +/** Parse crypto xform chain and set private session parameters. */ +static int +kasumi_session_configure(MB_MGR *mgr, void *priv_sess, + const struct rte_crypto_sym_xform *xform) +{ + const struct rte_crypto_sym_xform *auth_xform = NULL; + const struct rte_crypto_sym_xform *cipher_xform = NULL; + enum ipsec_mb_operation mode; + struct kasumi_session *sess = (struct kasumi_session *)priv_sess; + /* Select Crypto operation - hash then cipher / cipher then hash */ + int ret = ipsec_mb_parse_xform(xform, &mode, &auth_xform, + &cipher_xform, NULL); + + if (ret) + return ret; + + if (cipher_xform) { + /* Only KASUMI F8 supported */ + if (cipher_xform->cipher.algo != RTE_CRYPTO_CIPHER_KASUMI_F8) { + IPSEC_MB_LOG(ERR, "Unsupported cipher algorithm "); + return -ENOTSUP; + } + + sess->cipher_iv_offset = cipher_xform->cipher.iv.offset; + if (cipher_xform->cipher.iv.length != KASUMI_IV_LENGTH) { + IPSEC_MB_LOG(ERR, "Wrong IV length"); + return -EINVAL; + } + + /* Initialize key */ + IMB_KASUMI_INIT_F8_KEY_SCHED(mgr, + cipher_xform->cipher.key.data, + &sess->pKeySched_cipher); + } + + if (auth_xform) { + /* Only KASUMI F9 supported */ + if (auth_xform->auth.algo != RTE_CRYPTO_AUTH_KASUMI_F9) { + IPSEC_MB_LOG(ERR, "Unsupported authentication"); + return -ENOTSUP; + } + + if (auth_xform->auth.digest_length != KASUMI_DIGEST_LENGTH) { + IPSEC_MB_LOG(ERR, "Wrong digest length"); + return -EINVAL; + } + + sess->auth_op = auth_xform->auth.op; + + /* Initialize key */ + IMB_KASUMI_INIT_F9_KEY_SCHED(mgr, auth_xform->auth.key.data, + &sess->pKeySched_hash); + } + + sess->op = mode; + return ret; +} + +/** Encrypt/decrypt mbufs with same cipher key. */ +static uint8_t +process_kasumi_cipher_op(struct ipsec_mb_qp *qp, struct rte_crypto_op **ops, + struct kasumi_session *session, uint8_t num_ops) +{ + unsigned int i; + uint8_t processed_ops = 0; + const void *src[num_ops]; + void *dst[num_ops]; + uint8_t *iv_ptr; + uint64_t iv[num_ops]; + uint32_t num_bytes[num_ops]; + + for (i = 0; i < num_ops; i++) { + src[i] = rte_pktmbuf_mtod(ops[i]->sym->m_src, uint8_t *) + + (ops[i]->sym->cipher.data.offset >> 3); + dst[i] = ops[i]->sym->m_dst + ? rte_pktmbuf_mtod(ops[i]->sym->m_dst, uint8_t *) + + (ops[i]->sym->cipher.data.offset >> 3) + : rte_pktmbuf_mtod(ops[i]->sym->m_src, uint8_t *) + + (ops[i]->sym->cipher.data.offset >> 3); + iv_ptr = rte_crypto_op_ctod_offset(ops[i], uint8_t *, + session->cipher_iv_offset); + iv[i] = *((uint64_t *)(iv_ptr)); + num_bytes[i] = ops[i]->sym->cipher.data.length >> 3; + + processed_ops++; + } + + if (processed_ops != 0) + IMB_KASUMI_F8_N_BUFFER(qp->mb_mgr, &session->pKeySched_cipher, + iv, src, dst, num_bytes, + processed_ops); + + return processed_ops; +} + +/** Encrypt/decrypt mbuf (bit level function). */ +static uint8_t +process_kasumi_cipher_op_bit(struct ipsec_mb_qp *qp, struct rte_crypto_op *op, + struct kasumi_session *session) +{ + uint8_t *src, *dst; + uint8_t *iv_ptr; + uint64_t iv; + uint32_t length_in_bits, offset_in_bits; + + offset_in_bits = op->sym->cipher.data.offset; + src = rte_pktmbuf_mtod(op->sym->m_src, uint8_t *); + if (op->sym->m_dst == NULL) + dst = src; + else + dst = rte_pktmbuf_mtod(op->sym->m_dst, uint8_t *); + iv_ptr = rte_crypto_op_ctod_offset(op, uint8_t *, + session->cipher_iv_offset); + iv = *((uint64_t *)(iv_ptr)); + length_in_bits = op->sym->cipher.data.length; + + IMB_KASUMI_F8_1_BUFFER_BIT(qp->mb_mgr, &session->pKeySched_cipher, iv, + src, dst, length_in_bits, offset_in_bits); + + return 1; +} + +/** Generate/verify hash from mbufs with same hash key. */ +static int +process_kasumi_hash_op(struct ipsec_mb_qp *qp, struct rte_crypto_op **ops, + struct kasumi_session *session, uint8_t num_ops) +{ + unsigned int i; + uint8_t processed_ops = 0; + uint8_t *src, *dst; + uint32_t length_in_bits; + uint32_t num_bytes; + struct kasumi_qp_data *qp_data = ipsec_mb_get_qp_private_data(qp); + + for (i = 0; i < num_ops; i++) { + /* Data must be byte aligned */ + if ((ops[i]->sym->auth.data.offset % BYTE_LEN) != 0) { + ops[i]->status = RTE_CRYPTO_OP_STATUS_INVALID_ARGS; + IPSEC_MB_LOG(ERR, "Invalid Offset"); + break; + } + + length_in_bits = ops[i]->sym->auth.data.length; + + src = rte_pktmbuf_mtod(ops[i]->sym->m_src, uint8_t *) + + (ops[i]->sym->auth.data.offset >> 3); + /* Direction from next bit after end of message */ + num_bytes = length_in_bits >> 3; + + if (session->auth_op == RTE_CRYPTO_AUTH_OP_VERIFY) { + dst = qp_data->temp_digest; + IMB_KASUMI_F9_1_BUFFER(qp->mb_mgr, + &session->pKeySched_hash, src, + num_bytes, dst); + + /* Verify digest. */ + if (memcmp(dst, ops[i]->sym->auth.digest.data, + KASUMI_DIGEST_LENGTH) + != 0) + ops[i]->status + = RTE_CRYPTO_OP_STATUS_AUTH_FAILED; + } else { + dst = ops[i]->sym->auth.digest.data; + + IMB_KASUMI_F9_1_BUFFER(qp->mb_mgr, + &session->pKeySched_hash, src, + num_bytes, dst); + } + processed_ops++; + } + + return processed_ops; +} + +/** Process a batch of crypto ops which shares the same session. */ +static int +process_ops(struct rte_crypto_op **ops, struct kasumi_session *session, + struct ipsec_mb_qp *qp, uint8_t num_ops, + uint16_t *accumulated_enqueued_ops) +{ + unsigned int i; + unsigned int processed_ops; + + switch (session->op) { + case IPSEC_MB_OP_ENCRYPT_ONLY: + case IPSEC_MB_OP_DECRYPT_ONLY: + processed_ops + = process_kasumi_cipher_op(qp, ops, session, num_ops); + break; + case IPSEC_MB_OP_HASH_GEN_ONLY: + case IPSEC_MB_OP_HASH_VERIFY_ONLY: + processed_ops + = process_kasumi_hash_op(qp, ops, session, num_ops); + break; + case IPSEC_MB_OP_ENCRYPT_THEN_HASH_GEN: + case IPSEC_MB_OP_DECRYPT_THEN_HASH_VERIFY: + processed_ops + = process_kasumi_cipher_op(qp, ops, session, num_ops); + process_kasumi_hash_op(qp, ops, session, processed_ops); + break; + case IPSEC_MB_OP_HASH_VERIFY_THEN_DECRYPT: + case IPSEC_MB_OP_HASH_GEN_THEN_ENCRYPT: + processed_ops + = process_kasumi_hash_op(qp, ops, session, num_ops); + process_kasumi_cipher_op(qp, ops, session, processed_ops); + break; + default: + /* Operation not supported. */ + processed_ops = 0; + } + + for (i = 0; i < num_ops; i++) { + /* + * If there was no error/authentication failure, + * change status to successful. + */ + if (ops[i]->status == RTE_CRYPTO_OP_STATUS_NOT_PROCESSED) + ops[i]->status = RTE_CRYPTO_OP_STATUS_SUCCESS; + /* Free session if a session-less crypto op. */ + if (ops[i]->sess_type == RTE_CRYPTO_OP_SESSIONLESS) { + memset(session, 0, sizeof(struct kasumi_session)); + memset( + ops[i]->sym->session, 0, + rte_cryptodev_sym_get_existing_header_session_size( + ops[i]->sym->session)); + rte_mempool_put(qp->sess_mp_priv, session); + rte_mempool_put(qp->sess_mp, ops[i]->sym->session); + ops[i]->sym->session = NULL; + } + } + + *accumulated_enqueued_ops += i; + return processed_ops; +} + +/** Process a crypto op with length/offset in bits. */ +static int +process_op_bit(struct rte_crypto_op *op, struct kasumi_session *session, + struct ipsec_mb_qp *qp, uint16_t *accumulated_enqueued_ops) +{ + unsigned int processed_op; + + switch (session->op) { + /* case KASUMI_OP_ONLY_CIPHER: */ + case IPSEC_MB_OP_ENCRYPT_ONLY: + case IPSEC_MB_OP_DECRYPT_ONLY: + processed_op = process_kasumi_cipher_op_bit(qp, op, session); + break; + /* case KASUMI_OP_ONLY_AUTH: */ + case IPSEC_MB_OP_HASH_GEN_ONLY: + case IPSEC_MB_OP_HASH_VERIFY_ONLY: + processed_op = process_kasumi_hash_op(qp, &op, session, 1); + break; + /* case KASUMI_OP_CIPHER_AUTH: */ + case IPSEC_MB_OP_ENCRYPT_THEN_HASH_GEN: + processed_op = process_kasumi_cipher_op_bit(qp, op, session); + if (processed_op == 1) + process_kasumi_hash_op(qp, &op, session, 1); + break; + /* case KASUMI_OP_AUTH_CIPHER: */ + case IPSEC_MB_OP_HASH_VERIFY_THEN_DECRYPT: + processed_op = process_kasumi_hash_op(qp, &op, session, 1); + if (processed_op == 1) + process_kasumi_cipher_op_bit(qp, op, session); + break; + default: + /* Operation not supported. */ + processed_op = 0; + } + + /* + * If there was no error/authentication failure, + * change status to successful. + */ + if (op->status == RTE_CRYPTO_OP_STATUS_NOT_PROCESSED) + op->status = RTE_CRYPTO_OP_STATUS_SUCCESS; + + /* Free session if a session-less crypto op. */ + if (op->sess_type == RTE_CRYPTO_OP_SESSIONLESS) { + memset(op->sym->session, 0, sizeof(struct kasumi_session)); + rte_cryptodev_sym_session_free(op->sym->session); + op->sym->session = NULL; + } + + *accumulated_enqueued_ops += processed_op; + return processed_op; +} + +static uint16_t +kasumi_pmd_dequeue_burst(void *queue_pair, struct rte_crypto_op **ops, + uint16_t nb_ops) +{ + struct rte_crypto_op *c_ops[nb_ops]; + struct rte_crypto_op *curr_c_op = NULL; + + struct kasumi_session *prev_sess = NULL, *curr_sess = NULL; + struct ipsec_mb_qp *qp = queue_pair; + unsigned int i; + uint8_t burst_size = 0; + uint16_t enqueued_ops = 0; + uint8_t processed_ops; + unsigned int nb_dequeued; + + nb_dequeued = rte_ring_dequeue_burst(qp->ingress_queue, + (void **)ops, nb_ops, NULL); + for (i = 0; i < nb_dequeued; i++) { + curr_c_op = ops[i]; + +#ifdef RTE_LIBRTE_PMD_KASUMI_DEBUG + if (!rte_pktmbuf_is_contiguous(curr_c_op->sym->m_src) + || (curr_c_op->sym->m_dst != NULL + && !rte_pktmbuf_is_contiguous( + curr_c_op->sym->m_dst))) { + IPSEC_MB_LOG(ERR, + "PMD supports only contiguous mbufs, op (%p) provides noncontiguous mbuf as source/destination buffer.", + curr_c_op); + curr_c_op->status = RTE_CRYPTO_OP_STATUS_INVALID_ARGS; + break; + } +#endif + + /* Set status as enqueued (not processed yet) by default. */ + curr_c_op->status = RTE_CRYPTO_OP_STATUS_NOT_PROCESSED; + + curr_sess = (struct kasumi_session *) + ipsec_mb_get_session_private(qp, curr_c_op); + if (unlikely(curr_sess == NULL + || curr_sess->op == IPSEC_MB_OP_NOT_SUPPORTED)) { + curr_c_op->status + = RTE_CRYPTO_OP_STATUS_INVALID_SESSION; + break; + } + + /* If length/offset is at bit-level, process this buffer alone. + */ + if (((curr_c_op->sym->cipher.data.length % BYTE_LEN) != 0) + || ((ops[i]->sym->cipher.data.offset % BYTE_LEN) != 0)) { + /* Process the ops of the previous session. */ + if (prev_sess != NULL) { + processed_ops + = process_ops(c_ops, prev_sess, qp, + burst_size, &enqueued_ops); + if (processed_ops < burst_size) { + burst_size = 0; + break; + } + + burst_size = 0; + prev_sess = NULL; + } + + processed_ops = process_op_bit(curr_c_op, curr_sess, + qp, &enqueued_ops); + if (processed_ops != 1) + break; + + continue; + } + + /* Batch ops that share the same session. */ + if (prev_sess == NULL) { + prev_sess = curr_sess; + c_ops[burst_size++] = curr_c_op; + } else if (curr_sess == prev_sess) { + c_ops[burst_size++] = curr_c_op; + /* + * When there are enough ops to process in a batch, + * process them, and start a new batch. + */ + if (burst_size == KASUMI_MAX_BURST) { + processed_ops + = process_ops(c_ops, prev_sess, qp, + burst_size, &enqueued_ops); + if (processed_ops < burst_size) { + burst_size = 0; + break; + } + + burst_size = 0; + prev_sess = NULL; + } + } else { + /* + * Different session, process the ops + * of the previous session. + */ + processed_ops = process_ops( + c_ops, prev_sess, qp, burst_size, &enqueued_ops); + if (processed_ops < burst_size) { + burst_size = 0; + break; + } + + burst_size = 0; + prev_sess = curr_sess; + + c_ops[burst_size++] = curr_c_op; + } + } + + if (burst_size != 0) { + /* Process the crypto ops of the last session. */ + processed_ops = process_ops(c_ops, prev_sess, qp, burst_size, + &enqueued_ops); + } + qp->stats.dequeued_count += i; + + return i; +} + +struct rte_cryptodev_ops kasumi_pmd_ops = { + .dev_configure = ipsec_mb_pmd_config, + .dev_start = ipsec_mb_pmd_start, + .dev_stop = ipsec_mb_pmd_stop, + .dev_close = ipsec_mb_pmd_close, + + .stats_get = ipsec_mb_pmd_stats_get, + .stats_reset = ipsec_mb_pmd_stats_reset, + + .dev_infos_get = ipsec_mb_pmd_info_get, + + .queue_pair_setup = ipsec_mb_pmd_qp_setup, + .queue_pair_release = ipsec_mb_pmd_qp_release, + + .sym_session_get_size = ipsec_mb_pmd_sym_session_get_size, + .sym_session_configure = ipsec_mb_pmd_sym_session_configure, + .sym_session_clear = ipsec_mb_pmd_sym_session_clear +}; + +struct rte_cryptodev_ops *rte_kasumi_pmd_ops = &kasumi_pmd_ops; + +static int +cryptodev_kasumi_probe(struct rte_vdev_device *vdev) +{ + return cryptodev_ipsec_mb_create(vdev, IPSEC_MB_PMD_TYPE_KASUMI); +} + +static struct rte_vdev_driver cryptodev_kasumi_pmd_drv = { + .probe = cryptodev_kasumi_probe, + .remove = cryptodev_ipsec_mb_remove +}; + +static struct cryptodev_driver kasumi_crypto_drv; + +RTE_PMD_REGISTER_VDEV(CRYPTODEV_NAME_KASUMI_PMD, cryptodev_kasumi_pmd_drv); +RTE_PMD_REGISTER_ALIAS(CRYPTODEV_NAME_KASUMI_PMD, cryptodev_kasumi_pmd); +RTE_PMD_REGISTER_PARAM_STRING(CRYPTODEV_NAME_KASUMI_PMD, + "max_nb_queue_pairs= socket_id="); +RTE_PMD_REGISTER_CRYPTO_DRIVER(kasumi_crypto_drv, + cryptodev_kasumi_pmd_drv.driver, + pmd_driver_id_kasumi); + +/* Constructor function to register kasumi PMD */ +RTE_INIT(ipsec_mb_register_kasumi) +{ + struct ipsec_mb_pmd_data *kasumi_data + = &ipsec_mb_pmds[IPSEC_MB_PMD_TYPE_KASUMI]; + + kasumi_data->caps = kasumi_capabilities; + kasumi_data->dequeue_burst = kasumi_pmd_dequeue_burst; + kasumi_data->feature_flags = RTE_CRYPTODEV_FF_SYMMETRIC_CRYPTO + | RTE_CRYPTODEV_FF_SYM_OPERATION_CHAINING + | RTE_CRYPTODEV_FF_NON_BYTE_ALIGNED_DATA + | RTE_CRYPTODEV_FF_OOP_SGL_IN_LB_OUT + | RTE_CRYPTODEV_FF_SYM_SESSIONLESS + | RTE_CRYPTODEV_FF_OOP_LB_IN_LB_OUT; + kasumi_data->internals_priv_size = 0; + kasumi_data->ops = &kasumi_pmd_ops; + kasumi_data->qp_priv_size = sizeof(struct kasumi_qp_data); + kasumi_data->session_configure = kasumi_session_configure; + kasumi_data->session_priv_size = sizeof(struct kasumi_session); +} diff --git a/drivers/crypto/ipsec_mb/rte_ipsec_mb_pmd_private.h b/drivers/crypto/ipsec_mb/rte_ipsec_mb_pmd_private.h index a9de241714..20dcff40ee 100644 --- a/drivers/crypto/ipsec_mb/rte_ipsec_mb_pmd_private.h +++ b/drivers/crypto/ipsec_mb/rte_ipsec_mb_pmd_private.h @@ -37,6 +37,9 @@ extern RTE_DEFINE_PER_LCORE(MB_MGR *, mb_mgr); #define CRYPTODEV_NAME_AESNI_GCM_PMD crypto_aesni_gcm /**< IPSEC Multi buffer PMD aesni_gcm device name */ +#define CRYPTODEV_NAME_KASUMI_PMD crypto_kasumi +/**< IPSEC Multi buffer PMD kasumi device name */ + /** PMD LOGTYPE DRIVER, common to all PMDs */ extern int ipsec_mb_logtype_driver; #define IPSEC_MB_LOG(level, fmt, ...) \ @@ -47,6 +50,7 @@ extern int ipsec_mb_logtype_driver; enum ipsec_mb_pmd_types { IPSEC_MB_PMD_TYPE_AESNI_MB = 0, IPSEC_MB_PMD_TYPE_AESNI_GCM, + IPSEC_MB_PMD_TYPE_KASUMI, IPSEC_MB_N_PMD_TYPES }; @@ -67,6 +71,7 @@ enum ipsec_mb_operation { extern uint8_t pmd_driver_id_aesni_mb; extern uint8_t pmd_driver_id_aesni_gcm; +extern uint8_t pmd_driver_id_kasumi; /** Helper function. Gets driver ID based on PMD type */ static __rte_always_inline uint8_t @@ -77,6 +82,8 @@ ipsec_mb_get_driver_id(enum ipsec_mb_pmd_types pmd_type) return pmd_driver_id_aesni_mb; case IPSEC_MB_PMD_TYPE_AESNI_GCM: return pmd_driver_id_aesni_gcm; + case IPSEC_MB_PMD_TYPE_KASUMI: + return pmd_driver_id_kasumi; default: break; } diff --git a/drivers/crypto/kasumi/kasumi_pmd_private.h b/drivers/crypto/kasumi/kasumi_pmd_private.h deleted file mode 100644 index abedcd616d..0000000000 --- a/drivers/crypto/kasumi/kasumi_pmd_private.h +++ /dev/null @@ -1,81 +0,0 @@ -/* SPDX-License-Identifier: BSD-3-Clause - * Copyright(c) 2016-2018 Intel Corporation - */ - -#ifndef _KASUMI_PMD_PRIVATE_H_ -#define _KASUMI_PMD_PRIVATE_H_ - -#include - -#define CRYPTODEV_NAME_KASUMI_PMD crypto_kasumi -/**< KASUMI PMD device name */ - -/** KASUMI PMD LOGTYPE DRIVER */ -extern int kasumi_logtype_driver; - -#define KASUMI_LOG(level, fmt, ...) \ - rte_log(RTE_LOG_ ## level, kasumi_logtype_driver, \ - "%s() line %u: " fmt "\n", __func__, __LINE__, \ - ## __VA_ARGS__) - -#define KASUMI_DIGEST_LENGTH 4 - -/** private data structure for each virtual KASUMI device */ -struct kasumi_private { - unsigned max_nb_queue_pairs; - /**< Max number of queue pairs supported by device */ - MB_MGR *mgr; - /**< Multi-buffer instance */ -}; - -/** KASUMI buffer queue pair */ -struct kasumi_qp { - uint16_t id; - /**< Queue Pair Identifier */ - char name[RTE_CRYPTODEV_NAME_MAX_LEN]; - /**< Unique Queue Pair Name */ - struct rte_ring *processed_ops; - /**< Ring for placing processed ops */ - struct rte_mempool *sess_mp; - /**< Session Mempool */ - struct rte_mempool *sess_mp_priv; - /**< Session Private Data Mempool */ - struct rte_cryptodev_stats qp_stats; - /**< Queue pair statistics */ - uint8_t temp_digest[KASUMI_DIGEST_LENGTH]; - /**< Buffer used to store the digest generated - * by the driver when verifying a digest provided - * by the user (using authentication verify operation) - */ - MB_MGR *mgr; - /**< Multi-buffer instance */ -} __rte_cache_aligned; - -enum kasumi_operation { - KASUMI_OP_ONLY_CIPHER, - KASUMI_OP_ONLY_AUTH, - KASUMI_OP_CIPHER_AUTH, - KASUMI_OP_AUTH_CIPHER, - KASUMI_OP_NOT_SUPPORTED -}; - -/** KASUMI private session structure */ -struct kasumi_session { - /* Keys have to be 16-byte aligned */ - kasumi_key_sched_t pKeySched_cipher; - kasumi_key_sched_t pKeySched_hash; - enum kasumi_operation op; - enum rte_crypto_auth_operation auth_op; - uint16_t cipher_iv_offset; -} __rte_cache_aligned; - - -int -kasumi_set_session_parameters(MB_MGR *mgr, struct kasumi_session *sess, - const struct rte_crypto_sym_xform *xform); - - -/** device specific operations function pointer structure */ -extern struct rte_cryptodev_ops *rte_kasumi_pmd_ops; - -#endif /* _KASUMI_PMD_PRIVATE_H_ */ diff --git a/drivers/crypto/kasumi/meson.build b/drivers/crypto/kasumi/meson.build deleted file mode 100644 index e6e0f08c3d..0000000000 --- a/drivers/crypto/kasumi/meson.build +++ /dev/null @@ -1,24 +0,0 @@ -# SPDX-License-Identifier: BSD-3-Clause -# Copyright(c) 2018-2020 Intel Corporation - -IMB_required_ver = '0.53.0' -lib = cc.find_library('IPSec_MB', required: false) -if not lib.found() - build = false - reason = 'missing dependency, "libIPSec_MB"' -else - # version comes with quotes, so we split based on " and take the middle - imb_ver = cc.get_define('IMB_VERSION_STR', - prefix : '#include').split('"')[1] - - if (imb_ver == '') or (imb_ver.version_compare('<' + IMB_required_ver)) - reason = 'IPSec_MB version >= @0@ is required, found version @1@'.format( - IMB_required_ver, imb_ver) - build = false - endif - -endif - -ext_deps += lib -sources = files('rte_kasumi_pmd.c', 'rte_kasumi_pmd_ops.c') -deps += ['bus_vdev'] diff --git a/drivers/crypto/kasumi/rte_kasumi_pmd.c b/drivers/crypto/kasumi/rte_kasumi_pmd.c deleted file mode 100644 index 48b7db9e9b..0000000000 --- a/drivers/crypto/kasumi/rte_kasumi_pmd.c +++ /dev/null @@ -1,642 +0,0 @@ -/* SPDX-License-Identifier: BSD-3-Clause - * Copyright(c) 2016-2018 Intel Corporation - */ - -#include -#include -#include -#include -#include -#include -#include - -#include "kasumi_pmd_private.h" - -#define KASUMI_KEY_LENGTH 16 -#define KASUMI_IV_LENGTH 8 -#define KASUMI_MAX_BURST 4 -#define BYTE_LEN 8 - -static uint8_t cryptodev_driver_id; - -/** Get xform chain order. */ -static enum kasumi_operation -kasumi_get_mode(const struct rte_crypto_sym_xform *xform) -{ - if (xform == NULL) - return KASUMI_OP_NOT_SUPPORTED; - - if (xform->next) - if (xform->next->next != NULL) - return KASUMI_OP_NOT_SUPPORTED; - - if (xform->type == RTE_CRYPTO_SYM_XFORM_AUTH) { - if (xform->next == NULL) - return KASUMI_OP_ONLY_AUTH; - else if (xform->next->type == RTE_CRYPTO_SYM_XFORM_CIPHER) - return KASUMI_OP_AUTH_CIPHER; - else - return KASUMI_OP_NOT_SUPPORTED; - } - - if (xform->type == RTE_CRYPTO_SYM_XFORM_CIPHER) { - if (xform->next == NULL) - return KASUMI_OP_ONLY_CIPHER; - else if (xform->next->type == RTE_CRYPTO_SYM_XFORM_AUTH) - return KASUMI_OP_CIPHER_AUTH; - else - return KASUMI_OP_NOT_SUPPORTED; - } - - return KASUMI_OP_NOT_SUPPORTED; -} - - -/** Parse crypto xform chain and set private session parameters. */ -int -kasumi_set_session_parameters(MB_MGR *mgr, struct kasumi_session *sess, - const struct rte_crypto_sym_xform *xform) -{ - const struct rte_crypto_sym_xform *auth_xform = NULL; - const struct rte_crypto_sym_xform *cipher_xform = NULL; - enum kasumi_operation mode; - - /* Select Crypto operation - hash then cipher / cipher then hash */ - mode = kasumi_get_mode(xform); - - switch (mode) { - case KASUMI_OP_CIPHER_AUTH: - auth_xform = xform->next; - /* Fall-through */ - case KASUMI_OP_ONLY_CIPHER: - cipher_xform = xform; - break; - case KASUMI_OP_AUTH_CIPHER: - cipher_xform = xform->next; - /* Fall-through */ - case KASUMI_OP_ONLY_AUTH: - auth_xform = xform; - break; - case KASUMI_OP_NOT_SUPPORTED: - default: - KASUMI_LOG(ERR, "Unsupported operation chain order parameter"); - return -ENOTSUP; - } - - if (cipher_xform) { - /* Only KASUMI F8 supported */ - if (cipher_xform->cipher.algo != RTE_CRYPTO_CIPHER_KASUMI_F8) { - KASUMI_LOG(ERR, "Unsupported cipher algorithm "); - return -ENOTSUP; - } - - sess->cipher_iv_offset = cipher_xform->cipher.iv.offset; - if (cipher_xform->cipher.iv.length != KASUMI_IV_LENGTH) { - KASUMI_LOG(ERR, "Wrong IV length"); - return -EINVAL; - } - - /* Initialize key */ - IMB_KASUMI_INIT_F8_KEY_SCHED(mgr, cipher_xform->cipher.key.data, - &sess->pKeySched_cipher); - } - - if (auth_xform) { - /* Only KASUMI F9 supported */ - if (auth_xform->auth.algo != RTE_CRYPTO_AUTH_KASUMI_F9) { - KASUMI_LOG(ERR, "Unsupported authentication"); - return -ENOTSUP; - } - - if (auth_xform->auth.digest_length != KASUMI_DIGEST_LENGTH) { - KASUMI_LOG(ERR, "Wrong digest length"); - return -EINVAL; - } - - sess->auth_op = auth_xform->auth.op; - - /* Initialize key */ - IMB_KASUMI_INIT_F9_KEY_SCHED(mgr, auth_xform->auth.key.data, - &sess->pKeySched_hash); - } - - - sess->op = mode; - - return 0; -} - -/** Get KASUMI session. */ -static struct kasumi_session * -kasumi_get_session(struct kasumi_qp *qp, struct rte_crypto_op *op) -{ - struct kasumi_session *sess = NULL; - - if (op->sess_type == RTE_CRYPTO_OP_WITH_SESSION) { - if (likely(op->sym->session != NULL)) - sess = (struct kasumi_session *) - get_sym_session_private_data( - op->sym->session, - cryptodev_driver_id); - } else { - void *_sess = NULL; - void *_sess_private_data = NULL; - - if (rte_mempool_get(qp->sess_mp, (void **)&_sess)) - return NULL; - - if (rte_mempool_get(qp->sess_mp_priv, - (void **)&_sess_private_data)) - return NULL; - - sess = (struct kasumi_session *)_sess_private_data; - - if (unlikely(kasumi_set_session_parameters(qp->mgr, sess, - op->sym->xform) != 0)) { - rte_mempool_put(qp->sess_mp, _sess); - rte_mempool_put(qp->sess_mp_priv, _sess_private_data); - sess = NULL; - } - op->sym->session = (struct rte_cryptodev_sym_session *)_sess; - set_sym_session_private_data(op->sym->session, - cryptodev_driver_id, _sess_private_data); - } - - if (unlikely(sess == NULL)) - op->status = RTE_CRYPTO_OP_STATUS_INVALID_SESSION; - - return sess; -} - -/** Encrypt/decrypt mbufs with same cipher key. */ -static uint8_t -process_kasumi_cipher_op(struct kasumi_qp *qp, struct rte_crypto_op **ops, - struct kasumi_session *session, uint8_t num_ops) -{ - unsigned i; - uint8_t processed_ops = 0; - const void *src[num_ops]; - void *dst[num_ops]; - uint8_t *iv_ptr; - uint64_t iv[num_ops]; - uint32_t num_bytes[num_ops]; - - for (i = 0; i < num_ops; i++) { - src[i] = rte_pktmbuf_mtod(ops[i]->sym->m_src, uint8_t *) + - (ops[i]->sym->cipher.data.offset >> 3); - dst[i] = ops[i]->sym->m_dst ? - rte_pktmbuf_mtod(ops[i]->sym->m_dst, uint8_t *) + - (ops[i]->sym->cipher.data.offset >> 3) : - rte_pktmbuf_mtod(ops[i]->sym->m_src, uint8_t *) + - (ops[i]->sym->cipher.data.offset >> 3); - iv_ptr = rte_crypto_op_ctod_offset(ops[i], uint8_t *, - session->cipher_iv_offset); - iv[i] = *((uint64_t *)(iv_ptr)); - num_bytes[i] = ops[i]->sym->cipher.data.length >> 3; - - processed_ops++; - } - - if (processed_ops != 0) - IMB_KASUMI_F8_N_BUFFER(qp->mgr, &session->pKeySched_cipher, iv, - src, dst, num_bytes, processed_ops); - - return processed_ops; -} - -/** Encrypt/decrypt mbuf (bit level function). */ -static uint8_t -process_kasumi_cipher_op_bit(struct kasumi_qp *qp, struct rte_crypto_op *op, - struct kasumi_session *session) -{ - uint8_t *src, *dst; - uint8_t *iv_ptr; - uint64_t iv; - uint32_t length_in_bits, offset_in_bits; - - offset_in_bits = op->sym->cipher.data.offset; - src = rte_pktmbuf_mtod(op->sym->m_src, uint8_t *); - if (op->sym->m_dst == NULL) - dst = src; - else - dst = rte_pktmbuf_mtod(op->sym->m_dst, uint8_t *); - iv_ptr = rte_crypto_op_ctod_offset(op, uint8_t *, - session->cipher_iv_offset); - iv = *((uint64_t *)(iv_ptr)); - length_in_bits = op->sym->cipher.data.length; - - IMB_KASUMI_F8_1_BUFFER_BIT(qp->mgr, &session->pKeySched_cipher, iv, - src, dst, length_in_bits, offset_in_bits); - - return 1; -} - -/** Generate/verify hash from mbufs with same hash key. */ -static int -process_kasumi_hash_op(struct kasumi_qp *qp, struct rte_crypto_op **ops, - struct kasumi_session *session, - uint8_t num_ops) -{ - unsigned i; - uint8_t processed_ops = 0; - uint8_t *src, *dst; - uint32_t length_in_bits; - uint32_t num_bytes; - - for (i = 0; i < num_ops; i++) { - /* Data must be byte aligned */ - if ((ops[i]->sym->auth.data.offset % BYTE_LEN) != 0) { - ops[i]->status = RTE_CRYPTO_OP_STATUS_INVALID_ARGS; - KASUMI_LOG(ERR, "Invalid Offset"); - break; - } - - length_in_bits = ops[i]->sym->auth.data.length; - - src = rte_pktmbuf_mtod(ops[i]->sym->m_src, uint8_t *) + - (ops[i]->sym->auth.data.offset >> 3); - /* Direction from next bit after end of message */ - num_bytes = length_in_bits >> 3; - - if (session->auth_op == RTE_CRYPTO_AUTH_OP_VERIFY) { - dst = qp->temp_digest; - IMB_KASUMI_F9_1_BUFFER(qp->mgr, - &session->pKeySched_hash, src, - num_bytes, dst); - - /* Verify digest. */ - if (memcmp(dst, ops[i]->sym->auth.digest.data, - KASUMI_DIGEST_LENGTH) != 0) - ops[i]->status = RTE_CRYPTO_OP_STATUS_AUTH_FAILED; - } else { - dst = ops[i]->sym->auth.digest.data; - - IMB_KASUMI_F9_1_BUFFER(qp->mgr, - &session->pKeySched_hash, src, - num_bytes, dst); - } - processed_ops++; - } - - return processed_ops; -} - -/** Process a batch of crypto ops which shares the same session. */ -static int -process_ops(struct rte_crypto_op **ops, struct kasumi_session *session, - struct kasumi_qp *qp, uint8_t num_ops, - uint16_t *accumulated_enqueued_ops) -{ - unsigned i; - unsigned enqueued_ops, processed_ops; - - switch (session->op) { - case KASUMI_OP_ONLY_CIPHER: - processed_ops = process_kasumi_cipher_op(qp, ops, - session, num_ops); - break; - case KASUMI_OP_ONLY_AUTH: - processed_ops = process_kasumi_hash_op(qp, ops, session, - num_ops); - break; - case KASUMI_OP_CIPHER_AUTH: - processed_ops = process_kasumi_cipher_op(qp, ops, session, - num_ops); - process_kasumi_hash_op(qp, ops, session, processed_ops); - break; - case KASUMI_OP_AUTH_CIPHER: - processed_ops = process_kasumi_hash_op(qp, ops, session, - num_ops); - process_kasumi_cipher_op(qp, ops, session, processed_ops); - break; - default: - /* Operation not supported. */ - processed_ops = 0; - } - - for (i = 0; i < num_ops; i++) { - /* - * If there was no error/authentication failure, - * change status to successful. - */ - if (ops[i]->status == RTE_CRYPTO_OP_STATUS_NOT_PROCESSED) - ops[i]->status = RTE_CRYPTO_OP_STATUS_SUCCESS; - /* Free session if a session-less crypto op. */ - if (ops[i]->sess_type == RTE_CRYPTO_OP_SESSIONLESS) { - memset(session, 0, sizeof(struct kasumi_session)); - memset(ops[i]->sym->session, 0, - rte_cryptodev_sym_get_existing_header_session_size( - ops[i]->sym->session)); - rte_mempool_put(qp->sess_mp_priv, session); - rte_mempool_put(qp->sess_mp, ops[i]->sym->session); - ops[i]->sym->session = NULL; - } - } - - enqueued_ops = rte_ring_enqueue_burst(qp->processed_ops, - (void **)ops, processed_ops, NULL); - qp->qp_stats.enqueued_count += enqueued_ops; - *accumulated_enqueued_ops += enqueued_ops; - - return enqueued_ops; -} - -/** Process a crypto op with length/offset in bits. */ -static int -process_op_bit(struct rte_crypto_op *op, struct kasumi_session *session, - struct kasumi_qp *qp, uint16_t *accumulated_enqueued_ops) -{ - unsigned enqueued_op, processed_op; - - switch (session->op) { - case KASUMI_OP_ONLY_CIPHER: - processed_op = process_kasumi_cipher_op_bit(qp, op, - session); - break; - case KASUMI_OP_ONLY_AUTH: - processed_op = process_kasumi_hash_op(qp, &op, session, 1); - break; - case KASUMI_OP_CIPHER_AUTH: - processed_op = process_kasumi_cipher_op_bit(qp, op, session); - if (processed_op == 1) - process_kasumi_hash_op(qp, &op, session, 1); - break; - case KASUMI_OP_AUTH_CIPHER: - processed_op = process_kasumi_hash_op(qp, &op, session, 1); - if (processed_op == 1) - process_kasumi_cipher_op_bit(qp, op, session); - break; - default: - /* Operation not supported. */ - processed_op = 0; - } - - /* - * If there was no error/authentication failure, - * change status to successful. - */ - if (op->status == RTE_CRYPTO_OP_STATUS_NOT_PROCESSED) - op->status = RTE_CRYPTO_OP_STATUS_SUCCESS; - - /* Free session if a session-less crypto op. */ - if (op->sess_type == RTE_CRYPTO_OP_SESSIONLESS) { - memset(op->sym->session, 0, sizeof(struct kasumi_session)); - rte_cryptodev_sym_session_free(op->sym->session); - op->sym->session = NULL; - } - - enqueued_op = rte_ring_enqueue_burst(qp->processed_ops, (void **)&op, - processed_op, NULL); - qp->qp_stats.enqueued_count += enqueued_op; - *accumulated_enqueued_ops += enqueued_op; - - return enqueued_op; -} - -static uint16_t -kasumi_pmd_enqueue_burst(void *queue_pair, struct rte_crypto_op **ops, - uint16_t nb_ops) -{ - struct rte_crypto_op *c_ops[nb_ops]; - struct rte_crypto_op *curr_c_op; - - struct kasumi_session *prev_sess = NULL, *curr_sess = NULL; - struct kasumi_qp *qp = queue_pair; - unsigned i; - uint8_t burst_size = 0; - uint16_t enqueued_ops = 0; - uint8_t processed_ops; - - for (i = 0; i < nb_ops; i++) { - curr_c_op = ops[i]; - -#ifdef RTE_LIBRTE_PMD_KASUMI_DEBUG - if (!rte_pktmbuf_is_contiguous(curr_c_op->sym->m_src) || - (curr_c_op->sym->m_dst != NULL && - !rte_pktmbuf_is_contiguous( - curr_c_op->sym->m_dst))) { - KASUMI_LOG(ERR, "PMD supports only contiguous mbufs, " - "op (%p) provides noncontiguous mbuf as " - "source/destination buffer.", curr_c_op); - curr_c_op->status = RTE_CRYPTO_OP_STATUS_INVALID_ARGS; - break; - } -#endif - - /* Set status as enqueued (not processed yet) by default. */ - curr_c_op->status = RTE_CRYPTO_OP_STATUS_NOT_PROCESSED; - - curr_sess = kasumi_get_session(qp, curr_c_op); - if (unlikely(curr_sess == NULL || - curr_sess->op == KASUMI_OP_NOT_SUPPORTED)) { - curr_c_op->status = - RTE_CRYPTO_OP_STATUS_INVALID_SESSION; - break; - } - - /* If length/offset is at bit-level, process this buffer alone. */ - if (((curr_c_op->sym->cipher.data.length % BYTE_LEN) != 0) - || ((ops[i]->sym->cipher.data.offset - % BYTE_LEN) != 0)) { - /* Process the ops of the previous session. */ - if (prev_sess != NULL) { - processed_ops = process_ops(c_ops, prev_sess, - qp, burst_size, &enqueued_ops); - if (processed_ops < burst_size) { - burst_size = 0; - break; - } - - burst_size = 0; - prev_sess = NULL; - } - - processed_ops = process_op_bit(curr_c_op, curr_sess, - qp, &enqueued_ops); - if (processed_ops != 1) - break; - - continue; - } - - /* Batch ops that share the same session. */ - if (prev_sess == NULL) { - prev_sess = curr_sess; - c_ops[burst_size++] = curr_c_op; - } else if (curr_sess == prev_sess) { - c_ops[burst_size++] = curr_c_op; - /* - * When there are enough ops to process in a batch, - * process them, and start a new batch. - */ - if (burst_size == KASUMI_MAX_BURST) { - processed_ops = process_ops(c_ops, prev_sess, - qp, burst_size, &enqueued_ops); - if (processed_ops < burst_size) { - burst_size = 0; - break; - } - - burst_size = 0; - prev_sess = NULL; - } - } else { - /* - * Different session, process the ops - * of the previous session. - */ - processed_ops = process_ops(c_ops, prev_sess, - qp, burst_size, &enqueued_ops); - if (processed_ops < burst_size) { - burst_size = 0; - break; - } - - burst_size = 0; - prev_sess = curr_sess; - - c_ops[burst_size++] = curr_c_op; - } - } - - if (burst_size != 0) { - /* Process the crypto ops of the last session. */ - processed_ops = process_ops(c_ops, prev_sess, - qp, burst_size, &enqueued_ops); - } - - qp->qp_stats.enqueue_err_count += nb_ops - enqueued_ops; - return enqueued_ops; -} - -static uint16_t -kasumi_pmd_dequeue_burst(void *queue_pair, - struct rte_crypto_op **c_ops, uint16_t nb_ops) -{ - struct kasumi_qp *qp = queue_pair; - - unsigned nb_dequeued; - - nb_dequeued = rte_ring_dequeue_burst(qp->processed_ops, - (void **)c_ops, nb_ops, NULL); - qp->qp_stats.dequeued_count += nb_dequeued; - - return nb_dequeued; -} - -static int cryptodev_kasumi_remove(struct rte_vdev_device *vdev); - -static int -cryptodev_kasumi_create(const char *name, - struct rte_vdev_device *vdev, - struct rte_cryptodev_pmd_init_params *init_params) -{ - struct rte_cryptodev *dev; - struct kasumi_private *internals; - MB_MGR *mgr; - - dev = rte_cryptodev_pmd_create(name, &vdev->device, init_params); - if (dev == NULL) { - KASUMI_LOG(ERR, "failed to create cryptodev vdev"); - goto init_error; - } - - dev->driver_id = cryptodev_driver_id; - dev->dev_ops = rte_kasumi_pmd_ops; - - /* Register RX/TX burst functions for data path. */ - dev->dequeue_burst = kasumi_pmd_dequeue_burst; - dev->enqueue_burst = kasumi_pmd_enqueue_burst; - - dev->feature_flags = RTE_CRYPTODEV_FF_SYMMETRIC_CRYPTO | - RTE_CRYPTODEV_FF_SYM_OPERATION_CHAINING | - RTE_CRYPTODEV_FF_NON_BYTE_ALIGNED_DATA | - RTE_CRYPTODEV_FF_SYM_SESSIONLESS | - RTE_CRYPTODEV_FF_OOP_LB_IN_LB_OUT; - - mgr = alloc_mb_mgr(0); - if (mgr == NULL) - return -ENOMEM; - - if (rte_cpu_get_flag_enabled(RTE_CPUFLAG_AVX)) { - dev->feature_flags |= RTE_CRYPTODEV_FF_CPU_AVX; - init_mb_mgr_avx(mgr); - } else { - dev->feature_flags |= RTE_CRYPTODEV_FF_CPU_SSE; - init_mb_mgr_sse(mgr); - } - - internals = dev->data->dev_private; - - internals->max_nb_queue_pairs = init_params->max_nb_queue_pairs; - internals->mgr = mgr; - - return 0; -init_error: - KASUMI_LOG(ERR, "driver %s: failed", - init_params->name); - - cryptodev_kasumi_remove(vdev); - return -EFAULT; -} - -static int -cryptodev_kasumi_probe(struct rte_vdev_device *vdev) -{ - struct rte_cryptodev_pmd_init_params init_params = { - "", - sizeof(struct kasumi_private), - rte_socket_id(), - RTE_CRYPTODEV_PMD_DEFAULT_MAX_NB_QUEUE_PAIRS - }; - const char *name; - const char *input_args; - - name = rte_vdev_device_name(vdev); - if (name == NULL) - return -EINVAL; - input_args = rte_vdev_device_args(vdev); - - rte_cryptodev_pmd_parse_input_args(&init_params, input_args); - - return cryptodev_kasumi_create(name, vdev, &init_params); -} - -static int -cryptodev_kasumi_remove(struct rte_vdev_device *vdev) -{ - struct rte_cryptodev *cryptodev; - const char *name; - struct kasumi_private *internals; - - name = rte_vdev_device_name(vdev); - if (name == NULL) - return -EINVAL; - - cryptodev = rte_cryptodev_pmd_get_named_dev(name); - if (cryptodev == NULL) - return -ENODEV; - - internals = cryptodev->data->dev_private; - - free_mb_mgr(internals->mgr); - - return rte_cryptodev_pmd_destroy(cryptodev); -} - -static struct rte_vdev_driver cryptodev_kasumi_pmd_drv = { - .probe = cryptodev_kasumi_probe, - .remove = cryptodev_kasumi_remove -}; - -static struct cryptodev_driver kasumi_crypto_drv; - -RTE_PMD_REGISTER_VDEV(CRYPTODEV_NAME_KASUMI_PMD, cryptodev_kasumi_pmd_drv); -RTE_PMD_REGISTER_ALIAS(CRYPTODEV_NAME_KASUMI_PMD, cryptodev_kasumi_pmd); -RTE_PMD_REGISTER_PARAM_STRING(CRYPTODEV_NAME_KASUMI_PMD, - "max_nb_queue_pairs= " - "socket_id="); -RTE_PMD_REGISTER_CRYPTO_DRIVER(kasumi_crypto_drv, - cryptodev_kasumi_pmd_drv.driver, cryptodev_driver_id); - -RTE_LOG_REGISTER_DEFAULT(kasumi_logtype_driver, NOTICE); diff --git a/drivers/crypto/kasumi/rte_kasumi_pmd_ops.c b/drivers/crypto/kasumi/rte_kasumi_pmd_ops.c deleted file mode 100644 index 43376c1aa0..0000000000 --- a/drivers/crypto/kasumi/rte_kasumi_pmd_ops.c +++ /dev/null @@ -1,316 +0,0 @@ -/* SPDX-License-Identifier: BSD-3-Clause - * Copyright(c) 2016-2018 Intel Corporation - */ - -#include - -#include -#include -#include - -#include "kasumi_pmd_private.h" - -static const struct rte_cryptodev_capabilities kasumi_pmd_capabilities[] = { - { /* KASUMI (F9) */ - .op = RTE_CRYPTO_OP_TYPE_SYMMETRIC, - {.sym = { - .xform_type = RTE_CRYPTO_SYM_XFORM_AUTH, - {.auth = { - .algo = RTE_CRYPTO_AUTH_KASUMI_F9, - .block_size = 8, - .key_size = { - .min = 16, - .max = 16, - .increment = 0 - }, - .digest_size = { - .min = 4, - .max = 4, - .increment = 0 - }, - .iv_size = { 0 } - }, } - }, } - }, - { /* KASUMI (F8) */ - .op = RTE_CRYPTO_OP_TYPE_SYMMETRIC, - {.sym = { - .xform_type = RTE_CRYPTO_SYM_XFORM_CIPHER, - {.cipher = { - .algo = RTE_CRYPTO_CIPHER_KASUMI_F8, - .block_size = 8, - .key_size = { - .min = 16, - .max = 16, - .increment = 0 - }, - .iv_size = { - .min = 8, - .max = 8, - .increment = 0 - } - }, } - }, } - }, - RTE_CRYPTODEV_END_OF_CAPABILITIES_LIST() -}; - -/** Configure device */ -static int -kasumi_pmd_config(__rte_unused struct rte_cryptodev *dev, - __rte_unused struct rte_cryptodev_config *config) -{ - return 0; -} - -/** Start device */ -static int -kasumi_pmd_start(__rte_unused struct rte_cryptodev *dev) -{ - return 0; -} - -/** Stop device */ -static void -kasumi_pmd_stop(__rte_unused struct rte_cryptodev *dev) -{ -} - -/** Close device */ -static int -kasumi_pmd_close(__rte_unused struct rte_cryptodev *dev) -{ - return 0; -} - - -/** Get device statistics */ -static void -kasumi_pmd_stats_get(struct rte_cryptodev *dev, - struct rte_cryptodev_stats *stats) -{ - int qp_id; - - for (qp_id = 0; qp_id < dev->data->nb_queue_pairs; qp_id++) { - struct kasumi_qp *qp = dev->data->queue_pairs[qp_id]; - - stats->enqueued_count += qp->qp_stats.enqueued_count; - stats->dequeued_count += qp->qp_stats.dequeued_count; - - stats->enqueue_err_count += qp->qp_stats.enqueue_err_count; - stats->dequeue_err_count += qp->qp_stats.dequeue_err_count; - } -} - -/** Reset device statistics */ -static void -kasumi_pmd_stats_reset(struct rte_cryptodev *dev) -{ - int qp_id; - - for (qp_id = 0; qp_id < dev->data->nb_queue_pairs; qp_id++) { - struct kasumi_qp *qp = dev->data->queue_pairs[qp_id]; - - memset(&qp->qp_stats, 0, sizeof(qp->qp_stats)); - } -} - - -/** Get device info */ -static void -kasumi_pmd_info_get(struct rte_cryptodev *dev, - struct rte_cryptodev_info *dev_info) -{ - struct kasumi_private *internals = dev->data->dev_private; - - if (dev_info != NULL) { - dev_info->driver_id = dev->driver_id; - dev_info->max_nb_queue_pairs = internals->max_nb_queue_pairs; - /* No limit of number of sessions */ - dev_info->sym.max_nb_sessions = 0; - dev_info->feature_flags = dev->feature_flags; - dev_info->capabilities = kasumi_pmd_capabilities; - } -} - -/** Release queue pair */ -static int -kasumi_pmd_qp_release(struct rte_cryptodev *dev, uint16_t qp_id) -{ - struct kasumi_qp *qp = dev->data->queue_pairs[qp_id]; - - if (qp != NULL) { - rte_ring_free(qp->processed_ops); - rte_free(qp); - dev->data->queue_pairs[qp_id] = NULL; - } - return 0; -} - -/** set a unique name for the queue pair based on its name, dev_id and qp_id */ -static int -kasumi_pmd_qp_set_unique_name(struct rte_cryptodev *dev, - struct kasumi_qp *qp) -{ - unsigned n = snprintf(qp->name, sizeof(qp->name), - "kasumi_pmd_%u_qp_%u", - dev->data->dev_id, qp->id); - - if (n >= sizeof(qp->name)) - return -1; - - return 0; -} - -/** Create a ring to place processed ops on */ -static struct rte_ring * -kasumi_pmd_qp_create_processed_ops_ring(struct kasumi_qp *qp, - unsigned ring_size, int socket_id) -{ - struct rte_ring *r; - - r = rte_ring_lookup(qp->name); - if (r) { - if (rte_ring_get_size(r) == ring_size) { - KASUMI_LOG(INFO, "Reusing existing ring %s" - " for processed packets", - qp->name); - return r; - } - - KASUMI_LOG(ERR, "Unable to reuse existing ring %s" - " for processed packets", - qp->name); - return NULL; - } - - return rte_ring_create(qp->name, ring_size, socket_id, - RING_F_SP_ENQ | RING_F_SC_DEQ); -} - -/** Setup a queue pair */ -static int -kasumi_pmd_qp_setup(struct rte_cryptodev *dev, uint16_t qp_id, - const struct rte_cryptodev_qp_conf *qp_conf, - int socket_id) -{ - struct kasumi_qp *qp = NULL; - struct kasumi_private *internals = dev->data->dev_private; - - /* Free memory prior to re-allocation if needed. */ - if (dev->data->queue_pairs[qp_id] != NULL) - kasumi_pmd_qp_release(dev, qp_id); - - /* Allocate the queue pair data structure. */ - qp = rte_zmalloc_socket("KASUMI PMD Queue Pair", sizeof(*qp), - RTE_CACHE_LINE_SIZE, socket_id); - if (qp == NULL) - return (-ENOMEM); - - qp->id = qp_id; - dev->data->queue_pairs[qp_id] = qp; - - if (kasumi_pmd_qp_set_unique_name(dev, qp)) - goto qp_setup_cleanup; - - qp->processed_ops = kasumi_pmd_qp_create_processed_ops_ring(qp, - qp_conf->nb_descriptors, socket_id); - if (qp->processed_ops == NULL) - goto qp_setup_cleanup; - - qp->mgr = internals->mgr; - qp->sess_mp = qp_conf->mp_session; - qp->sess_mp_priv = qp_conf->mp_session_private; - - memset(&qp->qp_stats, 0, sizeof(qp->qp_stats)); - - return 0; - -qp_setup_cleanup: - rte_free(qp); - - return -1; -} - -/** Returns the size of the KASUMI session structure */ -static unsigned -kasumi_pmd_sym_session_get_size(struct rte_cryptodev *dev __rte_unused) -{ - return sizeof(struct kasumi_session); -} - -/** Configure a KASUMI session from a crypto xform chain */ -static int -kasumi_pmd_sym_session_configure(struct rte_cryptodev *dev, - struct rte_crypto_sym_xform *xform, - struct rte_cryptodev_sym_session *sess, - struct rte_mempool *mempool) -{ - void *sess_private_data; - int ret; - struct kasumi_private *internals = dev->data->dev_private; - - if (unlikely(sess == NULL)) { - KASUMI_LOG(ERR, "invalid session struct"); - return -EINVAL; - } - - if (rte_mempool_get(mempool, &sess_private_data)) { - KASUMI_LOG(ERR, - "Couldn't get object from session mempool"); - return -ENOMEM; - } - - ret = kasumi_set_session_parameters(internals->mgr, - sess_private_data, xform); - if (ret != 0) { - KASUMI_LOG(ERR, "failed configure session parameters"); - - /* Return session to mempool */ - rte_mempool_put(mempool, sess_private_data); - return ret; - } - - set_sym_session_private_data(sess, dev->driver_id, - sess_private_data); - - return 0; -} - -/** Clear the memory of session so it doesn't leave key material behind */ -static void -kasumi_pmd_sym_session_clear(struct rte_cryptodev *dev, - struct rte_cryptodev_sym_session *sess) -{ - uint8_t index = dev->driver_id; - void *sess_priv = get_sym_session_private_data(sess, index); - - /* Zero out the whole structure */ - if (sess_priv) { - memset(sess_priv, 0, sizeof(struct kasumi_session)); - struct rte_mempool *sess_mp = rte_mempool_from_obj(sess_priv); - set_sym_session_private_data(sess, index, NULL); - rte_mempool_put(sess_mp, sess_priv); - } -} - -struct rte_cryptodev_ops kasumi_pmd_ops = { - .dev_configure = kasumi_pmd_config, - .dev_start = kasumi_pmd_start, - .dev_stop = kasumi_pmd_stop, - .dev_close = kasumi_pmd_close, - - .stats_get = kasumi_pmd_stats_get, - .stats_reset = kasumi_pmd_stats_reset, - - .dev_infos_get = kasumi_pmd_info_get, - - .queue_pair_setup = kasumi_pmd_qp_setup, - .queue_pair_release = kasumi_pmd_qp_release, - - .sym_session_get_size = kasumi_pmd_sym_session_get_size, - .sym_session_configure = kasumi_pmd_sym_session_configure, - .sym_session_clear = kasumi_pmd_sym_session_clear -}; - -struct rte_cryptodev_ops *rte_kasumi_pmd_ops = &kasumi_pmd_ops; diff --git a/drivers/crypto/kasumi/version.map b/drivers/crypto/kasumi/version.map deleted file mode 100644 index 4a76d1d52d..0000000000 --- a/drivers/crypto/kasumi/version.map +++ /dev/null @@ -1,3 +0,0 @@ -DPDK_21 { - local: *; -}; diff --git a/drivers/crypto/meson.build b/drivers/crypto/meson.build index 0ba62e94ca..c1f854977c 100644 --- a/drivers/crypto/meson.build +++ b/drivers/crypto/meson.build @@ -13,7 +13,6 @@ drivers = [ 'ccp', 'dpaa_sec', 'dpaa2_sec', - 'kasumi', 'mvsam', 'nitrox', 'null', -- 2.25.1 -------------------------------------------------------------- Intel Research and Development Ireland Limited Registered in Ireland Registered Office: Collinstown Industrial Park, Leixlip, County Kildare Registered Number: 308263 This e-mail and any attachments may contain confidential material for the sole use of the intended recipient(s). Any review or distribution by others is strictly prohibited. If you are not the intended recipient, please contact the sender and delete all copies.