From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id A14C2A0C46; Fri, 18 Jun 2021 14:20:16 +0200 (CEST) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 1DFB7410EF; Fri, 18 Jun 2021 14:19:49 +0200 (CEST) Received: from mga07.intel.com (mga07.intel.com [134.134.136.100]) by mails.dpdk.org (Postfix) with ESMTP id D6864410EF for ; Fri, 18 Jun 2021 14:19:47 +0200 (CEST) IronPort-SDR: giiz4e8XQ9jLrc0Ch1ZRVXtdlqM1kb6ZcNSHCgSUy0FZjQdDSzR4z0Yq9UM1fPlMcmRtIsQ9nz h5GzX3Wb8yMA== X-IronPort-AV: E=McAfee;i="6200,9189,10018"; a="270392123" X-IronPort-AV: E=Sophos;i="5.83,283,1616482800"; d="scan'208";a="270392123" Received: from fmsmga001.fm.intel.com ([10.253.24.23]) by orsmga105.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 18 Jun 2021 05:19:47 -0700 IronPort-SDR: 3EY+gjZDi7yEAK81iaofojk/57an+e+EJ4yNONPvUM155bdslglOVH9fOcAueo34bIRPQo0hX2 b+CArTrS7aOA== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.83,283,1616482800"; d="scan'208";a="555560560" Received: from silpixa00400320.ir.intel.com ([10.237.214.214]) by fmsmga001.fm.intel.com with ESMTP; 18 Jun 2021 05:19:45 -0700 From: pbronowx To: dev@dpdk.org Cc: roy.fan.zhang@intel.com, thomas@monjalon.net, gakhil@marvell.com, ferruh.yigit@intel.com, declan.doherty@intel.com, pbronowx Date: Fri, 18 Jun 2021 12:18:03 +0000 Message-Id: <20210618121803.1189857-8-piotrx.bronowski@intel.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210618121803.1189857-1-piotrx.bronowski@intel.com> References: <20210618121803.1189857-1-piotrx.bronowski@intel.com> MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Subject: [dpdk-dev] [RFC 7/7] crypto/ipsec_mb: move zuc PMD to ipsec_mb framework X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" This patch removes crypto/zuc folder and gathers all zuc PMD implementation specific details into single file pmd_zuc.c in crypto/ipsec_mb folder. Signed-off-by: pbronowx --- drivers/crypto/ipsec_mb/meson.build | 3 +- .../{zuc/rte_zuc_pmd.c => ipsec_mb/pmd_zuc.c} | 456 ++++++++---------- .../ipsec_mb/rte_ipsec_mb_pmd_private.h | 7 + drivers/crypto/meson.build | 1 - drivers/crypto/zuc/meson.build | 24 - drivers/crypto/zuc/rte_zuc_pmd_ops.c | 322 ------------- drivers/crypto/zuc/version.map | 3 - drivers/crypto/zuc/zuc_pmd_private.h | 83 ---- 8 files changed, 198 insertions(+), 701 deletions(-) rename drivers/crypto/{zuc/rte_zuc_pmd.c => ipsec_mb/pmd_zuc.c} (52%) delete mode 100644 drivers/crypto/zuc/meson.build delete mode 100644 drivers/crypto/zuc/rte_zuc_pmd_ops.c delete mode 100644 drivers/crypto/zuc/version.map delete mode 100644 drivers/crypto/zuc/zuc_pmd_private.h diff --git a/drivers/crypto/ipsec_mb/meson.build b/drivers/crypto/ipsec_mb/meson.build index 5a09248039..9ed0a280cd 100644 --- a/drivers/crypto/ipsec_mb/meson.build +++ b/drivers/crypto/ipsec_mb/meson.build @@ -26,6 +26,7 @@ sources = files('rte_ipsec_mb_pmd.c', 'pmd_aesni_mb.c', 'pmd_aesni_gcm.c', 'pmd_kasumi.c', - 'pmd_snow3g.c' + 'pmd_snow3g.c', + 'pmd_zuc.c' ) deps += ['bus_vdev', 'net', 'security'] diff --git a/drivers/crypto/zuc/rte_zuc_pmd.c b/drivers/crypto/ipsec_mb/pmd_zuc.c similarity index 52% rename from drivers/crypto/zuc/rte_zuc_pmd.c rename to drivers/crypto/ipsec_mb/pmd_zuc.c index 42b669f188..7a3c417eec 100644 --- a/drivers/crypto/zuc/rte_zuc_pmd.c +++ b/drivers/crypto/ipsec_mb/pmd_zuc.c @@ -1,85 +1,111 @@ /* SPDX-License-Identifier: BSD-3-Clause - * Copyright(c) 2016-2018 Intel Corporation + * Copyright(c) 2015-2021 Intel Corporation */ -#include -#include -#include -#include -#include -#include -#include - -#include "zuc_pmd_private.h" -#define ZUC_MAX_BURST 16 -#define BYTE_LEN 8 +/** + * function used to process crypto workload using job API + */ +#include -static uint8_t cryptodev_driver_id; +#if defined(RTE_LIB_SECURITY) && (IMB_VERSION_NUM) >= IMB_VERSION(0, 54, 0) +#define AESNI_MB_DOCSIS_SEC_ENABLED 1 +#include +#include +#include +#endif -/** Get xform chain order. */ -static enum zuc_operation -zuc_get_mode(const struct rte_crypto_sym_xform *xform) -{ - if (xform == NULL) - return ZUC_OP_NOT_SUPPORTED; - - if (xform->next) - if (xform->next->next != NULL) - return ZUC_OP_NOT_SUPPORTED; - - if (xform->type == RTE_CRYPTO_SYM_XFORM_AUTH) { - if (xform->next == NULL) - return ZUC_OP_ONLY_AUTH; - else if (xform->next->type == RTE_CRYPTO_SYM_XFORM_CIPHER) - return ZUC_OP_AUTH_CIPHER; - else - return ZUC_OP_NOT_SUPPORTED; - } +#include "rte_ipsec_mb_pmd_private.h" - if (xform->type == RTE_CRYPTO_SYM_XFORM_CIPHER) { - if (xform->next == NULL) - return ZUC_OP_ONLY_CIPHER; - else if (xform->next->type == RTE_CRYPTO_SYM_XFORM_AUTH) - return ZUC_OP_CIPHER_AUTH; - else - return ZUC_OP_NOT_SUPPORTED; - } +#define ZUC_IV_KEY_LENGTH 16 +#define ZUC_DIGEST_LENGTH 4 +#define ZUC_MAX_BURST 16 +#define BYTE_LEN 8 - return ZUC_OP_NOT_SUPPORTED; -} +uint8_t pmd_driver_id_zuc; + +static const struct rte_cryptodev_capabilities zuc_capabilities[] = { + { /* ZUC (EIA3) */ + .op = RTE_CRYPTO_OP_TYPE_SYMMETRIC, + {.sym = { + .xform_type = RTE_CRYPTO_SYM_XFORM_AUTH, + {.auth = { + .algo = RTE_CRYPTO_AUTH_ZUC_EIA3, + .block_size = 16, + .key_size = { + .min = 16, + .max = 16, + .increment = 0 + }, + .digest_size = { + .min = 4, + .max = 4, + .increment = 0 + }, + .iv_size = { + .min = 16, + .max = 16, + .increment = 0 + } + }, } + }, } + }, + { /* ZUC (EEA3) */ + .op = RTE_CRYPTO_OP_TYPE_SYMMETRIC, + {.sym = { + .xform_type = RTE_CRYPTO_SYM_XFORM_CIPHER, + {.cipher = { + .algo = RTE_CRYPTO_CIPHER_ZUC_EEA3, + .block_size = 16, + .key_size = { + .min = 16, + .max = 16, + .increment = 0 + }, + .iv_size = { + .min = 16, + .max = 16, + .increment = 0 + }, + }, } + }, } + }, + RTE_CRYPTODEV_END_OF_CAPABILITIES_LIST() +}; +/** ZUC private session structure */ +struct zuc_session { + enum ipsec_mb_operation op; + enum rte_crypto_auth_operation auth_op; + uint8_t pKey_cipher[ZUC_IV_KEY_LENGTH]; + uint8_t pKey_hash[ZUC_IV_KEY_LENGTH]; + uint16_t cipher_iv_offset; + uint16_t auth_iv_offset; +} __rte_cache_aligned; + +struct zuc_qp_data { + + uint8_t temp_digest[ZUC_DIGEST_LENGTH]; + /* *< Buffers used to store the digest generated + * by the driver when verifying a digest provided + * by the user (using authentication verify operation) + */ +}; /** Parse crypto xform chain and set private session parameters. */ -int -zuc_set_session_parameters(struct zuc_session *sess, +static int +zuc_session_configure(__rte_unused IMB_MGR * mgr, void *zuc_sess, const struct rte_crypto_sym_xform *xform) { + struct zuc_session *sess = (struct zuc_session *) zuc_sess; const struct rte_crypto_sym_xform *auth_xform = NULL; const struct rte_crypto_sym_xform *cipher_xform = NULL; - enum zuc_operation mode; - + enum ipsec_mb_operation mode; /* Select Crypto operation - hash then cipher / cipher then hash */ - mode = zuc_get_mode(xform); + int ret = ipsec_mb_parse_xform(xform, &mode, &auth_xform, + &cipher_xform, NULL); - switch (mode) { - case ZUC_OP_CIPHER_AUTH: - auth_xform = xform->next; - - /* Fall-through */ - case ZUC_OP_ONLY_CIPHER: - cipher_xform = xform; - break; - case ZUC_OP_AUTH_CIPHER: - cipher_xform = xform->next; - /* Fall-through */ - case ZUC_OP_ONLY_AUTH: - auth_xform = xform; - break; - case ZUC_OP_NOT_SUPPORTED: - default: - ZUC_LOG(ERR, "Unsupported operation chain order parameter"); - return -ENOTSUP; - } + if (ret) + return ret; if (cipher_xform) { /* Only ZUC EEA3 supported */ @@ -87,7 +113,7 @@ zuc_set_session_parameters(struct zuc_session *sess, return -ENOTSUP; if (cipher_xform->cipher.iv.length != ZUC_IV_KEY_LENGTH) { - ZUC_LOG(ERR, "Wrong IV length"); + IPSEC_MB_LOG(ERR, "Wrong IV length"); return -EINVAL; } sess->cipher_iv_offset = cipher_xform->cipher.iv.offset; @@ -103,14 +129,14 @@ zuc_set_session_parameters(struct zuc_session *sess, return -ENOTSUP; if (auth_xform->auth.digest_length != ZUC_DIGEST_LENGTH) { - ZUC_LOG(ERR, "Wrong digest length"); + IPSEC_MB_LOG(ERR, "Wrong digest length"); return -EINVAL; } sess->auth_op = auth_xform->auth.op; if (auth_xform->auth.iv.length != ZUC_IV_KEY_LENGTH) { - ZUC_LOG(ERR, "Wrong IV length"); + IPSEC_MB_LOG(ERR, "Wrong IV length"); return -EINVAL; } sess->auth_iv_offset = auth_xform->auth.iv.offset; @@ -120,61 +146,17 @@ zuc_set_session_parameters(struct zuc_session *sess, ZUC_IV_KEY_LENGTH); } - sess->op = mode; - return 0; } -/** Get ZUC session. */ -static struct zuc_session * -zuc_get_session(struct zuc_qp *qp, struct rte_crypto_op *op) -{ - struct zuc_session *sess = NULL; - - if (op->sess_type == RTE_CRYPTO_OP_WITH_SESSION) { - if (likely(op->sym->session != NULL)) - sess = (struct zuc_session *)get_sym_session_private_data( - op->sym->session, - cryptodev_driver_id); - } else { - void *_sess = NULL; - void *_sess_private_data = NULL; - - if (rte_mempool_get(qp->sess_mp, (void **)&_sess)) - return NULL; - - if (rte_mempool_get(qp->sess_mp_priv, - (void **)&_sess_private_data)) - return NULL; - - sess = (struct zuc_session *)_sess_private_data; - - if (unlikely(zuc_set_session_parameters(sess, - op->sym->xform) != 0)) { - rte_mempool_put(qp->sess_mp, _sess); - rte_mempool_put(qp->sess_mp_priv, _sess_private_data); - sess = NULL; - } - op->sym->session = (struct rte_cryptodev_sym_session *)_sess; - set_sym_session_private_data(op->sym->session, - cryptodev_driver_id, _sess_private_data); - } - - if (unlikely(sess == NULL)) - op->status = RTE_CRYPTO_OP_STATUS_INVALID_SESSION; - - - return sess; -} - /** Encrypt/decrypt mbufs. */ static uint8_t -process_zuc_cipher_op(struct zuc_qp *qp, struct rte_crypto_op **ops, +process_zuc_cipher_op(struct ipsec_mb_qp *qp, struct rte_crypto_op **ops, struct zuc_session **sessions, uint8_t num_ops) { - unsigned i; + unsigned int i; uint8_t processed_ops = 0; const void *src[ZUC_MAX_BURST]; void *dst[ZUC_MAX_BURST]; @@ -188,7 +170,7 @@ process_zuc_cipher_op(struct zuc_qp *qp, struct rte_crypto_op **ops, || ((ops[i]->sym->cipher.data.offset % BYTE_LEN) != 0)) { ops[i]->status = RTE_CRYPTO_OP_STATUS_INVALID_ARGS; - ZUC_LOG(ERR, "Data Length or offset"); + IPSEC_MB_LOG(ERR, "Data Length or offset"); break; } @@ -199,9 +181,14 @@ process_zuc_cipher_op(struct zuc_qp *qp, struct rte_crypto_op **ops, (ops[i]->sym->m_dst != NULL && !rte_pktmbuf_is_contiguous( ops[i]->sym->m_dst))) { - ZUC_LOG(ERR, "PMD supports only contiguous mbufs, " - "op (%p) provides noncontiguous mbuf as " - "source/destination buffer.\n", ops[i]); + IPSEC_MB_LOG(ERR, "PMD supports only " + " contiguous mbufs, op (%p) " + "provides noncontiguous mbuf " + "as source/destination buffer.\n", + "PMD supports only contiguous mbufs, " + "op (%p) provides noncontiguous mbuf " + "as source/destination buffer.\n", + ops[i]); ops[i]->status = RTE_CRYPTO_OP_STATUS_INVALID_ARGS; break; } @@ -232,24 +219,26 @@ process_zuc_cipher_op(struct zuc_qp *qp, struct rte_crypto_op **ops, /** Generate/verify hash from mbufs. */ static int -process_zuc_hash_op(struct zuc_qp *qp, struct rte_crypto_op **ops, +process_zuc_hash_op(struct ipsec_mb_qp *qp, struct rte_crypto_op **ops, struct zuc_session **sessions, uint8_t num_ops) { unsigned int i; uint8_t processed_ops = 0; - uint8_t *src[ZUC_MAX_BURST] = { 0 }; + uint8_t *src[ZUC_MAX_BURST]; uint32_t *dst[ZUC_MAX_BURST]; - uint32_t length_in_bits[ZUC_MAX_BURST] = { 0 }; - uint8_t *iv[ZUC_MAX_BURST] = { 0 }; - const void *hash_keys[ZUC_MAX_BURST] = { 0 }; + uint32_t length_in_bits[ZUC_MAX_BURST]; + uint8_t *iv[ZUC_MAX_BURST]; + const void *hash_keys[ZUC_MAX_BURST]; struct zuc_session *sess; + struct zuc_qp_data *qp_data = ipsec_mb_get_qp_private_data(qp); + for (i = 0; i < num_ops; i++) { /* Data must be byte aligned */ if ((ops[i]->sym->auth.data.offset % BYTE_LEN) != 0) { ops[i]->status = RTE_CRYPTO_OP_STATUS_INVALID_ARGS; - ZUC_LOG(ERR, "Offset"); + IPSEC_MB_LOG(ERR, "Offset"); break; } @@ -264,7 +253,7 @@ process_zuc_hash_op(struct zuc_qp *qp, struct rte_crypto_op **ops, hash_keys[i] = sess->pKey_hash; if (sess->auth_op == RTE_CRYPTO_AUTH_OP_VERIFY) - dst[i] = (uint32_t *)qp->temp_digest; + dst[i] = (uint32_t *)qp_data->temp_digest; else dst[i] = (uint32_t *)ops[i]->sym->auth.digest.data; @@ -289,36 +278,39 @@ process_zuc_hash_op(struct zuc_qp *qp, struct rte_crypto_op **ops, if (sessions[i]->auth_op == RTE_CRYPTO_AUTH_OP_VERIFY) if (memcmp(dst[i], ops[i]->sym->auth.digest.data, ZUC_DIGEST_LENGTH) != 0) - ops[i]->status = RTE_CRYPTO_OP_STATUS_AUTH_FAILED; + ops[i]->status = + RTE_CRYPTO_OP_STATUS_AUTH_FAILED; return processed_ops; } /** Process a batch of crypto ops which shares the same operation type. */ static int -process_ops(struct rte_crypto_op **ops, enum zuc_operation op_type, +process_ops(struct rte_crypto_op **ops, enum ipsec_mb_operation op_type, struct zuc_session **sessions, - struct zuc_qp *qp, uint8_t num_ops, + struct ipsec_mb_qp *qp, uint8_t num_ops, uint16_t *accumulated_enqueued_ops) { - unsigned i; - unsigned enqueued_ops, processed_ops; + unsigned int i; + unsigned int processed_ops; switch (op_type) { - case ZUC_OP_ONLY_CIPHER: + case IPSEC_MB_OP_ENCRYPT_ONLY: + case IPSEC_MB_OP_DECRYPT_ONLY: processed_ops = process_zuc_cipher_op(qp, ops, sessions, num_ops); break; - case ZUC_OP_ONLY_AUTH: + case IPSEC_MB_OP_HASH_GEN_ONLY: + case IPSEC_MB_OP_HASH_VERIFY_ONLY: processed_ops = process_zuc_hash_op(qp, ops, sessions, num_ops); break; - case ZUC_OP_CIPHER_AUTH: + case IPSEC_MB_OP_ENCRYPT_THEN_HASH_GEN: processed_ops = process_zuc_cipher_op(qp, ops, sessions, num_ops); process_zuc_hash_op(qp, ops, sessions, processed_ops); break; - case ZUC_OP_AUTH_CIPHER: + case IPSEC_MB_OP_HASH_VERIFY_THEN_DECRYPT: processed_ops = process_zuc_hash_op(qp, ops, sessions, num_ops); process_zuc_cipher_op(qp, ops, sessions, processed_ops); @@ -347,35 +339,38 @@ process_ops(struct rte_crypto_op **ops, enum zuc_operation op_type, } } - enqueued_ops = rte_ring_enqueue_burst(qp->processed_ops, - (void **)ops, processed_ops, NULL); - qp->qp_stats.enqueued_count += enqueued_ops; - *accumulated_enqueued_ops += enqueued_ops; + *accumulated_enqueued_ops += processed_ops; - return enqueued_ops; + return processed_ops; } static uint16_t -zuc_pmd_enqueue_burst(void *queue_pair, struct rte_crypto_op **ops, - uint16_t nb_ops) +zuc_pmd_dequeue_burst(void *queue_pair, + struct rte_crypto_op **c_ops, uint16_t nb_ops) { - struct rte_crypto_op *c_ops[ZUC_MAX_BURST]; + struct rte_crypto_op *curr_c_op; struct zuc_session *curr_sess; struct zuc_session *sessions[ZUC_MAX_BURST]; - enum zuc_operation prev_zuc_op = ZUC_OP_NOT_SUPPORTED; - enum zuc_operation curr_zuc_op; - struct zuc_qp *qp = queue_pair; - unsigned i; + enum ipsec_mb_operation prev_zuc_op = IPSEC_MB_OP_NOT_SUPPORTED; + enum ipsec_mb_operation curr_zuc_op; + struct ipsec_mb_qp *qp = queue_pair; + unsigned int nb_dequeued; + unsigned int i; uint8_t burst_size = 0; uint16_t enqueued_ops = 0; uint8_t processed_ops; - for (i = 0; i < nb_ops; i++) { - curr_c_op = ops[i]; + nb_dequeued = rte_ring_dequeue_burst(qp->ingress_queue, + (void **)c_ops, nb_ops, NULL); + - curr_sess = zuc_get_session(qp, curr_c_op); + for (i = 0; i < nb_dequeued; i++) { + curr_c_op = c_ops[i]; + + curr_sess = (struct zuc_session *) + ipsec_mb_get_session_private(qp, curr_c_op); if (unlikely(curr_sess == NULL)) { curr_c_op->status = RTE_CRYPTO_OP_STATUS_INVALID_SESSION; @@ -441,143 +436,70 @@ zuc_pmd_enqueue_burst(void *queue_pair, struct rte_crypto_op **ops, &enqueued_ops); } - qp->qp_stats.enqueue_err_count += nb_ops - enqueued_ops; - return enqueued_ops; -} - -static uint16_t -zuc_pmd_dequeue_burst(void *queue_pair, - struct rte_crypto_op **c_ops, uint16_t nb_ops) -{ - struct zuc_qp *qp = queue_pair; - - unsigned nb_dequeued; - - nb_dequeued = rte_ring_dequeue_burst(qp->processed_ops, - (void **)c_ops, nb_ops, NULL); - qp->qp_stats.dequeued_count += nb_dequeued; - return nb_dequeued; + qp->stats.dequeued_count += i; + return i; } -static int cryptodev_zuc_remove(struct rte_vdev_device *vdev); +struct rte_cryptodev_ops zuc_pmd_ops = { + .dev_configure = ipsec_mb_pmd_config, + .dev_start = ipsec_mb_pmd_start, + .dev_stop = ipsec_mb_pmd_stop, + .dev_close = ipsec_mb_pmd_close, -static int -cryptodev_zuc_create(const char *name, - struct rte_vdev_device *vdev, - struct rte_cryptodev_pmd_init_params *init_params) -{ - struct rte_cryptodev *dev; - struct zuc_private *internals; - MB_MGR *mb_mgr; - - dev = rte_cryptodev_pmd_create(name, &vdev->device, init_params); - if (dev == NULL) { - ZUC_LOG(ERR, "failed to create cryptodev vdev"); - goto init_error; - } - - dev->feature_flags = RTE_CRYPTODEV_FF_SYMMETRIC_CRYPTO | - RTE_CRYPTODEV_FF_SYM_OPERATION_CHAINING | - RTE_CRYPTODEV_FF_NON_BYTE_ALIGNED_DATA | - RTE_CRYPTODEV_FF_SYM_SESSIONLESS | - RTE_CRYPTODEV_FF_OOP_LB_IN_LB_OUT; - - mb_mgr = alloc_mb_mgr(0); - if (mb_mgr == NULL) - return -ENOMEM; - - if (rte_cpu_get_flag_enabled(RTE_CPUFLAG_AVX512F)) { - dev->feature_flags |= RTE_CRYPTODEV_FF_CPU_AVX512; - init_mb_mgr_avx512(mb_mgr); - } else if (rte_cpu_get_flag_enabled(RTE_CPUFLAG_AVX2)) { - dev->feature_flags |= RTE_CRYPTODEV_FF_CPU_AVX2; - init_mb_mgr_avx2(mb_mgr); - } else if (rte_cpu_get_flag_enabled(RTE_CPUFLAG_AVX)) { - dev->feature_flags |= RTE_CRYPTODEV_FF_CPU_AVX; - init_mb_mgr_avx(mb_mgr); - } else { - dev->feature_flags |= RTE_CRYPTODEV_FF_CPU_SSE; - init_mb_mgr_sse(mb_mgr); - } + .stats_get = ipsec_mb_pmd_stats_get, + .stats_reset = ipsec_mb_pmd_stats_reset, - dev->driver_id = cryptodev_driver_id; - dev->dev_ops = rte_zuc_pmd_ops; + .dev_infos_get = ipsec_mb_pmd_info_get, - /* Register RX/TX burst functions for data path. */ - dev->dequeue_burst = zuc_pmd_dequeue_burst; - dev->enqueue_burst = zuc_pmd_enqueue_burst; + .queue_pair_setup = ipsec_mb_pmd_qp_setup, + .queue_pair_release = ipsec_mb_pmd_qp_release, - internals = dev->data->dev_private; - internals->mb_mgr = mb_mgr; - - internals->max_nb_queue_pairs = init_params->max_nb_queue_pairs; - - return 0; -init_error: - ZUC_LOG(ERR, "driver %s: failed", - init_params->name); + .sym_session_get_size = ipsec_mb_pmd_sym_session_get_size, + .sym_session_configure = ipsec_mb_pmd_sym_session_configure, + .sym_session_clear = ipsec_mb_pmd_sym_session_clear +}; - cryptodev_zuc_remove(vdev); - return -EFAULT; -} +struct rte_cryptodev_ops *rte_zuc_pmd_ops = &zuc_pmd_ops; static int cryptodev_zuc_probe(struct rte_vdev_device *vdev) { - struct rte_cryptodev_pmd_init_params init_params = { - "", - sizeof(struct zuc_private), - rte_socket_id(), - RTE_CRYPTODEV_PMD_DEFAULT_MAX_NB_QUEUE_PAIRS - }; - const char *name; - const char *input_args; - - name = rte_vdev_device_name(vdev); - if (name == NULL) - return -EINVAL; - input_args = rte_vdev_device_args(vdev); - - rte_cryptodev_pmd_parse_input_args(&init_params, input_args); - - return cryptodev_zuc_create(name, vdev, &init_params); -} - -static int -cryptodev_zuc_remove(struct rte_vdev_device *vdev) -{ - - struct rte_cryptodev *cryptodev; - const char *name; - struct zuc_private *internals; - - name = rte_vdev_device_name(vdev); - if (name == NULL) - return -EINVAL; - - cryptodev = rte_cryptodev_pmd_get_named_dev(name); - if (cryptodev == NULL) - return -ENODEV; - - internals = cryptodev->data->dev_private; - - free_mb_mgr(internals->mb_mgr); - - return rte_cryptodev_pmd_destroy(cryptodev); + return cryptodev_ipsec_mb_create(vdev, IPSEC_MB_PMD_TYPE_ZUC); } static struct rte_vdev_driver cryptodev_zuc_pmd_drv = { .probe = cryptodev_zuc_probe, - .remove = cryptodev_zuc_remove + .remove = cryptodev_ipsec_mb_remove + }; static struct cryptodev_driver zuc_crypto_drv; RTE_PMD_REGISTER_VDEV(CRYPTODEV_NAME_ZUC_PMD, cryptodev_zuc_pmd_drv); +RTE_PMD_REGISTER_ALIAS(CRYPTODEV_NAME_ZUC_PMD, cryptodev_zuc_pmd); RTE_PMD_REGISTER_PARAM_STRING(CRYPTODEV_NAME_ZUC_PMD, - "max_nb_queue_pairs= " - "socket_id="); + "max_nb_queue_pairs= socket_id="); RTE_PMD_REGISTER_CRYPTO_DRIVER(zuc_crypto_drv, cryptodev_zuc_pmd_drv.driver, - cryptodev_driver_id); -RTE_LOG_REGISTER_DEFAULT(zuc_logtype_driver, INFO); + pmd_driver_id_zuc); + +/* Constructor function to register zuc PMD */ +RTE_INIT(ipsec_mb_register_zuc) +{ + struct ipsec_mb_pmd_data *zuc_data + = &ipsec_mb_pmds[IPSEC_MB_PMD_TYPE_ZUC]; + + zuc_data->caps = zuc_capabilities; + zuc_data->dequeue_burst = zuc_pmd_dequeue_burst; + zuc_data->feature_flags = RTE_CRYPTODEV_FF_SYMMETRIC_CRYPTO + | RTE_CRYPTODEV_FF_SYM_OPERATION_CHAINING + | RTE_CRYPTODEV_FF_NON_BYTE_ALIGNED_DATA + | RTE_CRYPTODEV_FF_OOP_SGL_IN_LB_OUT + | RTE_CRYPTODEV_FF_SYM_SESSIONLESS + | RTE_CRYPTODEV_FF_OOP_LB_IN_LB_OUT; + zuc_data->internals_priv_size = 0; + zuc_data->ops = &zuc_pmd_ops; + zuc_data->qp_priv_size = sizeof(struct zuc_qp_data); + zuc_data->session_configure = zuc_session_configure; + zuc_data->session_priv_size = sizeof(struct zuc_session); +} diff --git a/drivers/crypto/ipsec_mb/rte_ipsec_mb_pmd_private.h b/drivers/crypto/ipsec_mb/rte_ipsec_mb_pmd_private.h index 9744129ea7..a81ab81f53 100644 --- a/drivers/crypto/ipsec_mb/rte_ipsec_mb_pmd_private.h +++ b/drivers/crypto/ipsec_mb/rte_ipsec_mb_pmd_private.h @@ -43,6 +43,9 @@ extern RTE_DEFINE_PER_LCORE(MB_MGR *, mb_mgr); #define CRYPTODEV_NAME_SNOW3G_PMD crypto_snow3g /**< IPSEC Multi buffer PMD snow3g device name */ +#define CRYPTODEV_NAME_ZUC_PMD crypto_zuc +/**< IPSEC Multi buffer PMD zuc device name */ + /** PMD LOGTYPE DRIVER, common to all PMDs */ extern int ipsec_mb_logtype_driver; #define IPSEC_MB_LOG(level, fmt, ...) \ @@ -55,6 +58,7 @@ enum ipsec_mb_pmd_types { IPSEC_MB_PMD_TYPE_AESNI_GCM, IPSEC_MB_PMD_TYPE_KASUMI, IPSEC_MB_PMD_TYPE_SNOW3G, + IPSEC_MB_PMD_TYPE_ZUC, IPSEC_MB_N_PMD_TYPES }; @@ -77,6 +81,7 @@ extern uint8_t pmd_driver_id_aesni_mb; extern uint8_t pmd_driver_id_aesni_gcm; extern uint8_t pmd_driver_id_kasumi; extern uint8_t pmd_driver_id_snow3g; +extern uint8_t pmd_driver_id_zuc; /** Helper function. Gets driver ID based on PMD type */ static __rte_always_inline uint8_t @@ -91,6 +96,8 @@ ipsec_mb_get_driver_id(enum ipsec_mb_pmd_types pmd_type) return pmd_driver_id_kasumi; case IPSEC_MB_PMD_TYPE_SNOW3G: return pmd_driver_id_snow3g; + case IPSEC_MB_PMD_TYPE_ZUC: + return pmd_driver_id_zuc; default: break; } diff --git a/drivers/crypto/meson.build b/drivers/crypto/meson.build index 2111be561b..cf27a05560 100644 --- a/drivers/crypto/meson.build +++ b/drivers/crypto/meson.build @@ -21,7 +21,6 @@ drivers = [ 'openssl', 'scheduler', 'virtio', - 'zuc', ] std_deps = ['cryptodev'] # cryptodev pulls in all other needed deps diff --git a/drivers/crypto/zuc/meson.build b/drivers/crypto/zuc/meson.build deleted file mode 100644 index a5f77a22d8..0000000000 --- a/drivers/crypto/zuc/meson.build +++ /dev/null @@ -1,24 +0,0 @@ -# SPDX-License-Identifier: BSD-3-Clause -# Copyright(c) 2018-2020 Intel Corporation - -IMB_required_ver = '0.53.0' -lib = cc.find_library('IPSec_MB', required: false) -if not lib.found() - build = false - reason = 'missing dependency, "libIPSec_MB"' -else - # version comes with quotes, so we split based on " and take the middle - imb_ver = cc.get_define('IMB_VERSION_STR', - prefix : '#include').split('"')[1] - - if (imb_ver == '') or (imb_ver.version_compare('<' + IMB_required_ver)) - reason = 'IPSec_MB version >= @0@ is required, found version @1@'.format( - IMB_required_ver, imb_ver) - build = false - endif - -endif - -ext_deps += lib -sources = files('rte_zuc_pmd.c', 'rte_zuc_pmd_ops.c') -deps += ['bus_vdev'] diff --git a/drivers/crypto/zuc/rte_zuc_pmd_ops.c b/drivers/crypto/zuc/rte_zuc_pmd_ops.c deleted file mode 100644 index dc9dc25ef8..0000000000 --- a/drivers/crypto/zuc/rte_zuc_pmd_ops.c +++ /dev/null @@ -1,322 +0,0 @@ -/* SPDX-License-Identifier: BSD-3-Clause - * Copyright(c) 2016-2018 Intel Corporation - */ - -#include - -#include -#include -#include - -#include "zuc_pmd_private.h" - -static const struct rte_cryptodev_capabilities zuc_pmd_capabilities[] = { - { /* ZUC (EIA3) */ - .op = RTE_CRYPTO_OP_TYPE_SYMMETRIC, - {.sym = { - .xform_type = RTE_CRYPTO_SYM_XFORM_AUTH, - {.auth = { - .algo = RTE_CRYPTO_AUTH_ZUC_EIA3, - .block_size = 16, - .key_size = { - .min = 16, - .max = 16, - .increment = 0 - }, - .digest_size = { - .min = 4, - .max = 4, - .increment = 0 - }, - .iv_size = { - .min = 16, - .max = 16, - .increment = 0 - } - }, } - }, } - }, - { /* ZUC (EEA3) */ - .op = RTE_CRYPTO_OP_TYPE_SYMMETRIC, - {.sym = { - .xform_type = RTE_CRYPTO_SYM_XFORM_CIPHER, - {.cipher = { - .algo = RTE_CRYPTO_CIPHER_ZUC_EEA3, - .block_size = 16, - .key_size = { - .min = 16, - .max = 16, - .increment = 0 - }, - .iv_size = { - .min = 16, - .max = 16, - .increment = 0 - }, - }, } - }, } - }, - RTE_CRYPTODEV_END_OF_CAPABILITIES_LIST() -}; - -/** Configure device */ -static int -zuc_pmd_config(__rte_unused struct rte_cryptodev *dev, - __rte_unused struct rte_cryptodev_config *config) -{ - return 0; -} - -/** Start device */ -static int -zuc_pmd_start(__rte_unused struct rte_cryptodev *dev) -{ - return 0; -} - -/** Stop device */ -static void -zuc_pmd_stop(__rte_unused struct rte_cryptodev *dev) -{ -} - -/** Close device */ -static int -zuc_pmd_close(__rte_unused struct rte_cryptodev *dev) -{ - return 0; -} - - -/** Get device statistics */ -static void -zuc_pmd_stats_get(struct rte_cryptodev *dev, - struct rte_cryptodev_stats *stats) -{ - int qp_id; - - for (qp_id = 0; qp_id < dev->data->nb_queue_pairs; qp_id++) { - struct zuc_qp *qp = dev->data->queue_pairs[qp_id]; - - stats->enqueued_count += qp->qp_stats.enqueued_count; - stats->dequeued_count += qp->qp_stats.dequeued_count; - - stats->enqueue_err_count += qp->qp_stats.enqueue_err_count; - stats->dequeue_err_count += qp->qp_stats.dequeue_err_count; - } -} - -/** Reset device statistics */ -static void -zuc_pmd_stats_reset(struct rte_cryptodev *dev) -{ - int qp_id; - - for (qp_id = 0; qp_id < dev->data->nb_queue_pairs; qp_id++) { - struct zuc_qp *qp = dev->data->queue_pairs[qp_id]; - - memset(&qp->qp_stats, 0, sizeof(qp->qp_stats)); - } -} - - -/** Get device info */ -static void -zuc_pmd_info_get(struct rte_cryptodev *dev, - struct rte_cryptodev_info *dev_info) -{ - struct zuc_private *internals = dev->data->dev_private; - - if (dev_info != NULL) { - dev_info->driver_id = dev->driver_id; - dev_info->max_nb_queue_pairs = internals->max_nb_queue_pairs; - /* No limit of number of sessions */ - dev_info->sym.max_nb_sessions = 0; - dev_info->feature_flags = dev->feature_flags; - dev_info->capabilities = zuc_pmd_capabilities; - } -} - -/** Release queue pair */ -static int -zuc_pmd_qp_release(struct rte_cryptodev *dev, uint16_t qp_id) -{ - if (dev->data->queue_pairs[qp_id] != NULL) { - struct zuc_qp *qp = dev->data->queue_pairs[qp_id]; - - if (qp->processed_ops) - rte_ring_free(qp->processed_ops); - - rte_free(dev->data->queue_pairs[qp_id]); - dev->data->queue_pairs[qp_id] = NULL; - } - return 0; -} - -/** set a unique name for the queue pair based on its name, dev_id and qp_id */ -static int -zuc_pmd_qp_set_unique_name(struct rte_cryptodev *dev, - struct zuc_qp *qp) -{ - unsigned n = snprintf(qp->name, sizeof(qp->name), - "zuc_pmd_%u_qp_%u", - dev->data->dev_id, qp->id); - - if (n >= sizeof(qp->name)) - return -1; - - return 0; -} - -/** Create a ring to place processed ops on */ -static struct rte_ring * -zuc_pmd_qp_create_processed_ops_ring(struct zuc_qp *qp, - unsigned ring_size, int socket_id) -{ - struct rte_ring *r; - - r = rte_ring_lookup(qp->name); - if (r) { - if (rte_ring_get_size(r) >= ring_size) { - ZUC_LOG(INFO, "Reusing existing ring %s" - " for processed packets", - qp->name); - return r; - } - - ZUC_LOG(ERR, "Unable to reuse existing ring %s" - " for processed packets", - qp->name); - return NULL; - } - - return rte_ring_create(qp->name, ring_size, socket_id, - RING_F_SP_ENQ | RING_F_SC_DEQ); -} - -/** Setup a queue pair */ -static int -zuc_pmd_qp_setup(struct rte_cryptodev *dev, uint16_t qp_id, - const struct rte_cryptodev_qp_conf *qp_conf, - int socket_id) -{ - struct zuc_qp *qp = NULL; - struct zuc_private *internals = dev->data->dev_private; - - /* Free memory prior to re-allocation if needed. */ - if (dev->data->queue_pairs[qp_id] != NULL) - zuc_pmd_qp_release(dev, qp_id); - - /* Allocate the queue pair data structure. */ - qp = rte_zmalloc_socket("ZUC PMD Queue Pair", sizeof(*qp), - RTE_CACHE_LINE_SIZE, socket_id); - if (qp == NULL) - return (-ENOMEM); - - qp->id = qp_id; - dev->data->queue_pairs[qp_id] = qp; - - if (zuc_pmd_qp_set_unique_name(dev, qp)) - goto qp_setup_cleanup; - - qp->processed_ops = zuc_pmd_qp_create_processed_ops_ring(qp, - qp_conf->nb_descriptors, socket_id); - if (qp->processed_ops == NULL) - goto qp_setup_cleanup; - - qp->mb_mgr = internals->mb_mgr; - qp->sess_mp = qp_conf->mp_session; - qp->sess_mp_priv = qp_conf->mp_session_private; - - memset(&qp->qp_stats, 0, sizeof(qp->qp_stats)); - - return 0; - -qp_setup_cleanup: - if (qp) - rte_free(qp); - - return -1; -} - -/** Returns the size of the ZUC session structure */ -static unsigned -zuc_pmd_sym_session_get_size(struct rte_cryptodev *dev __rte_unused) -{ - return sizeof(struct zuc_session); -} - -/** Configure a ZUC session from a crypto xform chain */ -static int -zuc_pmd_sym_session_configure(struct rte_cryptodev *dev __rte_unused, - struct rte_crypto_sym_xform *xform, - struct rte_cryptodev_sym_session *sess, - struct rte_mempool *mempool) -{ - void *sess_private_data; - int ret; - - if (unlikely(sess == NULL)) { - ZUC_LOG(ERR, "invalid session struct"); - return -EINVAL; - } - - if (rte_mempool_get(mempool, &sess_private_data)) { - ZUC_LOG(ERR, - "Couldn't get object from session mempool"); - - return -ENOMEM; - } - - ret = zuc_set_session_parameters(sess_private_data, xform); - if (ret != 0) { - ZUC_LOG(ERR, "failed configure session parameters"); - - /* Return session to mempool */ - rte_mempool_put(mempool, sess_private_data); - return ret; - } - - set_sym_session_private_data(sess, dev->driver_id, - sess_private_data); - - return 0; -} - -/** Clear the memory of session so it doesn't leave key material behind */ -static void -zuc_pmd_sym_session_clear(struct rte_cryptodev *dev, - struct rte_cryptodev_sym_session *sess) -{ - uint8_t index = dev->driver_id; - void *sess_priv = get_sym_session_private_data(sess, index); - - /* Zero out the whole structure */ - if (sess_priv) { - memset(sess_priv, 0, sizeof(struct zuc_session)); - struct rte_mempool *sess_mp = rte_mempool_from_obj(sess_priv); - set_sym_session_private_data(sess, index, NULL); - rte_mempool_put(sess_mp, sess_priv); - } -} - -struct rte_cryptodev_ops zuc_pmd_ops = { - .dev_configure = zuc_pmd_config, - .dev_start = zuc_pmd_start, - .dev_stop = zuc_pmd_stop, - .dev_close = zuc_pmd_close, - - .stats_get = zuc_pmd_stats_get, - .stats_reset = zuc_pmd_stats_reset, - - .dev_infos_get = zuc_pmd_info_get, - - .queue_pair_setup = zuc_pmd_qp_setup, - .queue_pair_release = zuc_pmd_qp_release, - - .sym_session_get_size = zuc_pmd_sym_session_get_size, - .sym_session_configure = zuc_pmd_sym_session_configure, - .sym_session_clear = zuc_pmd_sym_session_clear -}; - -struct rte_cryptodev_ops *rte_zuc_pmd_ops = &zuc_pmd_ops; diff --git a/drivers/crypto/zuc/version.map b/drivers/crypto/zuc/version.map deleted file mode 100644 index 4a76d1d52d..0000000000 --- a/drivers/crypto/zuc/version.map +++ /dev/null @@ -1,3 +0,0 @@ -DPDK_21 { - local: *; -}; diff --git a/drivers/crypto/zuc/zuc_pmd_private.h b/drivers/crypto/zuc/zuc_pmd_private.h deleted file mode 100644 index d8684891ee..0000000000 --- a/drivers/crypto/zuc/zuc_pmd_private.h +++ /dev/null @@ -1,83 +0,0 @@ -/* SPDX-License-Identifier: BSD-3-Clause - * Copyright(c) 2016-2018 Intel Corporation - */ - -#ifndef _ZUC_PMD_PRIVATE_H_ -#define _ZUC_PMD_PRIVATE_H_ - -#include - -#define CRYPTODEV_NAME_ZUC_PMD crypto_zuc -/**< ZUC PMD device name */ - -/** ZUC PMD LOGTYPE DRIVER */ -extern int zuc_logtype_driver; -#define ZUC_LOG(level, fmt, ...) \ - rte_log(RTE_LOG_ ## level, zuc_logtype_driver, \ - "%s()... line %u: " fmt "\n", __func__, __LINE__, \ - ## __VA_ARGS__) - -#define ZUC_IV_KEY_LENGTH 16 -#define ZUC_DIGEST_LENGTH 4 - -/** private data structure for each virtual ZUC device */ -struct zuc_private { - unsigned max_nb_queue_pairs; - /**< Max number of queue pairs supported by device */ - MB_MGR *mb_mgr; - /**< Multi-buffer instance */ -}; - -/** ZUC buffer queue pair */ -struct zuc_qp { - uint16_t id; - /**< Queue Pair Identifier */ - char name[RTE_CRYPTODEV_NAME_MAX_LEN]; - /**< Unique Queue Pair Name */ - struct rte_ring *processed_ops; - /**< Ring for placing processed ops */ - struct rte_mempool *sess_mp; - /**< Session Mempool */ - struct rte_mempool *sess_mp_priv; - /**< Session Private Data Mempool */ - struct rte_cryptodev_stats qp_stats; - /**< Queue pair statistics */ - uint8_t temp_digest[ZUC_DIGEST_LENGTH]; - /**< Buffer used to store the digest generated - * by the driver when verifying a digest provided - * by the user (using authentication verify operation) - */ - MB_MGR *mb_mgr; - /**< Multi-buffer instance */ -} __rte_cache_aligned; - -enum zuc_operation { - ZUC_OP_ONLY_CIPHER, - ZUC_OP_ONLY_AUTH, - ZUC_OP_CIPHER_AUTH, - ZUC_OP_AUTH_CIPHER, - ZUC_OP_NOT_SUPPORTED -}; - -/** ZUC private session structure */ -struct zuc_session { - enum zuc_operation op; - enum rte_crypto_auth_operation auth_op; - uint8_t pKey_cipher[ZUC_IV_KEY_LENGTH]; - uint8_t pKey_hash[ZUC_IV_KEY_LENGTH]; - uint16_t cipher_iv_offset; - uint16_t auth_iv_offset; -} __rte_cache_aligned; - - -extern int -zuc_set_session_parameters(struct zuc_session *sess, - const struct rte_crypto_sym_xform *xform); - - -/** device specific operations function pointer structure */ -extern struct rte_cryptodev_ops *rte_zuc_pmd_ops; - - - -#endif /* _ZUC_PMD_PRIVATE_H_ */ -- 2.25.1 -------------------------------------------------------------- Intel Research and Development Ireland Limited Registered in Ireland Registered Office: Collinstown Industrial Park, Leixlip, County Kildare Registered Number: 308263 This e-mail and any attachments may contain confidential material for the sole use of the intended recipient(s). Any review or distribution by others is strictly prohibited. If you are not the intended recipient, please contact the sender and delete all copies.