From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id 59A0DA04B6; Mon, 12 Oct 2020 13:19:47 +0200 (CEST) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id A43251D6B1; Mon, 12 Oct 2020 13:19:45 +0200 (CEST) Received: from mga06.intel.com (mga06.intel.com [134.134.136.31]) by dpdk.org (Postfix) with ESMTP id 5C92D1D6AB for ; Mon, 12 Oct 2020 13:19:44 +0200 (CEST) IronPort-SDR: /QzORMBzIBoeQH5DNkD7ETXBojBwVBTA6Z2E4wTEKqfcD784iJo6U9qlcsI4snA4sOkDVQDY12 q25X4ewjQQBg== X-IronPort-AV: E=McAfee;i="6000,8403,9771"; a="227369431" X-IronPort-AV: E=Sophos;i="5.77,366,1596524400"; d="scan'208";a="227369431" X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from fmsmga005.fm.intel.com ([10.253.24.32]) by orsmga104.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 12 Oct 2020 04:19:39 -0700 IronPort-SDR: IToeE9MglrAFpwExeVHN3xfelB6N2DsvFB4uRTqJxbum9/NF9ESaiDYmf+irM+YAIxaMrXJL5r qDyR8MMDRnEg== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.77,366,1596524400"; d="scan'208";a="520661355" Received: from silpixa00399593.ir.intel.com (HELO silpixa00399593.ger.corp.intel.com) ([10.237.223.27]) by fmsmga005.fm.intel.com with ESMTP; 12 Oct 2020 04:19:37 -0700 From: Pablo de Lara To: declan.doherty@intel.com Cc: dev@dpdk.org, Pablo de Lara , Fan Zhang Date: Mon, 12 Oct 2020 11:19:12 +0000 Message-Id: <20201012111913.705225-1-pablo.de.lara.guarch@intel.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20201009114029.190540-1-pablo.de.lara.guarch@intel.com> References: <20201009114029.190540-1-pablo.de.lara.guarch@intel.com> MIME-Version: 1.0 Content-Transfer-Encoding: 8bit Subject: [dpdk-dev] [PATCH v5 1/2] crypto/aesni_gcm: support SGL on AES-GMAC X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" Add Scatter-gather list support for AES-GMAC. Signed-off-by: Pablo de Lara Tested-by: Fan Zhang Acked-by: Fan Zhang --- Changes: - v4/v5: Rebased on top of crypto subtree - v2/v3: no change --- doc/guides/cryptodevs/aesni_gcm.rst | 2 - doc/guides/rel_notes/release_20_11.rst | 1 + drivers/crypto/aesni_gcm/aesni_gcm_ops.h | 27 ++++++++ drivers/crypto/aesni_gcm/aesni_gcm_pmd.c | 88 +++++++++++++++++++++++- 4 files changed, 115 insertions(+), 3 deletions(-) diff --git a/doc/guides/cryptodevs/aesni_gcm.rst b/doc/guides/cryptodevs/aesni_gcm.rst index 74e0de63a..0e146486e 100644 --- a/doc/guides/cryptodevs/aesni_gcm.rst +++ b/doc/guides/cryptodevs/aesni_gcm.rst @@ -31,8 +31,6 @@ Limitations ----------- * In out-of-place operations, chained destination mbufs are not supported. -* Chained mbufs are only supported by RTE_CRYPTO_AEAD_AES_GCM algorithm, - not RTE_CRYPTO_AUTH_AES_GMAC. * Cipher only is not supported. diff --git a/doc/guides/rel_notes/release_20_11.rst b/doc/guides/rel_notes/release_20_11.rst index c34ab5493..497a8ed9c 100644 --- a/doc/guides/rel_notes/release_20_11.rst +++ b/doc/guides/rel_notes/release_20_11.rst @@ -80,6 +80,7 @@ New Features * Added support for ZUC-EEA3/EIA3 algorithms. * Added support for SNOW3G-UEA2/UIA2 algorithms. * Added support for KASUMI-F8/F9 algorithms. + * Added SGL support AES-GMAC. * **Updated the OCTEON TX2 crypto PMD.** diff --git a/drivers/crypto/aesni_gcm/aesni_gcm_ops.h b/drivers/crypto/aesni_gcm/aesni_gcm_ops.h index 74acac09c..8a0d074b6 100644 --- a/drivers/crypto/aesni_gcm/aesni_gcm_ops.h +++ b/drivers/crypto/aesni_gcm/aesni_gcm_ops.h @@ -53,6 +53,23 @@ typedef void (*aesni_gcm_finalize_t)(const struct gcm_key_data *gcm_key_data, uint8_t *auth_tag, uint64_t auth_tag_len); +#if IMB_VERSION(0, 54, 0) < IMB_VERSION_NUM +typedef void (*aesni_gmac_init_t)(const struct gcm_key_data *gcm_key_data, + struct gcm_context_data *gcm_ctx_data, + const uint8_t *iv, + const uint64_t iv_len); + +typedef void (*aesni_gmac_update_t)(const struct gcm_key_data *gcm_key_data, + struct gcm_context_data *gcm_ctx_data, + const uint8_t *in, + const uint64_t plaintext_len); + +typedef void (*aesni_gmac_finalize_t)(const struct gcm_key_data *gcm_key_data, + struct gcm_context_data *gcm_ctx_data, + uint8_t *auth_tag, + const uint64_t auth_tag_len); +#endif + /** GCM library function pointer table */ struct aesni_gcm_ops { aesni_gcm_t enc; /**< GCM encode function pointer */ @@ -63,6 +80,11 @@ struct aesni_gcm_ops { aesni_gcm_update_t update_dec; aesni_gcm_finalize_t finalize_enc; aesni_gcm_finalize_t finalize_dec; +#if IMB_VERSION(0, 54, 0) < IMB_VERSION_NUM + aesni_gmac_init_t gmac_init; + aesni_gmac_update_t gmac_update; + aesni_gmac_finalize_t gmac_finalize; +#endif }; /** GCM per-session operation handlers */ @@ -72,6 +94,11 @@ struct aesni_gcm_session_ops { aesni_gcm_init_t init; aesni_gcm_update_t update; aesni_gcm_finalize_t finalize; +#if IMB_VERSION(0, 54, 0) < IMB_VERSION_NUM + aesni_gmac_init_t gmac_init; + aesni_gmac_update_t gmac_update; + aesni_gmac_finalize_t gmac_finalize; +#endif }; #endif /* _AESNI_GCM_OPS_H_ */ diff --git a/drivers/crypto/aesni_gcm/aesni_gcm_pmd.c b/drivers/crypto/aesni_gcm/aesni_gcm_pmd.c index 1d2a0ce00..aea599ebf 100644 --- a/drivers/crypto/aesni_gcm/aesni_gcm_pmd.c +++ b/drivers/crypto/aesni_gcm/aesni_gcm_pmd.c @@ -350,6 +350,76 @@ process_gcm_crypto_op(struct aesni_gcm_qp *qp, struct rte_crypto_op *op, &qp->gdata_ctx, tag, session->gen_digest_length); +#if IMB_VERSION(0, 54, 0) < IMB_VERSION_NUM + } else if (session->op == AESNI_GMAC_OP_GENERATE) { + qp->ops[session->key].gmac_init(&session->gdata_key, + &qp->gdata_ctx, + iv_ptr, + session->iv.length); + + qp->ops[session->key].gmac_update(&session->gdata_key, + &qp->gdata_ctx, src, + (uint64_t)part_len); + total_len = data_length - part_len; + + while (total_len) { + m_src = m_src->next; + + RTE_ASSERT(m_src != NULL); + + src = rte_pktmbuf_mtod(m_src, uint8_t *); + part_len = (m_src->data_len < total_len) ? + m_src->data_len : total_len; + + qp->ops[session->key].gmac_update(&session->gdata_key, + &qp->gdata_ctx, src, + (uint64_t)part_len); + total_len -= part_len; + } + + if (session->req_digest_length != session->gen_digest_length) + tag = qp->temp_digest; + else + tag = sym_op->auth.digest.data; + + qp->ops[session->key].gmac_finalize(&session->gdata_key, + &qp->gdata_ctx, + tag, + session->gen_digest_length); + } else { /* AESNI_GMAC_OP_VERIFY */ + qp->ops[session->key].gmac_init(&session->gdata_key, + &qp->gdata_ctx, + iv_ptr, + session->iv.length); + + qp->ops[session->key].gmac_update(&session->gdata_key, + &qp->gdata_ctx, src, + (uint64_t)part_len); + total_len = data_length - part_len; + + while (total_len) { + m_src = m_src->next; + + RTE_ASSERT(m_src != NULL); + + src = rte_pktmbuf_mtod(m_src, uint8_t *); + part_len = (m_src->data_len < total_len) ? + m_src->data_len : total_len; + + qp->ops[session->key].gmac_update(&session->gdata_key, + &qp->gdata_ctx, src, + (uint64_t)part_len); + total_len -= part_len; + } + + tag = qp->temp_digest; + + qp->ops[session->key].gmac_finalize(&session->gdata_key, + &qp->gdata_ctx, + tag, + session->gen_digest_length); + } +#else } else if (session->op == AESNI_GMAC_OP_GENERATE) { qp->ops[session->key].init(&session->gdata_key, &qp->gdata_ctx, @@ -381,6 +451,7 @@ process_gcm_crypto_op(struct aesni_gcm_qp *qp, struct rte_crypto_op *op, tag, session->gen_digest_length); } +#endif return 0; } @@ -769,7 +840,7 @@ aesni_gcm_create(const char *name, init_mb_mgr_avx2(mb_mgr); break; case RTE_AESNI_GCM_AVX512: - dev->feature_flags |= RTE_CRYPTODEV_FF_CPU_AVX2; + dev->feature_flags |= RTE_CRYPTODEV_FF_CPU_AVX512; init_mb_mgr_avx512(mb_mgr); break; default: @@ -791,6 +862,11 @@ aesni_gcm_create(const char *name, internals->ops[GCM_KEY_128].update_dec = mb_mgr->gcm128_dec_update; internals->ops[GCM_KEY_128].finalize_enc = mb_mgr->gcm128_enc_finalize; internals->ops[GCM_KEY_128].finalize_dec = mb_mgr->gcm128_dec_finalize; +#if IMB_VERSION(0, 54, 0) < IMB_VERSION_NUM + internals->ops[GCM_KEY_128].gmac_init = mb_mgr->gmac128_init; + internals->ops[GCM_KEY_128].gmac_update = mb_mgr->gmac128_update; + internals->ops[GCM_KEY_128].gmac_finalize = mb_mgr->gmac128_finalize; +#endif internals->ops[GCM_KEY_192].enc = mb_mgr->gcm192_enc; internals->ops[GCM_KEY_192].dec = mb_mgr->gcm192_dec; @@ -800,6 +876,11 @@ aesni_gcm_create(const char *name, internals->ops[GCM_KEY_192].update_dec = mb_mgr->gcm192_dec_update; internals->ops[GCM_KEY_192].finalize_enc = mb_mgr->gcm192_enc_finalize; internals->ops[GCM_KEY_192].finalize_dec = mb_mgr->gcm192_dec_finalize; +#if IMB_VERSION(0, 54, 0) < IMB_VERSION_NUM + internals->ops[GCM_KEY_192].gmac_init = mb_mgr->gmac192_init; + internals->ops[GCM_KEY_192].gmac_update = mb_mgr->gmac192_update; + internals->ops[GCM_KEY_192].gmac_finalize = mb_mgr->gmac192_finalize; +#endif internals->ops[GCM_KEY_256].enc = mb_mgr->gcm256_enc; internals->ops[GCM_KEY_256].dec = mb_mgr->gcm256_dec; @@ -809,6 +890,11 @@ aesni_gcm_create(const char *name, internals->ops[GCM_KEY_256].update_dec = mb_mgr->gcm256_dec_update; internals->ops[GCM_KEY_256].finalize_enc = mb_mgr->gcm256_enc_finalize; internals->ops[GCM_KEY_256].finalize_dec = mb_mgr->gcm256_dec_finalize; +#if IMB_VERSION(0, 54, 0) < IMB_VERSION_NUM + internals->ops[GCM_KEY_256].gmac_init = mb_mgr->gmac256_init; + internals->ops[GCM_KEY_256].gmac_update = mb_mgr->gmac256_update; + internals->ops[GCM_KEY_256].gmac_finalize = mb_mgr->gmac256_finalize; +#endif internals->max_nb_queue_pairs = init_params->max_nb_queue_pairs; -- 2.25.1