From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 67A1FA0C46; Fri, 18 Jun 2021 14:20:10 +0200 (CEST) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id E261441136; Fri, 18 Jun 2021 14:19:45 +0200 (CEST) Received: from mga07.intel.com (mga07.intel.com [134.134.136.100]) by mails.dpdk.org (Postfix) with ESMTP id BB6C74113D for ; Fri, 18 Jun 2021 14:19:44 +0200 (CEST) IronPort-SDR: 3DdDqTY/4lpfL26EqjC7bNwLtjf7PB1SgpS5ixlr0rXTIDACwTu4vpY3q66N1DlbmkVxSSUj1s kOVEKNM/VUUw== X-IronPort-AV: E=McAfee;i="6200,9189,10018"; a="270392104" X-IronPort-AV: E=Sophos;i="5.83,283,1616482800"; d="scan'208";a="270392104" Received: from fmsmga001.fm.intel.com ([10.253.24.23]) by orsmga105.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 18 Jun 2021 05:19:44 -0700 IronPort-SDR: 5+VT6Yz/qU/VZffnL683WyQ0sUNDyq9JXm5tOR924FH9Usx3Pv88l9wNO2QFel2Ks9zf11iylM +5eIGr3IML/Q== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.83,283,1616482800"; d="scan'208";a="555560549" Received: from silpixa00400320.ir.intel.com ([10.237.214.214]) by fmsmga001.fm.intel.com with ESMTP; 18 Jun 2021 05:19:42 -0700 From: pbronowx To: dev@dpdk.org Cc: roy.fan.zhang@intel.com, thomas@monjalon.net, gakhil@marvell.com, ferruh.yigit@intel.com, declan.doherty@intel.com, pbronowx , Damian Nowak , Kai Ji Date: Fri, 18 Jun 2021 12:18:02 +0000 Message-Id: <20210618121803.1189857-7-piotrx.bronowski@intel.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210618121803.1189857-1-piotrx.bronowski@intel.com> References: <20210618121803.1189857-1-piotrx.bronowski@intel.com> MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Subject: [dpdk-dev] [RFC 6/7] crypto/snow3g: add support for digest appended ops X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" This patch enable out-of-place auth-cipher operations where digest should be encrypted among with the rest of raw data. It also adds support for partially encrypted digest when using auth-cipher operations. Fixes: 7c87e2d7b359 ("crypto/snow3g: use IPsec library"). Merged to ipsec_mb framework. Signed-off-by: Damian Nowak Signed-off-by: Kai Ji Signed-off-by: pbronowx --- doc/guides/cryptodevs/features/snow3g.ini | 1 + drivers/crypto/ipsec_mb/pmd_snow3g.c | 140 ++++++++++++++++++++-- 2 files changed, 128 insertions(+), 13 deletions(-) diff --git a/doc/guides/cryptodevs/features/snow3g.ini b/doc/guides/cryptodevs/features/snow3g.ini index 14ac7e4b6d..4d4c5b579b 100644 --- a/doc/guides/cryptodevs/features/snow3g.ini +++ b/doc/guides/cryptodevs/features/snow3g.ini @@ -8,6 +8,7 @@ Symmetric crypto = Y Sym operation chaining = Y Symmetric sessionless = Y Non-Byte aligned data = Y +Digest encrypted = Y OOP LB In LB Out = Y ; diff --git a/drivers/crypto/ipsec_mb/pmd_snow3g.c b/drivers/crypto/ipsec_mb/pmd_snow3g.c index 2bd35be847..da8718c15b 100644 --- a/drivers/crypto/ipsec_mb/pmd_snow3g.c +++ b/drivers/crypto/ipsec_mb/pmd_snow3g.c @@ -160,6 +160,24 @@ snow3g_session_configure(MB_MGR *mgr, void *priv_sess, return 0; } +/** Check if conditions are met for digest-appended operations */ +static uint8_t * +snow3g_digest_appended_in_src(struct rte_crypto_op *op) +{ + unsigned int auth_size, cipher_size; + + auth_size = (op->sym->auth.data.offset >> 3) + + (op->sym->auth.data.length >> 3); + cipher_size = (op->sym->cipher.data.offset >> 3) + + (op->sym->cipher.data.length >> 3); + + if (auth_size < cipher_size) + return rte_pktmbuf_mtod_offset(op->sym->m_src, + uint8_t *, auth_size); + + return NULL; +} + /** Encrypt/decrypt mbufs with same cipher key. */ static uint8_t process_snow3g_cipher_op(struct ipsec_mb_qp *qp, struct rte_crypto_op **ops, @@ -170,27 +188,73 @@ process_snow3g_cipher_op(struct ipsec_mb_qp *qp, struct rte_crypto_op **ops, uint8_t processed_ops = 0; const void *src[SNOW3G_MAX_BURST]; void *dst[SNOW3G_MAX_BURST]; + uint8_t *digest_appended[SNOW3G_MAX_BURST]; const void *iv[SNOW3G_MAX_BURST]; uint32_t num_bytes[SNOW3G_MAX_BURST]; + uint32_t cipher_off, cipher_len; + int unencrypted_bytes = 0; for (i = 0; i < num_ops; i++) { - src[i] = rte_pktmbuf_mtod(ops[i]->sym->m_src, uint8_t *) + - (ops[i]->sym->cipher.data.offset >> 3); - dst[i] = ops[i]->sym->m_dst ? - rte_pktmbuf_mtod(ops[i]->sym->m_dst, uint8_t *) + - (ops[i]->sym->cipher.data.offset >> 3) : - rte_pktmbuf_mtod(ops[i]->sym->m_src, uint8_t *) + - (ops[i]->sym->cipher.data.offset >> 3); + + cipher_off = ops[i]->sym->cipher.data.offset >> 3; + cipher_len = ops[i]->sym->cipher.data.length >> 3; + src[i] = rte_pktmbuf_mtod_offset( + ops[i]->sym->m_src, uint8_t *, cipher_off); + + /* If out-of-place operation */ + if (ops[i]->sym->m_dst && + ops[i]->sym->m_src != ops[i]->sym->m_dst) { + dst[i] = rte_pktmbuf_mtod_offset( + ops[i]->sym->m_dst, uint8_t *, cipher_off); + + /* In case of out-of-place, auth-cipher operation + * with partial encryption of the digest, copy + * the remaining, unencrypted part. + */ + if (session->op == IPSEC_MB_OP_HASH_VERIFY_THEN_DECRYPT + || session->op == IPSEC_MB_OP_HASH_GEN_THEN_ENCRYPT) + unencrypted_bytes = + (ops[i]->sym->auth.data.offset >> 3) + + (ops[i]->sym->auth.data.length >> 3) + + (SNOW3G_DIGEST_LENGTH) - + cipher_off - cipher_len; + if (unencrypted_bytes > 0) + rte_memcpy( + rte_pktmbuf_mtod_offset( + ops[i]->sym->m_dst, uint8_t *, + cipher_off + cipher_len), + rte_pktmbuf_mtod_offset( + ops[i]->sym->m_src, uint8_t *, + cipher_off + cipher_len), + unencrypted_bytes); + } else + dst[i] = rte_pktmbuf_mtod_offset(ops[i]->sym->m_src, + uint8_t *, cipher_off); + iv[i] = rte_crypto_op_ctod_offset(ops[i], uint8_t *, session->cipher_iv_offset); - num_bytes[i] = ops[i]->sym->cipher.data.length >> 3; - + num_bytes[i] = cipher_len; processed_ops++; } IMB_SNOW3G_F8_N_BUFFER(qp->mb_mgr, &session->pKeySched_cipher, iv, src, dst, num_bytes, processed_ops); + /* Take care of the raw digest data in src buffer */ + for (i = 0; i < num_ops; i++) { + if ((session->op == IPSEC_MB_OP_HASH_VERIFY_THEN_DECRYPT || + session->op == IPSEC_MB_OP_HASH_GEN_THEN_ENCRYPT) && + ops[i]->sym->m_dst != NULL) { + digest_appended[i] = + snow3g_digest_appended_in_src(ops[i]); + /* Clear unencrypted digest from + * the src buffer + */ + if (digest_appended[i] != NULL) + memset(digest_appended[i], + 0, SNOW3G_DIGEST_LENGTH); + } + } return processed_ops; } @@ -203,6 +267,7 @@ process_snow3g_cipher_op_bit(struct ipsec_mb_qp *qp, uint8_t *src, *dst; uint8_t *iv; uint32_t length_in_bits, offset_in_bits; + int unencrypted_bytes = 0; offset_in_bits = op->sym->cipher.data.offset; src = rte_pktmbuf_mtod(op->sym->m_src, uint8_t *); @@ -211,10 +276,32 @@ process_snow3g_cipher_op_bit(struct ipsec_mb_qp *qp, IPSEC_MB_LOG(ERR, "bit-level in-place not supported\n"); return 0; } + length_in_bits = op->sym->cipher.data.length; dst = rte_pktmbuf_mtod(op->sym->m_dst, uint8_t *); + /* In case of out-of-place, auth-cipher operation + * with partial encryption of the digest, copy + * the remaining, unencrypted part. + */ + if (session->op == IPSEC_MB_OP_HASH_VERIFY_THEN_DECRYPT || + session->op == IPSEC_MB_OP_HASH_GEN_THEN_ENCRYPT) + unencrypted_bytes = + (op->sym->auth.data.offset >> 3) + + (op->sym->auth.data.length >> 3) + + (SNOW3G_DIGEST_LENGTH) - + (offset_in_bits >> 3) - + (length_in_bits >> 3); + if (unencrypted_bytes > 0) + rte_memcpy( + rte_pktmbuf_mtod_offset( + op->sym->m_dst, uint8_t *, + (length_in_bits >> 3)), + rte_pktmbuf_mtod_offset( + op->sym->m_src, uint8_t *, + (length_in_bits >> 3)), + unencrypted_bytes); + iv = rte_crypto_op_ctod_offset(op, uint8_t *, session->cipher_iv_offset); - length_in_bits = op->sym->cipher.data.length; IMB_SNOW3G_F8_1_BUFFER_BIT(qp->mb_mgr, &session->pKeySched_cipher, iv, src, dst, length_in_bits, offset_in_bits); @@ -233,6 +320,7 @@ process_snow3g_hash_op(struct ipsec_mb_qp *qp, struct rte_crypto_op **ops, uint8_t *src, *dst; uint32_t length_in_bits; uint8_t *iv; + uint8_t digest_appended = 0; struct snow3g_qp_data *qp_data = ipsec_mb_get_qp_private_data(qp); for (i = 0; i < num_ops; i++) { @@ -243,6 +331,8 @@ process_snow3g_hash_op(struct ipsec_mb_qp *qp, struct rte_crypto_op **ops, break; } + dst = NULL; + length_in_bits = ops[i]->sym->auth.data.length; src = rte_pktmbuf_mtod(ops[i]->sym->m_src, uint8_t *) + @@ -252,6 +342,15 @@ process_snow3g_hash_op(struct ipsec_mb_qp *qp, struct rte_crypto_op **ops, if (session->auth_op == RTE_CRYPTO_AUTH_OP_VERIFY) { dst = qp_data->temp_digest; + /* Handle auth cipher verify oop case*/ + if ((session->op == + IPSEC_MB_OP_ENCRYPT_THEN_HASH_GEN || + session->op == + IPSEC_MB_OP_DECRYPT_THEN_HASH_VERIFY) && + ops[i]->sym->m_dst != NULL) + src = rte_pktmbuf_mtod_offset( + ops[i]->sym->m_dst, uint8_t *, + ops[i]->sym->auth.data.offset >> 3); IMB_SNOW3G_F9_1_BUFFER(qp->mb_mgr, &session->pKeySched_hash, @@ -261,12 +360,26 @@ process_snow3g_hash_op(struct ipsec_mb_qp *qp, struct rte_crypto_op **ops, SNOW3G_DIGEST_LENGTH) != 0) ops[i]->status = RTE_CRYPTO_OP_STATUS_AUTH_FAILED; - } else { - dst = ops[i]->sym->auth.digest.data; + } else { + if (session->op == + IPSEC_MB_OP_HASH_VERIFY_THEN_DECRYPT || + session->op == + IPSEC_MB_OP_HASH_GEN_THEN_ENCRYPT) + dst = snow3g_digest_appended_in_src(ops[i]); + + if (dst != NULL) + digest_appended = 1; + else + dst = ops[i]->sym->auth.digest.data; IMB_SNOW3G_F9_1_BUFFER(qp->mb_mgr, &session->pKeySched_hash, iv, src, length_in_bits, dst); + + /* Copy back digest from src to auth.digest.data */ + if (digest_appended) + rte_memcpy(ops[i]->sym->auth.digest.data, + dst, SNOW3G_DIGEST_LENGTH); } processed_ops++; } @@ -582,7 +695,8 @@ RTE_INIT(ipsec_mb_register_snow3g) RTE_CRYPTODEV_FF_SYM_OPERATION_CHAINING | RTE_CRYPTODEV_FF_NON_BYTE_ALIGNED_DATA | RTE_CRYPTODEV_FF_SYM_SESSIONLESS | - RTE_CRYPTODEV_FF_OOP_LB_IN_LB_OUT; + RTE_CRYPTODEV_FF_OOP_LB_IN_LB_OUT | + RTE_CRYPTODEV_FF_DIGEST_ENCRYPTED; snow3g_data->internals_priv_size = 0; snow3g_data->ops = &snow3g_pmd_ops; snow3g_data->qp_priv_size = sizeof(struct snow3g_qp_data); -- 2.25.1 -------------------------------------------------------------- Intel Research and Development Ireland Limited Registered in Ireland Registered Office: Collinstown Industrial Park, Leixlip, County Kildare Registered Number: 308263 This e-mail and any attachments may contain confidential material for the sole use of the intended recipient(s). Any review or distribution by others is strictly prohibited. If you are not the intended recipient, please contact the sender and delete all copies.