[dpdk-dev] [PATCH 0/5] AESNI MB PMD changes

[dpdk-dev] [PATCH 0/5] AESNI MB PMD changes

[Pablo de Lara]

The Multi-buffer library supports full digest sizes for the HMAC
algorithms (except for MD5), from 0.50 version.
Also, since 0.50, keys larger than the algorithm block size
can be used for HMAC algorithms, performing a hash on the key.

Therefore, the AESNI MB PMD now supports any key size for
the HMAC algorithms and any truncated digest size for any
SHAx-HMAC and AES-CMAC algorithm.

Pablo de Lara (5):
  crypto/aesni_mb: support all truncated HMAC digest sizes
  crypto/aesni_mb: check for invalid digest size
  crypto/aesni_mb: fix truncated digest size for CMAC
  crypto/aesni_mb: support all truncated CMAC digest sizes
  crypto/aesni_mb: support large HMAC key sizes

 drivers/crypto/aesni_mb/aesni_mb_ops.h        |  61 ++++++
 drivers/crypto/aesni_mb/rte_aesni_mb_pmd.c    | 180 +++++++++++++++---
 .../crypto/aesni_mb/rte_aesni_mb_pmd_ops.c    |  60 +++++-
 .../aesni_mb/rte_aesni_mb_pmd_private.h       |  24 +--
 4 files changed, 285 insertions(+), 40 deletions(-)

-- 
2.17.1

[dpdk-dev] [PATCH 1/5] crypto/aesni_mb: support all truncated HMAC digest sizes

[Pablo de Lara]

HMAC algorithms (MD5 and SHAx) have different full digest sizes.
However, they are often truncated to a smaller size (such as in IPSec).
This commit allows a user to generate a digest of any size
up to the full size.

Signed-off-by: Pablo de Lara Guarch <pablo.de.lara.guarch@intel.com>
---
 drivers/crypto/aesni_mb/rte_aesni_mb_pmd.c    | 83 ++++++++++++++-----
 .../crypto/aesni_mb/rte_aesni_mb_pmd_ops.c    | 36 ++++++++
 .../aesni_mb/rte_aesni_mb_pmd_private.h       | 13 +--
 3 files changed, 105 insertions(+), 27 deletions(-)

diff --git a/drivers/crypto/aesni_mb/rte_aesni_mb_pmd.c b/drivers/crypto/aesni_mb/rte_aesni_mb_pmd.c
index 93dc7a443..6fbfab8b8 100644
--- a/drivers/crypto/aesni_mb/rte_aesni_mb_pmd.c
+++ b/drivers/crypto/aesni_mb/rte_aesni_mb_pmd.c
@@ -112,12 +112,17 @@ aesni_mb_set_session_auth_parameters(const struct aesni_mb_op_fns *mb_ops,
 		return -1;
 	}
 
+	/* Set the request digest size */
+	sess->auth.req_digest_len = xform->auth.digest_length;
+
 	/* Select auth generate/verify */
 	sess->auth.operation = xform->auth.op;
 
 	/* Set Authentication Parameters */
 	if (xform->auth.algo == RTE_CRYPTO_AUTH_AES_XCBC_MAC) {
 		sess->auth.algo = AES_XCBC;
+
+		sess->auth.gen_digest_len = sess->auth.req_digest_len;
 		(*mb_ops->aux.keyexp.aes_xcbc)(xform->auth.key.data,
 				sess->auth.xcbc.k1_expanded,
 				sess->auth.xcbc.k2, sess->auth.xcbc.k3);
@@ -126,6 +131,8 @@ aesni_mb_set_session_auth_parameters(const struct aesni_mb_op_fns *mb_ops,
 
 	if (xform->auth.algo == RTE_CRYPTO_AUTH_AES_CMAC) {
 		sess->auth.algo = AES_CMAC;
+
+		sess->auth.gen_digest_len = sess->auth.req_digest_len;
 		(*mb_ops->aux.keyexp.aes_cmac_expkey)(xform->auth.key.data,
 				sess->auth.cmac.expkey);
 
@@ -134,7 +141,6 @@ aesni_mb_set_session_auth_parameters(const struct aesni_mb_op_fns *mb_ops,
 		return 0;
 	}
 
-
 	switch (xform->auth.algo) {
 	case RTE_CRYPTO_AUTH_MD5_HMAC:
 		sess->auth.algo = MD5;
@@ -164,6 +170,26 @@ aesni_mb_set_session_auth_parameters(const struct aesni_mb_op_fns *mb_ops,
 		AESNI_MB_LOG(ERR, "Unsupported authentication algorithm selection");
 		return -ENOTSUP;
 	}
+	uint16_t trunc_digest_size =
+			get_truncated_digest_byte_length(sess->auth.algo);
+	uint16_t full_digest_size =
+			get_digest_byte_length(sess->auth.algo);
+
+#if IMB_VERSION_NUM >= IMB_VERSION(0, 50, 0)
+	if (sess->auth.req_digest_len > full_digest_size ||
+			sess->auth.req_digest_len == 0) {
+#else
+	if (sess->auth.req_digest_len != trunc_digest_size) {
+#endif
+		AESNI_MB_LOG(ERR, "Invalid digest size\n");
+		return -EINVAL;
+	}
+
+	if (sess->auth.req_digest_len != trunc_digest_size &&
+			sess->auth.req_digest_len != full_digest_size)
+		sess->auth.gen_digest_len = full_digest_size;
+	else
+		sess->auth.gen_digest_len = sess->auth.req_digest_len;
 
 	/* Calculate Authentication precomputes */
 	calculate_auth_precomputes(hash_oneblock_fn,
@@ -360,6 +386,9 @@ aesni_mb_set_session_aead_parameters(const struct aesni_mb_op_fns *mb_ops,
 	sess->iv.offset = xform->aead.iv.offset;
 	sess->iv.length = xform->aead.iv.length;
 
+	sess->auth.req_digest_len = xform->aead.digest_length;
+	sess->auth.gen_digest_len = sess->auth.req_digest_len;
+
 	/* Check key length and choose key expansion function for AES */
 
 	switch (xform->aead.key.length) {
@@ -397,19 +426,16 @@ aesni_mb_set_session_parameters(const struct aesni_mb_op_fns *mb_ops,
 		sess->chain_order = HASH_CIPHER;
 		auth_xform = xform;
 		cipher_xform = xform->next;
-		sess->auth.digest_len = xform->auth.digest_length;
 		break;
 	case AESNI_MB_OP_CIPHER_HASH:
 		sess->chain_order = CIPHER_HASH;
 		auth_xform = xform->next;
 		cipher_xform = xform;
-		sess->auth.digest_len = xform->auth.digest_length;
 		break;
 	case AESNI_MB_OP_HASH_ONLY:
 		sess->chain_order = HASH_CIPHER;
 		auth_xform = xform;
 		cipher_xform = NULL;
-		sess->auth.digest_len = xform->auth.digest_length;
 		break;
 	case AESNI_MB_OP_CIPHER_ONLY:
 		/*
@@ -428,13 +454,11 @@ aesni_mb_set_session_parameters(const struct aesni_mb_op_fns *mb_ops,
 	case AESNI_MB_OP_AEAD_CIPHER_HASH:
 		sess->chain_order = CIPHER_HASH;
 		sess->aead.aad_len = xform->aead.aad_length;
-		sess->auth.digest_len = xform->aead.digest_length;
 		aead_xform = xform;
 		break;
 	case AESNI_MB_OP_AEAD_HASH_CIPHER:
 		sess->chain_order = HASH_CIPHER;
 		sess->aead.aad_len = xform->aead.aad_length;
-		sess->auth.digest_len = xform->aead.digest_length;
 		aead_xform = xform;
 		break;
 	case AESNI_MB_OP_NOT_SUPPORTED:
@@ -641,21 +665,17 @@ set_mb_job_params(JOB_AES_HMAC *job, struct aesni_mb_qp *qp,
 			job->auth_tag_output = op->sym->aead.digest.data;
 		else
 			job->auth_tag_output = op->sym->auth.digest.data;
-	}
 
-	/*
-	 * Multi-buffer library current only support returning a truncated
-	 * digest length as specified in the relevant IPsec RFCs
-	 */
-	if (job->hash_alg != AES_CCM && job->hash_alg != AES_CMAC)
-		job->auth_tag_output_len_in_bytes =
-				get_truncated_digest_byte_length(job->hash_alg);
-	else
-		job->auth_tag_output_len_in_bytes = session->auth.digest_len;
+		if (session->auth.req_digest_len != session->auth.gen_digest_len) {
+			job->auth_tag_output = qp->temp_digests[*digest_idx];
+			*digest_idx = (*digest_idx + 1) % MAX_JOBS;
+		}
+	}
 
+	/* Set digest length */
+	job->auth_tag_output_len_in_bytes = session->auth.gen_digest_len;
 
 	/* Set IV parameters */
-
 	job->iv_len_in_bytes = session->iv.length;
 
 	/* Data  Parameter */
@@ -690,20 +710,37 @@ set_mb_job_params(JOB_AES_HMAC *job, struct aesni_mb_qp *qp,
 }
 
 static inline void
-verify_digest(struct aesni_mb_qp *qp __rte_unused, JOB_AES_HMAC *job,
-		struct rte_crypto_op *op) {
+verify_digest(JOB_AES_HMAC *job, struct rte_crypto_op *op,
+		struct aesni_mb_session *sess)
+{
 	/* Verify digest if required */
 	if (job->hash_alg == AES_CCM) {
 		if (memcmp(job->auth_tag_output, op->sym->aead.digest.data,
-				job->auth_tag_output_len_in_bytes) != 0)
+				sess->auth.req_digest_len) != 0)
 			op->status = RTE_CRYPTO_OP_STATUS_AUTH_FAILED;
 	} else {
 		if (memcmp(job->auth_tag_output, op->sym->auth.digest.data,
-				job->auth_tag_output_len_in_bytes) != 0)
+				sess->auth.req_digest_len) != 0)
 			op->status = RTE_CRYPTO_OP_STATUS_AUTH_FAILED;
 	}
 }
 
+static inline void
+generate_digest(JOB_AES_HMAC *job, struct rte_crypto_op *op,
+		struct aesni_mb_session *sess)
+{
+	/* No extra copy neeed */
+	if (likely(sess->auth.req_digest_len == sess->auth.gen_digest_len))
+		return;
+
+	/*
+	 * This can only happen for HMAC, so only digest
+	 * for authentication algos is required
+	 */
+	memcpy(op->sym->auth.digest.data, job->auth_tag_output,
+			sess->auth.req_digest_len);
+}
+
 /**
  * Process a completed job and return rte_mbuf which job processed
  *
@@ -730,7 +767,9 @@ post_process_mb_job(struct aesni_mb_qp *qp, JOB_AES_HMAC *job)
 			if (job->hash_alg != NULL_HASH) {
 				if (sess->auth.operation ==
 						RTE_CRYPTO_AUTH_OP_VERIFY)
-					verify_digest(qp, job, op);
+					verify_digest(job, op, sess);
+				else
+					generate_digest(job, op, sess);
 			}
 			break;
 		default:
diff --git a/drivers/crypto/aesni_mb/rte_aesni_mb_pmd_ops.c b/drivers/crypto/aesni_mb/rte_aesni_mb_pmd_ops.c
index ab26e5ae4..e8397803e 100644
--- a/drivers/crypto/aesni_mb/rte_aesni_mb_pmd_ops.c
+++ b/drivers/crypto/aesni_mb/rte_aesni_mb_pmd_ops.c
@@ -25,9 +25,15 @@ static const struct rte_cryptodev_capabilities aesni_mb_pmd_capabilities[] = {
 					.increment = 1
 				},
 				.digest_size = {
+#if IMB_VERSION_NUM >= IMB_VERSION(0, 50, 0)
+					.min = 1,
+					.max = 16,
+					.increment = 1
+#else
 					.min = 12,
 					.max = 12,
 					.increment = 0
+#endif
 				},
 				.iv_size = { 0 }
 			}, }
@@ -46,9 +52,15 @@ static const struct rte_cryptodev_capabilities aesni_mb_pmd_capabilities[] = {
 					.increment = 1
 				},
 				.digest_size = {
+#if IMB_VERSION_NUM >= IMB_VERSION(0, 50, 0)
+					.min = 1,
+					.max = 20,
+					.increment = 1
+#else
 					.min = 12,
 					.max = 12,
 					.increment = 0
+#endif
 				},
 				.iv_size = { 0 }
 			}, }
@@ -67,9 +79,15 @@ static const struct rte_cryptodev_capabilities aesni_mb_pmd_capabilities[] = {
 					.increment = 1
 				},
 				.digest_size = {
+#if IMB_VERSION_NUM >= IMB_VERSION(0, 50, 0)
+					.min = 1,
+					.max = 28,
+					.increment = 1
+#else
 					.min = 14,
 					.max = 14,
 					.increment = 0
+#endif
 				},
 				.iv_size = { 0 }
 			}, }
@@ -88,9 +106,15 @@ static const struct rte_cryptodev_capabilities aesni_mb_pmd_capabilities[] = {
 					.increment = 1
 				},
 				.digest_size = {
+#if IMB_VERSION_NUM >= IMB_VERSION(0, 50, 0)
+					.min = 1,
+					.max = 32,
+					.increment = 1
+#else
 					.min = 16,
 					.max = 16,
 					.increment = 0
+#endif
 				},
 				.iv_size = { 0 }
 			}, }
@@ -109,9 +133,15 @@ static const struct rte_cryptodev_capabilities aesni_mb_pmd_capabilities[] = {
 					.increment = 1
 				},
 				.digest_size = {
+#if IMB_VERSION_NUM >= IMB_VERSION(0, 50, 0)
+					.min = 1,
+					.max = 48,
+					.increment = 1
+#else
 					.min = 24,
 					.max = 24,
 					.increment = 0
+#endif
 				},
 				.iv_size = { 0 }
 			}, }
@@ -130,9 +160,15 @@ static const struct rte_cryptodev_capabilities aesni_mb_pmd_capabilities[] = {
 					.increment = 1
 				},
 				.digest_size = {
+#if IMB_VERSION_NUM >= IMB_VERSION(0, 50, 0)
+					.min = 1,
+					.max = 64,
+					.increment = 1
+#else
 					.min = 32,
 					.max = 32,
 					.increment = 0
+#endif
 				},
 				.iv_size = { 0 }
 			}, }
diff --git a/drivers/crypto/aesni_mb/rte_aesni_mb_pmd_private.h b/drivers/crypto/aesni_mb/rte_aesni_mb_pmd_private.h
index 70e9d18e5..cc5822a82 100644
--- a/drivers/crypto/aesni_mb/rte_aesni_mb_pmd_private.h
+++ b/drivers/crypto/aesni_mb/rte_aesni_mb_pmd_private.h
@@ -31,8 +31,8 @@ int aesni_mb_logtype_driver;
 #define HMAC_IPAD_VALUE			(0x36)
 #define HMAC_OPAD_VALUE			(0x5C)
 
-/* Maximum length for digest (SHA-512 truncated needs 32 bytes) */
-#define DIGEST_LENGTH_MAX 32
+/* Maximum length for digest */
+#define DIGEST_LENGTH_MAX 64
 static const unsigned auth_blocksize[] = {
 		[MD5]		= 64,
 		[SHA1]		= 64,
@@ -95,7 +95,8 @@ static const unsigned auth_digest_byte_lengths[] = {
 };
 
 /**
- * Get the output digest size in bytes for a specified authentication algorithm
+ * Get the full digest size in bytes for a specified authentication algorithm
+ * (if available in the Multi-buffer library)
  *
  * @Note: this function will not return a valid value for a non-valid
  * authentication algorithm
@@ -226,8 +227,10 @@ struct aesni_mb_session {
 			} cmac;
 			/**< Expanded XCBC authentication keys */
 		};
-	/** digest size */
-	uint16_t digest_len;
+	/** Generated digest size by the Multi-buffer library */
+	uint16_t gen_digest_len;
+	/** Requested digest size from Cryptodev */
+	uint16_t req_digest_len;
 
 	} auth;
 	struct {
-- 
2.17.1

[dpdk-dev] [PATCH 2/5] crypto/aesni_mb: check for invalid digest size

[Pablo de Lara]

When creating a crypto session, check if
ther requested digest size is supported for
AES-XCBC-MAC and AES-CCM.

Signed-off-by: Pablo de Lara <pablo.de.lara.guarch@intel.com>
---
 drivers/crypto/aesni_mb/rte_aesni_mb_pmd.c | 16 ++++++++++++++++
 1 file changed, 16 insertions(+)

diff --git a/drivers/crypto/aesni_mb/rte_aesni_mb_pmd.c b/drivers/crypto/aesni_mb/rte_aesni_mb_pmd.c
index 6fbfab8b8..54dcf7787 100644
--- a/drivers/crypto/aesni_mb/rte_aesni_mb_pmd.c
+++ b/drivers/crypto/aesni_mb/rte_aesni_mb_pmd.c
@@ -14,6 +14,9 @@
 
 #include "rte_aesni_mb_pmd_private.h"
 
+#define AES_CCM_DIGEST_MIN_LEN 4
+#define AES_CCM_DIGEST_MAX_LEN 16
+
 static uint8_t cryptodev_driver_id;
 
 typedef void (*hash_one_block_t)(const void *data, void *digest);
@@ -122,6 +125,12 @@ aesni_mb_set_session_auth_parameters(const struct aesni_mb_op_fns *mb_ops,
 	if (xform->auth.algo == RTE_CRYPTO_AUTH_AES_XCBC_MAC) {
 		sess->auth.algo = AES_XCBC;
 
+		uint16_t xcbc_mac_digest_len =
+			get_truncated_digest_byte_length(AES_XCBC);
+		if (sess->auth.req_digest_len != xcbc_mac_digest_len) {
+			AESNI_MB_LOG(ERR, "Invalid digest size\n");
+			return -EINVAL;
+		}
 		sess->auth.gen_digest_len = sess->auth.req_digest_len;
 		(*mb_ops->aux.keyexp.aes_xcbc)(xform->auth.key.data,
 				sess->auth.xcbc.k1_expanded,
@@ -387,6 +396,13 @@ aesni_mb_set_session_aead_parameters(const struct aesni_mb_op_fns *mb_ops,
 	sess->iv.length = xform->aead.iv.length;
 
 	sess->auth.req_digest_len = xform->aead.digest_length;
+	/* CCM digests must be between 4 and 16 and an even number */
+	if (sess->auth.req_digest_len < AES_CCM_DIGEST_MIN_LEN ||
+			sess->auth.req_digest_len > AES_CCM_DIGEST_MAX_LEN ||
+			(sess->auth.req_digest_len & 1) == 1) {
+		AESNI_MB_LOG(ERR, "Invalid digest size\n");
+		return -EINVAL;
+	}
 	sess->auth.gen_digest_len = sess->auth.req_digest_len;
 
 	/* Check key length and choose key expansion function for AES */
-- 
2.17.1

[dpdk-dev] [PATCH 3/5] crypto/aesni_mb: fix truncated digest size for CMAC

[Pablo de Lara]

The truncated digest size for AES-CMAC is 12 and not 16,
as the Multi-buffer library can output both 12 and 16 bytes.

Fixes: 6491dbbecebb ("crypto/aesni_mb: support AES CMAC")
Cc: stable@dpdk.org

Signed-off-by: Pablo de Lara <pablo.de.lara.guarch@intel.com>
---
 drivers/crypto/aesni_mb/rte_aesni_mb_pmd_private.h | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/drivers/crypto/aesni_mb/rte_aesni_mb_pmd_private.h b/drivers/crypto/aesni_mb/rte_aesni_mb_pmd_private.h
index cc5822a82..1e297f032 100644
--- a/drivers/crypto/aesni_mb/rte_aesni_mb_pmd_private.h
+++ b/drivers/crypto/aesni_mb/rte_aesni_mb_pmd_private.h
@@ -64,7 +64,7 @@ static const unsigned auth_truncated_digest_byte_lengths[] = {
 		[SHA_384]	= 24,
 		[SHA_512]	= 32,
 		[AES_XCBC]	= 12,
-		[AES_CMAC]	= 16,
+		[AES_CMAC]	= 12,
 		[AES_CCM]	= 8,
 		[NULL_HASH]	= 0
 };
-- 
2.17.1

[dpdk-dev] [PATCH 4/5] crypto/aesni_mb: support all truncated CMAC digest sizes

[Pablo de Lara]

The full digest size of CMAC algorithm is 16 bytes.
However, it is sometimes truncated to a smaller size (such as in IPSec).
This commit allows a user to generate a digest of any size
up to the full size.

Signed-off-by: Pablo de Lara <pablo.de.lara.guarch@intel.com>
---
 drivers/crypto/aesni_mb/rte_aesni_mb_pmd.c    | 26 ++++++++++++++++++-
 .../crypto/aesni_mb/rte_aesni_mb_pmd_ops.c    |  4 +--
 2 files changed, 27 insertions(+), 3 deletions(-)

diff --git a/drivers/crypto/aesni_mb/rte_aesni_mb_pmd.c b/drivers/crypto/aesni_mb/rte_aesni_mb_pmd.c
index 54dcf7787..007c3fb2b 100644
--- a/drivers/crypto/aesni_mb/rte_aesni_mb_pmd.c
+++ b/drivers/crypto/aesni_mb/rte_aesni_mb_pmd.c
@@ -141,7 +141,31 @@ aesni_mb_set_session_auth_parameters(const struct aesni_mb_op_fns *mb_ops,
 	if (xform->auth.algo == RTE_CRYPTO_AUTH_AES_CMAC) {
 		sess->auth.algo = AES_CMAC;
 
-		sess->auth.gen_digest_len = sess->auth.req_digest_len;
+		uint16_t cmac_digest_len = get_digest_byte_length(AES_CMAC);
+
+		if (sess->auth.req_digest_len > cmac_digest_len) {
+			AESNI_MB_LOG(ERR, "Invalid digest size\n");
+			return -EINVAL;
+		}
+		/*
+		 * Multi-buffer lib supports digest sizes from 4 to 16 bytes
+		 * in version 0.50 and sizes of 12 and 16 bytes,
+		 * in version 0.49.
+		 * If size requested is different, generate the full digest
+		 * (16 bytes) in a temporary location and then memcpy
+		 * the requested number of bytes.
+		 */
+#if IMB_VERSION_NUM >= IMB_VERSION(0, 50, 0)
+		if (sess->auth.req_digest_len < 4)
+#else
+		uint16_t cmac_trunc_digest_len =
+				get_truncated_digest_byte_length(AES_CMAC);
+		if (sess->auth.req_digest_len != cmac_digest_len &&
+				sess->auth.req_digest_len != cmac_trunc_digest_len)
+#endif
+			sess->auth.gen_digest_len = cmac_digest_len;
+		else
+			sess->auth.gen_digest_len = sess->auth.req_digest_len;
 		(*mb_ops->aux.keyexp.aes_cmac_expkey)(xform->auth.key.data,
 				sess->auth.cmac.expkey);
 
diff --git a/drivers/crypto/aesni_mb/rte_aesni_mb_pmd_ops.c b/drivers/crypto/aesni_mb/rte_aesni_mb_pmd_ops.c
index e8397803e..e41ba70fa 100644
--- a/drivers/crypto/aesni_mb/rte_aesni_mb_pmd_ops.c
+++ b/drivers/crypto/aesni_mb/rte_aesni_mb_pmd_ops.c
@@ -358,9 +358,9 @@ static const struct rte_cryptodev_capabilities aesni_mb_pmd_capabilities[] = {
 					.increment = 0
 				},
 				.digest_size = {
-					.min = 12,
+					.min = 1,
 					.max = 16,
-					.increment = 4
+					.increment = 1
 				},
 				.iv_size = { 0 }
 			}, }
-- 
2.17.1

[dpdk-dev] [PATCH 5/5] crypto/aesni_mb: support large HMAC key sizes

[Pablo de Lara]

Add support for SHAx-HMAC key sizes larger than the block size.
For these sizes, the input key is digested with the non-HMAC
version of the algorithm and used as the key.

Signed-off-by: Pablo de Lara <pablo.de.lara.guarch@intel.com>
---
 drivers/crypto/aesni_mb/aesni_mb_ops.h        | 61 +++++++++++++++++++
 drivers/crypto/aesni_mb/rte_aesni_mb_pmd.c    | 59 +++++++++++++++++-
 .../crypto/aesni_mb/rte_aesni_mb_pmd_ops.c    | 20 ++++++
 .../aesni_mb/rte_aesni_mb_pmd_private.h       |  9 ---
 4 files changed, 138 insertions(+), 11 deletions(-)

diff --git a/drivers/crypto/aesni_mb/aesni_mb_ops.h b/drivers/crypto/aesni_mb/aesni_mb_ops.h
index 5a1cba6cb..d224b7249 100644
--- a/drivers/crypto/aesni_mb/aesni_mb_ops.h
+++ b/drivers/crypto/aesni_mb/aesni_mb_ops.h
@@ -11,6 +11,15 @@
 
 #include <intel-ipsec-mb.h>
 
+/*
+ * IMB_VERSION_NUM macro was introduced in version Multi-buffer 0.50,
+ * so if macro is not defined, it means that the version is 0.49.
+ */
+#if !defined(IMB_VERSION_NUM)
+#define IMB_VERSION(a, b, c) (((a) << 16) + ((b) << 8) + (c))
+#define IMB_VERSION_NUM IMB_VERSION(0, 49, 0)
+#endif
+
 enum aesni_mb_vector_mode {
 	RTE_AESNI_MB_NOT_SUPPORTED = 0,
 	RTE_AESNI_MB_SSE,
@@ -88,6 +97,16 @@ struct aesni_mb_op_fns {
 			/**< AES CMAC key expansions */
 		} keyexp;
 		/**< Key expansion functions */
+#if IMB_VERSION_NUM >= IMB_VERSION(0, 50, 0)
+		struct {
+			hash_fn_t sha1;
+			hash_fn_t sha224;
+			hash_fn_t sha256;
+			hash_fn_t sha384;
+			hash_fn_t sha512;
+		} multi_block;
+		/** multi block hash functions */
+#endif
 	} aux;
 	/**< Auxiliary functions */
 };
@@ -104,7 +123,13 @@ static const struct aesni_mb_op_fns job_ops[] = {
 				},
 				.keyexp = {
 					NULL
+				},
+#if IMB_VERSION_NUM >= IMB_VERSION(0, 50, 0)
+				.multi_block = {
+					NULL
 				}
+#endif
+
 			}
 		},
 		[RTE_AESNI_MB_SSE] = {
@@ -131,7 +156,16 @@ static const struct aesni_mb_op_fns job_ops[] = {
 					aes_xcbc_expand_key_sse,
 					aes_cmac_subkey_gen_sse,
 					aes_keyexp_128_enc_sse
+				},
+#if IMB_VERSION_NUM >= IMB_VERSION(0, 50, 0)
+				.multi_block = {
+					sha1_sse,
+					sha224_sse,
+					sha256_sse,
+					sha384_sse,
+					sha512_sse
 				}
+#endif
 			}
 		},
 		[RTE_AESNI_MB_AVX] = {
@@ -158,7 +192,16 @@ static const struct aesni_mb_op_fns job_ops[] = {
 					aes_xcbc_expand_key_avx,
 					aes_cmac_subkey_gen_avx,
 					aes_keyexp_128_enc_avx
+				},
+#if IMB_VERSION_NUM >= IMB_VERSION(0, 50, 0)
+				.multi_block = {
+					sha1_avx,
+					sha224_avx,
+					sha256_avx,
+					sha384_avx,
+					sha512_avx
 				}
+#endif
 			}
 		},
 		[RTE_AESNI_MB_AVX2] = {
@@ -185,7 +228,16 @@ static const struct aesni_mb_op_fns job_ops[] = {
 					aes_xcbc_expand_key_avx2,
 					aes_cmac_subkey_gen_avx2,
 					aes_keyexp_128_enc_avx2
+				},
+#if IMB_VERSION_NUM >= IMB_VERSION(0, 50, 0)
+				.multi_block = {
+					sha1_avx2,
+					sha224_avx2,
+					sha256_avx2,
+					sha384_avx2,
+					sha512_avx2
 				}
+#endif
 			}
 		},
 		[RTE_AESNI_MB_AVX512] = {
@@ -212,7 +264,16 @@ static const struct aesni_mb_op_fns job_ops[] = {
 					aes_xcbc_expand_key_avx512,
 					aes_cmac_subkey_gen_avx512,
 					aes_keyexp_128_enc_avx512
+				},
+#if IMB_VERSION_NUM >= IMB_VERSION(0, 50, 0)
+				.multi_block = {
+					sha1_avx512,
+					sha224_avx512,
+					sha256_avx512,
+					sha384_avx512,
+					sha512_avx512
 				}
+#endif
 			}
 		}
 };
diff --git a/drivers/crypto/aesni_mb/rte_aesni_mb_pmd.c b/drivers/crypto/aesni_mb/rte_aesni_mb_pmd.c
index 007c3fb2b..b5a3692e6 100644
--- a/drivers/crypto/aesni_mb/rte_aesni_mb_pmd.c
+++ b/drivers/crypto/aesni_mb/rte_aesni_mb_pmd.c
@@ -16,7 +16,7 @@
 
 #define AES_CCM_DIGEST_MIN_LEN 4
 #define AES_CCM_DIGEST_MAX_LEN 16
-
+#define HMAC_MAX_BLOCK_SIZE 128
 static uint8_t cryptodev_driver_id;
 
 typedef void (*hash_one_block_t)(const void *data, void *digest);
@@ -104,6 +104,8 @@ aesni_mb_set_session_auth_parameters(const struct aesni_mb_op_fns *mb_ops,
 		const struct rte_crypto_sym_xform *xform)
 {
 	hash_one_block_t hash_oneblock_fn;
+	unsigned int key_larger_block_size = 0;
+	uint8_t hashed_key[HMAC_MAX_BLOCK_SIZE] = { 0 };
 
 	if (xform == NULL) {
 		sess->auth.algo = NULL_HASH;
@@ -182,22 +184,67 @@ aesni_mb_set_session_auth_parameters(const struct aesni_mb_op_fns *mb_ops,
 	case RTE_CRYPTO_AUTH_SHA1_HMAC:
 		sess->auth.algo = SHA1;
 		hash_oneblock_fn = mb_ops->aux.one_block.sha1;
+#if IMB_VERSION_NUM >= IMB_VERSION(0, 50, 0)
+		if (xform->auth.key.length > get_auth_algo_blocksize(SHA1)) {
+			mb_ops->aux.multi_block.sha1(
+				xform->auth.key.data,
+				xform->auth.key.length,
+				hashed_key);
+			key_larger_block_size = 1;
+		}
+#endif
 		break;
 	case RTE_CRYPTO_AUTH_SHA224_HMAC:
 		sess->auth.algo = SHA_224;
 		hash_oneblock_fn = mb_ops->aux.one_block.sha224;
+#if IMB_VERSION_NUM >= IMB_VERSION(0, 50, 0)
+		if (xform->auth.key.length > get_auth_algo_blocksize(SHA_224)) {
+			mb_ops->aux.multi_block.sha224(
+				xform->auth.key.data,
+				xform->auth.key.length,
+				hashed_key);
+			key_larger_block_size = 1;
+		}
+#endif
 		break;
 	case RTE_CRYPTO_AUTH_SHA256_HMAC:
 		sess->auth.algo = SHA_256;
 		hash_oneblock_fn = mb_ops->aux.one_block.sha256;
+#if IMB_VERSION_NUM >= IMB_VERSION(0, 50, 0)
+		if (xform->auth.key.length > get_auth_algo_blocksize(SHA_256)) {
+			mb_ops->aux.multi_block.sha256(
+				xform->auth.key.data,
+				xform->auth.key.length,
+				hashed_key);
+			key_larger_block_size = 1;
+		}
+#endif
 		break;
 	case RTE_CRYPTO_AUTH_SHA384_HMAC:
 		sess->auth.algo = SHA_384;
 		hash_oneblock_fn = mb_ops->aux.one_block.sha384;
+#if IMB_VERSION_NUM >= IMB_VERSION(0, 50, 0)
+		if (xform->auth.key.length > get_auth_algo_blocksize(SHA_384)) {
+			mb_ops->aux.multi_block.sha384(
+				xform->auth.key.data,
+				xform->auth.key.length,
+				hashed_key);
+			key_larger_block_size = 1;
+		}
+#endif
 		break;
 	case RTE_CRYPTO_AUTH_SHA512_HMAC:
 		sess->auth.algo = SHA_512;
 		hash_oneblock_fn = mb_ops->aux.one_block.sha512;
+#if IMB_VERSION_NUM >= IMB_VERSION(0, 50, 0)
+		if (xform->auth.key.length > get_auth_algo_blocksize(SHA_512)) {
+			mb_ops->aux.multi_block.sha512(
+				xform->auth.key.data,
+				xform->auth.key.length,
+				hashed_key);
+			key_larger_block_size = 1;
+		}
+#endif
 		break;
 	default:
 		AESNI_MB_LOG(ERR, "Unsupported authentication algorithm selection");
@@ -225,11 +272,19 @@ aesni_mb_set_session_auth_parameters(const struct aesni_mb_op_fns *mb_ops,
 		sess->auth.gen_digest_len = sess->auth.req_digest_len;
 
 	/* Calculate Authentication precomputes */
-	calculate_auth_precomputes(hash_oneblock_fn,
+	if (key_larger_block_size) {
+		calculate_auth_precomputes(hash_oneblock_fn,
+			sess->auth.pads.inner, sess->auth.pads.outer,
+			hashed_key,
+			xform->auth.key.length,
+			get_auth_algo_blocksize(sess->auth.algo));
+	} else {
+		calculate_auth_precomputes(hash_oneblock_fn,
 			sess->auth.pads.inner, sess->auth.pads.outer,
 			xform->auth.key.data,
 			xform->auth.key.length,
 			get_auth_algo_blocksize(sess->auth.algo));
+	}
 
 	return 0;
 }
diff --git a/drivers/crypto/aesni_mb/rte_aesni_mb_pmd_ops.c b/drivers/crypto/aesni_mb/rte_aesni_mb_pmd_ops.c
index e41ba70fa..4f0139b20 100644
--- a/drivers/crypto/aesni_mb/rte_aesni_mb_pmd_ops.c
+++ b/drivers/crypto/aesni_mb/rte_aesni_mb_pmd_ops.c
@@ -48,7 +48,11 @@ static const struct rte_cryptodev_capabilities aesni_mb_pmd_capabilities[] = {
 				.block_size = 64,
 				.key_size = {
 					.min = 1,
+#if IMB_VERSION_NUM >= IMB_VERSION(0, 50, 0)
+					.max = 65535,
+#else
 					.max = 64,
+#endif
 					.increment = 1
 				},
 				.digest_size = {
@@ -75,7 +79,11 @@ static const struct rte_cryptodev_capabilities aesni_mb_pmd_capabilities[] = {
 				.block_size = 64,
 				.key_size = {
 					.min = 1,
+#if IMB_VERSION_NUM >= IMB_VERSION(0, 50, 0)
+					.max = 65535,
+#else
 					.max = 64,
+#endif
 					.increment = 1
 				},
 				.digest_size = {
@@ -102,7 +110,11 @@ static const struct rte_cryptodev_capabilities aesni_mb_pmd_capabilities[] = {
 				.block_size = 64,
 				.key_size = {
 					.min = 1,
+#if IMB_VERSION_NUM >= IMB_VERSION(0, 50, 0)
+					.max = 65535,
+#else
 					.max = 64,
+#endif
 					.increment = 1
 				},
 				.digest_size = {
@@ -129,7 +141,11 @@ static const struct rte_cryptodev_capabilities aesni_mb_pmd_capabilities[] = {
 				.block_size = 128,
 				.key_size = {
 					.min = 1,
+#if IMB_VERSION_NUM >= IMB_VERSION(0, 50, 0)
+					.max = 65535,
+#else
 					.max = 128,
+#endif
 					.increment = 1
 				},
 				.digest_size = {
@@ -156,7 +172,11 @@ static const struct rte_cryptodev_capabilities aesni_mb_pmd_capabilities[] = {
 				.block_size = 128,
 				.key_size = {
 					.min = 1,
+#if IMB_VERSION_NUM >= IMB_VERSION(0, 50, 0)
+					.max = 65535,
+#else
 					.max = 128,
+#endif
 					.increment = 1
 				},
 				.digest_size = {
diff --git a/drivers/crypto/aesni_mb/rte_aesni_mb_pmd_private.h b/drivers/crypto/aesni_mb/rte_aesni_mb_pmd_private.h
index 1e297f032..8c027a87e 100644
--- a/drivers/crypto/aesni_mb/rte_aesni_mb_pmd_private.h
+++ b/drivers/crypto/aesni_mb/rte_aesni_mb_pmd_private.h
@@ -7,15 +7,6 @@
 
 #include "aesni_mb_ops.h"
 
-/*
- * IMB_VERSION_NUM macro was introduced in version Multi-buffer 0.50,
- * so if macro is not defined, it means that the version is 0.49.
- */
-#if !defined(IMB_VERSION_NUM)
-#define IMB_VERSION(a, b, c) (((a) << 16) + ((b) << 8) + (c))
-#define IMB_VERSION_NUM IMB_VERSION(0, 49, 0)
-#endif
-
 #define CRYPTODEV_NAME_AESNI_MB_PMD	crypto_aesni_mb
 /**< AES-NI Multi buffer PMD device name */
 
-- 
2.17.1

Re: [dpdk-dev] [PATCH 0/5] AESNI MB PMD changes

[Kovacevic, Marko]

> The Multi-buffer library supports full digest sizes for the HMAC algorithms
> (except for MD5), from 0.50 version.
> Also, since 0.50, keys larger than the algorithm block size can be used for
> HMAC algorithms, performing a hash on the key.
> 
> Therefore, the AESNI MB PMD now supports any key size for the HMAC
> algorithms and any truncated digest size for any SHAx-HMAC and AES-CMAC
> algorithm.
> 
> Pablo de Lara (5):
>   crypto/aesni_mb: support all truncated HMAC digest sizes
>   crypto/aesni_mb: check for invalid digest size
>   crypto/aesni_mb: fix truncated digest size for CMAC
>   crypto/aesni_mb: support all truncated CMAC digest sizes
>   crypto/aesni_mb: support large HMAC key sizes
> 
>  drivers/crypto/aesni_mb/aesni_mb_ops.h        |  61 ++++++
>  drivers/crypto/aesni_mb/rte_aesni_mb_pmd.c    | 180 +++++++++++++++---
>  .../crypto/aesni_mb/rte_aesni_mb_pmd_ops.c    |  60 +++++-
>  .../aesni_mb/rte_aesni_mb_pmd_private.h       |  24 +--
>  4 files changed, 285 insertions(+), 40 deletions(-)
> 

Series-Acked-by: Marko Kovacevic <marko.kovacevic@intel.com>

Re: [dpdk-dev] [PATCH 0/5] AESNI MB PMD changes

[Akhil Goyal]

On 8/14/2018 6:08 AM, Pablo de Lara wrote:
> The Multi-buffer library supports full digest sizes for the HMAC
> algorithms (except for MD5), from 0.50 version.
> Also, since 0.50, keys larger than the algorithm block size
> can be used for HMAC algorithms, performing a hash on the key.
>
> Therefore, the AESNI MB PMD now supports any key size for
> the HMAC algorithms and any truncated digest size for any
> SHAx-HMAC and AES-CMAC algorithm.
>
> Pablo de Lara (5):
>    crypto/aesni_mb: support all truncated HMAC digest sizes
>    crypto/aesni_mb: check for invalid digest size
>    crypto/aesni_mb: fix truncated digest size for CMAC
>    crypto/aesni_mb: support all truncated CMAC digest sizes
>    crypto/aesni_mb: support large HMAC key sizes
>
>   drivers/crypto/aesni_mb/aesni_mb_ops.h        |  61 ++++++
>   drivers/crypto/aesni_mb/rte_aesni_mb_pmd.c    | 180 +++++++++++++++---
>   .../crypto/aesni_mb/rte_aesni_mb_pmd_ops.c    |  60 +++++-
>   .../aesni_mb/rte_aesni_mb_pmd_private.h       |  24 +--
>   4 files changed, 285 insertions(+), 40 deletions(-)
>
Series applied to dpdk-next-crypto

Thanks