DPDK patches and discussions
 help / color / mirror / Atom feed
* [dpdk-dev] [PATCH 0/3] Add AES-GCM and cipher only offload support
@ 2020-07-24 11:00 Nagadheeraj Rottela
  2020-07-24 11:00 ` [dpdk-dev] [PATCH 1/3] test/crypto: replace NITROX PMD specific test suite Nagadheeraj Rottela
                   ` (3 more replies)
  0 siblings, 4 replies; 17+ messages in thread
From: Nagadheeraj Rottela @ 2020-07-24 11:00 UTC (permalink / raw)
  To: akhil.goyal; +Cc: dev, jsrikanth, Nagadheeraj Rottela

This patch set replaces the NITROX PMD specific test suite with generic
test suite and adds support for AES-GCM and cipher only offload.

Nagadheeraj Rottela (3):
  test/crypto: replace NITROX PMD specific test suite
  crypto/nitrox: support AES-GCM
  crypto/nitrox: support cipher only crypto operations

 app/test/test_cryptodev.c                     |  18 +-
 doc/guides/cryptodevs/features/nitrox.ini     |   3 +
 doc/guides/cryptodevs/nitrox.rst              |   6 +-
 drivers/crypto/nitrox/nitrox_sym.c            |  85 ++++-
 .../crypto/nitrox/nitrox_sym_capabilities.c   |  30 ++
 drivers/crypto/nitrox/nitrox_sym_ctx.h        |   5 +-
 drivers/crypto/nitrox/nitrox_sym_reqmgr.c     | 357 ++++++++++++++----
 7 files changed, 405 insertions(+), 99 deletions(-)

-- 
2.20.1


^ permalink raw reply	[flat|nested] 17+ messages in thread

* [dpdk-dev] [PATCH 1/3] test/crypto: replace NITROX PMD specific test suite
  2020-07-24 11:00 [dpdk-dev] [PATCH 0/3] Add AES-GCM and cipher only offload support Nagadheeraj Rottela
@ 2020-07-24 11:00 ` Nagadheeraj Rottela
  2020-07-24 11:00 ` [dpdk-dev] [PATCH 2/3] crypto/nitrox: support AES-GCM Nagadheeraj Rottela
                   ` (2 subsequent siblings)
  3 siblings, 0 replies; 17+ messages in thread
From: Nagadheeraj Rottela @ 2020-07-24 11:00 UTC (permalink / raw)
  To: akhil.goyal; +Cc: dev, jsrikanth, Nagadheeraj Rottela

Replace NITROX PMD specific tests with generic test suite.

Signed-off-by: Nagadheeraj Rottela <rnagadheeraj@marvell.com>
---
 app/test/test_cryptodev.c | 18 +-----------------
 1 file changed, 1 insertion(+), 17 deletions(-)

diff --git a/app/test/test_cryptodev.c b/app/test/test_cryptodev.c
index 70bf6fe2c..162134a5c 100644
--- a/app/test/test_cryptodev.c
+++ b/app/test/test_cryptodev.c
@@ -12665,22 +12665,6 @@ static struct unit_test_suite cryptodev_ccp_testsuite  = {
 	}
 };
 
-static struct unit_test_suite cryptodev_nitrox_testsuite  = {
-	.suite_name = "Crypto NITROX Unit Test Suite",
-	.setup = testsuite_setup,
-	.teardown = testsuite_teardown,
-	.unit_test_cases = {
-		TEST_CASE_ST(ut_setup, ut_teardown,
-			     test_device_configure_invalid_dev_id),
-		TEST_CASE_ST(ut_setup, ut_teardown,
-				test_device_configure_invalid_queue_pair_ids),
-		TEST_CASE_ST(ut_setup, ut_teardown, test_AES_chain_all),
-		TEST_CASE_ST(ut_setup, ut_teardown, test_3DES_chain_all),
-
-		TEST_CASES_END() /**< NULL terminate unit test array */
-	}
-};
-
 static int
 test_cryptodev_qat(void /*argv __rte_unused, int argc __rte_unused*/)
 {
@@ -13038,7 +13022,7 @@ test_cryptodev_nitrox(void)
 		return TEST_FAILED;
 	}
 
-	return unit_test_suite_runner(&cryptodev_nitrox_testsuite);
+	return unit_test_suite_runner(&cryptodev_testsuite);
 }
 
 REGISTER_TEST_COMMAND(cryptodev_qat_autotest, test_cryptodev_qat);
-- 
2.20.1


^ permalink raw reply	[flat|nested] 17+ messages in thread

* [dpdk-dev] [PATCH 2/3] crypto/nitrox: support AES-GCM
  2020-07-24 11:00 [dpdk-dev] [PATCH 0/3] Add AES-GCM and cipher only offload support Nagadheeraj Rottela
  2020-07-24 11:00 ` [dpdk-dev] [PATCH 1/3] test/crypto: replace NITROX PMD specific test suite Nagadheeraj Rottela
@ 2020-07-24 11:00 ` Nagadheeraj Rottela
  2020-07-24 11:00 ` [dpdk-dev] [PATCH 3/3] crypto/nitrox: support cipher only crypto operations Nagadheeraj Rottela
  2020-07-26 18:57 ` [dpdk-dev] [PATCH 0/3] Add AES-GCM and cipher only offload support Akhil Goyal
  3 siblings, 0 replies; 17+ messages in thread
From: Nagadheeraj Rottela @ 2020-07-24 11:00 UTC (permalink / raw)
  To: akhil.goyal; +Cc: dev, jsrikanth, Nagadheeraj Rottela

This patch adds AES-GCM AEAD algorithm.

Signed-off-by: Nagadheeraj Rottela <rnagadheeraj@marvell.com>
---
 doc/guides/cryptodevs/features/nitrox.ini     |   3 +
 doc/guides/cryptodevs/nitrox.rst              |   4 +
 drivers/crypto/nitrox/nitrox_sym.c            |  82 +++++++-
 .../crypto/nitrox/nitrox_sym_capabilities.c   |  30 +++
 drivers/crypto/nitrox/nitrox_sym_ctx.h        |   5 +-
 drivers/crypto/nitrox/nitrox_sym_reqmgr.c     | 182 +++++++++++++++---
 6 files changed, 268 insertions(+), 38 deletions(-)

diff --git a/doc/guides/cryptodevs/features/nitrox.ini b/doc/guides/cryptodevs/features/nitrox.ini
index 183494731..a1d6bcb4f 100644
--- a/doc/guides/cryptodevs/features/nitrox.ini
+++ b/doc/guides/cryptodevs/features/nitrox.ini
@@ -34,6 +34,9 @@ SHA256 HMAC  = Y
 ; Supported AEAD algorithms of the 'nitrox' crypto driver.
 ;
 [AEAD]
+AES GCM (128)  = Y
+AES GCM (192)  = Y
+AES GCM (256)  = Y
 
 ;
 ; Supported Asymmetric algorithms of the 'nitrox' crypto driver.
diff --git a/doc/guides/cryptodevs/nitrox.rst b/doc/guides/cryptodevs/nitrox.rst
index 85f5212b6..91fca905a 100644
--- a/doc/guides/cryptodevs/nitrox.rst
+++ b/doc/guides/cryptodevs/nitrox.rst
@@ -26,6 +26,10 @@ Hash algorithms:
 * ``RTE_CRYPTO_AUTH_SHA224_HMAC``
 * ``RTE_CRYPTO_AUTH_SHA256_HMAC``
 
+Supported AEAD algorithms:
+
+* ``RTE_CRYPTO_AEAD_AES_GCM``
+
 Limitations
 -----------
 
diff --git a/drivers/crypto/nitrox/nitrox_sym.c b/drivers/crypto/nitrox/nitrox_sym.c
index fad4a7a48..fe3ee6e23 100644
--- a/drivers/crypto/nitrox/nitrox_sym.c
+++ b/drivers/crypto/nitrox/nitrox_sym.c
@@ -20,6 +20,7 @@
 #define NPS_PKT_IN_INSTR_SIZE 64
 #define IV_FROM_DPTR 1
 #define FLEXI_CRYPTO_ENCRYPT_HMAC 0x33
+#define FLEXI_CRYPTO_MAX_AAD_LEN 512
 #define AES_KEYSIZE_128 16
 #define AES_KEYSIZE_192 24
 #define AES_KEYSIZE_256 32
@@ -297,6 +298,9 @@ get_crypto_chain_order(const struct rte_crypto_sym_xform *xform)
 			}
 		}
 		break;
+	case RTE_CRYPTO_SYM_XFORM_AEAD:
+		res = NITROX_CHAIN_COMBINED;
+		break;
 	default:
 		break;
 	}
@@ -431,17 +435,17 @@ get_flexi_auth_type(enum rte_crypto_auth_algorithm algo)
 }
 
 static bool
-auth_key_digest_is_valid(struct rte_crypto_auth_xform *xform,
-			 struct flexi_crypto_context *fctx)
+auth_key_is_valid(const uint8_t *data, uint16_t length,
+		  struct flexi_crypto_context *fctx)
 {
-	if (unlikely(!xform->key.data && xform->key.length)) {
+	if (unlikely(!data && length)) {
 		NITROX_LOG(ERR, "Invalid auth key\n");
 		return false;
 	}
 
-	if (unlikely(xform->key.length > sizeof(fctx->auth.opad))) {
+	if (unlikely(length > sizeof(fctx->auth.opad))) {
 		NITROX_LOG(ERR, "Invalid auth key length %d\n",
-			   xform->key.length);
+			   length);
 		return false;
 	}
 
@@ -459,11 +463,10 @@ configure_auth_ctx(struct rte_crypto_auth_xform *xform,
 	if (unlikely(type == AUTH_INVALID))
 		return -ENOTSUP;
 
-	if (unlikely(!auth_key_digest_is_valid(xform, fctx)))
+	if (unlikely(!auth_key_is_valid(xform->key.data, xform->key.length,
+					fctx)))
 		return -EINVAL;
 
-	ctx->auth_op = xform->op;
-	ctx->auth_algo = xform->algo;
 	ctx->digest_length = xform->digest_length;
 
 	fctx->flags = rte_be_to_cpu_64(fctx->flags);
@@ -476,6 +479,56 @@ configure_auth_ctx(struct rte_crypto_auth_xform *xform,
 	return 0;
 }
 
+static int
+configure_aead_ctx(struct rte_crypto_aead_xform *xform,
+		   struct nitrox_crypto_ctx *ctx)
+{
+	int aes_keylen;
+	struct flexi_crypto_context *fctx = &ctx->fctx;
+
+	if (unlikely(xform->aad_length > FLEXI_CRYPTO_MAX_AAD_LEN)) {
+		NITROX_LOG(ERR, "AAD length %d not supported\n",
+			   xform->aad_length);
+		return -ENOTSUP;
+	}
+
+	if (unlikely(xform->algo != RTE_CRYPTO_AEAD_AES_GCM))
+		return -ENOTSUP;
+
+	aes_keylen = flexi_aes_keylen(xform->key.length, true);
+	if (unlikely(aes_keylen < 0))
+		return -EINVAL;
+
+	if (unlikely(!auth_key_is_valid(xform->key.data, xform->key.length,
+					fctx)))
+		return -EINVAL;
+
+	if (unlikely(xform->iv.length > MAX_IV_LEN))
+		return -EINVAL;
+
+	fctx->flags = rte_be_to_cpu_64(fctx->flags);
+	fctx->w0.cipher_type = CIPHER_AES_GCM;
+	fctx->w0.aes_keylen = aes_keylen;
+	fctx->w0.iv_source = IV_FROM_DPTR;
+	fctx->w0.hash_type = AUTH_NULL;
+	fctx->w0.auth_input_type = 1;
+	fctx->w0.mac_len = xform->digest_length;
+	fctx->flags = rte_cpu_to_be_64(fctx->flags);
+	memset(fctx->crypto.key, 0, sizeof(fctx->crypto.key));
+	memcpy(fctx->crypto.key, xform->key.data, xform->key.length);
+	memset(&fctx->auth, 0, sizeof(fctx->auth));
+	memcpy(fctx->auth.opad, xform->key.data, xform->key.length);
+
+	ctx->opcode = FLEXI_CRYPTO_ENCRYPT_HMAC;
+	ctx->req_op = (xform->op == RTE_CRYPTO_AEAD_OP_ENCRYPT) ?
+			NITROX_OP_ENCRYPT : NITROX_OP_DECRYPT;
+	ctx->iv.offset = xform->iv.offset;
+	ctx->iv.length = xform->iv.length;
+	ctx->digest_length = xform->digest_length;
+	ctx->aad_length = xform->aad_length;
+	return 0;
+}
+
 static int
 nitrox_sym_dev_sess_configure(struct rte_cryptodev *cdev,
 			      struct rte_crypto_sym_xform *xform,
@@ -486,6 +539,8 @@ nitrox_sym_dev_sess_configure(struct rte_cryptodev *cdev,
 	struct nitrox_crypto_ctx *ctx;
 	struct rte_crypto_cipher_xform *cipher_xform = NULL;
 	struct rte_crypto_auth_xform *auth_xform = NULL;
+	struct rte_crypto_aead_xform *aead_xform = NULL;
+	int ret = -EINVAL;
 
 	if (rte_mempool_get(mempool, &mp_obj)) {
 		NITROX_LOG(ERR, "Couldn't allocate context\n");
@@ -503,8 +558,12 @@ nitrox_sym_dev_sess_configure(struct rte_cryptodev *cdev,
 		auth_xform = &xform->auth;
 		cipher_xform = &xform->next->cipher;
 		break;
+	case NITROX_CHAIN_COMBINED:
+		aead_xform = &xform->aead;
+		break;
 	default:
 		NITROX_LOG(ERR, "Crypto chain not supported\n");
+		ret = -ENOTSUP;
 		goto err;
 	}
 
@@ -518,12 +577,17 @@ nitrox_sym_dev_sess_configure(struct rte_cryptodev *cdev,
 		goto err;
 	}
 
+	if (aead_xform && unlikely(configure_aead_ctx(aead_xform, ctx))) {
+		NITROX_LOG(ERR, "Failed to configure aead ctx\n");
+		goto err;
+	}
+
 	ctx->iova = rte_mempool_virt2iova(ctx);
 	set_sym_session_private_data(sess, cdev->driver_id, ctx);
 	return 0;
 err:
 	rte_mempool_put(mempool, mp_obj);
-	return -EINVAL;
+	return ret;
 }
 
 static void
diff --git a/drivers/crypto/nitrox/nitrox_sym_capabilities.c b/drivers/crypto/nitrox/nitrox_sym_capabilities.c
index dc4df9185..a30cd9f8f 100644
--- a/drivers/crypto/nitrox/nitrox_sym_capabilities.c
+++ b/drivers/crypto/nitrox/nitrox_sym_capabilities.c
@@ -108,6 +108,36 @@ static const struct rte_cryptodev_capabilities nitrox_capabilities[] = {
 			}, }
 		}, }
 	},
+	{	/* AES GCM */
+		.op = RTE_CRYPTO_OP_TYPE_SYMMETRIC,
+		{.sym = {
+			.xform_type = RTE_CRYPTO_SYM_XFORM_AEAD,
+			{.aead = {
+				.algo = RTE_CRYPTO_AEAD_AES_GCM,
+				.block_size = 16,
+				.key_size = {
+					.min = 16,
+					.max = 32,
+					.increment = 8
+				},
+				.digest_size = {
+					.min = 1,
+					.max = 16,
+					.increment = 1
+				},
+				.aad_size = {
+					.min = 0,
+					.max = 512,
+					.increment = 1
+				},
+				.iv_size = {
+					.min = 12,
+					.max = 16,
+					.increment = 4
+				},
+			}, }
+		}, }
+	},
 	RTE_CRYPTODEV_END_OF_CAPABILITIES_LIST()
 };
 
diff --git a/drivers/crypto/nitrox/nitrox_sym_ctx.h b/drivers/crypto/nitrox/nitrox_sym_ctx.h
index 2985e519f..deb00fc1e 100644
--- a/drivers/crypto/nitrox/nitrox_sym_ctx.h
+++ b/drivers/crypto/nitrox/nitrox_sym_ctx.h
@@ -11,6 +11,7 @@
 
 #define AES_MAX_KEY_SIZE 32
 #define AES_BLOCK_SIZE 16
+#define AES_GCM_SALT_SIZE 4
 
 enum nitrox_chain {
 	NITROX_CHAIN_CIPHER_ONLY,
@@ -69,14 +70,14 @@ struct flexi_crypto_context {
 struct nitrox_crypto_ctx {
 	struct flexi_crypto_context fctx;
 	enum nitrox_chain nitrox_chain;
-	enum rte_crypto_auth_operation auth_op;
-	enum rte_crypto_auth_algorithm auth_algo;
 	struct {
 		uint16_t offset;
 		uint16_t length;
 	} iv;
 	rte_iova_t iova;
+	uint8_t salt[AES_GCM_SALT_SIZE];
 	uint16_t digest_length;
+	uint16_t aad_length;
 	uint8_t opcode;
 	uint8_t req_op;
 };
diff --git a/drivers/crypto/nitrox/nitrox_sym_reqmgr.c b/drivers/crypto/nitrox/nitrox_sym_reqmgr.c
index d9b426776..93d59b048 100644
--- a/drivers/crypto/nitrox/nitrox_sym_reqmgr.c
+++ b/drivers/crypto/nitrox/nitrox_sym_reqmgr.c
@@ -238,12 +238,13 @@ create_se_instr(struct nitrox_softreq *sr, uint8_t qno)
 }
 
 static void
-softreq_copy_iv(struct nitrox_softreq *sr)
+softreq_copy_iv(struct nitrox_softreq *sr, uint8_t salt_size)
 {
-	sr->iv.virt = rte_crypto_op_ctod_offset(sr->op, uint8_t *,
-						sr->ctx->iv.offset);
-	sr->iv.iova = rte_crypto_op_ctophys_offset(sr->op, sr->ctx->iv.offset);
-	sr->iv.len = sr->ctx->iv.length;
+	uint16_t offset = sr->ctx->iv.offset + salt_size;
+
+	sr->iv.virt = rte_crypto_op_ctod_offset(sr->op, uint8_t *, offset);
+	sr->iv.iova = rte_crypto_op_ctophys_offset(sr->op, offset);
+	sr->iv.len = sr->ctx->iv.length - salt_size;
 }
 
 static int
@@ -254,7 +255,7 @@ extract_cipher_auth_digest(struct nitrox_softreq *sr,
 	struct rte_mbuf *mdst = op->sym->m_dst ? op->sym->m_dst :
 					op->sym->m_src;
 
-	if (sr->ctx->auth_op == RTE_CRYPTO_AUTH_OP_VERIFY &&
+	if (sr->ctx->req_op == NITROX_OP_DECRYPT &&
 	    unlikely(!op->sym->auth.digest.data))
 		return -EINVAL;
 
@@ -352,6 +353,13 @@ create_cipher_auth_sglist(struct nitrox_softreq *sr,
 	if (unlikely(auth_only_len < 0))
 		return -EINVAL;
 
+	if (unlikely(
+		op->sym->cipher.data.offset + op->sym->cipher.data.length !=
+		op->sym->auth.data.offset + op->sym->auth.data.length)) {
+		NITROX_LOG(ERR, "Auth only data after cipher data not supported\n");
+		return -ENOTSUP;
+	}
+
 	err = create_sglist_from_mbuf(sgtbl, mbuf, op->sym->auth.data.offset,
 				      auth_only_len);
 	if (unlikely(err))
@@ -365,6 +373,41 @@ create_cipher_auth_sglist(struct nitrox_softreq *sr,
 	return 0;
 }
 
+static int
+create_combined_sglist(struct nitrox_softreq *sr, struct nitrox_sgtable *sgtbl,
+		       struct rte_mbuf *mbuf)
+{
+	struct rte_crypto_op *op = sr->op;
+
+	fill_sglist(sgtbl, sr->iv.len, sr->iv.iova, sr->iv.virt);
+	fill_sglist(sgtbl, sr->ctx->aad_length, op->sym->aead.aad.phys_addr,
+		    op->sym->aead.aad.data);
+	return create_sglist_from_mbuf(sgtbl, mbuf, op->sym->cipher.data.offset,
+				       op->sym->cipher.data.length);
+}
+
+static int
+create_aead_sglist(struct nitrox_softreq *sr, struct nitrox_sgtable *sgtbl,
+		   struct rte_mbuf *mbuf)
+{
+	int err;
+
+	switch (sr->ctx->nitrox_chain) {
+	case NITROX_CHAIN_CIPHER_AUTH:
+	case NITROX_CHAIN_AUTH_CIPHER:
+		err = create_cipher_auth_sglist(sr, sgtbl, mbuf);
+		break;
+	case NITROX_CHAIN_COMBINED:
+		err = create_combined_sglist(sr, sgtbl, mbuf);
+		break;
+	default:
+		err = -EINVAL;
+		break;
+	}
+
+	return err;
+}
+
 static void
 create_sgcomp(struct nitrox_sgtable *sgtbl)
 {
@@ -383,17 +426,16 @@ create_sgcomp(struct nitrox_sgtable *sgtbl)
 }
 
 static int
-create_cipher_auth_inbuf(struct nitrox_softreq *sr,
-			 struct nitrox_sglist *digest)
+create_aead_inbuf(struct nitrox_softreq *sr, struct nitrox_sglist *digest)
 {
 	int err;
 	struct nitrox_crypto_ctx *ctx = sr->ctx;
 
-	err = create_cipher_auth_sglist(sr, &sr->in, sr->op->sym->m_src);
+	err = create_aead_sglist(sr, &sr->in, sr->op->sym->m_src);
 	if (unlikely(err))
 		return err;
 
-	if (ctx->auth_op == RTE_CRYPTO_AUTH_OP_VERIFY)
+	if (ctx->req_op == NITROX_OP_DECRYPT)
 		fill_sglist(&sr->in, digest->len, digest->iova, digest->virt);
 
 	create_sgcomp(&sr->in);
@@ -402,25 +444,24 @@ create_cipher_auth_inbuf(struct nitrox_softreq *sr,
 }
 
 static int
-create_cipher_auth_oop_outbuf(struct nitrox_softreq *sr,
-			      struct nitrox_sglist *digest)
+create_aead_oop_outbuf(struct nitrox_softreq *sr, struct nitrox_sglist *digest)
 {
 	int err;
 	struct nitrox_crypto_ctx *ctx = sr->ctx;
 
-	err = create_cipher_auth_sglist(sr, &sr->out, sr->op->sym->m_dst);
+	err = create_aead_sglist(sr, &sr->out, sr->op->sym->m_dst);
 	if (unlikely(err))
 		return err;
 
-	if (ctx->auth_op == RTE_CRYPTO_AUTH_OP_GENERATE)
+	if (ctx->req_op == NITROX_OP_ENCRYPT)
 		fill_sglist(&sr->out, digest->len, digest->iova, digest->virt);
 
 	return 0;
 }
 
 static void
-create_cipher_auth_inplace_outbuf(struct nitrox_softreq *sr,
-				  struct nitrox_sglist *digest)
+create_aead_inplace_outbuf(struct nitrox_softreq *sr,
+			   struct nitrox_sglist *digest)
 {
 	int i, cnt;
 	struct nitrox_crypto_ctx *ctx = sr->ctx;
@@ -433,17 +474,16 @@ create_cipher_auth_inplace_outbuf(struct nitrox_softreq *sr,
 	}
 
 	sr->out.map_bufs_cnt = cnt;
-	if (ctx->auth_op == RTE_CRYPTO_AUTH_OP_GENERATE) {
+	if (ctx->req_op == NITROX_OP_ENCRYPT) {
 		fill_sglist(&sr->out, digest->len, digest->iova,
 			    digest->virt);
-	} else if (ctx->auth_op == RTE_CRYPTO_AUTH_OP_VERIFY) {
+	} else if (ctx->req_op == NITROX_OP_DECRYPT) {
 		sr->out.map_bufs_cnt--;
 	}
 }
 
 static int
-create_cipher_auth_outbuf(struct nitrox_softreq *sr,
-			  struct nitrox_sglist *digest)
+create_aead_outbuf(struct nitrox_softreq *sr, struct nitrox_sglist *digest)
 {
 	struct rte_crypto_op *op = sr->op;
 	int cnt = 0;
@@ -458,11 +498,11 @@ create_cipher_auth_outbuf(struct nitrox_softreq *sr,
 	if (op->sym->m_dst) {
 		int err;
 
-		err = create_cipher_auth_oop_outbuf(sr, digest);
+		err = create_aead_oop_outbuf(sr, digest);
 		if (unlikely(err))
 			return err;
 	} else {
-		create_cipher_auth_inplace_outbuf(sr, digest);
+		create_aead_inplace_outbuf(sr, digest);
 	}
 
 	cnt = sr->out.map_bufs_cnt;
@@ -516,16 +556,16 @@ process_cipher_auth_data(struct nitrox_softreq *sr)
 	int err;
 	struct nitrox_sglist digest;
 
-	softreq_copy_iv(sr);
+	softreq_copy_iv(sr, 0);
 	err = extract_cipher_auth_digest(sr, &digest);
 	if (unlikely(err))
 		return err;
 
-	err = create_cipher_auth_inbuf(sr, &digest);
+	err = create_aead_inbuf(sr, &digest);
 	if (unlikely(err))
 		return err;
 
-	err = create_cipher_auth_outbuf(sr, &digest);
+	err = create_aead_outbuf(sr, &digest);
 	if (unlikely(err))
 		return err;
 
@@ -534,6 +574,86 @@ process_cipher_auth_data(struct nitrox_softreq *sr)
 	return 0;
 }
 
+static int
+softreq_copy_salt(struct nitrox_softreq *sr)
+{
+	struct nitrox_crypto_ctx *ctx = sr->ctx;
+	uint8_t *addr;
+
+	if (unlikely(ctx->iv.length < AES_GCM_SALT_SIZE)) {
+		NITROX_LOG(ERR, "Invalid IV length %d\n", ctx->iv.length);
+		return -EINVAL;
+	}
+
+	addr = rte_crypto_op_ctod_offset(sr->op, uint8_t *, ctx->iv.offset);
+	if (!memcmp(ctx->salt, addr, AES_GCM_SALT_SIZE))
+		return 0;
+
+	memcpy(ctx->salt, addr, AES_GCM_SALT_SIZE);
+	memcpy(ctx->fctx.crypto.iv, addr, AES_GCM_SALT_SIZE);
+	return 0;
+}
+
+static int
+extract_combined_digest(struct nitrox_softreq *sr, struct nitrox_sglist *digest)
+{
+	struct rte_crypto_op *op = sr->op;
+	struct rte_mbuf *mdst = op->sym->m_dst ? op->sym->m_dst :
+		op->sym->m_src;
+
+	digest->len = sr->ctx->digest_length;
+	if (op->sym->aead.digest.data) {
+		digest->iova = op->sym->aead.digest.phys_addr;
+		digest->virt = op->sym->aead.digest.data;
+
+		return 0;
+	}
+
+	if (unlikely(rte_pktmbuf_data_len(mdst) < op->sym->aead.data.offset +
+	       op->sym->aead.data.length + digest->len))
+		return -EINVAL;
+
+	digest->iova = rte_pktmbuf_mtophys_offset(mdst,
+					op->sym->aead.data.offset +
+					op->sym->aead.data.length);
+	digest->virt = rte_pktmbuf_mtod_offset(mdst, uint8_t *,
+					op->sym->aead.data.offset +
+					op->sym->aead.data.length);
+
+	return 0;
+}
+
+static int
+process_combined_data(struct nitrox_softreq *sr)
+{
+	int err;
+	struct nitrox_sglist digest;
+	struct rte_crypto_op *op = sr->op;
+
+	err = softreq_copy_salt(sr);
+	if (unlikely(err))
+		return err;
+
+	softreq_copy_iv(sr, AES_GCM_SALT_SIZE);
+	err = extract_combined_digest(sr, &digest);
+	if (unlikely(err))
+		return err;
+
+	err = create_aead_inbuf(sr, &digest);
+	if (unlikely(err))
+		return err;
+
+	err = create_aead_outbuf(sr, &digest);
+	if (unlikely(err))
+		return err;
+
+	create_aead_gph(op->sym->aead.data.length, sr->iv.len,
+			op->sym->aead.data.length + sr->ctx->aad_length,
+			&sr->gph);
+
+	return 0;
+}
+
 static int
 process_softreq(struct nitrox_softreq *sr)
 {
@@ -545,6 +665,9 @@ process_softreq(struct nitrox_softreq *sr)
 	case NITROX_CHAIN_AUTH_CIPHER:
 		err = process_cipher_auth_data(sr);
 		break;
+	case NITROX_CHAIN_COMBINED:
+		err = process_combined_data(sr);
+		break;
 	default:
 		err = -EINVAL;
 		break;
@@ -558,10 +681,15 @@ nitrox_process_se_req(uint16_t qno, struct rte_crypto_op *op,
 		      struct nitrox_crypto_ctx *ctx,
 		      struct nitrox_softreq *sr)
 {
+	int err;
+
 	softreq_init(sr, sr->iova);
 	sr->ctx = ctx;
 	sr->op = op;
-	process_softreq(sr);
+	err = process_softreq(sr);
+	if (unlikely(err))
+		return err;
+
 	create_se_instr(sr, qno);
 	sr->timeout = rte_get_timer_cycles() + CMD_TIMEOUT * rte_get_timer_hz();
 	return 0;
@@ -577,7 +705,7 @@ nitrox_check_se_req(struct nitrox_softreq *sr, struct rte_crypto_op **op)
 	cc = *(volatile uint64_t *)(&sr->resp.completion);
 	orh = *(volatile uint64_t *)(&sr->resp.orh);
 	if (cc != PENDING_SIG)
-		err = 0;
+		err = orh & 0xff;
 	else if ((orh != PENDING_SIG) && (orh & 0xff))
 		err = orh & 0xff;
 	else if (rte_get_timer_cycles() >= sr->timeout)
-- 
2.20.1


^ permalink raw reply	[flat|nested] 17+ messages in thread

* [dpdk-dev] [PATCH 3/3] crypto/nitrox: support cipher only crypto operations
  2020-07-24 11:00 [dpdk-dev] [PATCH 0/3] Add AES-GCM and cipher only offload support Nagadheeraj Rottela
  2020-07-24 11:00 ` [dpdk-dev] [PATCH 1/3] test/crypto: replace NITROX PMD specific test suite Nagadheeraj Rottela
  2020-07-24 11:00 ` [dpdk-dev] [PATCH 2/3] crypto/nitrox: support AES-GCM Nagadheeraj Rottela
@ 2020-07-24 11:00 ` Nagadheeraj Rottela
  2020-09-22 19:11   ` Akhil Goyal
  2020-07-26 18:57 ` [dpdk-dev] [PATCH 0/3] Add AES-GCM and cipher only offload support Akhil Goyal
  3 siblings, 1 reply; 17+ messages in thread
From: Nagadheeraj Rottela @ 2020-07-24 11:00 UTC (permalink / raw)
  To: akhil.goyal; +Cc: dev, jsrikanth, Nagadheeraj Rottela

This patch adds cipher only crypto operation support.

Signed-off-by: Nagadheeraj Rottela <rnagadheeraj@marvell.com>
---
 doc/guides/cryptodevs/nitrox.rst          |   2 -
 drivers/crypto/nitrox/nitrox_sym.c        |   3 +
 drivers/crypto/nitrox/nitrox_sym_reqmgr.c | 189 ++++++++++++++++------
 3 files changed, 143 insertions(+), 51 deletions(-)

diff --git a/doc/guides/cryptodevs/nitrox.rst b/doc/guides/cryptodevs/nitrox.rst
index 91fca905a..095e545c6 100644
--- a/doc/guides/cryptodevs/nitrox.rst
+++ b/doc/guides/cryptodevs/nitrox.rst
@@ -33,8 +33,6 @@ Supported AEAD algorithms:
 Limitations
 -----------
 
-* AES_CBC Cipher Only combination is not supported.
-* 3DES Cipher Only combination is not supported.
 * Session-less APIs are not supported.
 
 Installation
diff --git a/drivers/crypto/nitrox/nitrox_sym.c b/drivers/crypto/nitrox/nitrox_sym.c
index fe3ee6e23..2768bdd2e 100644
--- a/drivers/crypto/nitrox/nitrox_sym.c
+++ b/drivers/crypto/nitrox/nitrox_sym.c
@@ -550,6 +550,9 @@ nitrox_sym_dev_sess_configure(struct rte_cryptodev *cdev,
 	ctx = mp_obj;
 	ctx->nitrox_chain = get_crypto_chain_order(xform);
 	switch (ctx->nitrox_chain) {
+	case NITROX_CHAIN_CIPHER_ONLY:
+		cipher_xform = &xform->cipher;
+		break;
 	case NITROX_CHAIN_CIPHER_AUTH:
 		cipher_xform = &xform->cipher;
 		auth_xform = &xform->next->auth;
diff --git a/drivers/crypto/nitrox/nitrox_sym_reqmgr.c b/drivers/crypto/nitrox/nitrox_sym_reqmgr.c
index 93d59b048..b5bbd1fd2 100644
--- a/drivers/crypto/nitrox/nitrox_sym_reqmgr.c
+++ b/drivers/crypto/nitrox/nitrox_sym_reqmgr.c
@@ -247,38 +247,6 @@ softreq_copy_iv(struct nitrox_softreq *sr, uint8_t salt_size)
 	sr->iv.len = sr->ctx->iv.length - salt_size;
 }
 
-static int
-extract_cipher_auth_digest(struct nitrox_softreq *sr,
-			   struct nitrox_sglist *digest)
-{
-	struct rte_crypto_op *op = sr->op;
-	struct rte_mbuf *mdst = op->sym->m_dst ? op->sym->m_dst :
-					op->sym->m_src;
-
-	if (sr->ctx->req_op == NITROX_OP_DECRYPT &&
-	    unlikely(!op->sym->auth.digest.data))
-		return -EINVAL;
-
-	digest->len = sr->ctx->digest_length;
-	if (op->sym->auth.digest.data) {
-		digest->iova = op->sym->auth.digest.phys_addr;
-		digest->virt = op->sym->auth.digest.data;
-		return 0;
-	}
-
-	if (unlikely(rte_pktmbuf_data_len(mdst) < op->sym->auth.data.offset +
-	       op->sym->auth.data.length + digest->len))
-		return -EINVAL;
-
-	digest->iova = rte_pktmbuf_mtophys_offset(mdst,
-					op->sym->auth.data.offset +
-					op->sym->auth.data.length);
-	digest->virt = rte_pktmbuf_mtod_offset(mdst, uint8_t *,
-					op->sym->auth.data.offset +
-					op->sym->auth.data.length);
-	return 0;
-}
-
 static void
 fill_sglist(struct nitrox_sgtable *sgtbl, uint16_t len, rte_iova_t iova,
 	    void *virt)
@@ -340,6 +308,143 @@ create_sglist_from_mbuf(struct nitrox_sgtable *sgtbl, struct rte_mbuf *mbuf,
 	return 0;
 }
 
+static void
+create_sgcomp(struct nitrox_sgtable *sgtbl)
+{
+	int i, j, nr_sgcomp;
+	struct nitrox_sgcomp *sgcomp = sgtbl->sgcomp;
+	struct nitrox_sglist *sglist = sgtbl->sglist;
+
+	nr_sgcomp = RTE_ALIGN_MUL_CEIL(sgtbl->map_bufs_cnt, 4) / 4;
+	sgtbl->nr_sgcomp = nr_sgcomp;
+	for (i = 0; i < nr_sgcomp; i++, sgcomp++) {
+		for (j = 0; j < 4; j++, sglist++) {
+			sgcomp->len[j] = rte_cpu_to_be_16(sglist->len);
+			sgcomp->iova[j] = rte_cpu_to_be_64(sglist->iova);
+		}
+	}
+}
+
+static int
+create_cipher_inbuf(struct nitrox_softreq *sr)
+{
+	int err;
+	struct rte_crypto_op *op = sr->op;
+
+	fill_sglist(&sr->in, sr->iv.len, sr->iv.iova, sr->iv.virt);
+	err = create_sglist_from_mbuf(&sr->in, op->sym->m_src,
+				      op->sym->cipher.data.offset,
+				      op->sym->cipher.data.length);
+	if (unlikely(err))
+		return err;
+
+	create_sgcomp(&sr->in);
+	sr->dptr = sr->iova + offsetof(struct nitrox_softreq, in.sgcomp);
+
+	return 0;
+}
+
+static int
+create_cipher_outbuf(struct nitrox_softreq *sr)
+{
+	struct rte_crypto_op *op = sr->op;
+	int err, cnt = 0;
+	struct rte_mbuf *m_dst = op->sym->m_dst ? op->sym->m_dst :
+		op->sym->m_src;
+
+	sr->resp.orh = PENDING_SIG;
+	sr->out.sglist[cnt].len = sizeof(sr->resp.orh);
+	sr->out.sglist[cnt].iova = sr->iova + offsetof(struct nitrox_softreq,
+						       resp.orh);
+	sr->out.sglist[cnt].virt = &sr->resp.orh;
+	cnt++;
+
+	sr->out.map_bufs_cnt = cnt;
+	fill_sglist(&sr->out, sr->iv.len, sr->iv.iova, sr->iv.virt);
+	err = create_sglist_from_mbuf(&sr->out, m_dst,
+				      op->sym->cipher.data.offset,
+				      op->sym->cipher.data.length);
+	if (unlikely(err))
+		return err;
+
+	cnt = sr->out.map_bufs_cnt;
+	sr->resp.completion = PENDING_SIG;
+	sr->out.sglist[cnt].len = sizeof(sr->resp.completion);
+	sr->out.sglist[cnt].iova = sr->iova + offsetof(struct nitrox_softreq,
+						     resp.completion);
+	sr->out.sglist[cnt].virt = &sr->resp.completion;
+	cnt++;
+
+	RTE_VERIFY(cnt <= MAX_SGBUF_CNT);
+	sr->out.map_bufs_cnt = cnt;
+
+	create_sgcomp(&sr->out);
+	sr->rptr = sr->iova + offsetof(struct nitrox_softreq, out.sgcomp);
+
+	return 0;
+}
+
+static void
+create_cipher_gph(uint32_t cryptlen, uint16_t ivlen, struct gphdr *gph)
+{
+	gph->param0 = rte_cpu_to_be_16(cryptlen);
+	gph->param1 = 0;
+	gph->param2 = rte_cpu_to_be_16(ivlen);
+	gph->param3 = 0;
+}
+
+static int
+process_cipher_data(struct nitrox_softreq *sr)
+{
+	struct rte_crypto_op *op = sr->op;
+	int err;
+
+	softreq_copy_iv(sr, 0);
+	err = create_cipher_inbuf(sr);
+	if (unlikely(err))
+		return err;
+
+	err = create_cipher_outbuf(sr);
+	if (unlikely(err))
+		return err;
+
+	create_cipher_gph(op->sym->cipher.data.length, sr->iv.len, &sr->gph);
+
+	return 0;
+}
+
+static int
+extract_cipher_auth_digest(struct nitrox_softreq *sr,
+			   struct nitrox_sglist *digest)
+{
+	struct rte_crypto_op *op = sr->op;
+	struct rte_mbuf *mdst = op->sym->m_dst ? op->sym->m_dst :
+					op->sym->m_src;
+
+	if (sr->ctx->req_op == NITROX_OP_DECRYPT &&
+	    unlikely(!op->sym->auth.digest.data))
+		return -EINVAL;
+
+	digest->len = sr->ctx->digest_length;
+	if (op->sym->auth.digest.data) {
+		digest->iova = op->sym->auth.digest.phys_addr;
+		digest->virt = op->sym->auth.digest.data;
+		return 0;
+	}
+
+	if (unlikely(rte_pktmbuf_data_len(mdst) < op->sym->auth.data.offset +
+	       op->sym->auth.data.length + digest->len))
+		return -EINVAL;
+
+	digest->iova = rte_pktmbuf_mtophys_offset(mdst,
+					op->sym->auth.data.offset +
+					op->sym->auth.data.length);
+	digest->virt = rte_pktmbuf_mtod_offset(mdst, uint8_t *,
+					op->sym->auth.data.offset +
+					op->sym->auth.data.length);
+	return 0;
+}
+
 static int
 create_cipher_auth_sglist(struct nitrox_softreq *sr,
 			  struct nitrox_sgtable *sgtbl, struct rte_mbuf *mbuf)
@@ -408,23 +513,6 @@ create_aead_sglist(struct nitrox_softreq *sr, struct nitrox_sgtable *sgtbl,
 	return err;
 }
 
-static void
-create_sgcomp(struct nitrox_sgtable *sgtbl)
-{
-	int i, j, nr_sgcomp;
-	struct nitrox_sgcomp *sgcomp = sgtbl->sgcomp;
-	struct nitrox_sglist *sglist = sgtbl->sglist;
-
-	nr_sgcomp = RTE_ALIGN_MUL_CEIL(sgtbl->map_bufs_cnt, 4) / 4;
-	sgtbl->nr_sgcomp = nr_sgcomp;
-	for (i = 0; i < nr_sgcomp; i++, sgcomp++) {
-		for (j = 0; j < 4; j++, sglist++) {
-			sgcomp->len[j] = rte_cpu_to_be_16(sglist->len);
-			sgcomp->iova[j] = rte_cpu_to_be_64(sglist->iova);
-		}
-	}
-}
-
 static int
 create_aead_inbuf(struct nitrox_softreq *sr, struct nitrox_sglist *digest)
 {
@@ -661,6 +749,9 @@ process_softreq(struct nitrox_softreq *sr)
 	int err = 0;
 
 	switch (ctx->nitrox_chain) {
+	case NITROX_CHAIN_CIPHER_ONLY:
+		err = process_cipher_data(sr);
+		break;
 	case NITROX_CHAIN_CIPHER_AUTH:
 	case NITROX_CHAIN_AUTH_CIPHER:
 		err = process_cipher_auth_data(sr);
-- 
2.20.1


^ permalink raw reply	[flat|nested] 17+ messages in thread

* Re: [dpdk-dev] [PATCH 0/3] Add AES-GCM and cipher only offload support
  2020-07-24 11:00 [dpdk-dev] [PATCH 0/3] Add AES-GCM and cipher only offload support Nagadheeraj Rottela
                   ` (2 preceding siblings ...)
  2020-07-24 11:00 ` [dpdk-dev] [PATCH 3/3] crypto/nitrox: support cipher only crypto operations Nagadheeraj Rottela
@ 2020-07-26 18:57 ` Akhil Goyal
  2020-07-28  6:41   ` Nagadheeraj Rottela
  3 siblings, 1 reply; 17+ messages in thread
From: Akhil Goyal @ 2020-07-26 18:57 UTC (permalink / raw)
  To: Nagadheeraj Rottela; +Cc: dev, jsrikanth

> Subject: [PATCH 0/3] Add AES-GCM and cipher only offload support
> 
> This patch set replaces the NITROX PMD specific test suite with generic
> test suite and adds support for AES-GCM and cipher only offload.
> 
I hope this series is for next release cycle.
From next time, please mention 20.11 in subject while 20.08 release cycle is still in progress.


^ permalink raw reply	[flat|nested] 17+ messages in thread

* Re: [dpdk-dev] [PATCH 0/3] Add AES-GCM and cipher only offload support
  2020-07-26 18:57 ` [dpdk-dev] [PATCH 0/3] Add AES-GCM and cipher only offload support Akhil Goyal
@ 2020-07-28  6:41   ` Nagadheeraj Rottela
  0 siblings, 0 replies; 17+ messages in thread
From: Nagadheeraj Rottela @ 2020-07-28  6:41 UTC (permalink / raw)
  To: Akhil Goyal; +Cc: dev, Srikanth Jampala

> -----Original Message-----
> From: Akhil Goyal <akhil.goyal@nxp.com>
> Sent: Monday, July 27, 2020 12:28 AM
> To: Nagadheeraj Rottela <rnagadheeraj@marvell.com>
> Cc: dev@dpdk.org; Srikanth Jampala <jsrikanth@marvell.com>
> Subject: [EXT] RE: [PATCH 0/3] Add AES-GCM and cipher only offload support
> 
> External Email
> 
> ----------------------------------------------------------------------
> > Subject: [PATCH 0/3] Add AES-GCM and cipher only offload support
> >
> > This patch set replaces the NITROX PMD specific test suite with
> > generic test suite and adds support for AES-GCM and cipher only offload.
> >
> I hope this series is for next release cycle.
> From next time, please mention 20.11 in subject while 20.08 release cycle is
> still in progress.
Sure, I will comply with this from the next time.

^ permalink raw reply	[flat|nested] 17+ messages in thread

* Re: [dpdk-dev] [PATCH 3/3] crypto/nitrox: support cipher only crypto operations
  2020-07-24 11:00 ` [dpdk-dev] [PATCH 3/3] crypto/nitrox: support cipher only crypto operations Nagadheeraj Rottela
@ 2020-09-22 19:11   ` Akhil Goyal
  2020-09-24 13:04     ` [dpdk-dev] [PATCH v2 0/3] Add AES-GCM and cipher only offload support Nagadheeraj Rottela
  0 siblings, 1 reply; 17+ messages in thread
From: Akhil Goyal @ 2020-09-22 19:11 UTC (permalink / raw)
  To: Nagadheeraj Rottela; +Cc: dev, jsrikanth

Hi Nagadheeraj,

> Subject: [PATCH 3/3] crypto/nitrox: support cipher only crypto operations
> 
> This patch adds cipher only crypto operation support.
> 
> Signed-off-by: Nagadheeraj Rottela <rnagadheeraj@marvell.com>
> ---

Could you please rebase this patch on latest TOT?
I see a conflict in the last patch.

You can also add an entry in release notes for aes-gcm and cipher only support.

Regards,
Akhil

^ permalink raw reply	[flat|nested] 17+ messages in thread

* [dpdk-dev] [PATCH v2 0/3] Add AES-GCM and cipher only offload support
  2020-09-22 19:11   ` Akhil Goyal
@ 2020-09-24 13:04     ` Nagadheeraj Rottela
  2020-09-24 13:04       ` [dpdk-dev] [PATCH v2 1/3] test/crypto: replace NITROX PMD specific test suite Nagadheeraj Rottela
                         ` (2 more replies)
  0 siblings, 3 replies; 17+ messages in thread
From: Nagadheeraj Rottela @ 2020-09-24 13:04 UTC (permalink / raw)
  To: akhil.goyal; +Cc: dev, jsrikanth, Nagadheeraj Rottela

This patch set replaces the NITROX PMD specific test suite with generic
test suite and adds support for AES-GCM and cipher only offload.
---
v2:
* Rebased patches to latest master and resolved merge conflict.
* Updated release notes.

Nagadheeraj Rottela (3):
  test/crypto: replace NITROX PMD specific test suite
  crypto/nitrox: support AES-GCM
  crypto/nitrox: support cipher only crypto operations

 app/test/test_cryptodev.c                     |  18 +-
 doc/guides/cryptodevs/features/nitrox.ini     |   3 +
 doc/guides/cryptodevs/nitrox.rst              |   6 +-
 doc/guides/rel_notes/release_20_11.rst        |   5 +
 drivers/crypto/nitrox/nitrox_sym.c            |  85 ++++-
 .../crypto/nitrox/nitrox_sym_capabilities.c   |  30 ++
 drivers/crypto/nitrox/nitrox_sym_ctx.h        |   5 +-
 drivers/crypto/nitrox/nitrox_sym_reqmgr.c     | 357 ++++++++++++++----
 8 files changed, 410 insertions(+), 99 deletions(-)

-- 
2.20.1


^ permalink raw reply	[flat|nested] 17+ messages in thread

* [dpdk-dev] [PATCH v2 1/3] test/crypto: replace NITROX PMD specific test suite
  2020-09-24 13:04     ` [dpdk-dev] [PATCH v2 0/3] Add AES-GCM and cipher only offload support Nagadheeraj Rottela
@ 2020-09-24 13:04       ` Nagadheeraj Rottela
  2020-10-06 20:07         ` Akhil Goyal
  2020-09-24 13:04       ` [dpdk-dev] [PATCH v2 2/3] crypto/nitrox: support AES-GCM Nagadheeraj Rottela
  2020-09-24 13:04       ` [dpdk-dev] [PATCH v2 3/3] crypto/nitrox: support cipher only crypto operations Nagadheeraj Rottela
  2 siblings, 1 reply; 17+ messages in thread
From: Nagadheeraj Rottela @ 2020-09-24 13:04 UTC (permalink / raw)
  To: akhil.goyal; +Cc: dev, jsrikanth, Nagadheeraj Rottela

Replace NITROX PMD specific tests with generic test suite.

Signed-off-by: Nagadheeraj Rottela <rnagadheeraj@marvell.com>
---
 app/test/test_cryptodev.c | 18 +-----------------
 1 file changed, 1 insertion(+), 17 deletions(-)

diff --git a/app/test/test_cryptodev.c b/app/test/test_cryptodev.c
index 70bf6fe2c..162134a5c 100644
--- a/app/test/test_cryptodev.c
+++ b/app/test/test_cryptodev.c
@@ -12665,22 +12665,6 @@ static struct unit_test_suite cryptodev_ccp_testsuite  = {
 	}
 };
 
-static struct unit_test_suite cryptodev_nitrox_testsuite  = {
-	.suite_name = "Crypto NITROX Unit Test Suite",
-	.setup = testsuite_setup,
-	.teardown = testsuite_teardown,
-	.unit_test_cases = {
-		TEST_CASE_ST(ut_setup, ut_teardown,
-			     test_device_configure_invalid_dev_id),
-		TEST_CASE_ST(ut_setup, ut_teardown,
-				test_device_configure_invalid_queue_pair_ids),
-		TEST_CASE_ST(ut_setup, ut_teardown, test_AES_chain_all),
-		TEST_CASE_ST(ut_setup, ut_teardown, test_3DES_chain_all),
-
-		TEST_CASES_END() /**< NULL terminate unit test array */
-	}
-};
-
 static int
 test_cryptodev_qat(void /*argv __rte_unused, int argc __rte_unused*/)
 {
@@ -13038,7 +13022,7 @@ test_cryptodev_nitrox(void)
 		return TEST_FAILED;
 	}
 
-	return unit_test_suite_runner(&cryptodev_nitrox_testsuite);
+	return unit_test_suite_runner(&cryptodev_testsuite);
 }
 
 REGISTER_TEST_COMMAND(cryptodev_qat_autotest, test_cryptodev_qat);
-- 
2.20.1


^ permalink raw reply	[flat|nested] 17+ messages in thread

* [dpdk-dev] [PATCH v2 2/3] crypto/nitrox: support AES-GCM
  2020-09-24 13:04     ` [dpdk-dev] [PATCH v2 0/3] Add AES-GCM and cipher only offload support Nagadheeraj Rottela
  2020-09-24 13:04       ` [dpdk-dev] [PATCH v2 1/3] test/crypto: replace NITROX PMD specific test suite Nagadheeraj Rottela
@ 2020-09-24 13:04       ` Nagadheeraj Rottela
  2020-10-06 20:04         ` Akhil Goyal
  2020-09-24 13:04       ` [dpdk-dev] [PATCH v2 3/3] crypto/nitrox: support cipher only crypto operations Nagadheeraj Rottela
  2 siblings, 1 reply; 17+ messages in thread
From: Nagadheeraj Rottela @ 2020-09-24 13:04 UTC (permalink / raw)
  To: akhil.goyal; +Cc: dev, jsrikanth, Nagadheeraj Rottela

This patch adds AES-GCM AEAD algorithm.

Signed-off-by: Nagadheeraj Rottela <rnagadheeraj@marvell.com>
---
 doc/guides/cryptodevs/features/nitrox.ini     |   3 +
 doc/guides/cryptodevs/nitrox.rst              |   4 +
 drivers/crypto/nitrox/nitrox_sym.c            |  82 +++++++-
 .../crypto/nitrox/nitrox_sym_capabilities.c   |  30 +++
 drivers/crypto/nitrox/nitrox_sym_ctx.h        |   5 +-
 drivers/crypto/nitrox/nitrox_sym_reqmgr.c     | 182 +++++++++++++++---
 6 files changed, 268 insertions(+), 38 deletions(-)

diff --git a/doc/guides/cryptodevs/features/nitrox.ini b/doc/guides/cryptodevs/features/nitrox.ini
index 183494731..a1d6bcb4f 100644
--- a/doc/guides/cryptodevs/features/nitrox.ini
+++ b/doc/guides/cryptodevs/features/nitrox.ini
@@ -34,6 +34,9 @@ SHA256 HMAC  = Y
 ; Supported AEAD algorithms of the 'nitrox' crypto driver.
 ;
 [AEAD]
+AES GCM (128)  = Y
+AES GCM (192)  = Y
+AES GCM (256)  = Y
 
 ;
 ; Supported Asymmetric algorithms of the 'nitrox' crypto driver.
diff --git a/doc/guides/cryptodevs/nitrox.rst b/doc/guides/cryptodevs/nitrox.rst
index 85f5212b6..91fca905a 100644
--- a/doc/guides/cryptodevs/nitrox.rst
+++ b/doc/guides/cryptodevs/nitrox.rst
@@ -26,6 +26,10 @@ Hash algorithms:
 * ``RTE_CRYPTO_AUTH_SHA224_HMAC``
 * ``RTE_CRYPTO_AUTH_SHA256_HMAC``
 
+Supported AEAD algorithms:
+
+* ``RTE_CRYPTO_AEAD_AES_GCM``
+
 Limitations
 -----------
 
diff --git a/drivers/crypto/nitrox/nitrox_sym.c b/drivers/crypto/nitrox/nitrox_sym.c
index fad4a7a48..fe3ee6e23 100644
--- a/drivers/crypto/nitrox/nitrox_sym.c
+++ b/drivers/crypto/nitrox/nitrox_sym.c
@@ -20,6 +20,7 @@
 #define NPS_PKT_IN_INSTR_SIZE 64
 #define IV_FROM_DPTR 1
 #define FLEXI_CRYPTO_ENCRYPT_HMAC 0x33
+#define FLEXI_CRYPTO_MAX_AAD_LEN 512
 #define AES_KEYSIZE_128 16
 #define AES_KEYSIZE_192 24
 #define AES_KEYSIZE_256 32
@@ -297,6 +298,9 @@ get_crypto_chain_order(const struct rte_crypto_sym_xform *xform)
 			}
 		}
 		break;
+	case RTE_CRYPTO_SYM_XFORM_AEAD:
+		res = NITROX_CHAIN_COMBINED;
+		break;
 	default:
 		break;
 	}
@@ -431,17 +435,17 @@ get_flexi_auth_type(enum rte_crypto_auth_algorithm algo)
 }
 
 static bool
-auth_key_digest_is_valid(struct rte_crypto_auth_xform *xform,
-			 struct flexi_crypto_context *fctx)
+auth_key_is_valid(const uint8_t *data, uint16_t length,
+		  struct flexi_crypto_context *fctx)
 {
-	if (unlikely(!xform->key.data && xform->key.length)) {
+	if (unlikely(!data && length)) {
 		NITROX_LOG(ERR, "Invalid auth key\n");
 		return false;
 	}
 
-	if (unlikely(xform->key.length > sizeof(fctx->auth.opad))) {
+	if (unlikely(length > sizeof(fctx->auth.opad))) {
 		NITROX_LOG(ERR, "Invalid auth key length %d\n",
-			   xform->key.length);
+			   length);
 		return false;
 	}
 
@@ -459,11 +463,10 @@ configure_auth_ctx(struct rte_crypto_auth_xform *xform,
 	if (unlikely(type == AUTH_INVALID))
 		return -ENOTSUP;
 
-	if (unlikely(!auth_key_digest_is_valid(xform, fctx)))
+	if (unlikely(!auth_key_is_valid(xform->key.data, xform->key.length,
+					fctx)))
 		return -EINVAL;
 
-	ctx->auth_op = xform->op;
-	ctx->auth_algo = xform->algo;
 	ctx->digest_length = xform->digest_length;
 
 	fctx->flags = rte_be_to_cpu_64(fctx->flags);
@@ -476,6 +479,56 @@ configure_auth_ctx(struct rte_crypto_auth_xform *xform,
 	return 0;
 }
 
+static int
+configure_aead_ctx(struct rte_crypto_aead_xform *xform,
+		   struct nitrox_crypto_ctx *ctx)
+{
+	int aes_keylen;
+	struct flexi_crypto_context *fctx = &ctx->fctx;
+
+	if (unlikely(xform->aad_length > FLEXI_CRYPTO_MAX_AAD_LEN)) {
+		NITROX_LOG(ERR, "AAD length %d not supported\n",
+			   xform->aad_length);
+		return -ENOTSUP;
+	}
+
+	if (unlikely(xform->algo != RTE_CRYPTO_AEAD_AES_GCM))
+		return -ENOTSUP;
+
+	aes_keylen = flexi_aes_keylen(xform->key.length, true);
+	if (unlikely(aes_keylen < 0))
+		return -EINVAL;
+
+	if (unlikely(!auth_key_is_valid(xform->key.data, xform->key.length,
+					fctx)))
+		return -EINVAL;
+
+	if (unlikely(xform->iv.length > MAX_IV_LEN))
+		return -EINVAL;
+
+	fctx->flags = rte_be_to_cpu_64(fctx->flags);
+	fctx->w0.cipher_type = CIPHER_AES_GCM;
+	fctx->w0.aes_keylen = aes_keylen;
+	fctx->w0.iv_source = IV_FROM_DPTR;
+	fctx->w0.hash_type = AUTH_NULL;
+	fctx->w0.auth_input_type = 1;
+	fctx->w0.mac_len = xform->digest_length;
+	fctx->flags = rte_cpu_to_be_64(fctx->flags);
+	memset(fctx->crypto.key, 0, sizeof(fctx->crypto.key));
+	memcpy(fctx->crypto.key, xform->key.data, xform->key.length);
+	memset(&fctx->auth, 0, sizeof(fctx->auth));
+	memcpy(fctx->auth.opad, xform->key.data, xform->key.length);
+
+	ctx->opcode = FLEXI_CRYPTO_ENCRYPT_HMAC;
+	ctx->req_op = (xform->op == RTE_CRYPTO_AEAD_OP_ENCRYPT) ?
+			NITROX_OP_ENCRYPT : NITROX_OP_DECRYPT;
+	ctx->iv.offset = xform->iv.offset;
+	ctx->iv.length = xform->iv.length;
+	ctx->digest_length = xform->digest_length;
+	ctx->aad_length = xform->aad_length;
+	return 0;
+}
+
 static int
 nitrox_sym_dev_sess_configure(struct rte_cryptodev *cdev,
 			      struct rte_crypto_sym_xform *xform,
@@ -486,6 +539,8 @@ nitrox_sym_dev_sess_configure(struct rte_cryptodev *cdev,
 	struct nitrox_crypto_ctx *ctx;
 	struct rte_crypto_cipher_xform *cipher_xform = NULL;
 	struct rte_crypto_auth_xform *auth_xform = NULL;
+	struct rte_crypto_aead_xform *aead_xform = NULL;
+	int ret = -EINVAL;
 
 	if (rte_mempool_get(mempool, &mp_obj)) {
 		NITROX_LOG(ERR, "Couldn't allocate context\n");
@@ -503,8 +558,12 @@ nitrox_sym_dev_sess_configure(struct rte_cryptodev *cdev,
 		auth_xform = &xform->auth;
 		cipher_xform = &xform->next->cipher;
 		break;
+	case NITROX_CHAIN_COMBINED:
+		aead_xform = &xform->aead;
+		break;
 	default:
 		NITROX_LOG(ERR, "Crypto chain not supported\n");
+		ret = -ENOTSUP;
 		goto err;
 	}
 
@@ -518,12 +577,17 @@ nitrox_sym_dev_sess_configure(struct rte_cryptodev *cdev,
 		goto err;
 	}
 
+	if (aead_xform && unlikely(configure_aead_ctx(aead_xform, ctx))) {
+		NITROX_LOG(ERR, "Failed to configure aead ctx\n");
+		goto err;
+	}
+
 	ctx->iova = rte_mempool_virt2iova(ctx);
 	set_sym_session_private_data(sess, cdev->driver_id, ctx);
 	return 0;
 err:
 	rte_mempool_put(mempool, mp_obj);
-	return -EINVAL;
+	return ret;
 }
 
 static void
diff --git a/drivers/crypto/nitrox/nitrox_sym_capabilities.c b/drivers/crypto/nitrox/nitrox_sym_capabilities.c
index dc4df9185..a30cd9f8f 100644
--- a/drivers/crypto/nitrox/nitrox_sym_capabilities.c
+++ b/drivers/crypto/nitrox/nitrox_sym_capabilities.c
@@ -108,6 +108,36 @@ static const struct rte_cryptodev_capabilities nitrox_capabilities[] = {
 			}, }
 		}, }
 	},
+	{	/* AES GCM */
+		.op = RTE_CRYPTO_OP_TYPE_SYMMETRIC,
+		{.sym = {
+			.xform_type = RTE_CRYPTO_SYM_XFORM_AEAD,
+			{.aead = {
+				.algo = RTE_CRYPTO_AEAD_AES_GCM,
+				.block_size = 16,
+				.key_size = {
+					.min = 16,
+					.max = 32,
+					.increment = 8
+				},
+				.digest_size = {
+					.min = 1,
+					.max = 16,
+					.increment = 1
+				},
+				.aad_size = {
+					.min = 0,
+					.max = 512,
+					.increment = 1
+				},
+				.iv_size = {
+					.min = 12,
+					.max = 16,
+					.increment = 4
+				},
+			}, }
+		}, }
+	},
 	RTE_CRYPTODEV_END_OF_CAPABILITIES_LIST()
 };
 
diff --git a/drivers/crypto/nitrox/nitrox_sym_ctx.h b/drivers/crypto/nitrox/nitrox_sym_ctx.h
index 2985e519f..deb00fc1e 100644
--- a/drivers/crypto/nitrox/nitrox_sym_ctx.h
+++ b/drivers/crypto/nitrox/nitrox_sym_ctx.h
@@ -11,6 +11,7 @@
 
 #define AES_MAX_KEY_SIZE 32
 #define AES_BLOCK_SIZE 16
+#define AES_GCM_SALT_SIZE 4
 
 enum nitrox_chain {
 	NITROX_CHAIN_CIPHER_ONLY,
@@ -69,14 +70,14 @@ struct flexi_crypto_context {
 struct nitrox_crypto_ctx {
 	struct flexi_crypto_context fctx;
 	enum nitrox_chain nitrox_chain;
-	enum rte_crypto_auth_operation auth_op;
-	enum rte_crypto_auth_algorithm auth_algo;
 	struct {
 		uint16_t offset;
 		uint16_t length;
 	} iv;
 	rte_iova_t iova;
+	uint8_t salt[AES_GCM_SALT_SIZE];
 	uint16_t digest_length;
+	uint16_t aad_length;
 	uint8_t opcode;
 	uint8_t req_op;
 };
diff --git a/drivers/crypto/nitrox/nitrox_sym_reqmgr.c b/drivers/crypto/nitrox/nitrox_sym_reqmgr.c
index 449224780..47f5244b1 100644
--- a/drivers/crypto/nitrox/nitrox_sym_reqmgr.c
+++ b/drivers/crypto/nitrox/nitrox_sym_reqmgr.c
@@ -238,12 +238,13 @@ create_se_instr(struct nitrox_softreq *sr, uint8_t qno)
 }
 
 static void
-softreq_copy_iv(struct nitrox_softreq *sr)
+softreq_copy_iv(struct nitrox_softreq *sr, uint8_t salt_size)
 {
-	sr->iv.virt = rte_crypto_op_ctod_offset(sr->op, uint8_t *,
-						sr->ctx->iv.offset);
-	sr->iv.iova = rte_crypto_op_ctophys_offset(sr->op, sr->ctx->iv.offset);
-	sr->iv.len = sr->ctx->iv.length;
+	uint16_t offset = sr->ctx->iv.offset + salt_size;
+
+	sr->iv.virt = rte_crypto_op_ctod_offset(sr->op, uint8_t *, offset);
+	sr->iv.iova = rte_crypto_op_ctophys_offset(sr->op, offset);
+	sr->iv.len = sr->ctx->iv.length - salt_size;
 }
 
 static int
@@ -254,7 +255,7 @@ extract_cipher_auth_digest(struct nitrox_softreq *sr,
 	struct rte_mbuf *mdst = op->sym->m_dst ? op->sym->m_dst :
 					op->sym->m_src;
 
-	if (sr->ctx->auth_op == RTE_CRYPTO_AUTH_OP_VERIFY &&
+	if (sr->ctx->req_op == NITROX_OP_DECRYPT &&
 	    unlikely(!op->sym->auth.digest.data))
 		return -EINVAL;
 
@@ -352,6 +353,13 @@ create_cipher_auth_sglist(struct nitrox_softreq *sr,
 	if (unlikely(auth_only_len < 0))
 		return -EINVAL;
 
+	if (unlikely(
+		op->sym->cipher.data.offset + op->sym->cipher.data.length !=
+		op->sym->auth.data.offset + op->sym->auth.data.length)) {
+		NITROX_LOG(ERR, "Auth only data after cipher data not supported\n");
+		return -ENOTSUP;
+	}
+
 	err = create_sglist_from_mbuf(sgtbl, mbuf, op->sym->auth.data.offset,
 				      auth_only_len);
 	if (unlikely(err))
@@ -365,6 +373,41 @@ create_cipher_auth_sglist(struct nitrox_softreq *sr,
 	return 0;
 }
 
+static int
+create_combined_sglist(struct nitrox_softreq *sr, struct nitrox_sgtable *sgtbl,
+		       struct rte_mbuf *mbuf)
+{
+	struct rte_crypto_op *op = sr->op;
+
+	fill_sglist(sgtbl, sr->iv.len, sr->iv.iova, sr->iv.virt);
+	fill_sglist(sgtbl, sr->ctx->aad_length, op->sym->aead.aad.phys_addr,
+		    op->sym->aead.aad.data);
+	return create_sglist_from_mbuf(sgtbl, mbuf, op->sym->cipher.data.offset,
+				       op->sym->cipher.data.length);
+}
+
+static int
+create_aead_sglist(struct nitrox_softreq *sr, struct nitrox_sgtable *sgtbl,
+		   struct rte_mbuf *mbuf)
+{
+	int err;
+
+	switch (sr->ctx->nitrox_chain) {
+	case NITROX_CHAIN_CIPHER_AUTH:
+	case NITROX_CHAIN_AUTH_CIPHER:
+		err = create_cipher_auth_sglist(sr, sgtbl, mbuf);
+		break;
+	case NITROX_CHAIN_COMBINED:
+		err = create_combined_sglist(sr, sgtbl, mbuf);
+		break;
+	default:
+		err = -EINVAL;
+		break;
+	}
+
+	return err;
+}
+
 static void
 create_sgcomp(struct nitrox_sgtable *sgtbl)
 {
@@ -383,17 +426,16 @@ create_sgcomp(struct nitrox_sgtable *sgtbl)
 }
 
 static int
-create_cipher_auth_inbuf(struct nitrox_softreq *sr,
-			 struct nitrox_sglist *digest)
+create_aead_inbuf(struct nitrox_softreq *sr, struct nitrox_sglist *digest)
 {
 	int err;
 	struct nitrox_crypto_ctx *ctx = sr->ctx;
 
-	err = create_cipher_auth_sglist(sr, &sr->in, sr->op->sym->m_src);
+	err = create_aead_sglist(sr, &sr->in, sr->op->sym->m_src);
 	if (unlikely(err))
 		return err;
 
-	if (ctx->auth_op == RTE_CRYPTO_AUTH_OP_VERIFY)
+	if (ctx->req_op == NITROX_OP_DECRYPT)
 		fill_sglist(&sr->in, digest->len, digest->iova, digest->virt);
 
 	create_sgcomp(&sr->in);
@@ -402,25 +444,24 @@ create_cipher_auth_inbuf(struct nitrox_softreq *sr,
 }
 
 static int
-create_cipher_auth_oop_outbuf(struct nitrox_softreq *sr,
-			      struct nitrox_sglist *digest)
+create_aead_oop_outbuf(struct nitrox_softreq *sr, struct nitrox_sglist *digest)
 {
 	int err;
 	struct nitrox_crypto_ctx *ctx = sr->ctx;
 
-	err = create_cipher_auth_sglist(sr, &sr->out, sr->op->sym->m_dst);
+	err = create_aead_sglist(sr, &sr->out, sr->op->sym->m_dst);
 	if (unlikely(err))
 		return err;
 
-	if (ctx->auth_op == RTE_CRYPTO_AUTH_OP_GENERATE)
+	if (ctx->req_op == NITROX_OP_ENCRYPT)
 		fill_sglist(&sr->out, digest->len, digest->iova, digest->virt);
 
 	return 0;
 }
 
 static void
-create_cipher_auth_inplace_outbuf(struct nitrox_softreq *sr,
-				  struct nitrox_sglist *digest)
+create_aead_inplace_outbuf(struct nitrox_softreq *sr,
+			   struct nitrox_sglist *digest)
 {
 	int i, cnt;
 	struct nitrox_crypto_ctx *ctx = sr->ctx;
@@ -433,17 +474,16 @@ create_cipher_auth_inplace_outbuf(struct nitrox_softreq *sr,
 	}
 
 	sr->out.map_bufs_cnt = cnt;
-	if (ctx->auth_op == RTE_CRYPTO_AUTH_OP_GENERATE) {
+	if (ctx->req_op == NITROX_OP_ENCRYPT) {
 		fill_sglist(&sr->out, digest->len, digest->iova,
 			    digest->virt);
-	} else if (ctx->auth_op == RTE_CRYPTO_AUTH_OP_VERIFY) {
+	} else if (ctx->req_op == NITROX_OP_DECRYPT) {
 		sr->out.map_bufs_cnt--;
 	}
 }
 
 static int
-create_cipher_auth_outbuf(struct nitrox_softreq *sr,
-			  struct nitrox_sglist *digest)
+create_aead_outbuf(struct nitrox_softreq *sr, struct nitrox_sglist *digest)
 {
 	struct rte_crypto_op *op = sr->op;
 	int cnt = 0;
@@ -458,11 +498,11 @@ create_cipher_auth_outbuf(struct nitrox_softreq *sr,
 	if (op->sym->m_dst) {
 		int err;
 
-		err = create_cipher_auth_oop_outbuf(sr, digest);
+		err = create_aead_oop_outbuf(sr, digest);
 		if (unlikely(err))
 			return err;
 	} else {
-		create_cipher_auth_inplace_outbuf(sr, digest);
+		create_aead_inplace_outbuf(sr, digest);
 	}
 
 	cnt = sr->out.map_bufs_cnt;
@@ -516,16 +556,16 @@ process_cipher_auth_data(struct nitrox_softreq *sr)
 	int err;
 	struct nitrox_sglist digest;
 
-	softreq_copy_iv(sr);
+	softreq_copy_iv(sr, 0);
 	err = extract_cipher_auth_digest(sr, &digest);
 	if (unlikely(err))
 		return err;
 
-	err = create_cipher_auth_inbuf(sr, &digest);
+	err = create_aead_inbuf(sr, &digest);
 	if (unlikely(err))
 		return err;
 
-	err = create_cipher_auth_outbuf(sr, &digest);
+	err = create_aead_outbuf(sr, &digest);
 	if (unlikely(err))
 		return err;
 
@@ -534,6 +574,86 @@ process_cipher_auth_data(struct nitrox_softreq *sr)
 	return 0;
 }
 
+static int
+softreq_copy_salt(struct nitrox_softreq *sr)
+{
+	struct nitrox_crypto_ctx *ctx = sr->ctx;
+	uint8_t *addr;
+
+	if (unlikely(ctx->iv.length < AES_GCM_SALT_SIZE)) {
+		NITROX_LOG(ERR, "Invalid IV length %d\n", ctx->iv.length);
+		return -EINVAL;
+	}
+
+	addr = rte_crypto_op_ctod_offset(sr->op, uint8_t *, ctx->iv.offset);
+	if (!memcmp(ctx->salt, addr, AES_GCM_SALT_SIZE))
+		return 0;
+
+	memcpy(ctx->salt, addr, AES_GCM_SALT_SIZE);
+	memcpy(ctx->fctx.crypto.iv, addr, AES_GCM_SALT_SIZE);
+	return 0;
+}
+
+static int
+extract_combined_digest(struct nitrox_softreq *sr, struct nitrox_sglist *digest)
+{
+	struct rte_crypto_op *op = sr->op;
+	struct rte_mbuf *mdst = op->sym->m_dst ? op->sym->m_dst :
+		op->sym->m_src;
+
+	digest->len = sr->ctx->digest_length;
+	if (op->sym->aead.digest.data) {
+		digest->iova = op->sym->aead.digest.phys_addr;
+		digest->virt = op->sym->aead.digest.data;
+
+		return 0;
+	}
+
+	if (unlikely(rte_pktmbuf_data_len(mdst) < op->sym->aead.data.offset +
+	       op->sym->aead.data.length + digest->len))
+		return -EINVAL;
+
+	digest->iova = rte_pktmbuf_mtophys_offset(mdst,
+					op->sym->aead.data.offset +
+					op->sym->aead.data.length);
+	digest->virt = rte_pktmbuf_mtod_offset(mdst, uint8_t *,
+					op->sym->aead.data.offset +
+					op->sym->aead.data.length);
+
+	return 0;
+}
+
+static int
+process_combined_data(struct nitrox_softreq *sr)
+{
+	int err;
+	struct nitrox_sglist digest;
+	struct rte_crypto_op *op = sr->op;
+
+	err = softreq_copy_salt(sr);
+	if (unlikely(err))
+		return err;
+
+	softreq_copy_iv(sr, AES_GCM_SALT_SIZE);
+	err = extract_combined_digest(sr, &digest);
+	if (unlikely(err))
+		return err;
+
+	err = create_aead_inbuf(sr, &digest);
+	if (unlikely(err))
+		return err;
+
+	err = create_aead_outbuf(sr, &digest);
+	if (unlikely(err))
+		return err;
+
+	create_aead_gph(op->sym->aead.data.length, sr->iv.len,
+			op->sym->aead.data.length + sr->ctx->aad_length,
+			&sr->gph);
+
+	return 0;
+}
+
 static int
 process_softreq(struct nitrox_softreq *sr)
 {
@@ -545,6 +665,9 @@ process_softreq(struct nitrox_softreq *sr)
 	case NITROX_CHAIN_AUTH_CIPHER:
 		err = process_cipher_auth_data(sr);
 		break;
+	case NITROX_CHAIN_COMBINED:
+		err = process_combined_data(sr);
+		break;
 	default:
 		err = -EINVAL;
 		break;
@@ -558,10 +681,15 @@ nitrox_process_se_req(uint16_t qno, struct rte_crypto_op *op,
 		      struct nitrox_crypto_ctx *ctx,
 		      struct nitrox_softreq *sr)
 {
+	int err;
+
 	softreq_init(sr, sr->iova);
 	sr->ctx = ctx;
 	sr->op = op;
-	process_softreq(sr);
+	err = process_softreq(sr);
+	if (unlikely(err))
+		return err;
+
 	create_se_instr(sr, qno);
 	sr->timeout = rte_get_timer_cycles() + CMD_TIMEOUT * rte_get_timer_hz();
 	return 0;
@@ -577,7 +705,7 @@ nitrox_check_se_req(struct nitrox_softreq *sr, struct rte_crypto_op **op)
 	cc = *(volatile uint64_t *)(&sr->resp.completion);
 	orh = *(volatile uint64_t *)(&sr->resp.orh);
 	if (cc != PENDING_SIG)
-		err = 0;
+		err = orh & 0xff;
 	else if ((orh != PENDING_SIG) && (orh & 0xff))
 		err = orh & 0xff;
 	else if (rte_get_timer_cycles() >= sr->timeout)
-- 
2.20.1


^ permalink raw reply	[flat|nested] 17+ messages in thread

* [dpdk-dev] [PATCH v2 3/3] crypto/nitrox: support cipher only crypto operations
  2020-09-24 13:04     ` [dpdk-dev] [PATCH v2 0/3] Add AES-GCM and cipher only offload support Nagadheeraj Rottela
  2020-09-24 13:04       ` [dpdk-dev] [PATCH v2 1/3] test/crypto: replace NITROX PMD specific test suite Nagadheeraj Rottela
  2020-09-24 13:04       ` [dpdk-dev] [PATCH v2 2/3] crypto/nitrox: support AES-GCM Nagadheeraj Rottela
@ 2020-09-24 13:04       ` Nagadheeraj Rottela
  2 siblings, 0 replies; 17+ messages in thread
From: Nagadheeraj Rottela @ 2020-09-24 13:04 UTC (permalink / raw)
  To: akhil.goyal; +Cc: dev, jsrikanth, Nagadheeraj Rottela

This patch adds cipher only crypto operation support.

Signed-off-by: Nagadheeraj Rottela <rnagadheeraj@marvell.com>
---
 doc/guides/cryptodevs/nitrox.rst          |   2 -
 doc/guides/rel_notes/release_20_11.rst    |   5 +
 drivers/crypto/nitrox/nitrox_sym.c        |   3 +
 drivers/crypto/nitrox/nitrox_sym_reqmgr.c | 191 ++++++++++++++++------
 4 files changed, 149 insertions(+), 52 deletions(-)

diff --git a/doc/guides/cryptodevs/nitrox.rst b/doc/guides/cryptodevs/nitrox.rst
index 91fca905a..095e545c6 100644
--- a/doc/guides/cryptodevs/nitrox.rst
+++ b/doc/guides/cryptodevs/nitrox.rst
@@ -33,8 +33,6 @@ Supported AEAD algorithms:
 Limitations
 -----------
 
-* AES_CBC Cipher Only combination is not supported.
-* 3DES Cipher Only combination is not supported.
 * Session-less APIs are not supported.
 
 Installation
diff --git a/doc/guides/rel_notes/release_20_11.rst b/doc/guides/rel_notes/release_20_11.rst
index 73ac08fb0..ddcf90356 100644
--- a/doc/guides/rel_notes/release_20_11.rst
+++ b/doc/guides/rel_notes/release_20_11.rst
@@ -55,6 +55,11 @@ New Features
      Also, make sure to start the actual text at the margin.
      =======================================================
 
+* **Updated Marvell NITROX symmetric crypto PMD.**
+
+  * Added cipher only offload support.
+  * Added AES-GCM support.
+
 
 Removed Items
 -------------
diff --git a/drivers/crypto/nitrox/nitrox_sym.c b/drivers/crypto/nitrox/nitrox_sym.c
index fe3ee6e23..2768bdd2e 100644
--- a/drivers/crypto/nitrox/nitrox_sym.c
+++ b/drivers/crypto/nitrox/nitrox_sym.c
@@ -550,6 +550,9 @@ nitrox_sym_dev_sess_configure(struct rte_cryptodev *cdev,
 	ctx = mp_obj;
 	ctx->nitrox_chain = get_crypto_chain_order(xform);
 	switch (ctx->nitrox_chain) {
+	case NITROX_CHAIN_CIPHER_ONLY:
+		cipher_xform = &xform->cipher;
+		break;
 	case NITROX_CHAIN_CIPHER_AUTH:
 		cipher_xform = &xform->cipher;
 		auth_xform = &xform->next->auth;
diff --git a/drivers/crypto/nitrox/nitrox_sym_reqmgr.c b/drivers/crypto/nitrox/nitrox_sym_reqmgr.c
index 47f5244b1..fe3ca25a0 100644
--- a/drivers/crypto/nitrox/nitrox_sym_reqmgr.c
+++ b/drivers/crypto/nitrox/nitrox_sym_reqmgr.c
@@ -247,38 +247,6 @@ softreq_copy_iv(struct nitrox_softreq *sr, uint8_t salt_size)
 	sr->iv.len = sr->ctx->iv.length - salt_size;
 }
 
-static int
-extract_cipher_auth_digest(struct nitrox_softreq *sr,
-			   struct nitrox_sglist *digest)
-{
-	struct rte_crypto_op *op = sr->op;
-	struct rte_mbuf *mdst = op->sym->m_dst ? op->sym->m_dst :
-					op->sym->m_src;
-
-	if (sr->ctx->req_op == NITROX_OP_DECRYPT &&
-	    unlikely(!op->sym->auth.digest.data))
-		return -EINVAL;
-
-	digest->len = sr->ctx->digest_length;
-	if (op->sym->auth.digest.data) {
-		digest->iova = op->sym->auth.digest.phys_addr;
-		digest->virt = op->sym->auth.digest.data;
-		return 0;
-	}
-
-	if (unlikely(rte_pktmbuf_data_len(mdst) < op->sym->auth.data.offset +
-	       op->sym->auth.data.length + digest->len))
-		return -EINVAL;
-
-	digest->iova = rte_pktmbuf_iova_offset(mdst,
-					op->sym->auth.data.offset +
-					op->sym->auth.data.length);
-	digest->virt = rte_pktmbuf_mtod_offset(mdst, uint8_t *,
-					op->sym->auth.data.offset +
-					op->sym->auth.data.length);
-	return 0;
-}
-
 static void
 fill_sglist(struct nitrox_sgtable *sgtbl, uint16_t len, rte_iova_t iova,
 	    void *virt)
@@ -340,6 +308,143 @@ create_sglist_from_mbuf(struct nitrox_sgtable *sgtbl, struct rte_mbuf *mbuf,
 	return 0;
 }
 
+static void
+create_sgcomp(struct nitrox_sgtable *sgtbl)
+{
+	int i, j, nr_sgcomp;
+	struct nitrox_sgcomp *sgcomp = sgtbl->sgcomp;
+	struct nitrox_sglist *sglist = sgtbl->sglist;
+
+	nr_sgcomp = RTE_ALIGN_MUL_CEIL(sgtbl->map_bufs_cnt, 4) / 4;
+	sgtbl->nr_sgcomp = nr_sgcomp;
+	for (i = 0; i < nr_sgcomp; i++, sgcomp++) {
+		for (j = 0; j < 4; j++, sglist++) {
+			sgcomp->len[j] = rte_cpu_to_be_16(sglist->len);
+			sgcomp->iova[j] = rte_cpu_to_be_64(sglist->iova);
+		}
+	}
+}
+
+static int
+create_cipher_inbuf(struct nitrox_softreq *sr)
+{
+	int err;
+	struct rte_crypto_op *op = sr->op;
+
+	fill_sglist(&sr->in, sr->iv.len, sr->iv.iova, sr->iv.virt);
+	err = create_sglist_from_mbuf(&sr->in, op->sym->m_src,
+				      op->sym->cipher.data.offset,
+				      op->sym->cipher.data.length);
+	if (unlikely(err))
+		return err;
+
+	create_sgcomp(&sr->in);
+	sr->dptr = sr->iova + offsetof(struct nitrox_softreq, in.sgcomp);
+
+	return 0;
+}
+
+static int
+create_cipher_outbuf(struct nitrox_softreq *sr)
+{
+	struct rte_crypto_op *op = sr->op;
+	int err, cnt = 0;
+	struct rte_mbuf *m_dst = op->sym->m_dst ? op->sym->m_dst :
+		op->sym->m_src;
+
+	sr->resp.orh = PENDING_SIG;
+	sr->out.sglist[cnt].len = sizeof(sr->resp.orh);
+	sr->out.sglist[cnt].iova = sr->iova + offsetof(struct nitrox_softreq,
+						       resp.orh);
+	sr->out.sglist[cnt].virt = &sr->resp.orh;
+	cnt++;
+
+	sr->out.map_bufs_cnt = cnt;
+	fill_sglist(&sr->out, sr->iv.len, sr->iv.iova, sr->iv.virt);
+	err = create_sglist_from_mbuf(&sr->out, m_dst,
+				      op->sym->cipher.data.offset,
+				      op->sym->cipher.data.length);
+	if (unlikely(err))
+		return err;
+
+	cnt = sr->out.map_bufs_cnt;
+	sr->resp.completion = PENDING_SIG;
+	sr->out.sglist[cnt].len = sizeof(sr->resp.completion);
+	sr->out.sglist[cnt].iova = sr->iova + offsetof(struct nitrox_softreq,
+						     resp.completion);
+	sr->out.sglist[cnt].virt = &sr->resp.completion;
+	cnt++;
+
+	RTE_VERIFY(cnt <= MAX_SGBUF_CNT);
+	sr->out.map_bufs_cnt = cnt;
+
+	create_sgcomp(&sr->out);
+	sr->rptr = sr->iova + offsetof(struct nitrox_softreq, out.sgcomp);
+
+	return 0;
+}
+
+static void
+create_cipher_gph(uint32_t cryptlen, uint16_t ivlen, struct gphdr *gph)
+{
+	gph->param0 = rte_cpu_to_be_16(cryptlen);
+	gph->param1 = 0;
+	gph->param2 = rte_cpu_to_be_16(ivlen);
+	gph->param3 = 0;
+}
+
+static int
+process_cipher_data(struct nitrox_softreq *sr)
+{
+	struct rte_crypto_op *op = sr->op;
+	int err;
+
+	softreq_copy_iv(sr, 0);
+	err = create_cipher_inbuf(sr);
+	if (unlikely(err))
+		return err;
+
+	err = create_cipher_outbuf(sr);
+	if (unlikely(err))
+		return err;
+
+	create_cipher_gph(op->sym->cipher.data.length, sr->iv.len, &sr->gph);
+
+	return 0;
+}
+
+static int
+extract_cipher_auth_digest(struct nitrox_softreq *sr,
+			   struct nitrox_sglist *digest)
+{
+	struct rte_crypto_op *op = sr->op;
+	struct rte_mbuf *mdst = op->sym->m_dst ? op->sym->m_dst :
+					op->sym->m_src;
+
+	if (sr->ctx->req_op == NITROX_OP_DECRYPT &&
+	    unlikely(!op->sym->auth.digest.data))
+		return -EINVAL;
+
+	digest->len = sr->ctx->digest_length;
+	if (op->sym->auth.digest.data) {
+		digest->iova = op->sym->auth.digest.phys_addr;
+		digest->virt = op->sym->auth.digest.data;
+		return 0;
+	}
+
+	if (unlikely(rte_pktmbuf_data_len(mdst) < op->sym->auth.data.offset +
+	       op->sym->auth.data.length + digest->len))
+		return -EINVAL;
+
+	digest->iova = rte_pktmbuf_iova_offset(mdst,
+					op->sym->auth.data.offset +
+					op->sym->auth.data.length);
+	digest->virt = rte_pktmbuf_mtod_offset(mdst, uint8_t *,
+					op->sym->auth.data.offset +
+					op->sym->auth.data.length);
+	return 0;
+}
+
 static int
 create_cipher_auth_sglist(struct nitrox_softreq *sr,
 			  struct nitrox_sgtable *sgtbl, struct rte_mbuf *mbuf)
@@ -408,23 +513,6 @@ create_aead_sglist(struct nitrox_softreq *sr, struct nitrox_sgtable *sgtbl,
 	return err;
 }
 
-static void
-create_sgcomp(struct nitrox_sgtable *sgtbl)
-{
-	int i, j, nr_sgcomp;
-	struct nitrox_sgcomp *sgcomp = sgtbl->sgcomp;
-	struct nitrox_sglist *sglist = sgtbl->sglist;
-
-	nr_sgcomp = RTE_ALIGN_MUL_CEIL(sgtbl->map_bufs_cnt, 4) / 4;
-	sgtbl->nr_sgcomp = nr_sgcomp;
-	for (i = 0; i < nr_sgcomp; i++, sgcomp++) {
-		for (j = 0; j < 4; j++, sglist++) {
-			sgcomp->len[j] = rte_cpu_to_be_16(sglist->len);
-			sgcomp->iova[j] = rte_cpu_to_be_64(sglist->iova);
-		}
-	}
-}
-
 static int
 create_aead_inbuf(struct nitrox_softreq *sr, struct nitrox_sglist *digest)
 {
@@ -613,7 +701,7 @@ extract_combined_digest(struct nitrox_softreq *sr, struct nitrox_sglist *digest)
 	       op->sym->aead.data.length + digest->len))
 		return -EINVAL;
 
-	digest->iova = rte_pktmbuf_mtophys_offset(mdst,
+	digest->iova = rte_pktmbuf_iova_offset(mdst,
 					op->sym->aead.data.offset +
 					op->sym->aead.data.length);
 	digest->virt = rte_pktmbuf_mtod_offset(mdst, uint8_t *,
@@ -661,6 +749,9 @@ process_softreq(struct nitrox_softreq *sr)
 	int err = 0;
 
 	switch (ctx->nitrox_chain) {
+	case NITROX_CHAIN_CIPHER_ONLY:
+		err = process_cipher_data(sr);
+		break;
 	case NITROX_CHAIN_CIPHER_AUTH:
 	case NITROX_CHAIN_AUTH_CIPHER:
 		err = process_cipher_auth_data(sr);
-- 
2.20.1


^ permalink raw reply	[flat|nested] 17+ messages in thread

* Re: [dpdk-dev] [PATCH v2 2/3] crypto/nitrox: support AES-GCM
  2020-09-24 13:04       ` [dpdk-dev] [PATCH v2 2/3] crypto/nitrox: support AES-GCM Nagadheeraj Rottela
@ 2020-10-06 20:04         ` Akhil Goyal
  0 siblings, 0 replies; 17+ messages in thread
From: Akhil Goyal @ 2020-10-06 20:04 UTC (permalink / raw)
  To: Nagadheeraj Rottela; +Cc: dev, jsrikanth

Hi Nagadheeraj,

> Subject: [PATCH v2 2/3] crypto/nitrox: support AES-GCM
> 
> This patch adds AES-GCM AEAD algorithm.
> 
> Signed-off-by: Nagadheeraj Rottela <rnagadheeraj@marvell.com>
> ---

This patch is showing compilation error while compiling individually.

./drivers/crypto/nitrox/nitrox_sym_reqmgr.c
../drivers/crypto/nitrox/nitrox_sym_reqmgr.c: In function 'extract_combined_digest':
../drivers/crypto/nitrox/nitrox_sym_reqmgr.c:616:17: error: implicit declaration of function 'rte_pktmbuf_mtophys_offset'; did you mean 'rte_pktmbuf_mtod_offset'? [-Werror=implicit-function-declaration]
  digest->iova = rte_pktmbuf_mtophys_offset(mdst,
                 ^~~~~~~~~~~~~~~~~~~~~~~~~~
                 rte_pktmbuf_mtod_offset
../drivers/crypto/nitrox/nitrox_sym_reqmgr.c:616:17: error: nested extern declaration of 'rte_pktmbuf_mtophys_offset' [-Werror=nested-externs]
cc1: all warnings being treated as errors

^ permalink raw reply	[flat|nested] 17+ messages in thread

* Re: [dpdk-dev] [PATCH v2 1/3] test/crypto: replace NITROX PMD specific test suite
  2020-09-24 13:04       ` [dpdk-dev] [PATCH v2 1/3] test/crypto: replace NITROX PMD specific test suite Nagadheeraj Rottela
@ 2020-10-06 20:07         ` Akhil Goyal
  2020-10-09  5:57           ` [dpdk-dev] [PATCH v3 0/2] Add AES-GCM and cipher only offload support Nagadheeraj Rottela
  0 siblings, 1 reply; 17+ messages in thread
From: Akhil Goyal @ 2020-10-06 20:07 UTC (permalink / raw)
  To: Nagadheeraj Rottela; +Cc: dev, jsrikanth

> Subject: [PATCH v2 1/3] test/crypto: replace NITROX PMD specific test suite
> 
> Replace NITROX PMD specific tests with generic test suite.
> 
> Signed-off-by: Nagadheeraj Rottela <rnagadheeraj@marvell.com>
> ---
Applied this patch to dpdk-next-crypto as is it unrelated to rest of the patches.

Please fix compilation error in the other two patches of the series.

Thanks!


^ permalink raw reply	[flat|nested] 17+ messages in thread

* [dpdk-dev] [PATCH v3 0/2] Add AES-GCM and cipher only offload support
  2020-10-06 20:07         ` Akhil Goyal
@ 2020-10-09  5:57           ` Nagadheeraj Rottela
  2020-10-09  5:57             ` [dpdk-dev] [PATCH v3 1/2] crypto/nitrox: support AES-GCM Nagadheeraj Rottela
                               ` (2 more replies)
  0 siblings, 3 replies; 17+ messages in thread
From: Nagadheeraj Rottela @ 2020-10-09  5:57 UTC (permalink / raw)
  To: akhil.goyal; +Cc: dev, jsrikanth, Nagadheeraj Rottela

This patchset adds support for AES-GCM and cipher only offload.
---
v3:
* Fixed compilation error while compiling individual patches.

v2:
* Rebased patches to latest master and resolved merge conflict.
* Updated release notes.


Nagadheeraj Rottela (2):
  crypto/nitrox: support AES-GCM
  crypto/nitrox: support cipher only crypto operations

 doc/guides/cryptodevs/features/nitrox.ini     |   3 +
 doc/guides/cryptodevs/nitrox.rst              |   6 +-
 doc/guides/rel_notes/release_20_11.rst        |   5 +
 drivers/crypto/nitrox/nitrox_sym.c            |  85 ++++-
 .../crypto/nitrox/nitrox_sym_capabilities.c   |  30 ++
 drivers/crypto/nitrox/nitrox_sym_ctx.h        |   5 +-
 drivers/crypto/nitrox/nitrox_sym_reqmgr.c     | 357 ++++++++++++++----
 7 files changed, 409 insertions(+), 82 deletions(-)

-- 
2.20.1


^ permalink raw reply	[flat|nested] 17+ messages in thread

* [dpdk-dev] [PATCH v3 1/2] crypto/nitrox: support AES-GCM
  2020-10-09  5:57           ` [dpdk-dev] [PATCH v3 0/2] Add AES-GCM and cipher only offload support Nagadheeraj Rottela
@ 2020-10-09  5:57             ` Nagadheeraj Rottela
  2020-10-09  5:57             ` [dpdk-dev] [PATCH v3 2/2] crypto/nitrox: support cipher only crypto operations Nagadheeraj Rottela
  2020-10-09 12:13             ` [dpdk-dev] [PATCH v3 0/2] Add AES-GCM and cipher only offload support Akhil Goyal
  2 siblings, 0 replies; 17+ messages in thread
From: Nagadheeraj Rottela @ 2020-10-09  5:57 UTC (permalink / raw)
  To: akhil.goyal; +Cc: dev, jsrikanth, Nagadheeraj Rottela

This patch adds AES-GCM AEAD algorithm.

Signed-off-by: Nagadheeraj Rottela <rnagadheeraj@marvell.com>
---
 doc/guides/cryptodevs/features/nitrox.ini     |   3 +
 doc/guides/cryptodevs/nitrox.rst              |   4 +
 drivers/crypto/nitrox/nitrox_sym.c            |  82 +++++++-
 .../crypto/nitrox/nitrox_sym_capabilities.c   |  30 +++
 drivers/crypto/nitrox/nitrox_sym_ctx.h        |   5 +-
 drivers/crypto/nitrox/nitrox_sym_reqmgr.c     | 182 +++++++++++++++---
 6 files changed, 268 insertions(+), 38 deletions(-)

diff --git a/doc/guides/cryptodevs/features/nitrox.ini b/doc/guides/cryptodevs/features/nitrox.ini
index 183494731..a1d6bcb4f 100644
--- a/doc/guides/cryptodevs/features/nitrox.ini
+++ b/doc/guides/cryptodevs/features/nitrox.ini
@@ -34,6 +34,9 @@ SHA256 HMAC  = Y
 ; Supported AEAD algorithms of the 'nitrox' crypto driver.
 ;
 [AEAD]
+AES GCM (128)  = Y
+AES GCM (192)  = Y
+AES GCM (256)  = Y
 
 ;
 ; Supported Asymmetric algorithms of the 'nitrox' crypto driver.
diff --git a/doc/guides/cryptodevs/nitrox.rst b/doc/guides/cryptodevs/nitrox.rst
index 85f5212b6..91fca905a 100644
--- a/doc/guides/cryptodevs/nitrox.rst
+++ b/doc/guides/cryptodevs/nitrox.rst
@@ -26,6 +26,10 @@ Hash algorithms:
 * ``RTE_CRYPTO_AUTH_SHA224_HMAC``
 * ``RTE_CRYPTO_AUTH_SHA256_HMAC``
 
+Supported AEAD algorithms:
+
+* ``RTE_CRYPTO_AEAD_AES_GCM``
+
 Limitations
 -----------
 
diff --git a/drivers/crypto/nitrox/nitrox_sym.c b/drivers/crypto/nitrox/nitrox_sym.c
index fad4a7a48..fe3ee6e23 100644
--- a/drivers/crypto/nitrox/nitrox_sym.c
+++ b/drivers/crypto/nitrox/nitrox_sym.c
@@ -20,6 +20,7 @@
 #define NPS_PKT_IN_INSTR_SIZE 64
 #define IV_FROM_DPTR 1
 #define FLEXI_CRYPTO_ENCRYPT_HMAC 0x33
+#define FLEXI_CRYPTO_MAX_AAD_LEN 512
 #define AES_KEYSIZE_128 16
 #define AES_KEYSIZE_192 24
 #define AES_KEYSIZE_256 32
@@ -297,6 +298,9 @@ get_crypto_chain_order(const struct rte_crypto_sym_xform *xform)
 			}
 		}
 		break;
+	case RTE_CRYPTO_SYM_XFORM_AEAD:
+		res = NITROX_CHAIN_COMBINED;
+		break;
 	default:
 		break;
 	}
@@ -431,17 +435,17 @@ get_flexi_auth_type(enum rte_crypto_auth_algorithm algo)
 }
 
 static bool
-auth_key_digest_is_valid(struct rte_crypto_auth_xform *xform,
-			 struct flexi_crypto_context *fctx)
+auth_key_is_valid(const uint8_t *data, uint16_t length,
+		  struct flexi_crypto_context *fctx)
 {
-	if (unlikely(!xform->key.data && xform->key.length)) {
+	if (unlikely(!data && length)) {
 		NITROX_LOG(ERR, "Invalid auth key\n");
 		return false;
 	}
 
-	if (unlikely(xform->key.length > sizeof(fctx->auth.opad))) {
+	if (unlikely(length > sizeof(fctx->auth.opad))) {
 		NITROX_LOG(ERR, "Invalid auth key length %d\n",
-			   xform->key.length);
+			   length);
 		return false;
 	}
 
@@ -459,11 +463,10 @@ configure_auth_ctx(struct rte_crypto_auth_xform *xform,
 	if (unlikely(type == AUTH_INVALID))
 		return -ENOTSUP;
 
-	if (unlikely(!auth_key_digest_is_valid(xform, fctx)))
+	if (unlikely(!auth_key_is_valid(xform->key.data, xform->key.length,
+					fctx)))
 		return -EINVAL;
 
-	ctx->auth_op = xform->op;
-	ctx->auth_algo = xform->algo;
 	ctx->digest_length = xform->digest_length;
 
 	fctx->flags = rte_be_to_cpu_64(fctx->flags);
@@ -476,6 +479,56 @@ configure_auth_ctx(struct rte_crypto_auth_xform *xform,
 	return 0;
 }
 
+static int
+configure_aead_ctx(struct rte_crypto_aead_xform *xform,
+		   struct nitrox_crypto_ctx *ctx)
+{
+	int aes_keylen;
+	struct flexi_crypto_context *fctx = &ctx->fctx;
+
+	if (unlikely(xform->aad_length > FLEXI_CRYPTO_MAX_AAD_LEN)) {
+		NITROX_LOG(ERR, "AAD length %d not supported\n",
+			   xform->aad_length);
+		return -ENOTSUP;
+	}
+
+	if (unlikely(xform->algo != RTE_CRYPTO_AEAD_AES_GCM))
+		return -ENOTSUP;
+
+	aes_keylen = flexi_aes_keylen(xform->key.length, true);
+	if (unlikely(aes_keylen < 0))
+		return -EINVAL;
+
+	if (unlikely(!auth_key_is_valid(xform->key.data, xform->key.length,
+					fctx)))
+		return -EINVAL;
+
+	if (unlikely(xform->iv.length > MAX_IV_LEN))
+		return -EINVAL;
+
+	fctx->flags = rte_be_to_cpu_64(fctx->flags);
+	fctx->w0.cipher_type = CIPHER_AES_GCM;
+	fctx->w0.aes_keylen = aes_keylen;
+	fctx->w0.iv_source = IV_FROM_DPTR;
+	fctx->w0.hash_type = AUTH_NULL;
+	fctx->w0.auth_input_type = 1;
+	fctx->w0.mac_len = xform->digest_length;
+	fctx->flags = rte_cpu_to_be_64(fctx->flags);
+	memset(fctx->crypto.key, 0, sizeof(fctx->crypto.key));
+	memcpy(fctx->crypto.key, xform->key.data, xform->key.length);
+	memset(&fctx->auth, 0, sizeof(fctx->auth));
+	memcpy(fctx->auth.opad, xform->key.data, xform->key.length);
+
+	ctx->opcode = FLEXI_CRYPTO_ENCRYPT_HMAC;
+	ctx->req_op = (xform->op == RTE_CRYPTO_AEAD_OP_ENCRYPT) ?
+			NITROX_OP_ENCRYPT : NITROX_OP_DECRYPT;
+	ctx->iv.offset = xform->iv.offset;
+	ctx->iv.length = xform->iv.length;
+	ctx->digest_length = xform->digest_length;
+	ctx->aad_length = xform->aad_length;
+	return 0;
+}
+
 static int
 nitrox_sym_dev_sess_configure(struct rte_cryptodev *cdev,
 			      struct rte_crypto_sym_xform *xform,
@@ -486,6 +539,8 @@ nitrox_sym_dev_sess_configure(struct rte_cryptodev *cdev,
 	struct nitrox_crypto_ctx *ctx;
 	struct rte_crypto_cipher_xform *cipher_xform = NULL;
 	struct rte_crypto_auth_xform *auth_xform = NULL;
+	struct rte_crypto_aead_xform *aead_xform = NULL;
+	int ret = -EINVAL;
 
 	if (rte_mempool_get(mempool, &mp_obj)) {
 		NITROX_LOG(ERR, "Couldn't allocate context\n");
@@ -503,8 +558,12 @@ nitrox_sym_dev_sess_configure(struct rte_cryptodev *cdev,
 		auth_xform = &xform->auth;
 		cipher_xform = &xform->next->cipher;
 		break;
+	case NITROX_CHAIN_COMBINED:
+		aead_xform = &xform->aead;
+		break;
 	default:
 		NITROX_LOG(ERR, "Crypto chain not supported\n");
+		ret = -ENOTSUP;
 		goto err;
 	}
 
@@ -518,12 +577,17 @@ nitrox_sym_dev_sess_configure(struct rte_cryptodev *cdev,
 		goto err;
 	}
 
+	if (aead_xform && unlikely(configure_aead_ctx(aead_xform, ctx))) {
+		NITROX_LOG(ERR, "Failed to configure aead ctx\n");
+		goto err;
+	}
+
 	ctx->iova = rte_mempool_virt2iova(ctx);
 	set_sym_session_private_data(sess, cdev->driver_id, ctx);
 	return 0;
 err:
 	rte_mempool_put(mempool, mp_obj);
-	return -EINVAL;
+	return ret;
 }
 
 static void
diff --git a/drivers/crypto/nitrox/nitrox_sym_capabilities.c b/drivers/crypto/nitrox/nitrox_sym_capabilities.c
index dc4df9185..a30cd9f8f 100644
--- a/drivers/crypto/nitrox/nitrox_sym_capabilities.c
+++ b/drivers/crypto/nitrox/nitrox_sym_capabilities.c
@@ -108,6 +108,36 @@ static const struct rte_cryptodev_capabilities nitrox_capabilities[] = {
 			}, }
 		}, }
 	},
+	{	/* AES GCM */
+		.op = RTE_CRYPTO_OP_TYPE_SYMMETRIC,
+		{.sym = {
+			.xform_type = RTE_CRYPTO_SYM_XFORM_AEAD,
+			{.aead = {
+				.algo = RTE_CRYPTO_AEAD_AES_GCM,
+				.block_size = 16,
+				.key_size = {
+					.min = 16,
+					.max = 32,
+					.increment = 8
+				},
+				.digest_size = {
+					.min = 1,
+					.max = 16,
+					.increment = 1
+				},
+				.aad_size = {
+					.min = 0,
+					.max = 512,
+					.increment = 1
+				},
+				.iv_size = {
+					.min = 12,
+					.max = 16,
+					.increment = 4
+				},
+			}, }
+		}, }
+	},
 	RTE_CRYPTODEV_END_OF_CAPABILITIES_LIST()
 };
 
diff --git a/drivers/crypto/nitrox/nitrox_sym_ctx.h b/drivers/crypto/nitrox/nitrox_sym_ctx.h
index 2985e519f..deb00fc1e 100644
--- a/drivers/crypto/nitrox/nitrox_sym_ctx.h
+++ b/drivers/crypto/nitrox/nitrox_sym_ctx.h
@@ -11,6 +11,7 @@
 
 #define AES_MAX_KEY_SIZE 32
 #define AES_BLOCK_SIZE 16
+#define AES_GCM_SALT_SIZE 4
 
 enum nitrox_chain {
 	NITROX_CHAIN_CIPHER_ONLY,
@@ -69,14 +70,14 @@ struct flexi_crypto_context {
 struct nitrox_crypto_ctx {
 	struct flexi_crypto_context fctx;
 	enum nitrox_chain nitrox_chain;
-	enum rte_crypto_auth_operation auth_op;
-	enum rte_crypto_auth_algorithm auth_algo;
 	struct {
 		uint16_t offset;
 		uint16_t length;
 	} iv;
 	rte_iova_t iova;
+	uint8_t salt[AES_GCM_SALT_SIZE];
 	uint16_t digest_length;
+	uint16_t aad_length;
 	uint8_t opcode;
 	uint8_t req_op;
 };
diff --git a/drivers/crypto/nitrox/nitrox_sym_reqmgr.c b/drivers/crypto/nitrox/nitrox_sym_reqmgr.c
index 449224780..113ce5d11 100644
--- a/drivers/crypto/nitrox/nitrox_sym_reqmgr.c
+++ b/drivers/crypto/nitrox/nitrox_sym_reqmgr.c
@@ -238,12 +238,13 @@ create_se_instr(struct nitrox_softreq *sr, uint8_t qno)
 }
 
 static void
-softreq_copy_iv(struct nitrox_softreq *sr)
+softreq_copy_iv(struct nitrox_softreq *sr, uint8_t salt_size)
 {
-	sr->iv.virt = rte_crypto_op_ctod_offset(sr->op, uint8_t *,
-						sr->ctx->iv.offset);
-	sr->iv.iova = rte_crypto_op_ctophys_offset(sr->op, sr->ctx->iv.offset);
-	sr->iv.len = sr->ctx->iv.length;
+	uint16_t offset = sr->ctx->iv.offset + salt_size;
+
+	sr->iv.virt = rte_crypto_op_ctod_offset(sr->op, uint8_t *, offset);
+	sr->iv.iova = rte_crypto_op_ctophys_offset(sr->op, offset);
+	sr->iv.len = sr->ctx->iv.length - salt_size;
 }
 
 static int
@@ -254,7 +255,7 @@ extract_cipher_auth_digest(struct nitrox_softreq *sr,
 	struct rte_mbuf *mdst = op->sym->m_dst ? op->sym->m_dst :
 					op->sym->m_src;
 
-	if (sr->ctx->auth_op == RTE_CRYPTO_AUTH_OP_VERIFY &&
+	if (sr->ctx->req_op == NITROX_OP_DECRYPT &&
 	    unlikely(!op->sym->auth.digest.data))
 		return -EINVAL;
 
@@ -352,6 +353,13 @@ create_cipher_auth_sglist(struct nitrox_softreq *sr,
 	if (unlikely(auth_only_len < 0))
 		return -EINVAL;
 
+	if (unlikely(
+		op->sym->cipher.data.offset + op->sym->cipher.data.length !=
+		op->sym->auth.data.offset + op->sym->auth.data.length)) {
+		NITROX_LOG(ERR, "Auth only data after cipher data not supported\n");
+		return -ENOTSUP;
+	}
+
 	err = create_sglist_from_mbuf(sgtbl, mbuf, op->sym->auth.data.offset,
 				      auth_only_len);
 	if (unlikely(err))
@@ -365,6 +373,41 @@ create_cipher_auth_sglist(struct nitrox_softreq *sr,
 	return 0;
 }
 
+static int
+create_combined_sglist(struct nitrox_softreq *sr, struct nitrox_sgtable *sgtbl,
+		       struct rte_mbuf *mbuf)
+{
+	struct rte_crypto_op *op = sr->op;
+
+	fill_sglist(sgtbl, sr->iv.len, sr->iv.iova, sr->iv.virt);
+	fill_sglist(sgtbl, sr->ctx->aad_length, op->sym->aead.aad.phys_addr,
+		    op->sym->aead.aad.data);
+	return create_sglist_from_mbuf(sgtbl, mbuf, op->sym->cipher.data.offset,
+				       op->sym->cipher.data.length);
+}
+
+static int
+create_aead_sglist(struct nitrox_softreq *sr, struct nitrox_sgtable *sgtbl,
+		   struct rte_mbuf *mbuf)
+{
+	int err;
+
+	switch (sr->ctx->nitrox_chain) {
+	case NITROX_CHAIN_CIPHER_AUTH:
+	case NITROX_CHAIN_AUTH_CIPHER:
+		err = create_cipher_auth_sglist(sr, sgtbl, mbuf);
+		break;
+	case NITROX_CHAIN_COMBINED:
+		err = create_combined_sglist(sr, sgtbl, mbuf);
+		break;
+	default:
+		err = -EINVAL;
+		break;
+	}
+
+	return err;
+}
+
 static void
 create_sgcomp(struct nitrox_sgtable *sgtbl)
 {
@@ -383,17 +426,16 @@ create_sgcomp(struct nitrox_sgtable *sgtbl)
 }
 
 static int
-create_cipher_auth_inbuf(struct nitrox_softreq *sr,
-			 struct nitrox_sglist *digest)
+create_aead_inbuf(struct nitrox_softreq *sr, struct nitrox_sglist *digest)
 {
 	int err;
 	struct nitrox_crypto_ctx *ctx = sr->ctx;
 
-	err = create_cipher_auth_sglist(sr, &sr->in, sr->op->sym->m_src);
+	err = create_aead_sglist(sr, &sr->in, sr->op->sym->m_src);
 	if (unlikely(err))
 		return err;
 
-	if (ctx->auth_op == RTE_CRYPTO_AUTH_OP_VERIFY)
+	if (ctx->req_op == NITROX_OP_DECRYPT)
 		fill_sglist(&sr->in, digest->len, digest->iova, digest->virt);
 
 	create_sgcomp(&sr->in);
@@ -402,25 +444,24 @@ create_cipher_auth_inbuf(struct nitrox_softreq *sr,
 }
 
 static int
-create_cipher_auth_oop_outbuf(struct nitrox_softreq *sr,
-			      struct nitrox_sglist *digest)
+create_aead_oop_outbuf(struct nitrox_softreq *sr, struct nitrox_sglist *digest)
 {
 	int err;
 	struct nitrox_crypto_ctx *ctx = sr->ctx;
 
-	err = create_cipher_auth_sglist(sr, &sr->out, sr->op->sym->m_dst);
+	err = create_aead_sglist(sr, &sr->out, sr->op->sym->m_dst);
 	if (unlikely(err))
 		return err;
 
-	if (ctx->auth_op == RTE_CRYPTO_AUTH_OP_GENERATE)
+	if (ctx->req_op == NITROX_OP_ENCRYPT)
 		fill_sglist(&sr->out, digest->len, digest->iova, digest->virt);
 
 	return 0;
 }
 
 static void
-create_cipher_auth_inplace_outbuf(struct nitrox_softreq *sr,
-				  struct nitrox_sglist *digest)
+create_aead_inplace_outbuf(struct nitrox_softreq *sr,
+			   struct nitrox_sglist *digest)
 {
 	int i, cnt;
 	struct nitrox_crypto_ctx *ctx = sr->ctx;
@@ -433,17 +474,16 @@ create_cipher_auth_inplace_outbuf(struct nitrox_softreq *sr,
 	}
 
 	sr->out.map_bufs_cnt = cnt;
-	if (ctx->auth_op == RTE_CRYPTO_AUTH_OP_GENERATE) {
+	if (ctx->req_op == NITROX_OP_ENCRYPT) {
 		fill_sglist(&sr->out, digest->len, digest->iova,
 			    digest->virt);
-	} else if (ctx->auth_op == RTE_CRYPTO_AUTH_OP_VERIFY) {
+	} else if (ctx->req_op == NITROX_OP_DECRYPT) {
 		sr->out.map_bufs_cnt--;
 	}
 }
 
 static int
-create_cipher_auth_outbuf(struct nitrox_softreq *sr,
-			  struct nitrox_sglist *digest)
+create_aead_outbuf(struct nitrox_softreq *sr, struct nitrox_sglist *digest)
 {
 	struct rte_crypto_op *op = sr->op;
 	int cnt = 0;
@@ -458,11 +498,11 @@ create_cipher_auth_outbuf(struct nitrox_softreq *sr,
 	if (op->sym->m_dst) {
 		int err;
 
-		err = create_cipher_auth_oop_outbuf(sr, digest);
+		err = create_aead_oop_outbuf(sr, digest);
 		if (unlikely(err))
 			return err;
 	} else {
-		create_cipher_auth_inplace_outbuf(sr, digest);
+		create_aead_inplace_outbuf(sr, digest);
 	}
 
 	cnt = sr->out.map_bufs_cnt;
@@ -516,16 +556,16 @@ process_cipher_auth_data(struct nitrox_softreq *sr)
 	int err;
 	struct nitrox_sglist digest;
 
-	softreq_copy_iv(sr);
+	softreq_copy_iv(sr, 0);
 	err = extract_cipher_auth_digest(sr, &digest);
 	if (unlikely(err))
 		return err;
 
-	err = create_cipher_auth_inbuf(sr, &digest);
+	err = create_aead_inbuf(sr, &digest);
 	if (unlikely(err))
 		return err;
 
-	err = create_cipher_auth_outbuf(sr, &digest);
+	err = create_aead_outbuf(sr, &digest);
 	if (unlikely(err))
 		return err;
 
@@ -534,6 +574,86 @@ process_cipher_auth_data(struct nitrox_softreq *sr)
 	return 0;
 }
 
+static int
+softreq_copy_salt(struct nitrox_softreq *sr)
+{
+	struct nitrox_crypto_ctx *ctx = sr->ctx;
+	uint8_t *addr;
+
+	if (unlikely(ctx->iv.length < AES_GCM_SALT_SIZE)) {
+		NITROX_LOG(ERR, "Invalid IV length %d\n", ctx->iv.length);
+		return -EINVAL;
+	}
+
+	addr = rte_crypto_op_ctod_offset(sr->op, uint8_t *, ctx->iv.offset);
+	if (!memcmp(ctx->salt, addr, AES_GCM_SALT_SIZE))
+		return 0;
+
+	memcpy(ctx->salt, addr, AES_GCM_SALT_SIZE);
+	memcpy(ctx->fctx.crypto.iv, addr, AES_GCM_SALT_SIZE);
+	return 0;
+}
+
+static int
+extract_combined_digest(struct nitrox_softreq *sr, struct nitrox_sglist *digest)
+{
+	struct rte_crypto_op *op = sr->op;
+	struct rte_mbuf *mdst = op->sym->m_dst ? op->sym->m_dst :
+		op->sym->m_src;
+
+	digest->len = sr->ctx->digest_length;
+	if (op->sym->aead.digest.data) {
+		digest->iova = op->sym->aead.digest.phys_addr;
+		digest->virt = op->sym->aead.digest.data;
+
+		return 0;
+	}
+
+	if (unlikely(rte_pktmbuf_data_len(mdst) < op->sym->aead.data.offset +
+	       op->sym->aead.data.length + digest->len))
+		return -EINVAL;
+
+	digest->iova = rte_pktmbuf_iova_offset(mdst,
+					op->sym->aead.data.offset +
+					op->sym->aead.data.length);
+	digest->virt = rte_pktmbuf_mtod_offset(mdst, uint8_t *,
+					op->sym->aead.data.offset +
+					op->sym->aead.data.length);
+
+	return 0;
+}
+
+static int
+process_combined_data(struct nitrox_softreq *sr)
+{
+	int err;
+	struct nitrox_sglist digest;
+	struct rte_crypto_op *op = sr->op;
+
+	err = softreq_copy_salt(sr);
+	if (unlikely(err))
+		return err;
+
+	softreq_copy_iv(sr, AES_GCM_SALT_SIZE);
+	err = extract_combined_digest(sr, &digest);
+	if (unlikely(err))
+		return err;
+
+	err = create_aead_inbuf(sr, &digest);
+	if (unlikely(err))
+		return err;
+
+	err = create_aead_outbuf(sr, &digest);
+	if (unlikely(err))
+		return err;
+
+	create_aead_gph(op->sym->aead.data.length, sr->iv.len,
+			op->sym->aead.data.length + sr->ctx->aad_length,
+			&sr->gph);
+
+	return 0;
+}
+
 static int
 process_softreq(struct nitrox_softreq *sr)
 {
@@ -545,6 +665,9 @@ process_softreq(struct nitrox_softreq *sr)
 	case NITROX_CHAIN_AUTH_CIPHER:
 		err = process_cipher_auth_data(sr);
 		break;
+	case NITROX_CHAIN_COMBINED:
+		err = process_combined_data(sr);
+		break;
 	default:
 		err = -EINVAL;
 		break;
@@ -558,10 +681,15 @@ nitrox_process_se_req(uint16_t qno, struct rte_crypto_op *op,
 		      struct nitrox_crypto_ctx *ctx,
 		      struct nitrox_softreq *sr)
 {
+	int err;
+
 	softreq_init(sr, sr->iova);
 	sr->ctx = ctx;
 	sr->op = op;
-	process_softreq(sr);
+	err = process_softreq(sr);
+	if (unlikely(err))
+		return err;
+
 	create_se_instr(sr, qno);
 	sr->timeout = rte_get_timer_cycles() + CMD_TIMEOUT * rte_get_timer_hz();
 	return 0;
@@ -577,7 +705,7 @@ nitrox_check_se_req(struct nitrox_softreq *sr, struct rte_crypto_op **op)
 	cc = *(volatile uint64_t *)(&sr->resp.completion);
 	orh = *(volatile uint64_t *)(&sr->resp.orh);
 	if (cc != PENDING_SIG)
-		err = 0;
+		err = orh & 0xff;
 	else if ((orh != PENDING_SIG) && (orh & 0xff))
 		err = orh & 0xff;
 	else if (rte_get_timer_cycles() >= sr->timeout)
-- 
2.20.1


^ permalink raw reply	[flat|nested] 17+ messages in thread

* [dpdk-dev] [PATCH v3 2/2] crypto/nitrox: support cipher only crypto operations
  2020-10-09  5:57           ` [dpdk-dev] [PATCH v3 0/2] Add AES-GCM and cipher only offload support Nagadheeraj Rottela
  2020-10-09  5:57             ` [dpdk-dev] [PATCH v3 1/2] crypto/nitrox: support AES-GCM Nagadheeraj Rottela
@ 2020-10-09  5:57             ` Nagadheeraj Rottela
  2020-10-09 12:13             ` [dpdk-dev] [PATCH v3 0/2] Add AES-GCM and cipher only offload support Akhil Goyal
  2 siblings, 0 replies; 17+ messages in thread
From: Nagadheeraj Rottela @ 2020-10-09  5:57 UTC (permalink / raw)
  To: akhil.goyal; +Cc: dev, jsrikanth, Nagadheeraj Rottela

This patch adds cipher only crypto operation support.

Signed-off-by: Nagadheeraj Rottela <rnagadheeraj@marvell.com>
---
 doc/guides/cryptodevs/nitrox.rst          |   2 -
 doc/guides/rel_notes/release_20_11.rst    |   5 +
 drivers/crypto/nitrox/nitrox_sym.c        |   3 +
 drivers/crypto/nitrox/nitrox_sym_reqmgr.c | 189 ++++++++++++++++------
 4 files changed, 148 insertions(+), 51 deletions(-)

diff --git a/doc/guides/cryptodevs/nitrox.rst b/doc/guides/cryptodevs/nitrox.rst
index 91fca905a..095e545c6 100644
--- a/doc/guides/cryptodevs/nitrox.rst
+++ b/doc/guides/cryptodevs/nitrox.rst
@@ -33,8 +33,6 @@ Supported AEAD algorithms:
 Limitations
 -----------
 
-* AES_CBC Cipher Only combination is not supported.
-* 3DES Cipher Only combination is not supported.
 * Session-less APIs are not supported.
 
 Installation
diff --git a/doc/guides/rel_notes/release_20_11.rst b/doc/guides/rel_notes/release_20_11.rst
index 73ac08fb0..ddcf90356 100644
--- a/doc/guides/rel_notes/release_20_11.rst
+++ b/doc/guides/rel_notes/release_20_11.rst
@@ -55,6 +55,11 @@ New Features
      Also, make sure to start the actual text at the margin.
      =======================================================
 
+* **Updated Marvell NITROX symmetric crypto PMD.**
+
+  * Added cipher only offload support.
+  * Added AES-GCM support.
+
 
 Removed Items
 -------------
diff --git a/drivers/crypto/nitrox/nitrox_sym.c b/drivers/crypto/nitrox/nitrox_sym.c
index fe3ee6e23..2768bdd2e 100644
--- a/drivers/crypto/nitrox/nitrox_sym.c
+++ b/drivers/crypto/nitrox/nitrox_sym.c
@@ -550,6 +550,9 @@ nitrox_sym_dev_sess_configure(struct rte_cryptodev *cdev,
 	ctx = mp_obj;
 	ctx->nitrox_chain = get_crypto_chain_order(xform);
 	switch (ctx->nitrox_chain) {
+	case NITROX_CHAIN_CIPHER_ONLY:
+		cipher_xform = &xform->cipher;
+		break;
 	case NITROX_CHAIN_CIPHER_AUTH:
 		cipher_xform = &xform->cipher;
 		auth_xform = &xform->next->auth;
diff --git a/drivers/crypto/nitrox/nitrox_sym_reqmgr.c b/drivers/crypto/nitrox/nitrox_sym_reqmgr.c
index 113ce5d11..fe3ca25a0 100644
--- a/drivers/crypto/nitrox/nitrox_sym_reqmgr.c
+++ b/drivers/crypto/nitrox/nitrox_sym_reqmgr.c
@@ -247,38 +247,6 @@ softreq_copy_iv(struct nitrox_softreq *sr, uint8_t salt_size)
 	sr->iv.len = sr->ctx->iv.length - salt_size;
 }
 
-static int
-extract_cipher_auth_digest(struct nitrox_softreq *sr,
-			   struct nitrox_sglist *digest)
-{
-	struct rte_crypto_op *op = sr->op;
-	struct rte_mbuf *mdst = op->sym->m_dst ? op->sym->m_dst :
-					op->sym->m_src;
-
-	if (sr->ctx->req_op == NITROX_OP_DECRYPT &&
-	    unlikely(!op->sym->auth.digest.data))
-		return -EINVAL;
-
-	digest->len = sr->ctx->digest_length;
-	if (op->sym->auth.digest.data) {
-		digest->iova = op->sym->auth.digest.phys_addr;
-		digest->virt = op->sym->auth.digest.data;
-		return 0;
-	}
-
-	if (unlikely(rte_pktmbuf_data_len(mdst) < op->sym->auth.data.offset +
-	       op->sym->auth.data.length + digest->len))
-		return -EINVAL;
-
-	digest->iova = rte_pktmbuf_iova_offset(mdst,
-					op->sym->auth.data.offset +
-					op->sym->auth.data.length);
-	digest->virt = rte_pktmbuf_mtod_offset(mdst, uint8_t *,
-					op->sym->auth.data.offset +
-					op->sym->auth.data.length);
-	return 0;
-}
-
 static void
 fill_sglist(struct nitrox_sgtable *sgtbl, uint16_t len, rte_iova_t iova,
 	    void *virt)
@@ -340,6 +308,143 @@ create_sglist_from_mbuf(struct nitrox_sgtable *sgtbl, struct rte_mbuf *mbuf,
 	return 0;
 }
 
+static void
+create_sgcomp(struct nitrox_sgtable *sgtbl)
+{
+	int i, j, nr_sgcomp;
+	struct nitrox_sgcomp *sgcomp = sgtbl->sgcomp;
+	struct nitrox_sglist *sglist = sgtbl->sglist;
+
+	nr_sgcomp = RTE_ALIGN_MUL_CEIL(sgtbl->map_bufs_cnt, 4) / 4;
+	sgtbl->nr_sgcomp = nr_sgcomp;
+	for (i = 0; i < nr_sgcomp; i++, sgcomp++) {
+		for (j = 0; j < 4; j++, sglist++) {
+			sgcomp->len[j] = rte_cpu_to_be_16(sglist->len);
+			sgcomp->iova[j] = rte_cpu_to_be_64(sglist->iova);
+		}
+	}
+}
+
+static int
+create_cipher_inbuf(struct nitrox_softreq *sr)
+{
+	int err;
+	struct rte_crypto_op *op = sr->op;
+
+	fill_sglist(&sr->in, sr->iv.len, sr->iv.iova, sr->iv.virt);
+	err = create_sglist_from_mbuf(&sr->in, op->sym->m_src,
+				      op->sym->cipher.data.offset,
+				      op->sym->cipher.data.length);
+	if (unlikely(err))
+		return err;
+
+	create_sgcomp(&sr->in);
+	sr->dptr = sr->iova + offsetof(struct nitrox_softreq, in.sgcomp);
+
+	return 0;
+}
+
+static int
+create_cipher_outbuf(struct nitrox_softreq *sr)
+{
+	struct rte_crypto_op *op = sr->op;
+	int err, cnt = 0;
+	struct rte_mbuf *m_dst = op->sym->m_dst ? op->sym->m_dst :
+		op->sym->m_src;
+
+	sr->resp.orh = PENDING_SIG;
+	sr->out.sglist[cnt].len = sizeof(sr->resp.orh);
+	sr->out.sglist[cnt].iova = sr->iova + offsetof(struct nitrox_softreq,
+						       resp.orh);
+	sr->out.sglist[cnt].virt = &sr->resp.orh;
+	cnt++;
+
+	sr->out.map_bufs_cnt = cnt;
+	fill_sglist(&sr->out, sr->iv.len, sr->iv.iova, sr->iv.virt);
+	err = create_sglist_from_mbuf(&sr->out, m_dst,
+				      op->sym->cipher.data.offset,
+				      op->sym->cipher.data.length);
+	if (unlikely(err))
+		return err;
+
+	cnt = sr->out.map_bufs_cnt;
+	sr->resp.completion = PENDING_SIG;
+	sr->out.sglist[cnt].len = sizeof(sr->resp.completion);
+	sr->out.sglist[cnt].iova = sr->iova + offsetof(struct nitrox_softreq,
+						     resp.completion);
+	sr->out.sglist[cnt].virt = &sr->resp.completion;
+	cnt++;
+
+	RTE_VERIFY(cnt <= MAX_SGBUF_CNT);
+	sr->out.map_bufs_cnt = cnt;
+
+	create_sgcomp(&sr->out);
+	sr->rptr = sr->iova + offsetof(struct nitrox_softreq, out.sgcomp);
+
+	return 0;
+}
+
+static void
+create_cipher_gph(uint32_t cryptlen, uint16_t ivlen, struct gphdr *gph)
+{
+	gph->param0 = rte_cpu_to_be_16(cryptlen);
+	gph->param1 = 0;
+	gph->param2 = rte_cpu_to_be_16(ivlen);
+	gph->param3 = 0;
+}
+
+static int
+process_cipher_data(struct nitrox_softreq *sr)
+{
+	struct rte_crypto_op *op = sr->op;
+	int err;
+
+	softreq_copy_iv(sr, 0);
+	err = create_cipher_inbuf(sr);
+	if (unlikely(err))
+		return err;
+
+	err = create_cipher_outbuf(sr);
+	if (unlikely(err))
+		return err;
+
+	create_cipher_gph(op->sym->cipher.data.length, sr->iv.len, &sr->gph);
+
+	return 0;
+}
+
+static int
+extract_cipher_auth_digest(struct nitrox_softreq *sr,
+			   struct nitrox_sglist *digest)
+{
+	struct rte_crypto_op *op = sr->op;
+	struct rte_mbuf *mdst = op->sym->m_dst ? op->sym->m_dst :
+					op->sym->m_src;
+
+	if (sr->ctx->req_op == NITROX_OP_DECRYPT &&
+	    unlikely(!op->sym->auth.digest.data))
+		return -EINVAL;
+
+	digest->len = sr->ctx->digest_length;
+	if (op->sym->auth.digest.data) {
+		digest->iova = op->sym->auth.digest.phys_addr;
+		digest->virt = op->sym->auth.digest.data;
+		return 0;
+	}
+
+	if (unlikely(rte_pktmbuf_data_len(mdst) < op->sym->auth.data.offset +
+	       op->sym->auth.data.length + digest->len))
+		return -EINVAL;
+
+	digest->iova = rte_pktmbuf_iova_offset(mdst,
+					op->sym->auth.data.offset +
+					op->sym->auth.data.length);
+	digest->virt = rte_pktmbuf_mtod_offset(mdst, uint8_t *,
+					op->sym->auth.data.offset +
+					op->sym->auth.data.length);
+	return 0;
+}
+
 static int
 create_cipher_auth_sglist(struct nitrox_softreq *sr,
 			  struct nitrox_sgtable *sgtbl, struct rte_mbuf *mbuf)
@@ -408,23 +513,6 @@ create_aead_sglist(struct nitrox_softreq *sr, struct nitrox_sgtable *sgtbl,
 	return err;
 }
 
-static void
-create_sgcomp(struct nitrox_sgtable *sgtbl)
-{
-	int i, j, nr_sgcomp;
-	struct nitrox_sgcomp *sgcomp = sgtbl->sgcomp;
-	struct nitrox_sglist *sglist = sgtbl->sglist;
-
-	nr_sgcomp = RTE_ALIGN_MUL_CEIL(sgtbl->map_bufs_cnt, 4) / 4;
-	sgtbl->nr_sgcomp = nr_sgcomp;
-	for (i = 0; i < nr_sgcomp; i++, sgcomp++) {
-		for (j = 0; j < 4; j++, sglist++) {
-			sgcomp->len[j] = rte_cpu_to_be_16(sglist->len);
-			sgcomp->iova[j] = rte_cpu_to_be_64(sglist->iova);
-		}
-	}
-}
-
 static int
 create_aead_inbuf(struct nitrox_softreq *sr, struct nitrox_sglist *digest)
 {
@@ -661,6 +749,9 @@ process_softreq(struct nitrox_softreq *sr)
 	int err = 0;
 
 	switch (ctx->nitrox_chain) {
+	case NITROX_CHAIN_CIPHER_ONLY:
+		err = process_cipher_data(sr);
+		break;
 	case NITROX_CHAIN_CIPHER_AUTH:
 	case NITROX_CHAIN_AUTH_CIPHER:
 		err = process_cipher_auth_data(sr);
-- 
2.20.1


^ permalink raw reply	[flat|nested] 17+ messages in thread

* Re: [dpdk-dev] [PATCH v3 0/2] Add AES-GCM and cipher only offload support
  2020-10-09  5:57           ` [dpdk-dev] [PATCH v3 0/2] Add AES-GCM and cipher only offload support Nagadheeraj Rottela
  2020-10-09  5:57             ` [dpdk-dev] [PATCH v3 1/2] crypto/nitrox: support AES-GCM Nagadheeraj Rottela
  2020-10-09  5:57             ` [dpdk-dev] [PATCH v3 2/2] crypto/nitrox: support cipher only crypto operations Nagadheeraj Rottela
@ 2020-10-09 12:13             ` Akhil Goyal
  2 siblings, 0 replies; 17+ messages in thread
From: Akhil Goyal @ 2020-10-09 12:13 UTC (permalink / raw)
  To: Nagadheeraj Rottela; +Cc: dev, jsrikanth

> This patchset adds support for AES-GCM and cipher only offload.
> ---
> v3:
> * Fixed compilation error while compiling individual patches.
> 
> v2:
> * Rebased patches to latest master and resolved merge conflict.
> * Updated release notes.
> 

Release notes should be split across the patches. I forgot to comment on prev version.

Fixed it while applying the patches.

Applied to dpdk-next-crypto

Thanks.

^ permalink raw reply	[flat|nested] 17+ messages in thread

end of thread, other threads:[~2020-10-09 12:14 UTC | newest]

Thread overview: 17+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2020-07-24 11:00 [dpdk-dev] [PATCH 0/3] Add AES-GCM and cipher only offload support Nagadheeraj Rottela
2020-07-24 11:00 ` [dpdk-dev] [PATCH 1/3] test/crypto: replace NITROX PMD specific test suite Nagadheeraj Rottela
2020-07-24 11:00 ` [dpdk-dev] [PATCH 2/3] crypto/nitrox: support AES-GCM Nagadheeraj Rottela
2020-07-24 11:00 ` [dpdk-dev] [PATCH 3/3] crypto/nitrox: support cipher only crypto operations Nagadheeraj Rottela
2020-09-22 19:11   ` Akhil Goyal
2020-09-24 13:04     ` [dpdk-dev] [PATCH v2 0/3] Add AES-GCM and cipher only offload support Nagadheeraj Rottela
2020-09-24 13:04       ` [dpdk-dev] [PATCH v2 1/3] test/crypto: replace NITROX PMD specific test suite Nagadheeraj Rottela
2020-10-06 20:07         ` Akhil Goyal
2020-10-09  5:57           ` [dpdk-dev] [PATCH v3 0/2] Add AES-GCM and cipher only offload support Nagadheeraj Rottela
2020-10-09  5:57             ` [dpdk-dev] [PATCH v3 1/2] crypto/nitrox: support AES-GCM Nagadheeraj Rottela
2020-10-09  5:57             ` [dpdk-dev] [PATCH v3 2/2] crypto/nitrox: support cipher only crypto operations Nagadheeraj Rottela
2020-10-09 12:13             ` [dpdk-dev] [PATCH v3 0/2] Add AES-GCM and cipher only offload support Akhil Goyal
2020-09-24 13:04       ` [dpdk-dev] [PATCH v2 2/3] crypto/nitrox: support AES-GCM Nagadheeraj Rottela
2020-10-06 20:04         ` Akhil Goyal
2020-09-24 13:04       ` [dpdk-dev] [PATCH v2 3/3] crypto/nitrox: support cipher only crypto operations Nagadheeraj Rottela
2020-07-26 18:57 ` [dpdk-dev] [PATCH 0/3] Add AES-GCM and cipher only offload support Akhil Goyal
2020-07-28  6:41   ` Nagadheeraj Rottela

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).