[PATCH 00/25] New features and improvements in cnxk crypto PMD

[PATCH 00/25] New features and improvements in cnxk crypto PMD

[Anoob Joseph]

New features and fixes to cnxk crypto PMDs
- Support for more algorithms in lookaside crypto & protocol
- Support for copy & set DF bit
- Support for CPT CTX update
- Support for security session stats in cn10k

Ankur Dwivedi (1):
  crypto/cnxk: add security session stats get

Anoob Joseph (17):
  common/cnxk: define minor opcodes for MISC opcode
  common/cnxk: add aes-xcbc key derive
  common/cnxk: fix reset of fields
  common/cnxk: verify input args
  crypto/cnxk: clear session data before populating
  crypto/cnxk: update max sec crypto caps
  crypto/cnxk: account for CPT CTX updates and flush delays
  crypto/cnxk: use struct sizes for ctx writes
  crypto/cnxk: add skip for unsupported cases
  crypto/cnxk: handle null chained ops
  crypto/cnxk: fix inflight cnt calculation
  crypto/cnxk: use atomics to access cpt res
  crypto/cnxk: add more info on command timeout
  crypto/cnxk: fix extend tail calculation
  crypto/cnxk: add aes xcbc and null cipher
  crypto/cnxk: add copy and set DF
  crypto/cnxk: add aes cmac

Archana Muniganti (1):
  common/cnxk: add bit fields for params

Shijith Thotton (1):
  crypto/cnxk: only enable queues that are allocated

Tejasree Kondoj (5):
  crypto/cnxk: add lookaside IPsec AES-CBC-HMAC-SHA256 support
  crypto/cnxk: write CPT CTX through microcode op
  crypto/cnxk: support cnxk lookaside IPsec HMAC-SHA384/512
  crypto/cnxk: add context reload for IV
  crypto/cnxk: support lookaside IPsec AES-CTR

 doc/guides/cryptodevs/cnxk.rst                    |  50 ++++-
 doc/guides/cryptodevs/features/cn10k.ini          |  37 ++--
 doc/guides/cryptodevs/features/cn9k.ini           |  37 ++--
 doc/guides/rel_notes/release_22_03.rst            |  10 +
 drivers/common/cnxk/cnxk_security.c               |  92 ++++++--
 drivers/common/cnxk/hw/cpt.h                      |  15 ++
 drivers/common/cnxk/meson.build                   |   1 +
 drivers/common/cnxk/roc_aes.c                     | 208 ++++++++++++++++++
 drivers/common/cnxk/roc_aes.h                     |  14 ++
 drivers/common/cnxk/roc_api.h                     |   3 +
 drivers/common/cnxk/roc_cpt.c                     |   4 +-
 drivers/common/cnxk/roc_cpt.h                     |  24 +--
 drivers/common/cnxk/roc_ie_on.h                   |  40 +++-
 drivers/common/cnxk/roc_se.h                      |  14 +-
 drivers/common/cnxk/version.map                   |   1 +
 drivers/crypto/cnxk/cn10k_cryptodev_ops.c         |  36 ++--
 drivers/crypto/cnxk/cn10k_ipsec.c                 | 231 ++++++++++++++++----
 drivers/crypto/cnxk/cn10k_ipsec.h                 |  25 ++-
 drivers/crypto/cnxk/cn10k_ipsec_la_ops.h          |  28 ++-
 drivers/crypto/cnxk/cn9k_cryptodev_ops.c          |  29 ++-
 drivers/crypto/cnxk/cn9k_ipsec.c                  | 221 ++++++++++++++-----
 drivers/crypto/cnxk/cn9k_ipsec_la_ops.h           |   6 +-
 drivers/crypto/cnxk/cnxk_cryptodev.h              |   2 +-
 drivers/crypto/cnxk/cnxk_cryptodev_capabilities.c | 155 +++++++++++++-
 drivers/crypto/cnxk/cnxk_cryptodev_ops.c          | 245 +++++++++++++---------
 drivers/crypto/cnxk/cnxk_cryptodev_ops.h          |  17 +-
 drivers/crypto/cnxk/cnxk_cryptodev_sec.c          |   1 +
 drivers/crypto/cnxk/cnxk_ipsec.h                  |  19 +-
 drivers/crypto/cnxk/cnxk_se.h                     |  66 ++++--
 29 files changed, 1283 insertions(+), 348 deletions(-)
 create mode 100644 drivers/common/cnxk/roc_aes.c
 create mode 100644 drivers/common/cnxk/roc_aes.h

-- 
2.7.4

[PATCH 01/25] common/cnxk: define minor opcodes for MISC opcode

[Anoob Joseph]

MISC CPT instruction behaves differently based on minor opcode.
Define the missing minor opcodes for MISC major opcode.

Signed-off-by: Aakash Sasidharan <asasidharan@marvell.com>
Signed-off-by: Anoob Joseph <anoobj@marvell.com>
---
 drivers/common/cnxk/roc_se.h | 6 +++++-
 1 file changed, 5 insertions(+), 1 deletion(-)

diff --git a/drivers/common/cnxk/roc_se.h b/drivers/common/cnxk/roc_se.h
index 5be832f..253575a 100644
--- a/drivers/common/cnxk/roc_se.h
+++ b/drivers/common/cnxk/roc_se.h
@@ -15,7 +15,11 @@
 #define ROC_SE_MAJOR_OP_HMAC	   0x35
 #define ROC_SE_MAJOR_OP_ZUC_SNOW3G 0x37
 #define ROC_SE_MAJOR_OP_KASUMI	   0x38
-#define ROC_SE_MAJOR_OP_MISC	   0x01
+
+#define ROC_SE_MAJOR_OP_MISC		 0x01
+#define ROC_SE_MISC_MINOR_OP_PASSTHROUGH 0x03
+#define ROC_SE_MISC_MINOR_OP_DUMMY	 0x04
+#define ROC_SE_MISC_MINOR_OP_HW_SUPPORT	 0x08
 
 #define ROC_SE_MAX_AAD_SIZE 64
 #define ROC_SE_MAX_MAC_LEN  64
-- 
2.7.4

[PATCH 02/25] common/cnxk: add aes-xcbc key derive

[Anoob Joseph]

Add support for AES-XCBC key derivation.


Signed-off-by: Anoob Joseph <anoobj@marvell.com>
---
 drivers/common/cnxk/meson.build |   1 +
 drivers/common/cnxk/roc_aes.c   | 208 ++++++++++++++++++++++++++++++++++++++++
 drivers/common/cnxk/roc_aes.h   |  14 +++
 drivers/common/cnxk/roc_api.h   |   3 +
 drivers/common/cnxk/roc_cpt.h   |  24 ++---
 drivers/common/cnxk/version.map |   1 +
 6 files changed, 239 insertions(+), 12 deletions(-)
 create mode 100644 drivers/common/cnxk/roc_aes.c
 create mode 100644 drivers/common/cnxk/roc_aes.h

diff --git a/drivers/common/cnxk/meson.build b/drivers/common/cnxk/meson.build
index 4928f7e..4995cfd 100644
--- a/drivers/common/cnxk/meson.build
+++ b/drivers/common/cnxk/meson.build
@@ -12,6 +12,7 @@ config_flag_fmt = 'RTE_LIBRTE_@0@_COMMON'
 deps = ['eal', 'pci', 'bus_pci', 'mbuf', 'security']
 sources = files(
         'roc_ae.c',
+        'roc_aes.c',
         'roc_ae_fpm_tables.c',
         'roc_bphy.c',
         'roc_bphy_cgx.c',
diff --git a/drivers/common/cnxk/roc_aes.c b/drivers/common/cnxk/roc_aes.c
new file mode 100644
index 0000000..f821c8b
--- /dev/null
+++ b/drivers/common/cnxk/roc_aes.c
@@ -0,0 +1,208 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright (c) 2021 Marvell.
+ */
+
+#include "roc_api.h"
+
+#define KEY_WORD_LEN	 (ROC_CPT_AES_XCBC_KEY_LENGTH / sizeof(uint32_t))
+#define KEY_ROUNDS	 10			/* (Nr+1)*Nb */
+#define KEY_SCHEDULE_LEN ((KEY_ROUNDS + 1) * 4) /* (Nr+1)*Nb words */
+
+/*
+ * AES 128 implementation based on NIST FIPS 197 suitable for LittleEndian
+ * https://nvlpubs.nist.gov/nistpubs/FIPS/NIST.FIPS.197.pdf
+ */
+
+/* Sbox from NIST FIPS 197 */
+static uint8_t Sbox[] = {
+	0x63, 0x7c, 0x77, 0x7b, 0xf2, 0x6b, 0x6f, 0xc5, 0x30, 0x01, 0x67, 0x2b,
+	0xfe, 0xd7, 0xab, 0x76, 0xca, 0x82, 0xc9, 0x7d, 0xfa, 0x59, 0x47, 0xf0,
+	0xad, 0xd4, 0xa2, 0xaf, 0x9c, 0xa4, 0x72, 0xc0, 0xb7, 0xfd, 0x93, 0x26,
+	0x36, 0x3f, 0xf7, 0xcc, 0x34, 0xa5, 0xe5, 0xf1, 0x71, 0xd8, 0x31, 0x15,
+	0x04, 0xc7, 0x23, 0xc3, 0x18, 0x96, 0x05, 0x9a, 0x07, 0x12, 0x80, 0xe2,
+	0xeb, 0x27, 0xb2, 0x75, 0x09, 0x83, 0x2c, 0x1a, 0x1b, 0x6e, 0x5a, 0xa0,
+	0x52, 0x3b, 0xd6, 0xb3, 0x29, 0xe3, 0x2f, 0x84, 0x53, 0xd1, 0x00, 0xed,
+	0x20, 0xfc, 0xb1, 0x5b, 0x6a, 0xcb, 0xbe, 0x39, 0x4a, 0x4c, 0x58, 0xcf,
+	0xd0, 0xef, 0xaa, 0xfb, 0x43, 0x4d, 0x33, 0x85, 0x45, 0xf9, 0x02, 0x7f,
+	0x50, 0x3c, 0x9f, 0xa8, 0x51, 0xa3, 0x40, 0x8f, 0x92, 0x9d, 0x38, 0xf5,
+	0xbc, 0xb6, 0xda, 0x21, 0x10, 0xff, 0xf3, 0xd2, 0xcd, 0x0c, 0x13, 0xec,
+	0x5f, 0x97, 0x44, 0x17, 0xc4, 0xa7, 0x7e, 0x3d, 0x64, 0x5d, 0x19, 0x73,
+	0x60, 0x81, 0x4f, 0xdc, 0x22, 0x2a, 0x90, 0x88, 0x46, 0xee, 0xb8, 0x14,
+	0xde, 0x5e, 0x0b, 0xdb, 0xe0, 0x32, 0x3a, 0x0a, 0x49, 0x06, 0x24, 0x5c,
+	0xc2, 0xd3, 0xac, 0x62, 0x91, 0x95, 0xe4, 0x79, 0xe7, 0xc8, 0x37, 0x6d,
+	0x8d, 0xd5, 0x4e, 0xa9, 0x6c, 0x56, 0xf4, 0xea, 0x65, 0x7a, 0xae, 0x08,
+	0xba, 0x78, 0x25, 0x2e, 0x1c, 0xa6, 0xb4, 0xc6, 0xe8, 0xdd, 0x74, 0x1f,
+	0x4b, 0xbd, 0x8b, 0x8a, 0x70, 0x3e, 0xb5, 0x66, 0x48, 0x03, 0xf6, 0x0e,
+	0x61, 0x35, 0x57, 0xb9, 0x86, 0xc1, 0x1d, 0x9e, 0xe1, 0xf8, 0x98, 0x11,
+	0x69, 0xd9, 0x8e, 0x94, 0x9b, 0x1e, 0x87, 0xe9, 0xce, 0x55, 0x28, 0xdf,
+	0x8c, 0xa1, 0x89, 0x0d, 0xbf, 0xe6, 0x42, 0x68, 0x41, 0x99, 0x2d, 0x0f,
+	0xb0, 0x54, 0xbb, 0x16,
+};
+
+/* Substitute a byte with Sbox[byte]. Do it for a word for 4 bytes */
+static uint32_t
+sub_word(uint32_t word)
+{
+	word = (Sbox[(word >> 24) & 0xFF] << 24) |
+	       (Sbox[(word >> 16) & 0xFF] << 16) |
+	       (Sbox[(word >> 8) & 0xFF] << 8) | Sbox[word & 0xFF];
+	return word;
+}
+
+/* Rotate a word by one byte */
+static uint32_t
+rot_word(uint32_t word)
+{
+	return ((word >> 8) & 0xFFFFFF) | (word << 24);
+}
+
+/*
+ * Multiply with power of 2 and polynomial reduce the result using AES
+ * polynomial
+ */
+static uint8_t
+Xtime(uint8_t byte, uint8_t pow)
+{
+	uint32_t w = byte;
+
+	while (pow) {
+		w = w << 1;
+		if (w >> 8)
+			w ^= 0x11b;
+		pow--;
+	}
+
+	return (uint8_t)w;
+}
+
+/*
+ * Multiply a byte with another number such that the result is polynomial
+ * reduced in the GF8 space
+ */
+static uint8_t
+GF8mul(uint8_t byte, uint32_t mp)
+{
+	uint8_t pow, mul = 0;
+
+	while (mp) {
+		pow = ffs(mp) - 1;
+		mul ^= Xtime(byte, pow);
+		mp ^= (1 << pow);
+	}
+	return mul;
+}
+
+static void
+aes_key_expand(const uint8_t *key, uint32_t *ks)
+{
+	unsigned int i = 4;
+	uint32_t temp;
+
+	/* Skip key in ks */
+	memcpy(ks, key, KEY_WORD_LEN * sizeof(uint32_t));
+
+	while (i < KEY_SCHEDULE_LEN) {
+		temp = ks[i - 1];
+		if ((i & 0x3) == 0) {
+			temp = rot_word(temp);
+			temp = sub_word(temp);
+			temp ^= (uint32_t)GF8mul(1, 1 << ((i >> 2) - 1));
+		}
+		ks[i] = ks[i - 4] ^ temp;
+		i++;
+	}
+}
+
+/* Shift Rows(columns in state in this implementation) */
+static void
+shift_word(uint8_t *sRc, uint8_t c, int count)
+{
+	/* rotate across non-consecutive locations */
+	while (count) {
+		uint8_t t = sRc[c];
+
+		sRc[c] = sRc[0x4 + c];
+		sRc[0x4 + c] = sRc[0x8 + c];
+		sRc[0x8 + c] = sRc[0xc + c];
+		sRc[0xc + c] = t;
+		count--;
+	}
+}
+
+/* Mix Columns(rows in state in this implementation) */
+static void
+mix_columns(uint8_t *sRc)
+{
+	uint8_t new_st[4];
+	int i;
+
+	for (i = 0; i < 4; i++)
+		new_st[i] = GF8mul(sRc[i], 0x2) ^
+			    GF8mul(sRc[(i + 1) & 0x3], 0x3) ^
+			    sRc[(i + 2) & 0x3] ^ sRc[(i + 3) & 0x3];
+	for (i = 0; i < 4; i++)
+		sRc[i] = new_st[i];
+}
+
+static void
+cipher(uint8_t *in, uint8_t *out, uint32_t *ks)
+{
+	uint32_t state[KEY_WORD_LEN];
+	unsigned int i, round;
+
+	memcpy(state, in, sizeof(state));
+
+	/* AddRoundKey(state, w[0, Nb-1]) // See Sec. 5.1.4 */
+	for (i = 0; i < KEY_WORD_LEN; i++)
+		state[i] ^= ks[i];
+
+	for (round = 1; round < KEY_ROUNDS; round++) {
+		/* SubBytes(state) // See Sec. 5.1.1 */
+		for (i = 0; i < KEY_WORD_LEN; i++)
+			state[i] = sub_word(state[i]);
+
+		/* ShiftRows(state) // See Sec. 5.1.2 */
+		for (i = 0; i < KEY_WORD_LEN; i++)
+			shift_word((uint8_t *)state, i, i);
+
+		/* MixColumns(state) // See Sec. 5.1.3 */
+		for (i = 0; i < KEY_WORD_LEN; i++)
+			mix_columns((uint8_t *)&state[i]);
+
+		/* AddRoundKey(state, w[round*Nb, (round+1)*Nb-1]) */
+		for (i = 0; i < KEY_WORD_LEN; i++)
+			state[i] ^= ks[round * 4 + i];
+	}
+
+	/* SubBytes(state) */
+	for (i = 0; i < KEY_WORD_LEN; i++)
+		state[i] = sub_word(state[i]);
+
+	/* ShiftRows(state) */
+	for (i = 0; i < KEY_WORD_LEN; i++)
+		shift_word((uint8_t *)state, i, i);
+
+	/* AddRoundKey(state, w[Nr*Nb, (Nr+1)*Nb-1]) */
+	for (i = 0; i < KEY_WORD_LEN; i++)
+		state[i] ^= ks[KEY_ROUNDS * 4 + i];
+	memcpy(out, state, KEY_WORD_LEN * sizeof(uint32_t));
+}
+
+void
+roc_aes_xcbc_key_derive(const uint8_t *auth_key, uint8_t *derived_key)
+{
+	uint32_t aes_ks[KEY_SCHEDULE_LEN] = {0};
+	uint8_t k1[16] = {[0 ... 15] = 0x01};
+	uint8_t k2[16] = {[0 ... 15] = 0x02};
+	uint8_t k3[16] = {[0 ... 15] = 0x03};
+
+	aes_key_expand(auth_key, aes_ks);
+
+	cipher(k1, derived_key, aes_ks);
+	derived_key += sizeof(k1);
+
+	cipher(k2, derived_key, aes_ks);
+	derived_key += sizeof(k2);
+
+	cipher(k3, derived_key, aes_ks);
+}
diff --git a/drivers/common/cnxk/roc_aes.h b/drivers/common/cnxk/roc_aes.h
new file mode 100644
index 0000000..9540391
--- /dev/null
+++ b/drivers/common/cnxk/roc_aes.h
@@ -0,0 +1,14 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright (c) 2021 Marvell.
+ */
+
+#ifndef _ROC_AES_H_
+#define _ROC_AES_H_
+
+/*
+ * Derive k1, k2, k3 from 128 bit AES key
+ */
+void __roc_api roc_aes_xcbc_key_derive(const uint8_t *auth_key,
+				       uint8_t *derived_key);
+
+#endif /* _ROC_AES_H_ */
diff --git a/drivers/common/cnxk/roc_api.h b/drivers/common/cnxk/roc_api.h
index e7aaa07..cf4d487 100644
--- a/drivers/common/cnxk/roc_api.h
+++ b/drivers/common/cnxk/roc_api.h
@@ -90,6 +90,9 @@
 /* DPI */
 #include "roc_dpi.h"
 
+/* AES */
+#include "roc_aes.h"
+
 /* HASH computation */
 #include "roc_hash.h"
 
diff --git a/drivers/common/cnxk/roc_cpt.h b/drivers/common/cnxk/roc_cpt.h
index 12e6b81..99cb8b2 100644
--- a/drivers/common/cnxk/roc_cpt.h
+++ b/drivers/common/cnxk/roc_cpt.h
@@ -49,18 +49,18 @@
 #define ROC_CPT_AES_CBC_IV_LEN	 16
 #define ROC_CPT_SHA1_HMAC_LEN	 12
 #define ROC_CPT_SHA2_HMAC_LEN	 16
-#define ROC_CPT_AUTH_KEY_LEN_MAX 64
-
-#define ROC_CPT_DES3_KEY_LEN	  24
-#define ROC_CPT_AES128_KEY_LEN	  16
-#define ROC_CPT_AES192_KEY_LEN	  24
-#define ROC_CPT_AES256_KEY_LEN	  32
-#define ROC_CPT_MD5_KEY_LENGTH	  16
-#define ROC_CPT_SHA1_KEY_LENGTH	  20
-#define ROC_CPT_SHA256_KEY_LENGTH 32
-#define ROC_CPT_SHA384_KEY_LENGTH 48
-#define ROC_CPT_SHA512_KEY_LENGTH 64
-#define ROC_CPT_AUTH_KEY_LEN_MAX  64
+
+#define ROC_CPT_DES3_KEY_LEN	    24
+#define ROC_CPT_AES128_KEY_LEN	    16
+#define ROC_CPT_AES192_KEY_LEN	    24
+#define ROC_CPT_AES256_KEY_LEN	    32
+#define ROC_CPT_MD5_KEY_LENGTH	    16
+#define ROC_CPT_SHA1_KEY_LENGTH	    20
+#define ROC_CPT_SHA256_KEY_LENGTH   32
+#define ROC_CPT_SHA384_KEY_LENGTH   48
+#define ROC_CPT_SHA512_KEY_LENGTH   64
+#define ROC_CPT_AES_XCBC_KEY_LENGTH 16
+#define ROC_CPT_AUTH_KEY_LEN_MAX    64
 
 #define ROC_CPT_DES_BLOCK_LENGTH 8
 #define ROC_CPT_AES_BLOCK_LENGTH 16
diff --git a/drivers/common/cnxk/version.map b/drivers/common/cnxk/version.map
index 07c6720..b31e8eb 100644
--- a/drivers/common/cnxk/version.map
+++ b/drivers/common/cnxk/version.map
@@ -26,6 +26,7 @@ INTERNAL {
 	roc_ae_ec_grp_put;
 	roc_ae_fpm_get;
 	roc_ae_fpm_put;
+	roc_aes_xcbc_key_derive;
 	roc_bphy_cgx_dev_fini;
 	roc_bphy_cgx_dev_init;
 	roc_bphy_cgx_fec_set;
-- 
2.7.4

[PATCH 03/25] common/cnxk: add bit fields for params

[Anoob Joseph]

From: Archana Muniganti <marchana@marvell.com>

Added new structure with bit fields for params.


Signed-off-by: Archana Muniganti <marchana@marvell.com>
---
 drivers/common/cnxk/roc_ie_on.h  | 30 +++++++++++++++++++++++++++++-
 drivers/crypto/cnxk/cn9k_ipsec.c | 16 +++++++++++++---
 2 files changed, 42 insertions(+), 4 deletions(-)

diff --git a/drivers/common/cnxk/roc_ie_on.h b/drivers/common/cnxk/roc_ie_on.h
index 53591c6..817ef33 100644
--- a/drivers/common/cnxk/roc_ie_on.h
+++ b/drivers/common/cnxk/roc_ie_on.h
@@ -21,7 +21,6 @@ enum roc_ie_on_ucc_ipsec {
 };
 
 /* Helper macros */
-#define ROC_IE_ON_PER_PKT_IV   BIT(11)
 #define ROC_IE_ON_INB_RPTR_HDR 0x8
 
 enum {
@@ -102,6 +101,35 @@ struct roc_ie_on_ip_template {
 	};
 };
 
+union roc_on_ipsec_outb_param1 {
+	uint16_t u16;
+	struct {
+		uint16_t frag_num : 4;
+		uint16_t rsvd_4_6 : 3;
+		uint16_t gre_select : 1;
+		uint16_t dsiv : 1;
+		uint16_t ikev2 : 1;
+		uint16_t min_frag_size : 1;
+		uint16_t per_pkt_iv : 1;
+		uint16_t tfc_pad_enable : 1;
+		uint16_t tfc_dummy_pkt : 1;
+		uint16_t rfc_or_override_mode : 1;
+		uint16_t custom_hdr_or_p99 : 1;
+	} s;
+};
+
+union roc_on_ipsec_inb_param2 {
+	uint16_t u16;
+	struct {
+		uint16_t rsvd_0_10 : 11;
+		uint16_t gre_select : 1;
+		uint16_t ikev2 : 1;
+		uint16_t udp_cksum : 1;
+		uint16_t ctx_addr_sel : 1;
+		uint16_t custom_hdr_or_p99 : 1;
+	} s;
+};
+
 struct roc_ie_on_sa_ctl {
 	uint64_t spi : 32;
 	uint64_t exp_proto_inter_frag : 8;
diff --git a/drivers/crypto/cnxk/cn9k_ipsec.c b/drivers/crypto/cnxk/cn9k_ipsec.c
index a81130b..6455ef9 100644
--- a/drivers/crypto/cnxk/cn9k_ipsec.c
+++ b/drivers/crypto/cnxk/cn9k_ipsec.c
@@ -280,6 +280,7 @@ cn9k_ipsec_outb_sa_create(struct cnxk_cpt_qp *qp,
 	struct rte_crypto_sym_xform *auth_xform = crypto_xform->next;
 	struct roc_ie_on_ip_template *template = NULL;
 	struct roc_cpt *roc_cpt = qp->lf.roc_cpt;
+	union roc_on_ipsec_outb_param1 param1;
 	struct cnxk_cpt_inst_tmpl *inst_tmpl;
 	struct roc_ie_on_outb_sa *out_sa;
 	struct cn9k_sec_session *sess;
@@ -407,8 +408,12 @@ cn9k_ipsec_outb_sa_create(struct cnxk_cpt_qp *qp,
 	w4.u64 = 0;
 	w4.s.opcode_major = ROC_IE_ON_MAJOR_OP_PROCESS_OUTBOUND_IPSEC;
 	w4.s.opcode_minor = ctx_len >> 3;
-	w4.s.param1 = BIT(9);
-	w4.s.param1 |= ROC_IE_ON_PER_PKT_IV;
+
+	param1.u16 = 0;
+	param1.s.ikev2 = 1;
+	param1.s.per_pkt_iv = 1;
+	w4.s.param1 = param1.u16;
+
 	inst_tmpl->w4 = w4.u64;
 
 	w7.u64 = 0;
@@ -428,6 +433,7 @@ cn9k_ipsec_inb_sa_create(struct cnxk_cpt_qp *qp,
 {
 	struct rte_crypto_sym_xform *auth_xform = crypto_xform;
 	struct roc_cpt *roc_cpt = qp->lf.roc_cpt;
+	union roc_on_ipsec_inb_param2 param2;
 	struct cnxk_cpt_inst_tmpl *inst_tmpl;
 	struct roc_ie_on_inb_sa *in_sa;
 	struct cn9k_sec_session *sess;
@@ -478,7 +484,11 @@ cn9k_ipsec_inb_sa_create(struct cnxk_cpt_qp *qp,
 	w4.u64 = 0;
 	w4.s.opcode_major = ROC_IE_ON_MAJOR_OP_PROCESS_INBOUND_IPSEC;
 	w4.s.opcode_minor = ctx_len >> 3;
-	w4.s.param2 = BIT(12);
+
+	param2.u16 = 0;
+	param2.s.ikev2 = 1;
+	w4.s.param2 = param2.u16;
+
 	inst_tmpl->w4 = w4.u64;
 
 	w7.u64 = 0;
-- 
2.7.4

[PATCH 04/25] common/cnxk: fix reset of fields

[Anoob Joseph]

Copy DF/DSCP fields would get set based on ipsec_xform in the code
preceding this. Setting it again would cause the options to be reset.

Fixes: 78d03027f2cc ("common/cnxk: add IPsec common code")
Cc: schalla@marvell.com

Signed-off-by: Anoob Joseph <anoobj@marvell.com>
---
 drivers/common/cnxk/cnxk_security.c | 4 ----
 1 file changed, 4 deletions(-)

diff --git a/drivers/common/cnxk/cnxk_security.c b/drivers/common/cnxk/cnxk_security.c
index 30562b4..787138b 100644
--- a/drivers/common/cnxk/cnxk_security.c
+++ b/drivers/common/cnxk/cnxk_security.c
@@ -444,10 +444,6 @@ cnxk_ot_ipsec_outb_sa_fill(struct roc_ot_ipsec_outb_sa *sa,
 		return -EINVAL;
 	}
 
-	/* Default options of DSCP and Flow label/DF */
-	sa->w2.s.dscp_src = ROC_IE_OT_SA_COPY_FROM_SA;
-	sa->w2.s.ipv4_df_src_or_ipv6_flw_lbl_src = ROC_IE_OT_SA_COPY_FROM_SA;
-
 skip_tunnel_info:
 	/* ESN */
 	sa->w0.s.esn_en = !!ipsec_xfrm->options.esn;
-- 
2.7.4

[PATCH 05/25] common/cnxk: verify input args

[Anoob Joseph]

Add input arg verification.

Signed-off-by: Anoob Joseph <anoobj@marvell.com>
---
 drivers/common/cnxk/hw/cpt.h  | 2 ++
 drivers/common/cnxk/roc_cpt.c | 4 +++-
 2 files changed, 5 insertions(+), 1 deletion(-)

diff --git a/drivers/common/cnxk/hw/cpt.h b/drivers/common/cnxk/hw/cpt.h
index 919f842..ccc7af4 100644
--- a/drivers/common/cnxk/hw/cpt.h
+++ b/drivers/common/cnxk/hw/cpt.h
@@ -64,6 +64,7 @@ union cpt_lf_ctx_flush {
 	struct {
 		uint64_t cptr : 46;
 		uint64_t inval : 1;
+		uint64_t reserved_47_63 : 17;
 	} s;
 };
 
@@ -71,6 +72,7 @@ union cpt_lf_ctx_reload {
 	uint64_t u;
 	struct {
 		uint64_t cptr : 46;
+		uint64_t reserved_46_63 : 18;
 	} s;
 };
 
diff --git a/drivers/common/cnxk/roc_cpt.c b/drivers/common/cnxk/roc_cpt.c
index 8f8e6d3..1bc7a29 100644
--- a/drivers/common/cnxk/roc_cpt.c
+++ b/drivers/common/cnxk/roc_cpt.c
@@ -681,8 +681,10 @@ roc_cpt_lf_ctx_flush(struct roc_cpt_lf *lf, void *cptr, bool inval)
 {
 	union cpt_lf_ctx_flush reg;
 
-	if (lf == NULL)
+	if (lf == NULL) {
+		plt_err("Could not trigger CTX flush");
 		return -ENOTSUP;
+	}
 
 	reg.u = 0;
 	reg.s.inval = inval;
-- 
2.7.4

[PATCH 06/25] crypto/cnxk: only enable queues that are allocated

[Anoob Joseph]

From: Shijith Thotton <sthotton@marvell.com>

Only enable/disable queue pairs that are allocated during cryptodev
start/stop.

Fixes: 6a95dbc1a291 ("crypto/cnxk: add dev start and dev stop")

Signed-off-by: Shijith Thotton <sthotton@marvell.com>
---
 drivers/crypto/cnxk/cnxk_cryptodev_ops.c | 13 +++++++++++--
 1 file changed, 11 insertions(+), 2 deletions(-)

diff --git a/drivers/crypto/cnxk/cnxk_cryptodev_ops.c b/drivers/crypto/cnxk/cnxk_cryptodev_ops.c
index a2281fb..21ee09f 100644
--- a/drivers/crypto/cnxk/cnxk_cryptodev_ops.c
+++ b/drivers/crypto/cnxk/cnxk_cryptodev_ops.c
@@ -100,8 +100,13 @@ cnxk_cpt_dev_start(struct rte_cryptodev *dev)
 	uint16_t nb_lf = roc_cpt->nb_lf;
 	uint16_t qp_id;
 
-	for (qp_id = 0; qp_id < nb_lf; qp_id++)
+	for (qp_id = 0; qp_id < nb_lf; qp_id++) {
+		/* Application may not setup all queue pair */
+		if (roc_cpt->lf[qp_id] == NULL)
+			continue;
+
 		roc_cpt_iq_enable(roc_cpt->lf[qp_id]);
+	}
 
 	return 0;
 }
@@ -114,8 +119,12 @@ cnxk_cpt_dev_stop(struct rte_cryptodev *dev)
 	uint16_t nb_lf = roc_cpt->nb_lf;
 	uint16_t qp_id;
 
-	for (qp_id = 0; qp_id < nb_lf; qp_id++)
+	for (qp_id = 0; qp_id < nb_lf; qp_id++) {
+		if (roc_cpt->lf[qp_id] == NULL)
+			continue;
+
 		roc_cpt_iq_disable(roc_cpt->lf[qp_id]);
+	}
 }
 
 int
-- 
2.7.4

[PATCH 07/25] crypto/cnxk: add lookaside IPsec AES-CBC-HMAC-SHA256 support

[Anoob Joseph]

From: Tejasree Kondoj <ktejasree@marvell.com>

Adding AES-CBC-HMAC-SHA256 support to lookaside IPsec PMD.

Signed-off-by: Tejasree Kondoj <ktejasree@marvell.com>
---
 doc/guides/cryptodevs/cnxk.rst                    | 39 +++++++++++++++++++----
 doc/guides/rel_notes/release_22_03.rst            |  4 +++
 drivers/common/cnxk/cnxk_security.c               | 14 ++++++++
 drivers/crypto/cnxk/cn10k_ipsec.c                 |  3 ++
 drivers/crypto/cnxk/cnxk_cryptodev_capabilities.c | 20 ++++++++++++
 drivers/crypto/cnxk/cnxk_ipsec.h                  |  3 +-
 6 files changed, 75 insertions(+), 8 deletions(-)

diff --git a/doc/guides/cryptodevs/cnxk.rst b/doc/guides/cryptodevs/cnxk.rst
index 23cc823..8c4c4ea 100644
--- a/doc/guides/cryptodevs/cnxk.rst
+++ b/doc/guides/cryptodevs/cnxk.rst
@@ -246,14 +246,27 @@ CN9XX Features supported
 * IPv4
 * IPv6
 * ESP
+* ESN
+* Anti-replay
 * Tunnel mode
 * Transport mode(IPv4)
 * UDP Encapsulation
+
+AEAD algorithms
++++++++++++++++
+
 * AES-128/192/256-GCM
-* AES-128/192/256-CBC-SHA1-HMAC
-* AES-128/192/256-CBC-SHA256-128-HMAC
-* ESN
-* Anti-replay
+
+Cipher algorithms
++++++++++++++++++
+
+* AES-128/192/256-CBC
+
+Auth algorithms
++++++++++++++++
+
+* SHA1-HMAC
+* SHA256-128-HMAC
 
 CN10XX Features supported
 ~~~~~~~~~~~~~~~~~~~~~~~~~
@@ -263,6 +276,20 @@ CN10XX Features supported
 * Tunnel mode
 * Transport mode
 * UDP Encapsulation
+
+AEAD algorithms
++++++++++++++++
+
 * AES-128/192/256-GCM
-* AES-128/192/256-CBC-NULL
-* AES-128/192/256-CBC-SHA1-HMAC
+
+Cipher algorithms
++++++++++++++++++
+
+* AES-128/192/256-CBC
+
+Auth algorithms
++++++++++++++++
+
+* NULL
+* SHA1-HMAC
+* SHA256-128-HMAC
diff --git a/doc/guides/rel_notes/release_22_03.rst b/doc/guides/rel_notes/release_22_03.rst
index 6d99d1e..1639b0e 100644
--- a/doc/guides/rel_notes/release_22_03.rst
+++ b/doc/guides/rel_notes/release_22_03.rst
@@ -55,6 +55,10 @@ New Features
      Also, make sure to start the actual text at the margin.
      =======================================================
 
+* **Updated Marvell cnxk crypto PMD.**
+
+  * Added SHA256-HMAC support in lookaside protocol (IPsec) for CN10K.
+
 
 Removed Items
 -------------
diff --git a/drivers/common/cnxk/cnxk_security.c b/drivers/common/cnxk/cnxk_security.c
index 787138b..f39bc1e 100644
--- a/drivers/common/cnxk/cnxk_security.c
+++ b/drivers/common/cnxk/cnxk_security.c
@@ -32,6 +32,10 @@ ipsec_hmac_opad_ipad_gen(struct rte_crypto_sym_xform *auth_xform,
 		roc_hash_sha1_gen(opad, (uint32_t *)&hmac_opad_ipad[0]);
 		roc_hash_sha1_gen(ipad, (uint32_t *)&hmac_opad_ipad[24]);
 		break;
+	case RTE_CRYPTO_AUTH_SHA256_HMAC:
+		roc_hash_sha256_gen(opad, (uint32_t *)&hmac_opad_ipad[0]);
+		roc_hash_sha256_gen(ipad, (uint32_t *)&hmac_opad_ipad[64]);
+		break;
 	default:
 		break;
 	}
@@ -129,6 +133,16 @@ ot_ipsec_sa_common_param_fill(union roc_ot_ipsec_sa_word2 *w2,
 			     i++)
 				tmp_key[i] = rte_be_to_cpu_64(tmp_key[i]);
 			break;
+		case RTE_CRYPTO_AUTH_SHA256_HMAC:
+			w2->s.auth_type = ROC_IE_OT_SA_AUTH_SHA2_256;
+			ipsec_hmac_opad_ipad_gen(auth_xfrm, hmac_opad_ipad);
+
+			tmp_key = (uint64_t *)hmac_opad_ipad;
+			for (i = 0; i < (int)(ROC_CTX_MAX_OPAD_IPAD_LEN /
+					      sizeof(uint64_t));
+			     i++)
+				tmp_key[i] = rte_be_to_cpu_64(tmp_key[i]);
+			break;
 		default:
 			return -ENOTSUP;
 		}
diff --git a/drivers/crypto/cnxk/cn10k_ipsec.c b/drivers/crypto/cnxk/cn10k_ipsec.c
index 27df1dc..93eab1b 100644
--- a/drivers/crypto/cnxk/cn10k_ipsec.c
+++ b/drivers/crypto/cnxk/cn10k_ipsec.c
@@ -65,6 +65,9 @@ cn10k_ipsec_outb_sa_create(struct roc_cpt *roc_cpt,
 		if (crypto_xfrm->type == RTE_CRYPTO_SYM_XFORM_AEAD) {
 			sa->iv_offset = crypto_xfrm->aead.iv.offset;
 			sa->iv_length = crypto_xfrm->aead.iv.length;
+		} else {
+			sa->iv_offset = crypto_xfrm->cipher.iv.offset;
+			sa->iv_length = crypto_xfrm->cipher.iv.length;
 		}
 	}
 #else
diff --git a/drivers/crypto/cnxk/cnxk_cryptodev_capabilities.c b/drivers/crypto/cnxk/cnxk_cryptodev_capabilities.c
index 59b63ed..7d22626 100644
--- a/drivers/crypto/cnxk/cnxk_cryptodev_capabilities.c
+++ b/drivers/crypto/cnxk/cnxk_cryptodev_capabilities.c
@@ -797,6 +797,26 @@ static const struct rte_cryptodev_capabilities sec_caps_sha1_sha2[] = {
 			}, }
 		}, }
 	},
+	{	/* SHA256 HMAC */
+		.op = RTE_CRYPTO_OP_TYPE_SYMMETRIC,
+		{.sym = {
+			.xform_type = RTE_CRYPTO_SYM_XFORM_AUTH,
+			{.auth = {
+				.algo = RTE_CRYPTO_AUTH_SHA256_HMAC,
+				.block_size = 64,
+				.key_size = {
+					.min = 1,
+					.max = 1024,
+					.increment = 1
+				},
+				.digest_size = {
+					.min = 16,
+					.max = 16,
+					.increment = 0
+				},
+			}, }
+		}, }
+	},
 };
 
 static const struct rte_security_capability sec_caps_templ[] = {
diff --git a/drivers/crypto/cnxk/cnxk_ipsec.h b/drivers/crypto/cnxk/cnxk_ipsec.h
index dddb414..f4a1012 100644
--- a/drivers/crypto/cnxk/cnxk_ipsec.h
+++ b/drivers/crypto/cnxk/cnxk_ipsec.h
@@ -46,8 +46,7 @@ ipsec_xform_auth_verify(struct rte_crypto_sym_xform *crypto_xform)
 	if (crypto_xform->auth.algo == RTE_CRYPTO_AUTH_SHA1_HMAC) {
 		if (keylen >= 20 && keylen <= 64)
 			return 0;
-	} else if (roc_model_is_cn9k() &&
-		   (crypto_xform->auth.algo == RTE_CRYPTO_AUTH_SHA256_HMAC)) {
+	} else if (crypto_xform->auth.algo == RTE_CRYPTO_AUTH_SHA256_HMAC) {
 		if (keylen >= 32 && keylen <= 64)
 			return 0;
 	}
-- 
2.7.4

[PATCH 08/25] crypto/cnxk: clear session data before populating

[Anoob Joseph]

Clear session data before populating fields to not have garbage data.

Signed-off-by: Anoob Joseph <anoobj@marvell.com>
---
 drivers/crypto/cnxk/cn10k_ipsec.c | 2 ++
 1 file changed, 2 insertions(+)

diff --git a/drivers/crypto/cnxk/cn10k_ipsec.c b/drivers/crypto/cnxk/cn10k_ipsec.c
index 93eab1b..1bd127e 100644
--- a/drivers/crypto/cnxk/cn10k_ipsec.c
+++ b/drivers/crypto/cnxk/cn10k_ipsec.c
@@ -130,6 +130,8 @@ cn10k_ipsec_inb_sa_create(struct roc_cpt *roc_cpt,
 	sa = &sess->sa;
 	in_sa = &sa->in_sa;
 
+	memset(in_sa, 0, sizeof(struct roc_ot_ipsec_inb_sa));
+
 	/* Translate security parameters to SA */
 	ret = cnxk_ot_ipsec_inb_sa_fill(in_sa, ipsec_xfrm, crypto_xfrm);
 	if (ret)
-- 
2.7.4

[PATCH 09/25] crypto/cnxk: update max sec crypto caps

[Anoob Joseph]

Update the macro to include newly added ciphers. Updated the functions
populating caps to throw error when max is exceeded.

Signed-off-by: Anoob Joseph <anoobj@marvell.com>
---
 drivers/crypto/cnxk/cnxk_cryptodev.h              | 2 +-
 drivers/crypto/cnxk/cnxk_cryptodev_capabilities.c | 8 ++++++--
 2 files changed, 7 insertions(+), 3 deletions(-)

diff --git a/drivers/crypto/cnxk/cnxk_cryptodev.h b/drivers/crypto/cnxk/cnxk_cryptodev.h
index cfb9d29..2e0f467 100644
--- a/drivers/crypto/cnxk/cnxk_cryptodev.h
+++ b/drivers/crypto/cnxk/cnxk_cryptodev.h
@@ -11,7 +11,7 @@
 #include "roc_cpt.h"
 
 #define CNXK_CPT_MAX_CAPS	 34
-#define CNXK_SEC_CRYPTO_MAX_CAPS 4
+#define CNXK_SEC_CRYPTO_MAX_CAPS 6
 #define CNXK_SEC_MAX_CAPS	 5
 #define CNXK_AE_EC_ID_MAX	 8
 /**
diff --git a/drivers/crypto/cnxk/cnxk_cryptodev_capabilities.c b/drivers/crypto/cnxk/cnxk_cryptodev_capabilities.c
index 7d22626..8305341 100644
--- a/drivers/crypto/cnxk/cnxk_cryptodev_capabilities.c
+++ b/drivers/crypto/cnxk/cnxk_cryptodev_capabilities.c
@@ -943,8 +943,10 @@ static void
 sec_caps_add(struct rte_cryptodev_capabilities cnxk_caps[], int *cur_pos,
 	     const struct rte_cryptodev_capabilities *caps, int nb_caps)
 {
-	if (*cur_pos + nb_caps > CNXK_SEC_CRYPTO_MAX_CAPS)
+	if (*cur_pos + nb_caps > CNXK_SEC_CRYPTO_MAX_CAPS) {
+		rte_panic("Could not add sec crypto caps");
 		return;
+	}
 
 	memcpy(&cnxk_caps[*cur_pos], caps, nb_caps * sizeof(caps[0]));
 	*cur_pos += nb_caps;
@@ -957,8 +959,10 @@ cn10k_sec_crypto_caps_update(struct rte_cryptodev_capabilities cnxk_caps[],
 	const struct rte_cryptodev_capabilities *cap;
 	unsigned int i;
 
-	if ((CNXK_CPT_MAX_CAPS - *cur_pos) < 1)
+	if ((CNXK_SEC_CRYPTO_MAX_CAPS - *cur_pos) < 1) {
+		rte_panic("Could not add sec crypto caps");
 		return;
+	}
 
 	/* NULL auth */
 	for (i = 0; i < RTE_DIM(caps_null); i++) {
-- 
2.7.4

[PATCH 10/25] crypto/cnxk: write CPT CTX through microcode op

[Anoob Joseph]

From: Tejasree Kondoj <ktejasree@marvell.com>

Adding support to write CPT CTX through microcode op(SET_CTX) for
cn10k lookaside PMD.

Signed-off-by: Tejasree Kondoj <ktejasree@marvell.com>
---
 drivers/crypto/cnxk/cn10k_ipsec.c | 121 ++++++++++++++++++++++++++++----------
 1 file changed, 89 insertions(+), 32 deletions(-)

diff --git a/drivers/crypto/cnxk/cn10k_ipsec.c b/drivers/crypto/cnxk/cn10k_ipsec.c
index 1bd127e..a11a6b7 100644
--- a/drivers/crypto/cnxk/cn10k_ipsec.c
+++ b/drivers/crypto/cnxk/cn10k_ipsec.c
@@ -2,18 +2,19 @@
  * Copyright(C) 2021 Marvell.
  */
 
-#include <rte_malloc.h>
 #include <cryptodev_pmd.h>
 #include <rte_esp.h>
 #include <rte_ip.h>
+#include <rte_malloc.h>
 #include <rte_security.h>
 #include <rte_security_driver.h>
 #include <rte_udp.h>
 
+#include "cn10k_ipsec.h"
 #include "cnxk_cryptodev.h"
+#include "cnxk_cryptodev_ops.h"
 #include "cnxk_ipsec.h"
 #include "cnxk_security.h"
-#include "cn10k_ipsec.h"
 
 #include "roc_api.h"
 
@@ -32,36 +33,46 @@ ipsec_cpt_inst_w7_get(struct roc_cpt *roc_cpt, void *sa)
 }
 
 static int
-cn10k_ipsec_outb_sa_create(struct roc_cpt *roc_cpt,
+cn10k_ipsec_outb_sa_create(struct roc_cpt *roc_cpt, struct roc_cpt_lf *lf,
 			   struct rte_security_ipsec_xform *ipsec_xfrm,
 			   struct rte_crypto_sym_xform *crypto_xfrm,
 			   struct rte_security_session *sec_sess)
 {
 	union roc_ot_ipsec_outb_param1 param1;
-	struct roc_ot_ipsec_outb_sa *out_sa;
+	struct roc_ot_ipsec_outb_sa *sa_dptr;
 	struct cnxk_ipsec_outb_rlens rlens;
 	struct cn10k_sec_session *sess;
 	struct cn10k_ipsec_sa *sa;
 	union cpt_inst_w4 inst_w4;
-	int ret;
+	void *out_sa;
+	int ret = 0;
 
 	sess = get_sec_session_private_data(sec_sess);
 	sa = &sess->sa;
 	out_sa = &sa->out_sa;
 
-	memset(out_sa, 0, sizeof(struct roc_ot_ipsec_outb_sa));
+	/* Allocate memory to be used as dptr for CPT ucode WRITE_SA op */
+	sa_dptr = plt_zmalloc(ROC_NIX_INL_OT_IPSEC_OUTB_HW_SZ, 0);
+	if (sa_dptr == NULL) {
+		plt_err("Couldn't allocate memory for SA dptr");
+		return -ENOMEM;
+	}
+
+	memset(sa_dptr, 0, sizeof(struct roc_ot_ipsec_outb_sa));
 
 	/* Translate security parameters to SA */
-	ret = cnxk_ot_ipsec_outb_sa_fill(out_sa, ipsec_xfrm, crypto_xfrm);
-	if (ret)
-		return ret;
+	ret = cnxk_ot_ipsec_outb_sa_fill(sa_dptr, ipsec_xfrm, crypto_xfrm);
+	if (ret) {
+		plt_err("Could not fill outbound session parameters");
+		goto sa_dptr_free;
+	}
 
 	sa->inst.w7 = ipsec_cpt_inst_w7_get(roc_cpt, sa);
 
 #ifdef LA_IPSEC_DEBUG
 	/* Use IV from application in debug mode */
 	if (ipsec_xfrm->options.iv_gen_disable == 1) {
-		out_sa->w2.s.iv_src = ROC_IE_OT_SA_IV_SRC_FROM_SA;
+		sa_dptr->w2.s.iv_src = ROC_IE_OT_SA_IV_SRC_FROM_SA;
 		if (crypto_xfrm->type == RTE_CRYPTO_SYM_XFORM_AEAD) {
 			sa->iv_offset = crypto_xfrm->aead.iv.offset;
 			sa->iv_length = crypto_xfrm->aead.iv.length;
@@ -73,14 +84,15 @@ cn10k_ipsec_outb_sa_create(struct roc_cpt *roc_cpt,
 #else
 	if (ipsec_xfrm->options.iv_gen_disable != 0) {
 		plt_err("Application provided IV not supported");
-		return -ENOTSUP;
+		ret = -ENOTSUP;
+		goto sa_dptr_free;
 	}
 #endif
 
 	/* Get Rlen calculation data */
 	ret = cnxk_ipsec_outb_rlens_get(&rlens, ipsec_xfrm, crypto_xfrm);
 	if (ret)
-		return ret;
+		goto sa_dptr_free;
 
 	sa->max_extended_len = rlens.max_extended_len;
 
@@ -110,37 +122,61 @@ cn10k_ipsec_outb_sa_create(struct roc_cpt *roc_cpt,
 
 	sa->inst.w4 = inst_w4.u64;
 
-	return 0;
+	memset(out_sa, 0, sizeof(struct roc_ot_ipsec_outb_sa));
+
+	/* Copy word0 from sa_dptr to populate ctx_push_sz ctx_size fields */
+	memcpy(out_sa, sa_dptr, 8);
+
+	/* Write session using microcode opcode */
+	ret = roc_cpt_ctx_write(lf, sa_dptr, out_sa,
+				ROC_NIX_INL_OT_IPSEC_OUTB_HW_SZ);
+	if (ret) {
+		plt_err("Could not write outbound session to hardware");
+		goto sa_dptr_free;
+	}
+
+	/* Trigger CTX flush to write dirty data back to DRAM */
+	roc_cpt_lf_ctx_flush(lf, out_sa, false);
+
+sa_dptr_free:
+	plt_free(sa_dptr);
+
+	return ret;
 }
 
 static int
-cn10k_ipsec_inb_sa_create(struct roc_cpt *roc_cpt,
+cn10k_ipsec_inb_sa_create(struct roc_cpt *roc_cpt, struct roc_cpt_lf *lf,
 			  struct rte_security_ipsec_xform *ipsec_xfrm,
 			  struct rte_crypto_sym_xform *crypto_xfrm,
 			  struct rte_security_session *sec_sess)
 {
 	union roc_ot_ipsec_inb_param1 param1;
-	struct roc_ot_ipsec_inb_sa *in_sa;
+	struct roc_ot_ipsec_inb_sa *sa_dptr;
 	struct cn10k_sec_session *sess;
 	struct cn10k_ipsec_sa *sa;
 	union cpt_inst_w4 inst_w4;
-	int ret;
+	void *in_sa;
+	int ret = 0;
 
 	sess = get_sec_session_private_data(sec_sess);
 	sa = &sess->sa;
 	in_sa = &sa->in_sa;
 
-	memset(in_sa, 0, sizeof(struct roc_ot_ipsec_inb_sa));
-
-	/* Translate security parameters to SA */
-	ret = cnxk_ot_ipsec_inb_sa_fill(in_sa, ipsec_xfrm, crypto_xfrm);
-	if (ret)
-		return ret;
+	/* Allocate memory to be used as dptr for CPT ucode WRITE_SA op */
+	sa_dptr = plt_zmalloc(ROC_NIX_INL_OT_IPSEC_INB_HW_SZ, 0);
+	if (sa_dptr == NULL) {
+		plt_err("Couldn't allocate memory for SA dptr");
+		return -ENOMEM;
+	}
 
-	/* TODO add support for antireplay */
-	sa->in_sa.w0.s.ar_win = 0;
+	memset(sa_dptr, 0, sizeof(struct roc_ot_ipsec_inb_sa));
 
-	/* TODO add support for udp encap */
+	/* Translate security parameters to SA */
+	ret = cnxk_ot_ipsec_inb_sa_fill(sa_dptr, ipsec_xfrm, crypto_xfrm);
+	if (ret) {
+		plt_err("Could not fill inbound session parameters");
+		goto sa_dptr_free;
+	}
 
 	sa->inst.w7 = ipsec_cpt_inst_w7_get(roc_cpt, sa);
 
@@ -173,7 +209,26 @@ cn10k_ipsec_inb_sa_create(struct roc_cpt *roc_cpt,
 
 	sa->inst.w4 = inst_w4.u64;
 
-	return 0;
+	memset(in_sa, 0, sizeof(struct roc_ot_ipsec_inb_sa));
+
+	/* Copy word0 from sa_dptr to populate ctx_push_sz ctx_size fields */
+	memcpy(in_sa, sa_dptr, 8);
+
+	/* Write session using microcode opcode */
+	ret = roc_cpt_ctx_write(lf, sa_dptr, in_sa,
+				ROC_NIX_INL_OT_IPSEC_INB_HW_SZ);
+	if (ret) {
+		plt_err("Could not write inbound session to hardware");
+		goto sa_dptr_free;
+	}
+
+	/* Trigger CTX flush to write dirty data back to DRAM */
+	roc_cpt_lf_ctx_flush(lf, in_sa, false);
+
+sa_dptr_free:
+	plt_free(sa_dptr);
+
+	return ret;
 }
 
 static int
@@ -185,12 +240,11 @@ cn10k_ipsec_session_create(void *dev,
 	struct rte_cryptodev *crypto_dev = dev;
 	struct roc_cpt *roc_cpt;
 	struct cnxk_cpt_vf *vf;
+	struct cnxk_cpt_qp *qp;
 	int ret;
 
-	vf = crypto_dev->data->dev_private;
-	roc_cpt = &vf->cpt;
-
-	if (crypto_dev->data->queue_pairs[0] == NULL) {
+	qp = crypto_dev->data->queue_pairs[0];
+	if (qp == NULL) {
 		plt_err("Setup cpt queue pair before creating security session");
 		return -EPERM;
 	}
@@ -199,11 +253,14 @@ cn10k_ipsec_session_create(void *dev,
 	if (ret)
 		return ret;
 
+	vf = crypto_dev->data->dev_private;
+	roc_cpt = &vf->cpt;
+
 	if (ipsec_xfrm->direction == RTE_SECURITY_IPSEC_SA_DIR_INGRESS)
-		return cn10k_ipsec_inb_sa_create(roc_cpt, ipsec_xfrm,
+		return cn10k_ipsec_inb_sa_create(roc_cpt, &qp->lf, ipsec_xfrm,
 						 crypto_xfrm, sess);
 	else
-		return cn10k_ipsec_outb_sa_create(roc_cpt, ipsec_xfrm,
+		return cn10k_ipsec_outb_sa_create(roc_cpt, &qp->lf, ipsec_xfrm,
 						  crypto_xfrm, sess);
 }
 
-- 
2.7.4

[PATCH 11/25] crypto/cnxk: support cnxk lookaside IPsec HMAC-SHA384/512

[Anoob Joseph]

From: Tejasree Kondoj <ktejasree@marvell.com>

Adding HMAC-SHA384/512 support to cnxk lookaside IPsec.

Signed-off-by: Tejasree Kondoj <ktejasree@marvell.com>
---
 doc/guides/cryptodevs/cnxk.rst                    |  4 ++
 doc/guides/rel_notes/release_22_03.rst            |  2 +
 drivers/common/cnxk/cnxk_security.c               | 36 +++++++++------
 drivers/crypto/cnxk/cn9k_ipsec.c                  | 55 ++++++++++++++++++-----
 drivers/crypto/cnxk/cnxk_cryptodev.h              |  2 +-
 drivers/crypto/cnxk/cnxk_cryptodev_capabilities.c | 40 +++++++++++++++++
 drivers/crypto/cnxk/cnxk_ipsec.h                  |  6 +++
 7 files changed, 118 insertions(+), 27 deletions(-)

diff --git a/doc/guides/cryptodevs/cnxk.rst b/doc/guides/cryptodevs/cnxk.rst
index 8c4c4ea..c49a779 100644
--- a/doc/guides/cryptodevs/cnxk.rst
+++ b/doc/guides/cryptodevs/cnxk.rst
@@ -267,6 +267,8 @@ Auth algorithms
 
 * SHA1-HMAC
 * SHA256-128-HMAC
+* SHA384-192-HMAC
+* SHA512-256-HMAC
 
 CN10XX Features supported
 ~~~~~~~~~~~~~~~~~~~~~~~~~
@@ -293,3 +295,5 @@ Auth algorithms
 * NULL
 * SHA1-HMAC
 * SHA256-128-HMAC
+* SHA384-192-HMAC
+* SHA512-256-HMAC
diff --git a/doc/guides/rel_notes/release_22_03.rst b/doc/guides/rel_notes/release_22_03.rst
index 1639b0e..8df9092 100644
--- a/doc/guides/rel_notes/release_22_03.rst
+++ b/doc/guides/rel_notes/release_22_03.rst
@@ -58,6 +58,8 @@ New Features
 * **Updated Marvell cnxk crypto PMD.**
 
   * Added SHA256-HMAC support in lookaside protocol (IPsec) for CN10K.
+  * Added SHA384-HMAC support in lookaside protocol (IPsec) for CN9K & CN10K.
+  * Added SHA512-HMAC support in lookaside protocol (IPsec) for CN9K & CN10K.
 
 
 Removed Items
diff --git a/drivers/common/cnxk/cnxk_security.c b/drivers/common/cnxk/cnxk_security.c
index f39bc1e..1c86f82 100644
--- a/drivers/common/cnxk/cnxk_security.c
+++ b/drivers/common/cnxk/cnxk_security.c
@@ -36,6 +36,14 @@ ipsec_hmac_opad_ipad_gen(struct rte_crypto_sym_xform *auth_xform,
 		roc_hash_sha256_gen(opad, (uint32_t *)&hmac_opad_ipad[0]);
 		roc_hash_sha256_gen(ipad, (uint32_t *)&hmac_opad_ipad[64]);
 		break;
+	case RTE_CRYPTO_AUTH_SHA384_HMAC:
+		roc_hash_sha512_gen(opad, (uint64_t *)&hmac_opad_ipad[0], 384);
+		roc_hash_sha512_gen(ipad, (uint64_t *)&hmac_opad_ipad[64], 384);
+		break;
+	case RTE_CRYPTO_AUTH_SHA512_HMAC:
+		roc_hash_sha512_gen(opad, (uint64_t *)&hmac_opad_ipad[0], 512);
+		roc_hash_sha512_gen(ipad, (uint64_t *)&hmac_opad_ipad[64], 512);
+		break;
 	default:
 		break;
 	}
@@ -125,28 +133,28 @@ ot_ipsec_sa_common_param_fill(union roc_ot_ipsec_sa_word2 *w2,
 			break;
 		case RTE_CRYPTO_AUTH_SHA1_HMAC:
 			w2->s.auth_type = ROC_IE_OT_SA_AUTH_SHA1;
-			ipsec_hmac_opad_ipad_gen(auth_xfrm, hmac_opad_ipad);
-
-			tmp_key = (uint64_t *)hmac_opad_ipad;
-			for (i = 0; i < (int)(ROC_CTX_MAX_OPAD_IPAD_LEN /
-					      sizeof(uint64_t));
-			     i++)
-				tmp_key[i] = rte_be_to_cpu_64(tmp_key[i]);
 			break;
 		case RTE_CRYPTO_AUTH_SHA256_HMAC:
 			w2->s.auth_type = ROC_IE_OT_SA_AUTH_SHA2_256;
-			ipsec_hmac_opad_ipad_gen(auth_xfrm, hmac_opad_ipad);
-
-			tmp_key = (uint64_t *)hmac_opad_ipad;
-			for (i = 0; i < (int)(ROC_CTX_MAX_OPAD_IPAD_LEN /
-					      sizeof(uint64_t));
-			     i++)
-				tmp_key[i] = rte_be_to_cpu_64(tmp_key[i]);
+			break;
+		case RTE_CRYPTO_AUTH_SHA384_HMAC:
+			w2->s.auth_type = ROC_IE_OT_SA_AUTH_SHA2_384;
+			break;
+		case RTE_CRYPTO_AUTH_SHA512_HMAC:
+			w2->s.auth_type = ROC_IE_OT_SA_AUTH_SHA2_512;
 			break;
 		default:
 			return -ENOTSUP;
 		}
 
+		ipsec_hmac_opad_ipad_gen(auth_xfrm, hmac_opad_ipad);
+
+		tmp_key = (uint64_t *)hmac_opad_ipad;
+		for (i = 0;
+		     i < (int)(ROC_CTX_MAX_OPAD_IPAD_LEN / sizeof(uint64_t));
+		     i++)
+			tmp_key[i] = rte_be_to_cpu_64(tmp_key[i]);
+
 		key = cipher_xfrm->cipher.key.data;
 		length = cipher_xfrm->cipher.key.length;
 	}
diff --git a/drivers/crypto/cnxk/cn9k_ipsec.c b/drivers/crypto/cnxk/cn9k_ipsec.c
index 6455ef9..395b0d5 100644
--- a/drivers/crypto/cnxk/cn9k_ipsec.c
+++ b/drivers/crypto/cnxk/cn9k_ipsec.c
@@ -321,14 +321,23 @@ cn9k_ipsec_outb_sa_create(struct cnxk_cpt_qp *qp,
 	    ctl->auth_type == ROC_IE_ON_SA_AUTH_NULL) {
 		template = &out_sa->aes_gcm.template;
 		ctx_len = offsetof(struct roc_ie_on_outb_sa, aes_gcm.template);
-	} else if (ctl->auth_type == ROC_IE_ON_SA_AUTH_SHA1) {
-		template = &out_sa->sha1.template;
-		ctx_len = offsetof(struct roc_ie_on_outb_sa, sha1.template);
-	} else if (ctl->auth_type == ROC_IE_ON_SA_AUTH_SHA2_256) {
-		template = &out_sa->sha2.template;
-		ctx_len = offsetof(struct roc_ie_on_outb_sa, sha2.template);
 	} else {
-		return -EINVAL;
+		switch (ctl->auth_type) {
+		case ROC_IE_ON_SA_AUTH_SHA1:
+			template = &out_sa->sha1.template;
+			ctx_len = offsetof(struct roc_ie_on_outb_sa,
+					   sha1.template);
+			break;
+		case ROC_IE_ON_SA_AUTH_SHA2_256:
+		case ROC_IE_ON_SA_AUTH_SHA2_384:
+		case ROC_IE_ON_SA_AUTH_SHA2_512:
+			template = &out_sa->sha2.template;
+			ctx_len = offsetof(struct roc_ie_on_outb_sa,
+					   sha2.template);
+			break;
+		default:
+			return -EINVAL;
+		}
 	}
 
 	ip4 = (struct rte_ipv4_hdr *)&template->ip4.ipv4_hdr;
@@ -397,10 +406,22 @@ cn9k_ipsec_outb_sa_create(struct cnxk_cpt_qp *qp,
 		auth_key = auth_xform->auth.key.data;
 		auth_key_len = auth_xform->auth.key.length;
 
-		if (auth_xform->auth.algo == RTE_CRYPTO_AUTH_SHA1_HMAC)
+		switch (auth_xform->auth.algo) {
+		case RTE_CRYPTO_AUTH_NULL:
+			break;
+		case RTE_CRYPTO_AUTH_SHA1_HMAC:
 			memcpy(out_sa->sha1.hmac_key, auth_key, auth_key_len);
-		else if (auth_xform->auth.algo == RTE_CRYPTO_AUTH_SHA256_HMAC)
+			break;
+		case RTE_CRYPTO_AUTH_SHA256_HMAC:
+		case RTE_CRYPTO_AUTH_SHA384_HMAC:
+		case RTE_CRYPTO_AUTH_SHA512_HMAC:
 			memcpy(out_sa->sha2.hmac_key, auth_key, auth_key_len);
+			break;
+		default:
+			plt_err("Unsupported auth algorithm %u",
+				auth_xform->auth.algo);
+			return -ENOTSUP;
+		}
 	}
 
 	inst_tmpl = &sa->inst;
@@ -466,16 +487,26 @@ cn9k_ipsec_inb_sa_create(struct cnxk_cpt_qp *qp,
 		auth_key = auth_xform->auth.key.data;
 		auth_key_len = auth_xform->auth.key.length;
 
-		if (auth_xform->auth.algo == RTE_CRYPTO_AUTH_SHA1_HMAC) {
+		switch (auth_xform->auth.algo) {
+		case RTE_CRYPTO_AUTH_NULL:
+			break;
+		case RTE_CRYPTO_AUTH_SHA1_HMAC:
 			memcpy(in_sa->sha1_or_gcm.hmac_key, auth_key,
 			       auth_key_len);
 			ctx_len = offsetof(struct roc_ie_on_inb_sa,
 					   sha1_or_gcm.selector);
-		} else if (auth_xform->auth.algo ==
-			   RTE_CRYPTO_AUTH_SHA256_HMAC) {
+			break;
+		case RTE_CRYPTO_AUTH_SHA256_HMAC:
+		case RTE_CRYPTO_AUTH_SHA384_HMAC:
+		case RTE_CRYPTO_AUTH_SHA512_HMAC:
 			memcpy(in_sa->sha2.hmac_key, auth_key, auth_key_len);
 			ctx_len = offsetof(struct roc_ie_on_inb_sa,
 					   sha2.selector);
+			break;
+		default:
+			plt_err("Unsupported auth algorithm %u",
+				auth_xform->auth.algo);
+			return -ENOTSUP;
 		}
 	}
 
diff --git a/drivers/crypto/cnxk/cnxk_cryptodev.h b/drivers/crypto/cnxk/cnxk_cryptodev.h
index 2e0f467..f701c26 100644
--- a/drivers/crypto/cnxk/cnxk_cryptodev.h
+++ b/drivers/crypto/cnxk/cnxk_cryptodev.h
@@ -11,7 +11,7 @@
 #include "roc_cpt.h"
 
 #define CNXK_CPT_MAX_CAPS	 34
-#define CNXK_SEC_CRYPTO_MAX_CAPS 6
+#define CNXK_SEC_CRYPTO_MAX_CAPS 8
 #define CNXK_SEC_MAX_CAPS	 5
 #define CNXK_AE_EC_ID_MAX	 8
 /**
diff --git a/drivers/crypto/cnxk/cnxk_cryptodev_capabilities.c b/drivers/crypto/cnxk/cnxk_cryptodev_capabilities.c
index 8305341..9a55474 100644
--- a/drivers/crypto/cnxk/cnxk_cryptodev_capabilities.c
+++ b/drivers/crypto/cnxk/cnxk_cryptodev_capabilities.c
@@ -817,6 +817,46 @@ static const struct rte_cryptodev_capabilities sec_caps_sha1_sha2[] = {
 			}, }
 		}, }
 	},
+	{	/* SHA384 HMAC */
+		.op = RTE_CRYPTO_OP_TYPE_SYMMETRIC,
+		{.sym = {
+			.xform_type = RTE_CRYPTO_SYM_XFORM_AUTH,
+			{.auth = {
+				.algo = RTE_CRYPTO_AUTH_SHA384_HMAC,
+				.block_size = 64,
+				.key_size = {
+					.min = 48,
+					.max = 48,
+					.increment = 0
+				},
+				.digest_size = {
+					.min = 24,
+					.max = 24,
+					.increment = 0
+					},
+			}, }
+		}, }
+	},
+	{	/* SHA512 HMAC */
+		.op = RTE_CRYPTO_OP_TYPE_SYMMETRIC,
+		{.sym = {
+			.xform_type = RTE_CRYPTO_SYM_XFORM_AUTH,
+			{.auth = {
+				.algo = RTE_CRYPTO_AUTH_SHA512_HMAC,
+				.block_size = 128,
+				.key_size = {
+					.min = 64,
+					.max = 64,
+					.increment = 0
+				},
+				.digest_size = {
+					.min = 32,
+					.max = 32,
+					.increment = 0
+				},
+			}, }
+		}, }
+	},
 };
 
 static const struct rte_security_capability sec_caps_templ[] = {
diff --git a/drivers/crypto/cnxk/cnxk_ipsec.h b/drivers/crypto/cnxk/cnxk_ipsec.h
index f4a1012..426eaa8 100644
--- a/drivers/crypto/cnxk/cnxk_ipsec.h
+++ b/drivers/crypto/cnxk/cnxk_ipsec.h
@@ -49,6 +49,12 @@ ipsec_xform_auth_verify(struct rte_crypto_sym_xform *crypto_xform)
 	} else if (crypto_xform->auth.algo == RTE_CRYPTO_AUTH_SHA256_HMAC) {
 		if (keylen >= 32 && keylen <= 64)
 			return 0;
+	} else if (crypto_xform->auth.algo == RTE_CRYPTO_AUTH_SHA384_HMAC) {
+		if (keylen == 48)
+			return 0;
+	} else if (crypto_xform->auth.algo == RTE_CRYPTO_AUTH_SHA512_HMAC) {
+		if (keylen == 64)
+			return 0;
 	}
 
 	return -ENOTSUP;
-- 
2.7.4

[PATCH 12/25] crypto/cnxk: account for CPT CTX updates and flush delays

[Anoob Joseph]

CPT CTX write with microcode would require CPT flush to complete to have
DRAM updated with the SA. Since datapath requires SA direction field,
introduce a new flag for the same.

Session destroy path is also updated to clear sa.valid bit using CTX
reload operation.

Session is updated with marker to differentiate s/w immutable and s/w
mutable portions.

Signed-off-by: Anoob Joseph <anoobj@marvell.com>
---
 drivers/crypto/cnxk/cn10k_cryptodev_ops.c |  4 +--
 drivers/crypto/cnxk/cn10k_ipsec.c         | 60 ++++++++++++++++++++++++-------
 drivers/crypto/cnxk/cn10k_ipsec.h         | 25 ++++++++-----
 drivers/crypto/cnxk/cn10k_ipsec_la_ops.h  | 18 +++++-----
 4 files changed, 76 insertions(+), 31 deletions(-)

diff --git a/drivers/crypto/cnxk/cn10k_cryptodev_ops.c b/drivers/crypto/cnxk/cn10k_cryptodev_ops.c
index d25a17c..7617bdc 100644
--- a/drivers/crypto/cnxk/cn10k_cryptodev_ops.c
+++ b/drivers/crypto/cnxk/cn10k_cryptodev_ops.c
@@ -53,7 +53,6 @@ cpt_sec_inst_fill(struct rte_crypto_op *op, struct cn10k_sec_session *sess,
 		  struct cpt_inflight_req *infl_req, struct cpt_inst_s *inst)
 {
 	struct rte_crypto_sym_op *sym_op = op->sym;
-	union roc_ot_ipsec_sa_word2 *w2;
 	struct cn10k_ipsec_sa *sa;
 	int ret;
 
@@ -68,9 +67,8 @@ cpt_sec_inst_fill(struct rte_crypto_op *op, struct cn10k_sec_session *sess,
 	}
 
 	sa = &sess->sa;
-	w2 = (union roc_ot_ipsec_sa_word2 *)&sa->in_sa.w2;
 
-	if (w2->s.dir == ROC_IE_SA_DIR_OUTBOUND)
+	if (sa->is_outbound)
 		ret = process_outb_sa(op, sa, inst);
 	else {
 		infl_req->op_flags |= CPT_OP_FLAGS_IPSEC_DIR_INBOUND;
diff --git a/drivers/crypto/cnxk/cn10k_ipsec.c b/drivers/crypto/cnxk/cn10k_ipsec.c
index a11a6b7..b4acbac 100644
--- a/drivers/crypto/cnxk/cn10k_ipsec.c
+++ b/drivers/crypto/cnxk/cn10k_ipsec.c
@@ -67,7 +67,7 @@ cn10k_ipsec_outb_sa_create(struct roc_cpt *roc_cpt, struct roc_cpt_lf *lf,
 		goto sa_dptr_free;
 	}
 
-	sa->inst.w7 = ipsec_cpt_inst_w7_get(roc_cpt, sa);
+	sa->inst.w7 = ipsec_cpt_inst_w7_get(roc_cpt, out_sa);
 
 #ifdef LA_IPSEC_DEBUG
 	/* Use IV from application in debug mode */
@@ -89,6 +89,8 @@ cn10k_ipsec_outb_sa_create(struct roc_cpt *roc_cpt, struct roc_cpt_lf *lf,
 	}
 #endif
 
+	sa->is_outbound = true;
+
 	/* Get Rlen calculation data */
 	ret = cnxk_ipsec_outb_rlens_get(&rlens, ipsec_xfrm, crypto_xfrm);
 	if (ret)
@@ -127,6 +129,8 @@ cn10k_ipsec_outb_sa_create(struct roc_cpt *roc_cpt, struct roc_cpt_lf *lf,
 	/* Copy word0 from sa_dptr to populate ctx_push_sz ctx_size fields */
 	memcpy(out_sa, sa_dptr, 8);
 
+	plt_atomic_thread_fence(__ATOMIC_SEQ_CST);
+
 	/* Write session using microcode opcode */
 	ret = roc_cpt_ctx_write(lf, sa_dptr, out_sa,
 				ROC_NIX_INL_OT_IPSEC_OUTB_HW_SZ);
@@ -135,9 +139,11 @@ cn10k_ipsec_outb_sa_create(struct roc_cpt *roc_cpt, struct roc_cpt_lf *lf,
 		goto sa_dptr_free;
 	}
 
-	/* Trigger CTX flush to write dirty data back to DRAM */
+	/* Trigger CTX flush so that data is written back to DRAM */
 	roc_cpt_lf_ctx_flush(lf, out_sa, false);
 
+	plt_atomic_thread_fence(__ATOMIC_SEQ_CST);
+
 sa_dptr_free:
 	plt_free(sa_dptr);
 
@@ -178,7 +184,8 @@ cn10k_ipsec_inb_sa_create(struct roc_cpt *roc_cpt, struct roc_cpt_lf *lf,
 		goto sa_dptr_free;
 	}
 
-	sa->inst.w7 = ipsec_cpt_inst_w7_get(roc_cpt, sa);
+	sa->is_outbound = false;
+	sa->inst.w7 = ipsec_cpt_inst_w7_get(roc_cpt, in_sa);
 
 	/* pre-populate CPT INST word 4 */
 	inst_w4.u64 = 0;
@@ -214,6 +221,8 @@ cn10k_ipsec_inb_sa_create(struct roc_cpt *roc_cpt, struct roc_cpt_lf *lf,
 	/* Copy word0 from sa_dptr to populate ctx_push_sz ctx_size fields */
 	memcpy(in_sa, sa_dptr, 8);
 
+	plt_atomic_thread_fence(__ATOMIC_SEQ_CST);
+
 	/* Write session using microcode opcode */
 	ret = roc_cpt_ctx_write(lf, sa_dptr, in_sa,
 				ROC_NIX_INL_OT_IPSEC_INB_HW_SZ);
@@ -222,9 +231,11 @@ cn10k_ipsec_inb_sa_create(struct roc_cpt *roc_cpt, struct roc_cpt_lf *lf,
 		goto sa_dptr_free;
 	}
 
-	/* Trigger CTX flush to write dirty data back to DRAM */
+	/* Trigger CTX flush so that data is written back to DRAM */
 	roc_cpt_lf_ctx_flush(lf, in_sa, false);
 
+	plt_atomic_thread_fence(__ATOMIC_SEQ_CST);
+
 sa_dptr_free:
 	plt_free(sa_dptr);
 
@@ -300,21 +311,46 @@ cn10k_sec_session_create(void *device, struct rte_security_session_conf *conf,
 }
 
 static int
-cn10k_sec_session_destroy(void *device __rte_unused,
-			  struct rte_security_session *sess)
+cn10k_sec_session_destroy(void *dev, struct rte_security_session *sec_sess)
 {
-	struct cn10k_sec_session *priv;
+	struct rte_cryptodev *crypto_dev = dev;
+	union roc_ot_ipsec_sa_word2 *w2;
+	struct cn10k_sec_session *sess;
 	struct rte_mempool *sess_mp;
+	struct cn10k_ipsec_sa *sa;
+	struct cnxk_cpt_qp *qp;
+	struct roc_cpt_lf *lf;
 
-	priv = get_sec_session_private_data(sess);
+	sess = get_sec_session_private_data(sec_sess);
+	if (sess == NULL)
+		return 0;
 
-	if (priv == NULL)
+	qp = crypto_dev->data->queue_pairs[0];
+	if (qp == NULL)
 		return 0;
 
-	sess_mp = rte_mempool_from_obj(priv);
+	lf = &qp->lf;
 
-	set_sec_session_private_data(sess, NULL);
-	rte_mempool_put(sess_mp, priv);
+	sa = &sess->sa;
+
+	/* Trigger CTX flush to write dirty data back to DRAM */
+	roc_cpt_lf_ctx_flush(lf, &sa->in_sa, false);
+
+	/* Wait for 1 ms so that flush is complete */
+	rte_delay_ms(1);
+
+	w2 = (union roc_ot_ipsec_sa_word2 *)&sa->in_sa.w2;
+	w2->s.valid = 0;
+
+	plt_atomic_thread_fence(__ATOMIC_SEQ_CST);
+
+	/* Trigger CTX reload to fetch new data from DRAM */
+	roc_cpt_lf_ctx_reload(lf, &sa->in_sa);
+
+	sess_mp = rte_mempool_from_obj(sess);
+
+	set_sec_session_private_data(sec_sess, NULL);
+	rte_mempool_put(sess_mp, sess);
 
 	return 0;
 }
diff --git a/drivers/crypto/cnxk/cn10k_ipsec.h b/drivers/crypto/cnxk/cn10k_ipsec.h
index 86cd248..8be1fee 100644
--- a/drivers/crypto/cnxk/cn10k_ipsec.h
+++ b/drivers/crypto/cnxk/cn10k_ipsec.h
@@ -7,24 +7,33 @@
 
 #include <rte_security.h>
 
+#include "roc_api.h"
+
 #include "cnxk_ipsec.h"
 
-#define CN10K_IPSEC_SA_CTX_HDR_SIZE 1
+typedef void *CN10K_SA_CONTEXT_MARKER[0];
 
 struct cn10k_ipsec_sa {
-	union {
-		/** Inbound SA */
-		struct roc_ot_ipsec_inb_sa in_sa;
-		/** Outbound SA */
-		struct roc_ot_ipsec_outb_sa out_sa;
-	};
 	/** Pre-populated CPT inst words */
 	struct cnxk_cpt_inst_tmpl inst;
 	uint16_t max_extended_len;
 	uint16_t iv_offset;
 	uint8_t iv_length;
 	bool ip_csum_enable;
-};
+	bool is_outbound;
+
+	/**
+	 * End of SW mutable area
+	 */
+	CN10K_SA_CONTEXT_MARKER sw_area_end __rte_aligned(ROC_ALIGN);
+
+	union {
+		/** Inbound SA */
+		struct roc_ot_ipsec_inb_sa in_sa;
+		/** Outbound SA */
+		struct roc_ot_ipsec_outb_sa out_sa;
+	};
+} __rte_aligned(ROC_ALIGN);
 
 struct cn10k_sec_session {
 	struct cn10k_ipsec_sa sa;
diff --git a/drivers/crypto/cnxk/cn10k_ipsec_la_ops.h b/drivers/crypto/cnxk/cn10k_ipsec_la_ops.h
index 881fbd1..cab6a50 100644
--- a/drivers/crypto/cnxk/cn10k_ipsec_la_ops.h
+++ b/drivers/crypto/cnxk/cn10k_ipsec_la_ops.h
@@ -54,6 +54,7 @@ process_outb_sa(struct rte_crypto_op *cop, struct cn10k_ipsec_sa *sess,
 	struct rte_crypto_sym_op *sym_op = cop->sym;
 	struct rte_mbuf *m_src = sym_op->m_src;
 	uint64_t inst_w4_u64 = sess->inst.w4;
+	uint64_t dptr;
 
 	if (unlikely(rte_pktmbuf_tailroom(m_src) < sess->max_extended_len)) {
 		plt_dp_err("Not enough tail room");
@@ -76,10 +77,10 @@ process_outb_sa(struct rte_crypto_op *cop, struct cn10k_ipsec_sa *sess,
 		inst_w4_u64 &= ~BIT_ULL(32);
 
 	/* Prepare CPT instruction */
-	inst->w4.u64 = inst_w4_u64;
-	inst->w4.s.dlen = rte_pktmbuf_pkt_len(m_src);
-	inst->dptr = rte_pktmbuf_iova(m_src);
-	inst->rptr = inst->dptr;
+	inst->w4.u64 = inst_w4_u64 | rte_pktmbuf_pkt_len(m_src);
+	dptr = rte_pktmbuf_iova(m_src);
+	inst->dptr = dptr;
+	inst->rptr = dptr;
 
 	return 0;
 }
@@ -90,12 +91,13 @@ process_inb_sa(struct rte_crypto_op *cop, struct cn10k_ipsec_sa *sa,
 {
 	struct rte_crypto_sym_op *sym_op = cop->sym;
 	struct rte_mbuf *m_src = sym_op->m_src;
+	uint64_t dptr;
 
 	/* Prepare CPT instruction */
-	inst->w4.u64 = sa->inst.w4;
-	inst->w4.s.dlen = rte_pktmbuf_pkt_len(m_src);
-	inst->dptr = rte_pktmbuf_iova(m_src);
-	inst->rptr = inst->dptr;
+	inst->w4.u64 = sa->inst.w4 | rte_pktmbuf_pkt_len(m_src);
+	dptr = rte_pktmbuf_iova(m_src);
+	inst->dptr = dptr;
+	inst->rptr = dptr;
 
 	return 0;
 }
-- 
2.7.4

[PATCH 13/25] crypto/cnxk: use struct sizes for ctx writes

[Anoob Joseph]

CTX writes only require the lengths are 8B aligned. Use the struct size
directly.

Signed-off-by: Anoob Joseph <anoobj@marvell.com>
---
 drivers/crypto/cnxk/cn10k_ipsec.c | 12 ++++--------
 1 file changed, 4 insertions(+), 8 deletions(-)

diff --git a/drivers/crypto/cnxk/cn10k_ipsec.c b/drivers/crypto/cnxk/cn10k_ipsec.c
index b4acbac..0832b53 100644
--- a/drivers/crypto/cnxk/cn10k_ipsec.c
+++ b/drivers/crypto/cnxk/cn10k_ipsec.c
@@ -52,14 +52,12 @@ cn10k_ipsec_outb_sa_create(struct roc_cpt *roc_cpt, struct roc_cpt_lf *lf,
 	out_sa = &sa->out_sa;
 
 	/* Allocate memory to be used as dptr for CPT ucode WRITE_SA op */
-	sa_dptr = plt_zmalloc(ROC_NIX_INL_OT_IPSEC_OUTB_HW_SZ, 0);
+	sa_dptr = plt_zmalloc(sizeof(struct roc_ot_ipsec_outb_sa), 8);
 	if (sa_dptr == NULL) {
 		plt_err("Couldn't allocate memory for SA dptr");
 		return -ENOMEM;
 	}
 
-	memset(sa_dptr, 0, sizeof(struct roc_ot_ipsec_outb_sa));
-
 	/* Translate security parameters to SA */
 	ret = cnxk_ot_ipsec_outb_sa_fill(sa_dptr, ipsec_xfrm, crypto_xfrm);
 	if (ret) {
@@ -133,7 +131,7 @@ cn10k_ipsec_outb_sa_create(struct roc_cpt *roc_cpt, struct roc_cpt_lf *lf,
 
 	/* Write session using microcode opcode */
 	ret = roc_cpt_ctx_write(lf, sa_dptr, out_sa,
-				ROC_NIX_INL_OT_IPSEC_OUTB_HW_SZ);
+				sizeof(struct roc_ot_ipsec_outb_sa));
 	if (ret) {
 		plt_err("Could not write outbound session to hardware");
 		goto sa_dptr_free;
@@ -169,14 +167,12 @@ cn10k_ipsec_inb_sa_create(struct roc_cpt *roc_cpt, struct roc_cpt_lf *lf,
 	in_sa = &sa->in_sa;
 
 	/* Allocate memory to be used as dptr for CPT ucode WRITE_SA op */
-	sa_dptr = plt_zmalloc(ROC_NIX_INL_OT_IPSEC_INB_HW_SZ, 0);
+	sa_dptr = plt_zmalloc(sizeof(struct roc_ot_ipsec_inb_sa), 8);
 	if (sa_dptr == NULL) {
 		plt_err("Couldn't allocate memory for SA dptr");
 		return -ENOMEM;
 	}
 
-	memset(sa_dptr, 0, sizeof(struct roc_ot_ipsec_inb_sa));
-
 	/* Translate security parameters to SA */
 	ret = cnxk_ot_ipsec_inb_sa_fill(sa_dptr, ipsec_xfrm, crypto_xfrm);
 	if (ret) {
@@ -225,7 +221,7 @@ cn10k_ipsec_inb_sa_create(struct roc_cpt *roc_cpt, struct roc_cpt_lf *lf,
 
 	/* Write session using microcode opcode */
 	ret = roc_cpt_ctx_write(lf, sa_dptr, in_sa,
-				ROC_NIX_INL_OT_IPSEC_INB_HW_SZ);
+				sizeof(struct roc_ot_ipsec_inb_sa));
 	if (ret) {
 		plt_err("Could not write inbound session to hardware");
 		goto sa_dptr_free;
-- 
2.7.4

[PATCH 14/25] crypto/cnxk: add security session stats get

[Anoob Joseph]

From: Ankur Dwivedi <adwivedi@marvell.com>

Adds the security session stats get op for cn10k.


Signed-off-by: Ankur Dwivedi <adwivedi@marvell.com>
---
 drivers/crypto/cnxk/cn10k_ipsec.c                 | 55 +++++++++++++++++++++++
 drivers/crypto/cnxk/cnxk_cryptodev_capabilities.c |  1 +
 drivers/crypto/cnxk/cnxk_cryptodev_sec.c          |  1 +
 3 files changed, 57 insertions(+)

diff --git a/drivers/crypto/cnxk/cn10k_ipsec.c b/drivers/crypto/cnxk/cn10k_ipsec.c
index 0832b53..a93c211 100644
--- a/drivers/crypto/cnxk/cn10k_ipsec.c
+++ b/drivers/crypto/cnxk/cn10k_ipsec.c
@@ -122,6 +122,12 @@ cn10k_ipsec_outb_sa_create(struct roc_cpt *roc_cpt, struct roc_cpt_lf *lf,
 
 	sa->inst.w4 = inst_w4.u64;
 
+	if (ipsec_xfrm->options.stats == 1) {
+		/* Enable mib counters */
+		sa_dptr->w0.s.count_mib_bytes = 1;
+		sa_dptr->w0.s.count_mib_pkts = 1;
+	}
+
 	memset(out_sa, 0, sizeof(struct roc_ot_ipsec_outb_sa));
 
 	/* Copy word0 from sa_dptr to populate ctx_push_sz ctx_size fields */
@@ -212,6 +218,12 @@ cn10k_ipsec_inb_sa_create(struct roc_cpt *roc_cpt, struct roc_cpt_lf *lf,
 
 	sa->inst.w4 = inst_w4.u64;
 
+	if (ipsec_xfrm->options.stats == 1) {
+		/* Enable mib counters */
+		sa_dptr->w0.s.count_mib_bytes = 1;
+		sa_dptr->w0.s.count_mib_pkts = 1;
+	}
+
 	memset(in_sa, 0, sizeof(struct roc_ot_ipsec_inb_sa));
 
 	/* Copy word0 from sa_dptr to populate ctx_push_sz ctx_size fields */
@@ -357,6 +369,48 @@ cn10k_sec_session_get_size(void *device __rte_unused)
 	return sizeof(struct cn10k_sec_session);
 }
 
+static int
+cn10k_sec_session_stats_get(void *device, struct rte_security_session *sess,
+			    struct rte_security_stats *stats)
+{
+	struct rte_cryptodev *crypto_dev = device;
+	struct roc_ot_ipsec_outb_sa *out_sa;
+	struct roc_ot_ipsec_inb_sa *in_sa;
+	union roc_ot_ipsec_sa_word2 *w2;
+	struct cn10k_sec_session *priv;
+	struct cn10k_ipsec_sa *sa;
+	struct cnxk_cpt_qp *qp;
+
+	priv = get_sec_session_private_data(sess);
+	if (priv == NULL)
+		return -EINVAL;
+
+	qp = crypto_dev->data->queue_pairs[0];
+	if (qp == NULL)
+		return -EINVAL;
+
+	sa = &priv->sa;
+	w2 = (union roc_ot_ipsec_sa_word2 *)&sa->in_sa.w2;
+
+	stats->protocol = RTE_SECURITY_PROTOCOL_IPSEC;
+
+	if (w2->s.dir == ROC_IE_SA_DIR_OUTBOUND) {
+		out_sa = &sa->out_sa;
+		roc_cpt_lf_ctx_flush(&qp->lf, out_sa, false);
+		rte_delay_ms(1);
+		stats->ipsec.opackets = out_sa->ctx.mib_pkts;
+		stats->ipsec.obytes = out_sa->ctx.mib_octs;
+	} else {
+		in_sa = &sa->in_sa;
+		roc_cpt_lf_ctx_flush(&qp->lf, in_sa, false);
+		rte_delay_ms(1);
+		stats->ipsec.ipackets = in_sa->ctx.mib_pkts;
+		stats->ipsec.ibytes = in_sa->ctx.mib_octs;
+	}
+
+	return 0;
+}
+
 /* Update platform specific security ops */
 void
 cn10k_sec_ops_override(void)
@@ -365,4 +419,5 @@ cn10k_sec_ops_override(void)
 	cnxk_sec_ops.session_create = cn10k_sec_session_create;
 	cnxk_sec_ops.session_destroy = cn10k_sec_session_destroy;
 	cnxk_sec_ops.session_get_size = cn10k_sec_session_get_size;
+	cnxk_sec_ops.session_stats_get = cn10k_sec_session_stats_get;
 }
diff --git a/drivers/crypto/cnxk/cnxk_cryptodev_capabilities.c b/drivers/crypto/cnxk/cnxk_cryptodev_capabilities.c
index 9a55474..0fdd91a 100644
--- a/drivers/crypto/cnxk/cnxk_cryptodev_capabilities.c
+++ b/drivers/crypto/cnxk/cnxk_cryptodev_capabilities.c
@@ -1073,6 +1073,7 @@ cn10k_sec_caps_update(struct rte_security_capability *sec_cap)
 	}
 	sec_cap->ipsec.options.ip_csum_enable = 1;
 	sec_cap->ipsec.options.l4_csum_enable = 1;
+	sec_cap->ipsec.options.stats = 1;
 }
 
 static void
diff --git a/drivers/crypto/cnxk/cnxk_cryptodev_sec.c b/drivers/crypto/cnxk/cnxk_cryptodev_sec.c
index 2021d5c..e5a5d2d 100644
--- a/drivers/crypto/cnxk/cnxk_cryptodev_sec.c
+++ b/drivers/crypto/cnxk/cnxk_cryptodev_sec.c
@@ -15,6 +15,7 @@ struct rte_security_ops cnxk_sec_ops = {
 	.session_create = NULL,
 	.session_destroy = NULL,
 	.session_get_size = NULL,
+	.session_stats_get = NULL,
 	.set_pkt_metadata = NULL,
 	.get_userdata = NULL,
 	.capabilities_get = cnxk_crypto_sec_capabilities_get
-- 
2.7.4

[PATCH 15/25] crypto/cnxk: add skip for unsupported cases

[Anoob Joseph]

Add skip for transport mode tests that are not supported. Also, updated the
transport mode path to configure IP version as v4.

Signed-off-by: Anoob Joseph <anoobj@marvell.com>
---
 drivers/crypto/cnxk/cn9k_ipsec.c | 53 +++++++++++++++++++++++++++++++++++-----
 1 file changed, 47 insertions(+), 6 deletions(-)

diff --git a/drivers/crypto/cnxk/cn9k_ipsec.c b/drivers/crypto/cnxk/cn9k_ipsec.c
index 395b0d5..3c6c8e9 100644
--- a/drivers/crypto/cnxk/cn9k_ipsec.c
+++ b/drivers/crypto/cnxk/cn9k_ipsec.c
@@ -141,11 +141,10 @@ ipsec_sa_ctl_set(struct rte_security_ipsec_xform *ipsec,
 			return -EINVAL;
 	}
 
-	ctl->inner_ip_ver = ctl->outer_ip_ver;
-
-	if (ipsec->mode == RTE_SECURITY_IPSEC_SA_MODE_TRANSPORT)
+	if (ipsec->mode == RTE_SECURITY_IPSEC_SA_MODE_TRANSPORT) {
 		ctl->ipsec_mode = ROC_IE_SA_MODE_TRANSPORT;
-	else if (ipsec->mode == RTE_SECURITY_IPSEC_SA_MODE_TUNNEL)
+		ctl->outer_ip_ver = ROC_IE_SA_IP_VERSION_4;
+	} else if (ipsec->mode == RTE_SECURITY_IPSEC_SA_MODE_TUNNEL)
 		ctl->ipsec_mode = ROC_IE_SA_MODE_TUNNEL;
 	else
 		return -EINVAL;
@@ -548,7 +547,8 @@ cn9k_ipsec_inb_sa_create(struct cnxk_cpt_qp *qp,
 }
 
 static inline int
-cn9k_ipsec_xform_verify(struct rte_security_ipsec_xform *ipsec)
+cn9k_ipsec_xform_verify(struct rte_security_ipsec_xform *ipsec,
+			struct rte_crypto_sym_xform *crypto)
 {
 	if (ipsec->life.bytes_hard_limit != 0 ||
 	    ipsec->life.bytes_soft_limit != 0 ||
@@ -556,6 +556,47 @@ cn9k_ipsec_xform_verify(struct rte_security_ipsec_xform *ipsec)
 	    ipsec->life.packets_soft_limit != 0)
 		return -ENOTSUP;
 
+	if (ipsec->mode == RTE_SECURITY_IPSEC_SA_MODE_TRANSPORT) {
+		enum rte_crypto_sym_xform_type type = crypto->type;
+
+		if (type == RTE_CRYPTO_SYM_XFORM_AEAD) {
+			if ((crypto->aead.algo == RTE_CRYPTO_AEAD_AES_GCM) &&
+			    (crypto->aead.key.length == 32)) {
+				plt_err("Transport mode AES-256-GCM is not supported");
+				return -ENOTSUP;
+			}
+		} else {
+			struct rte_crypto_cipher_xform *cipher;
+			struct rte_crypto_auth_xform *auth;
+
+			if (crypto->type == RTE_CRYPTO_SYM_XFORM_CIPHER) {
+				cipher = &crypto->cipher;
+				auth = &crypto->next->auth;
+			} else {
+				cipher = &crypto->next->cipher;
+				auth = &crypto->auth;
+			}
+
+			if ((cipher->algo == RTE_CRYPTO_CIPHER_AES_CBC) &&
+			    (auth->algo == RTE_CRYPTO_AUTH_SHA256_HMAC)) {
+				plt_err("Transport mode AES-CBC SHA1 HMAC 256 is not supported");
+				return -ENOTSUP;
+			}
+
+			if ((cipher->algo == RTE_CRYPTO_CIPHER_AES_CBC) &&
+			    (auth->algo == RTE_CRYPTO_AUTH_SHA384_HMAC)) {
+				plt_err("Transport mode AES-CBC SHA2 HMAC 384 is not supported");
+				return -ENOTSUP;
+			}
+
+			if ((cipher->algo == RTE_CRYPTO_CIPHER_AES_CBC) &&
+			    (auth->algo == RTE_CRYPTO_AUTH_SHA512_HMAC)) {
+				plt_err("Transport mode AES-CBC SHA2 HMAC 512 is not supported");
+				return -ENOTSUP;
+			}
+		}
+	}
+
 	return 0;
 }
 
@@ -580,7 +621,7 @@ cn9k_ipsec_session_create(void *dev,
 	if (ret)
 		return ret;
 
-	ret = cn9k_ipsec_xform_verify(ipsec_xform);
+	ret = cn9k_ipsec_xform_verify(ipsec_xform, crypto_xform);
 	if (ret)
 		return ret;
 
-- 
2.7.4

[PATCH 16/25] crypto/cnxk: add context reload for IV

[Anoob Joseph]

From: Tejasree Kondoj <ktejasree@marvell.com>

Adding context reload in datapath for IV in debug mode.

Signed-off-by: Tejasree Kondoj <ktejasree@marvell.com>
---
 drivers/crypto/cnxk/cn10k_cryptodev_ops.c |  7 ++++---
 drivers/crypto/cnxk/cn10k_ipsec_la_ops.h  | 10 ++++++++--
 2 files changed, 12 insertions(+), 5 deletions(-)

diff --git a/drivers/crypto/cnxk/cn10k_cryptodev_ops.c b/drivers/crypto/cnxk/cn10k_cryptodev_ops.c
index 7617bdc..638268e 100644
--- a/drivers/crypto/cnxk/cn10k_cryptodev_ops.c
+++ b/drivers/crypto/cnxk/cn10k_cryptodev_ops.c
@@ -49,7 +49,8 @@ cn10k_cpt_sym_temp_sess_create(struct cnxk_cpt_qp *qp, struct rte_crypto_op *op)
 }
 
 static __rte_always_inline int __rte_hot
-cpt_sec_inst_fill(struct rte_crypto_op *op, struct cn10k_sec_session *sess,
+cpt_sec_inst_fill(struct cnxk_cpt_qp *qp, struct rte_crypto_op *op,
+		  struct cn10k_sec_session *sess,
 		  struct cpt_inflight_req *infl_req, struct cpt_inst_s *inst)
 {
 	struct rte_crypto_sym_op *sym_op = op->sym;
@@ -69,7 +70,7 @@ cpt_sec_inst_fill(struct rte_crypto_op *op, struct cn10k_sec_session *sess,
 	sa = &sess->sa;
 
 	if (sa->is_outbound)
-		ret = process_outb_sa(op, sa, inst);
+		ret = process_outb_sa(&qp->lf, op, sa, inst);
 	else {
 		infl_req->op_flags |= CPT_OP_FLAGS_IPSEC_DIR_INBOUND;
 		ret = process_inb_sa(op, sa, inst);
@@ -122,7 +123,7 @@ cn10k_cpt_fill_inst(struct cnxk_cpt_qp *qp, struct rte_crypto_op *ops[],
 		if (op->sess_type == RTE_CRYPTO_OP_SECURITY_SESSION) {
 			sec_sess = get_sec_session_private_data(
 				sym_op->sec_session);
-			ret = cpt_sec_inst_fill(op, sec_sess, infl_req,
+			ret = cpt_sec_inst_fill(qp, op, sec_sess, infl_req,
 						&inst[0]);
 			if (unlikely(ret))
 				return 0;
diff --git a/drivers/crypto/cnxk/cn10k_ipsec_la_ops.h b/drivers/crypto/cnxk/cn10k_ipsec_la_ops.h
index cab6a50..f2d8122 100644
--- a/drivers/crypto/cnxk/cn10k_ipsec_la_ops.h
+++ b/drivers/crypto/cnxk/cn10k_ipsec_la_ops.h
@@ -48,8 +48,8 @@ ipsec_po_sa_aes_gcm_iv_set(struct cn10k_ipsec_sa *sess,
 }
 
 static __rte_always_inline int
-process_outb_sa(struct rte_crypto_op *cop, struct cn10k_ipsec_sa *sess,
-		struct cpt_inst_s *inst)
+process_outb_sa(struct roc_cpt_lf *lf, struct rte_crypto_op *cop,
+		struct cn10k_ipsec_sa *sess, struct cpt_inst_s *inst)
 {
 	struct rte_crypto_sym_op *sym_op = cop->sym;
 	struct rte_mbuf *m_src = sym_op->m_src;
@@ -61,6 +61,8 @@ process_outb_sa(struct rte_crypto_op *cop, struct cn10k_ipsec_sa *sess,
 		return -ENOMEM;
 	}
 
+	RTE_SET_USED(lf);
+
 #ifdef LA_IPSEC_DEBUG
 	if (sess->out_sa.w2.s.iv_src == ROC_IE_OT_SA_IV_SRC_FROM_SA) {
 		if (sess->out_sa.w2.s.enc_type == ROC_IE_OT_SA_ENC_AES_GCM)
@@ -68,6 +70,10 @@ process_outb_sa(struct rte_crypto_op *cop, struct cn10k_ipsec_sa *sess,
 		else
 			ipsec_po_sa_iv_set(sess, cop);
 	}
+
+	/* Trigger CTX reload to fetch new data from DRAM */
+	roc_cpt_lf_ctx_reload(lf, &sess->out_sa);
+	rte_delay_ms(1);
 #endif
 
 	if (m_src->ol_flags & RTE_MBUF_F_TX_IP_CKSUM)
-- 
2.7.4

[PATCH 17/25] crypto/cnxk: handle null chained ops

[Anoob Joseph]

Verification doesn't cover cases when NULL auth/cipher is provided as a
chain. Removed the separate function for verification and added a
replacement function which calls the appropriate downstream functions.

Signed-off-by: Anoob Joseph <anoobj@marvell.com>
---
 drivers/crypto/cnxk/cnxk_cryptodev_ops.c | 189 ++++++++++++++++---------------
 drivers/crypto/cnxk/cnxk_cryptodev_ops.h |  10 --
 drivers/crypto/cnxk/cnxk_se.h            |   6 +
 3 files changed, 102 insertions(+), 103 deletions(-)

diff --git a/drivers/crypto/cnxk/cnxk_cryptodev_ops.c b/drivers/crypto/cnxk/cnxk_cryptodev_ops.c
index 21ee09f..b02f070 100644
--- a/drivers/crypto/cnxk/cnxk_cryptodev_ops.c
+++ b/drivers/crypto/cnxk/cnxk_cryptodev_ops.c
@@ -418,84 +418,121 @@ cnxk_cpt_sym_session_get_size(struct rte_cryptodev *dev __rte_unused)
 }
 
 static int
-sym_xform_verify(struct rte_crypto_sym_xform *xform)
+cnxk_sess_fill(struct rte_crypto_sym_xform *xform, struct cnxk_se_sess *sess)
 {
-	if (xform->type == RTE_CRYPTO_SYM_XFORM_AUTH &&
-	    xform->auth.algo == RTE_CRYPTO_AUTH_NULL &&
-	    xform->auth.op == RTE_CRYPTO_AUTH_OP_VERIFY)
-		return -ENOTSUP;
+	struct rte_crypto_sym_xform *aead_xfrm = NULL;
+	struct rte_crypto_sym_xform *c_xfrm = NULL;
+	struct rte_crypto_sym_xform *a_xfrm = NULL;
+	bool ciph_then_auth;
 
-	if (xform->type == RTE_CRYPTO_SYM_XFORM_CIPHER && xform->next == NULL)
-		return CNXK_CPT_CIPHER;
+	if (xform == NULL)
+		return -EINVAL;
 
-	if (xform->type == RTE_CRYPTO_SYM_XFORM_AUTH && xform->next == NULL)
-		return CNXK_CPT_AUTH;
+	if (xform->type == RTE_CRYPTO_SYM_XFORM_CIPHER) {
+		c_xfrm = xform;
+		a_xfrm = xform->next;
+		ciph_then_auth = true;
+	} else if (xform->type == RTE_CRYPTO_SYM_XFORM_AUTH) {
+		c_xfrm = xform->next;
+		a_xfrm = xform;
+		ciph_then_auth = false;
+	} else {
+		aead_xfrm = xform;
+	}
 
-	if (xform->type == RTE_CRYPTO_SYM_XFORM_AEAD && xform->next == NULL)
-		return CNXK_CPT_AEAD;
+	if (c_xfrm != NULL && c_xfrm->type != RTE_CRYPTO_SYM_XFORM_CIPHER) {
+		plt_dp_err("Invalid type in cipher xform");
+		return -EINVAL;
+	}
 
-	if (xform->next == NULL)
-		return -EIO;
+	if (a_xfrm != NULL && a_xfrm->type != RTE_CRYPTO_SYM_XFORM_AUTH) {
+		plt_dp_err("Invalid type in auth xform");
+		return -EINVAL;
+	}
+
+	if (aead_xfrm != NULL && aead_xfrm->type != RTE_CRYPTO_SYM_XFORM_AEAD) {
+		plt_dp_err("Invalid type in AEAD xform");
+		return -EINVAL;
+	}
 
-	if (xform->type == RTE_CRYPTO_SYM_XFORM_CIPHER &&
-	    xform->cipher.algo == RTE_CRYPTO_CIPHER_3DES_CBC &&
-	    xform->next->type == RTE_CRYPTO_SYM_XFORM_AUTH &&
-	    xform->next->auth.algo == RTE_CRYPTO_AUTH_SHA1)
+	if ((c_xfrm == NULL || c_xfrm->cipher.algo == RTE_CRYPTO_CIPHER_NULL) &&
+	    a_xfrm != NULL && a_xfrm->auth.algo == RTE_CRYPTO_AUTH_NULL &&
+	    a_xfrm->auth.op == RTE_CRYPTO_AUTH_OP_VERIFY) {
+		plt_dp_err("Null cipher + null auth verify is not supported");
 		return -ENOTSUP;
+	}
+
+	/* Cipher only */
+	if (c_xfrm != NULL &&
+	    (a_xfrm == NULL || a_xfrm->auth.algo == RTE_CRYPTO_AUTH_NULL)) {
+		if (fill_sess_cipher(c_xfrm, sess))
+			return -ENOTSUP;
+		else
+			return 0;
+	}
+
+	/* Auth only */
+	if (a_xfrm != NULL &&
+	    (c_xfrm == NULL || c_xfrm->cipher.algo == RTE_CRYPTO_CIPHER_NULL)) {
+		if (fill_sess_auth(a_xfrm, sess))
+			return -ENOTSUP;
+		else
+			return 0;
+	}
+
+	/* AEAD */
+	if (aead_xfrm != NULL) {
+		if (fill_sess_aead(aead_xfrm, sess))
+			return -ENOTSUP;
+		else
+			return 0;
+	}
+
+	/* Chained ops */
+	if (c_xfrm == NULL || a_xfrm == NULL) {
+		plt_dp_err("Invalid xforms");
+		return -EINVAL;
+	}
 
-	if (xform->type == RTE_CRYPTO_SYM_XFORM_AUTH &&
-	    xform->auth.algo == RTE_CRYPTO_AUTH_SHA1 &&
-	    xform->next->type == RTE_CRYPTO_SYM_XFORM_CIPHER &&
-	    xform->next->cipher.algo == RTE_CRYPTO_CIPHER_3DES_CBC)
+	if (c_xfrm->cipher.algo == RTE_CRYPTO_CIPHER_3DES_CBC &&
+	    a_xfrm->auth.algo == RTE_CRYPTO_AUTH_SHA1) {
+		plt_dp_err("3DES-CBC + SHA1 is not supported");
 		return -ENOTSUP;
+	}
 
-	if (xform->type == RTE_CRYPTO_SYM_XFORM_CIPHER &&
-	    xform->cipher.op == RTE_CRYPTO_CIPHER_OP_ENCRYPT &&
-	    xform->next->type == RTE_CRYPTO_SYM_XFORM_AUTH &&
-	    xform->next->auth.op == RTE_CRYPTO_AUTH_OP_GENERATE)
-		return CNXK_CPT_CIPHER_ENC_AUTH_GEN;
-
-	if (xform->type == RTE_CRYPTO_SYM_XFORM_AUTH &&
-	    xform->auth.op == RTE_CRYPTO_AUTH_OP_VERIFY &&
-	    xform->next->type == RTE_CRYPTO_SYM_XFORM_CIPHER &&
-	    xform->next->cipher.op == RTE_CRYPTO_CIPHER_OP_DECRYPT)
-		return CNXK_CPT_AUTH_VRFY_CIPHER_DEC;
-
-	if (xform->type == RTE_CRYPTO_SYM_XFORM_AUTH &&
-	    xform->auth.op == RTE_CRYPTO_AUTH_OP_GENERATE &&
-	    xform->next->type == RTE_CRYPTO_SYM_XFORM_CIPHER &&
-	    xform->next->cipher.op == RTE_CRYPTO_CIPHER_OP_ENCRYPT) {
-		switch (xform->auth.algo) {
-		case RTE_CRYPTO_AUTH_SHA1_HMAC:
-			switch (xform->next->cipher.algo) {
-			case RTE_CRYPTO_CIPHER_AES_CBC:
-				return CNXK_CPT_AUTH_GEN_CIPHER_ENC;
-			default:
-				return -ENOTSUP;
-			}
-		default:
+	/* Cipher then auth */
+	if (ciph_then_auth) {
+		if (fill_sess_cipher(c_xfrm, sess))
 			return -ENOTSUP;
-		}
+		if (fill_sess_auth(a_xfrm, sess))
+			return -ENOTSUP;
+		else
+			return 0;
 	}
 
-	if (xform->type == RTE_CRYPTO_SYM_XFORM_CIPHER &&
-	    xform->cipher.op == RTE_CRYPTO_CIPHER_OP_DECRYPT &&
-	    xform->next->type == RTE_CRYPTO_SYM_XFORM_AUTH &&
-	    xform->next->auth.op == RTE_CRYPTO_AUTH_OP_VERIFY) {
-		switch (xform->cipher.algo) {
-		case RTE_CRYPTO_CIPHER_AES_CBC:
-			switch (xform->next->auth.algo) {
-			case RTE_CRYPTO_AUTH_SHA1_HMAC:
-				return CNXK_CPT_CIPHER_DEC_AUTH_VRFY;
+	/* else */
+
+	if (c_xfrm->cipher.op == RTE_CRYPTO_CIPHER_OP_ENCRYPT) {
+		switch (a_xfrm->auth.algo) {
+		case RTE_CRYPTO_AUTH_SHA1_HMAC:
+			switch (c_xfrm->cipher.algo) {
+			case RTE_CRYPTO_CIPHER_AES_CBC:
+				break;
 			default:
 				return -ENOTSUP;
 			}
+			break;
 		default:
 			return -ENOTSUP;
 		}
 	}
 
-	return -ENOTSUP;
+	if (fill_sess_auth(a_xfrm, sess))
+		return -ENOTSUP;
+	if (fill_sess_cipher(c_xfrm, sess))
+		return -ENOTSUP;
+	else
+		return 0;
 }
 
 static uint64_t
@@ -524,10 +561,6 @@ sym_session_configure(struct roc_cpt *roc_cpt, int driver_id,
 	void *priv;
 	int ret;
 
-	ret = sym_xform_verify(xform);
-	if (unlikely(ret < 0))
-		return ret;
-
 	if (unlikely(rte_mempool_get(pool, &priv))) {
 		plt_dp_err("Could not allocate session private data");
 		return -ENOMEM;
@@ -537,37 +570,7 @@ sym_session_configure(struct roc_cpt *roc_cpt, int driver_id,
 
 	sess_priv = priv;
 
-	switch (ret) {
-	case CNXK_CPT_CIPHER:
-		ret = fill_sess_cipher(xform, sess_priv);
-		break;
-	case CNXK_CPT_AUTH:
-		if (xform->auth.algo == RTE_CRYPTO_AUTH_AES_GMAC)
-			ret = fill_sess_gmac(xform, sess_priv);
-		else
-			ret = fill_sess_auth(xform, sess_priv);
-		break;
-	case CNXK_CPT_AEAD:
-		ret = fill_sess_aead(xform, sess_priv);
-		break;
-	case CNXK_CPT_CIPHER_ENC_AUTH_GEN:
-	case CNXK_CPT_CIPHER_DEC_AUTH_VRFY:
-		ret = fill_sess_cipher(xform, sess_priv);
-		if (ret < 0)
-			break;
-		ret = fill_sess_auth(xform->next, sess_priv);
-		break;
-	case CNXK_CPT_AUTH_VRFY_CIPHER_DEC:
-	case CNXK_CPT_AUTH_GEN_CIPHER_ENC:
-		ret = fill_sess_auth(xform, sess_priv);
-		if (ret < 0)
-			break;
-		ret = fill_sess_cipher(xform->next, sess_priv);
-		break;
-	default:
-		ret = -1;
-	}
-
+	ret = cnxk_sess_fill(xform, sess_priv);
 	if (ret)
 		goto priv_put;
 
@@ -592,7 +595,7 @@ sym_session_configure(struct roc_cpt *roc_cpt, int driver_id,
 priv_put:
 	rte_mempool_put(pool, priv);
 
-	return -ENOTSUP;
+	return ret;
 }
 
 int
diff --git a/drivers/crypto/cnxk/cnxk_cryptodev_ops.h b/drivers/crypto/cnxk/cnxk_cryptodev_ops.h
index 0d36365..ca363bb 100644
--- a/drivers/crypto/cnxk/cnxk_cryptodev_ops.h
+++ b/drivers/crypto/cnxk/cnxk_cryptodev_ops.h
@@ -30,16 +30,6 @@ struct cpt_qp_meta_info {
 	int mlen;
 };
 
-enum sym_xform_type {
-	CNXK_CPT_CIPHER = 1,
-	CNXK_CPT_AUTH,
-	CNXK_CPT_AEAD,
-	CNXK_CPT_CIPHER_ENC_AUTH_GEN,
-	CNXK_CPT_AUTH_VRFY_CIPHER_DEC,
-	CNXK_CPT_AUTH_GEN_CIPHER_ENC,
-	CNXK_CPT_CIPHER_DEC_AUTH_VRFY
-};
-
 #define CPT_OP_FLAGS_METABUF	       (1 << 1)
 #define CPT_OP_FLAGS_AUTH_VERIFY       (1 << 0)
 #define CPT_OP_FLAGS_IPSEC_DIR_INBOUND (1 << 2)
diff --git a/drivers/crypto/cnxk/cnxk_se.h b/drivers/crypto/cnxk/cnxk_se.h
index 37237de..a8cd2c5 100644
--- a/drivers/crypto/cnxk/cnxk_se.h
+++ b/drivers/crypto/cnxk/cnxk_se.h
@@ -36,6 +36,9 @@ struct cnxk_se_sess {
 	struct roc_se_ctx roc_se_ctx;
 } __rte_cache_aligned;
 
+static __rte_always_inline int
+fill_sess_gmac(struct rte_crypto_sym_xform *xform, struct cnxk_se_sess *sess);
+
 static inline void
 cpt_pack_iv(uint8_t *iv_src, uint8_t *iv_dst)
 {
@@ -1808,6 +1811,9 @@ fill_sess_auth(struct rte_crypto_sym_xform *xform, struct cnxk_se_sess *sess)
 	roc_se_auth_type auth_type = 0; /* NULL Auth type */
 	uint8_t zsk_flag = 0, aes_gcm = 0, is_null = 0;
 
+	if (xform->auth.algo == RTE_CRYPTO_AUTH_AES_GMAC)
+		return fill_sess_gmac(xform, sess);
+
 	if (xform->next != NULL &&
 	    xform->next->type == RTE_CRYPTO_SYM_XFORM_CIPHER &&
 	    xform->next->cipher.op == RTE_CRYPTO_CIPHER_OP_ENCRYPT) {
-- 
2.7.4

[PATCH 18/25] crypto/cnxk: fix inflight cnt calculation

[Anoob Joseph]

Inflight count calculation is updated to cover wrap around cases where
head can become smaller than tail.


Reported-by: Kiran Kumar K <kirankumark@marvell.com>
Signed-off-by: Anoob Joseph <anoobj@marvell.com>
---
 drivers/crypto/cnxk/cnxk_cryptodev_ops.h | 6 +++++-
 1 file changed, 5 insertions(+), 1 deletion(-)

diff --git a/drivers/crypto/cnxk/cnxk_cryptodev_ops.h b/drivers/crypto/cnxk/cnxk_cryptodev_ops.h
index ca363bb..0336ae1 100644
--- a/drivers/crypto/cnxk/cnxk_cryptodev_ops.h
+++ b/drivers/crypto/cnxk/cnxk_cryptodev_ops.h
@@ -156,7 +156,11 @@ pending_queue_retreat(uint64_t *index, const uint64_t mask, uint64_t nb_entry)
 static __rte_always_inline uint64_t
 pending_queue_infl_cnt(uint64_t head, uint64_t tail, const uint64_t mask)
 {
-	return (head - tail) & mask;
+	/*
+	 * Mask is nb_desc - 1. Add nb_desc to head and mask to account for
+	 * cases when tail > head, which happens during wrap around.
+	 */
+	return ((head + mask + 1) - tail) & mask;
 }
 
 static __rte_always_inline uint64_t
-- 
2.7.4

[PATCH 19/25] crypto/cnxk: use atomics to access cpt res

[Anoob Joseph]

The memory would be updated by hardware. Use atomics to read the same..

Signed-off-by: Anoob Joseph <anoobj@marvell.com>
---
 drivers/common/cnxk/hw/cpt.h              |  2 ++
 drivers/crypto/cnxk/cn10k_cryptodev_ops.c | 24 ++++++++++++++++--------
 drivers/crypto/cnxk/cn9k_cryptodev_ops.c  | 28 +++++++++++++++++++---------
 3 files changed, 37 insertions(+), 17 deletions(-)

diff --git a/drivers/common/cnxk/hw/cpt.h b/drivers/common/cnxk/hw/cpt.h
index ccc7af4..412dd76 100644
--- a/drivers/common/cnxk/hw/cpt.h
+++ b/drivers/common/cnxk/hw/cpt.h
@@ -215,6 +215,8 @@ union cpt_res_s {
 
 		uint64_t reserved_64_127;
 	} cn9k;
+
+	uint64_t u64[2];
 };
 
 /* [CN10K, .) */
diff --git a/drivers/crypto/cnxk/cn10k_cryptodev_ops.c b/drivers/crypto/cnxk/cn10k_cryptodev_ops.c
index 638268e..f8240e1 100644
--- a/drivers/crypto/cnxk/cn10k_cryptodev_ops.c
+++ b/drivers/crypto/cnxk/cn10k_cryptodev_ops.c
@@ -111,6 +111,10 @@ cn10k_cpt_fill_inst(struct cnxk_cpt_qp *qp, struct rte_crypto_op *ops[],
 	uint64_t w7;
 	int ret;
 
+	const union cpt_res_s res = {
+		.cn10k.compcode = CPT_COMP_NOT_DONE,
+	};
+
 	op = ops[0];
 
 	inst[0].w0.u64 = 0;
@@ -174,7 +178,7 @@ cn10k_cpt_fill_inst(struct cnxk_cpt_qp *qp, struct rte_crypto_op *ops[],
 	}
 
 	inst[0].res_addr = (uint64_t)&infl_req->res;
-	infl_req->res.cn10k.compcode = CPT_COMP_NOT_DONE;
+	__atomic_store_n(&infl_req->res.u64[0], res.u64[0], __ATOMIC_RELAXED);
 	infl_req->cop = op;
 
 	inst[0].w7.u64 = w7;
@@ -395,9 +399,9 @@ cn10k_cpt_sec_ucc_process(struct rte_crypto_op *cop,
 static inline void
 cn10k_cpt_dequeue_post_process(struct cnxk_cpt_qp *qp,
 			       struct rte_crypto_op *cop,
-			       struct cpt_inflight_req *infl_req)
+			       struct cpt_inflight_req *infl_req,
+			       struct cpt_cn10k_res_s *res)
 {
-	struct cpt_cn10k_res_s *res = (struct cpt_cn10k_res_s *)&infl_req->res;
 	const uint8_t uc_compcode = res->uc_compcode;
 	const uint8_t compcode = res->compcode;
 	unsigned int sz;
@@ -495,12 +499,15 @@ cn10k_cpt_crypto_adapter_dequeue(uintptr_t get_work1)
 	struct cpt_inflight_req *infl_req;
 	struct rte_crypto_op *cop;
 	struct cnxk_cpt_qp *qp;
+	union cpt_res_s res;
 
 	infl_req = (struct cpt_inflight_req *)(get_work1);
 	cop = infl_req->cop;
 	qp = infl_req->qp;
 
-	cn10k_cpt_dequeue_post_process(qp, infl_req->cop, infl_req);
+	res.u64[0] = __atomic_load_n(&infl_req->res.u64[0], __ATOMIC_RELAXED);
+
+	cn10k_cpt_dequeue_post_process(qp, infl_req->cop, infl_req, &res.cn10k);
 
 	if (unlikely(infl_req->op_flags & CPT_OP_FLAGS_METABUF))
 		rte_mempool_put(qp->meta_info.pool, infl_req->mdata);
@@ -515,9 +522,9 @@ cn10k_cpt_dequeue_burst(void *qptr, struct rte_crypto_op **ops, uint16_t nb_ops)
 	struct cpt_inflight_req *infl_req;
 	struct cnxk_cpt_qp *qp = qptr;
 	struct pending_queue *pend_q;
-	struct cpt_cn10k_res_s *res;
 	uint64_t infl_cnt, pq_tail;
 	struct rte_crypto_op *cop;
+	union cpt_res_s res;
 	int i;
 
 	pend_q = &qp->pend_q;
@@ -534,9 +541,10 @@ cn10k_cpt_dequeue_burst(void *qptr, struct rte_crypto_op **ops, uint16_t nb_ops)
 	for (i = 0; i < nb_ops; i++) {
 		infl_req = &pend_q->req_queue[pq_tail];
 
-		res = (struct cpt_cn10k_res_s *)&infl_req->res;
+		res.u64[0] = __atomic_load_n(&infl_req->res.u64[0],
+					     __ATOMIC_RELAXED);
 
-		if (unlikely(res->compcode == CPT_COMP_NOT_DONE)) {
+		if (unlikely(res.cn10k.compcode == CPT_COMP_NOT_DONE)) {
 			if (unlikely(rte_get_timer_cycles() >
 				     pend_q->time_out)) {
 				plt_err("Request timed out");
@@ -553,7 +561,7 @@ cn10k_cpt_dequeue_burst(void *qptr, struct rte_crypto_op **ops, uint16_t nb_ops)
 
 		ops[i] = cop;
 
-		cn10k_cpt_dequeue_post_process(qp, cop, infl_req);
+		cn10k_cpt_dequeue_post_process(qp, cop, infl_req, &res.cn10k);
 
 		if (unlikely(infl_req->op_flags & CPT_OP_FLAGS_METABUF))
 			rte_mempool_put(qp->meta_info.pool, infl_req->mdata);
diff --git a/drivers/crypto/cnxk/cn9k_cryptodev_ops.c b/drivers/crypto/cnxk/cn9k_cryptodev_ops.c
index 449208d..cf80d47 100644
--- a/drivers/crypto/cnxk/cn9k_cryptodev_ops.c
+++ b/drivers/crypto/cnxk/cn9k_cryptodev_ops.c
@@ -221,6 +221,10 @@ cn9k_cpt_enqueue_burst(void *qptr, struct rte_crypto_op **ops, uint16_t nb_ops)
 	uint64_t head;
 	int ret;
 
+	const union cpt_res_s res = {
+		.cn10k.compcode = CPT_COMP_NOT_DONE,
+	};
+
 	pend_q = &qp->pend_q;
 
 	const uint64_t lmt_base = qp->lf.lmt_base;
@@ -274,10 +278,12 @@ cn9k_cpt_enqueue_burst(void *qptr, struct rte_crypto_op **ops, uint16_t nb_ops)
 		infl_req_1->op_flags = 0;
 		infl_req_2->op_flags = 0;
 
-		infl_req_1->res.cn9k.compcode = CPT_COMP_NOT_DONE;
+		__atomic_store_n(&infl_req_1->res.u64[0], res.u64[0],
+				 __ATOMIC_RELAXED);
 		inst[0].res_addr = (uint64_t)&infl_req_1->res;
 
-		infl_req_2->res.cn9k.compcode = CPT_COMP_NOT_DONE;
+		__atomic_store_n(&infl_req_2->res.u64[0], res.u64[0],
+				 __ATOMIC_RELAXED);
 		inst[1].res_addr = (uint64_t)&infl_req_2->res;
 
 		ret = cn9k_cpt_inst_prep(qp, op_1, infl_req_1, &inst[0]);
@@ -410,9 +416,9 @@ cn9k_cpt_sec_post_process(struct rte_crypto_op *cop,
 
 static inline void
 cn9k_cpt_dequeue_post_process(struct cnxk_cpt_qp *qp, struct rte_crypto_op *cop,
-			      struct cpt_inflight_req *infl_req)
+			      struct cpt_inflight_req *infl_req,
+			      struct cpt_cn9k_res_s *res)
 {
-	struct cpt_cn9k_res_s *res = (struct cpt_cn9k_res_s *)&infl_req->res;
 	unsigned int sz;
 
 	if (likely(res->compcode == CPT_COMP_GOOD)) {
@@ -492,12 +498,15 @@ cn9k_cpt_crypto_adapter_dequeue(uintptr_t get_work1)
 	struct cpt_inflight_req *infl_req;
 	struct rte_crypto_op *cop;
 	struct cnxk_cpt_qp *qp;
+	union cpt_res_s res;
 
 	infl_req = (struct cpt_inflight_req *)(get_work1);
 	cop = infl_req->cop;
 	qp = infl_req->qp;
 
-	cn9k_cpt_dequeue_post_process(qp, infl_req->cop, infl_req);
+	res.u64[0] = __atomic_load_n(&infl_req->res.u64[0], __ATOMIC_RELAXED);
+
+	cn9k_cpt_dequeue_post_process(qp, infl_req->cop, infl_req, &res.cn9k);
 
 	if (unlikely(infl_req->op_flags & CPT_OP_FLAGS_METABUF))
 		rte_mempool_put(qp->meta_info.pool, infl_req->mdata);
@@ -512,9 +521,9 @@ cn9k_cpt_dequeue_burst(void *qptr, struct rte_crypto_op **ops, uint16_t nb_ops)
 	struct cpt_inflight_req *infl_req;
 	struct cnxk_cpt_qp *qp = qptr;
 	struct pending_queue *pend_q;
-	struct cpt_cn9k_res_s *res;
 	uint64_t infl_cnt, pq_tail;
 	struct rte_crypto_op *cop;
+	union cpt_res_s res;
 	int i;
 
 	pend_q = &qp->pend_q;
@@ -531,9 +540,10 @@ cn9k_cpt_dequeue_burst(void *qptr, struct rte_crypto_op **ops, uint16_t nb_ops)
 	for (i = 0; i < nb_ops; i++) {
 		infl_req = &pend_q->req_queue[pq_tail];
 
-		res = (struct cpt_cn9k_res_s *)&infl_req->res;
+		res.u64[0] = __atomic_load_n(&infl_req->res.u64[0],
+					     __ATOMIC_RELAXED);
 
-		if (unlikely(res->compcode == CPT_COMP_NOT_DONE)) {
+		if (unlikely(res.cn9k.compcode == CPT_COMP_NOT_DONE)) {
 			if (unlikely(rte_get_timer_cycles() >
 				     pend_q->time_out)) {
 				plt_err("Request timed out");
@@ -550,7 +560,7 @@ cn9k_cpt_dequeue_burst(void *qptr, struct rte_crypto_op **ops, uint16_t nb_ops)
 
 		ops[i] = cop;
 
-		cn9k_cpt_dequeue_post_process(qp, cop, infl_req);
+		cn9k_cpt_dequeue_post_process(qp, cop, infl_req, &res.cn9k);
 
 		if (unlikely(infl_req->op_flags & CPT_OP_FLAGS_METABUF))
 			rte_mempool_put(qp->meta_info.pool, infl_req->mdata);
-- 
2.7.4

[PATCH 20/25] crypto/cnxk: add more info on command timeout

[Anoob Joseph]

Print more info when command timeout happens. Print software and
hardware queue information.

Signed-off-by: Anoob Joseph <anoobj@marvell.com>
Signed-off-by: Tejasree Kondoj <ktejasree@marvell.com>
---
 drivers/common/cnxk/hw/cpt.h              | 11 ++++++++
 drivers/crypto/cnxk/cn10k_cryptodev_ops.c |  1 +
 drivers/crypto/cnxk/cn9k_cryptodev_ops.c  |  1 +
 drivers/crypto/cnxk/cnxk_cryptodev_ops.c  | 43 +++++++++++++++++++++++++++++++
 drivers/crypto/cnxk/cnxk_cryptodev_ops.h  |  1 +
 5 files changed, 57 insertions(+)

diff --git a/drivers/common/cnxk/hw/cpt.h b/drivers/common/cnxk/hw/cpt.h
index 412dd76..4d9c896 100644
--- a/drivers/common/cnxk/hw/cpt.h
+++ b/drivers/common/cnxk/hw/cpt.h
@@ -91,6 +91,17 @@ union cpt_lf_inprog {
 	} s;
 };
 
+union cpt_lf_q_inst_ptr {
+	uint64_t u;
+	struct cpt_lf_q_inst_ptr_s {
+		uint64_t dq_ptr : 20;
+		uint64_t reserved_20_31 : 12;
+		uint64_t nq_ptr : 20;
+		uint64_t reserved_52_62 : 11;
+		uint64_t xq_xor : 1;
+	} s;
+};
+
 union cpt_lf_q_base {
 	uint64_t u;
 	struct cpt_lf_q_base_s {
diff --git a/drivers/crypto/cnxk/cn10k_cryptodev_ops.c b/drivers/crypto/cnxk/cn10k_cryptodev_ops.c
index f8240e1..1905ea3 100644
--- a/drivers/crypto/cnxk/cn10k_cryptodev_ops.c
+++ b/drivers/crypto/cnxk/cn10k_cryptodev_ops.c
@@ -548,6 +548,7 @@ cn10k_cpt_dequeue_burst(void *qptr, struct rte_crypto_op **ops, uint16_t nb_ops)
 			if (unlikely(rte_get_timer_cycles() >
 				     pend_q->time_out)) {
 				plt_err("Request timed out");
+				cnxk_cpt_dump_on_err(qp);
 				pend_q->time_out = rte_get_timer_cycles() +
 						   DEFAULT_COMMAND_TIMEOUT *
 							   rte_get_timer_hz();
diff --git a/drivers/crypto/cnxk/cn9k_cryptodev_ops.c b/drivers/crypto/cnxk/cn9k_cryptodev_ops.c
index cf80d47..ac1953b 100644
--- a/drivers/crypto/cnxk/cn9k_cryptodev_ops.c
+++ b/drivers/crypto/cnxk/cn9k_cryptodev_ops.c
@@ -547,6 +547,7 @@ cn9k_cpt_dequeue_burst(void *qptr, struct rte_crypto_op **ops, uint16_t nb_ops)
 			if (unlikely(rte_get_timer_cycles() >
 				     pend_q->time_out)) {
 				plt_err("Request timed out");
+				cnxk_cpt_dump_on_err(qp);
 				pend_q->time_out = rte_get_timer_cycles() +
 						   DEFAULT_COMMAND_TIMEOUT *
 							   rte_get_timer_hz();
diff --git a/drivers/crypto/cnxk/cnxk_cryptodev_ops.c b/drivers/crypto/cnxk/cnxk_cryptodev_ops.c
index b02f070..f63dcdd 100644
--- a/drivers/crypto/cnxk/cnxk_cryptodev_ops.c
+++ b/drivers/crypto/cnxk/cnxk_cryptodev_ops.c
@@ -703,3 +703,46 @@ cnxk_ae_session_cfg(struct rte_cryptodev *dev,
 
 	return 0;
 }
+
+void
+cnxk_cpt_dump_on_err(struct cnxk_cpt_qp *qp)
+{
+	struct pending_queue *pend_q = &qp->pend_q;
+	uint64_t inflight, enq_ptr, deq_ptr, insts;
+	union cpt_lf_q_inst_ptr inst_ptr;
+	union cpt_lf_inprog lf_inprog;
+
+	plt_print("Lcore ID: %d, LF/QP ID: %d", rte_lcore_id(), qp->lf.lf_id);
+	plt_print("");
+	plt_print("S/w pending queue:");
+	plt_print("\tHead: %ld", pend_q->head);
+	plt_print("\tTail: %ld", pend_q->tail);
+	plt_print("\tMask: 0x%lx", pend_q->pq_mask);
+	plt_print("\tInflight count: %ld",
+		  pending_queue_infl_cnt(pend_q->head, pend_q->tail,
+					 pend_q->pq_mask));
+
+	plt_print("");
+	plt_print("H/w pending queue:");
+
+	lf_inprog.u = plt_read64(qp->lf.rbase + CPT_LF_INPROG);
+	inflight = lf_inprog.s.inflight;
+	plt_print("\tInflight in engines: %ld", inflight);
+
+	inst_ptr.u = plt_read64(qp->lf.rbase + CPT_LF_Q_INST_PTR);
+
+	enq_ptr = inst_ptr.s.nq_ptr;
+	deq_ptr = inst_ptr.s.dq_ptr;
+
+	if (enq_ptr >= deq_ptr)
+		insts = enq_ptr - deq_ptr;
+	else
+		insts = (enq_ptr + pend_q->pq_mask + 1 + 320 + 40) - deq_ptr;
+
+	plt_print("\tNQ ptr: 0x%lx", enq_ptr);
+	plt_print("\tDQ ptr: 0x%lx", deq_ptr);
+	plt_print("Insts waiting in CPT: %ld", insts);
+
+	plt_print("");
+	roc_cpt_afs_print(qp->lf.roc_cpt);
+}
diff --git a/drivers/crypto/cnxk/cnxk_cryptodev_ops.h b/drivers/crypto/cnxk/cnxk_cryptodev_ops.h
index 0336ae1..e521f07 100644
--- a/drivers/crypto/cnxk/cnxk_cryptodev_ops.h
+++ b/drivers/crypto/cnxk/cnxk_cryptodev_ops.h
@@ -122,6 +122,7 @@ int cnxk_ae_session_cfg(struct rte_cryptodev *dev,
 			struct rte_crypto_asym_xform *xform,
 			struct rte_cryptodev_asym_session *sess,
 			struct rte_mempool *pool);
+void cnxk_cpt_dump_on_err(struct cnxk_cpt_qp *qp);
 
 static inline union rte_event_crypto_metadata *
 cnxk_event_crypto_mdata_get(struct rte_crypto_op *op)
-- 
2.7.4

[PATCH 21/25] crypto/cnxk: support lookaside IPsec AES-CTR

[Anoob Joseph]

From: Tejasree Kondoj <ktejasree@marvell.com>

Adding AES-CTR support to cnxk CPT in
lookaside IPsec mode.

Signed-off-by: Tejasree Kondoj <ktejasree@marvell.com>
---
 doc/guides/cryptodevs/cnxk.rst                    |  2 ++
 doc/guides/rel_notes/release_22_03.rst            |  1 +
 drivers/common/cnxk/cnxk_security.c               |  6 ++++++
 drivers/crypto/cnxk/cn9k_ipsec.c                  |  3 +++
 drivers/crypto/cnxk/cnxk_cryptodev.h              |  2 +-
 drivers/crypto/cnxk/cnxk_cryptodev_capabilities.c | 20 ++++++++++++++++++++
 drivers/crypto/cnxk/cnxk_ipsec.h                  |  3 ++-
 7 files changed, 35 insertions(+), 2 deletions(-)

diff --git a/doc/guides/cryptodevs/cnxk.rst b/doc/guides/cryptodevs/cnxk.rst
index c49a779..1239155 100644
--- a/doc/guides/cryptodevs/cnxk.rst
+++ b/doc/guides/cryptodevs/cnxk.rst
@@ -261,6 +261,7 @@ Cipher algorithms
 +++++++++++++++++
 
 * AES-128/192/256-CBC
+* AES-128/192/256-CTR
 
 Auth algorithms
 +++++++++++++++
@@ -288,6 +289,7 @@ Cipher algorithms
 +++++++++++++++++
 
 * AES-128/192/256-CBC
+* AES-128/192/256-CTR
 
 Auth algorithms
 +++++++++++++++
diff --git a/doc/guides/rel_notes/release_22_03.rst b/doc/guides/rel_notes/release_22_03.rst
index 8df9092..4b272e4 100644
--- a/doc/guides/rel_notes/release_22_03.rst
+++ b/doc/guides/rel_notes/release_22_03.rst
@@ -60,6 +60,7 @@ New Features
   * Added SHA256-HMAC support in lookaside protocol (IPsec) for CN10K.
   * Added SHA384-HMAC support in lookaside protocol (IPsec) for CN9K & CN10K.
   * Added SHA512-HMAC support in lookaside protocol (IPsec) for CN9K & CN10K.
+  * Added AES-CTR support in lookaside protocol (IPsec) for CN9K & CN10K.
 
 
 Removed Items
diff --git a/drivers/common/cnxk/cnxk_security.c b/drivers/common/cnxk/cnxk_security.c
index 1c86f82..0d4baa9 100644
--- a/drivers/common/cnxk/cnxk_security.c
+++ b/drivers/common/cnxk/cnxk_security.c
@@ -123,6 +123,9 @@ ot_ipsec_sa_common_param_fill(union roc_ot_ipsec_sa_word2 *w2,
 		case RTE_CRYPTO_CIPHER_AES_CBC:
 			w2->s.enc_type = ROC_IE_OT_SA_ENC_AES_CBC;
 			break;
+		case RTE_CRYPTO_CIPHER_AES_CTR:
+			w2->s.enc_type = ROC_IE_OT_SA_ENC_AES_CTR;
+			break;
 		default:
 			return -ENOTSUP;
 		}
@@ -630,6 +633,9 @@ onf_ipsec_sa_common_param_fill(struct roc_ie_onf_sa_ctl *ctl, uint8_t *salt,
 		case RTE_CRYPTO_CIPHER_AES_CBC:
 			ctl->enc_type = ROC_IE_ON_SA_ENC_AES_CBC;
 			break;
+		case RTE_CRYPTO_CIPHER_AES_CTR:
+			ctl->enc_type = ROC_IE_ON_SA_ENC_AES_CTR;
+			break;
 		default:
 			return -ENOTSUP;
 		}
diff --git a/drivers/crypto/cnxk/cn9k_ipsec.c b/drivers/crypto/cnxk/cn9k_ipsec.c
index 3c6c8e9..65f46b2 100644
--- a/drivers/crypto/cnxk/cn9k_ipsec.c
+++ b/drivers/crypto/cnxk/cn9k_ipsec.c
@@ -166,6 +166,9 @@ ipsec_sa_ctl_set(struct rte_security_ipsec_xform *ipsec,
 	} else if (cipher_xform->cipher.algo == RTE_CRYPTO_CIPHER_AES_CBC) {
 		ctl->enc_type = ROC_IE_ON_SA_ENC_AES_CBC;
 		aes_key_len = cipher_xform->cipher.key.length;
+	} else if (cipher_xform->cipher.algo == RTE_CRYPTO_CIPHER_AES_CTR) {
+		ctl->enc_type = ROC_IE_ON_SA_ENC_AES_CTR;
+		aes_key_len = cipher_xform->cipher.key.length;
 	} else {
 		return -ENOTSUP;
 	}
diff --git a/drivers/crypto/cnxk/cnxk_cryptodev.h b/drivers/crypto/cnxk/cnxk_cryptodev.h
index f701c26..4a1e377 100644
--- a/drivers/crypto/cnxk/cnxk_cryptodev.h
+++ b/drivers/crypto/cnxk/cnxk_cryptodev.h
@@ -11,7 +11,7 @@
 #include "roc_cpt.h"
 
 #define CNXK_CPT_MAX_CAPS	 34
-#define CNXK_SEC_CRYPTO_MAX_CAPS 8
+#define CNXK_SEC_CRYPTO_MAX_CAPS 9
 #define CNXK_SEC_MAX_CAPS	 5
 #define CNXK_AE_EC_ID_MAX	 8
 /**
diff --git a/drivers/crypto/cnxk/cnxk_cryptodev_capabilities.c b/drivers/crypto/cnxk/cnxk_cryptodev_capabilities.c
index 0fdd91a..fae433e 100644
--- a/drivers/crypto/cnxk/cnxk_cryptodev_capabilities.c
+++ b/drivers/crypto/cnxk/cnxk_cryptodev_capabilities.c
@@ -754,6 +754,26 @@ static const struct rte_cryptodev_capabilities sec_caps_aes[] = {
 			}, }
 		}, }
 	},
+	{	/* AES CTR */
+		.op = RTE_CRYPTO_OP_TYPE_SYMMETRIC,
+		{.sym = {
+			.xform_type = RTE_CRYPTO_SYM_XFORM_CIPHER,
+			{.cipher = {
+				.algo = RTE_CRYPTO_CIPHER_AES_CTR,
+				.block_size = 16,
+				.key_size = {
+					.min = 16,
+					.max = 32,
+					.increment = 8
+				},
+				.iv_size = {
+					.min = 12,
+					.max = 16,
+					.increment = 4
+				}
+			}, }
+		}, }
+	},
 	{	/* AES CBC */
 		.op = RTE_CRYPTO_OP_TYPE_SYMMETRIC,
 		{.sym = {
diff --git a/drivers/crypto/cnxk/cnxk_ipsec.h b/drivers/crypto/cnxk/cnxk_ipsec.h
index 426eaa8..f5a51b5 100644
--- a/drivers/crypto/cnxk/cnxk_ipsec.h
+++ b/drivers/crypto/cnxk/cnxk_ipsec.h
@@ -20,7 +20,8 @@ struct cnxk_cpt_inst_tmpl {
 static inline int
 ipsec_xform_cipher_verify(struct rte_crypto_sym_xform *crypto_xform)
 {
-	if (crypto_xform->cipher.algo == RTE_CRYPTO_CIPHER_AES_CBC) {
+	if (crypto_xform->cipher.algo == RTE_CRYPTO_CIPHER_AES_CBC ||
+	    crypto_xform->cipher.algo == RTE_CRYPTO_CIPHER_AES_CTR) {
 		switch (crypto_xform->cipher.key.length) {
 		case 16:
 		case 24:
-- 
2.7.4

[PATCH 22/25] crypto/cnxk: fix extend tail calculation

[Anoob Joseph]

If the packet size to be incremented after IPsec processing is less
than size of hdr (size incremented before submitting), then extend_tail
can become negative. Allow negative values for the variable.

Fixes: 2474f57fc45c ("crypto/cnxk: add cn9k lookaside IPsec datapath")
Cc: marchana@marvell.com

Signed-off-by: Anoob Joseph <anoobj@marvell.com>
---
 drivers/crypto/cnxk/cn9k_ipsec_la_ops.h | 6 ++++--
 1 file changed, 4 insertions(+), 2 deletions(-)

diff --git a/drivers/crypto/cnxk/cn9k_ipsec_la_ops.h b/drivers/crypto/cnxk/cn9k_ipsec_la_ops.h
index 2dc8913..2b0261e 100644
--- a/drivers/crypto/cnxk/cn9k_ipsec_la_ops.h
+++ b/drivers/crypto/cnxk/cn9k_ipsec_la_ops.h
@@ -77,9 +77,10 @@ process_outb_sa(struct rte_crypto_op *cop, struct cn9k_ipsec_sa *sa,
 	const unsigned int hdr_len = sizeof(struct roc_ie_on_outb_hdr);
 	struct rte_crypto_sym_op *sym_op = cop->sym;
 	struct rte_mbuf *m_src = sym_op->m_src;
-	uint32_t dlen, rlen, extend_tail;
 	struct roc_ie_on_outb_sa *out_sa;
 	struct roc_ie_on_outb_hdr *hdr;
+	uint32_t dlen, rlen;
+	int32_t extend_tail;
 
 	out_sa = &sa->out_sa;
 
@@ -88,7 +89,8 @@ process_outb_sa(struct rte_crypto_op *cop, struct cn9k_ipsec_sa *sa,
 
 	extend_tail = rlen - dlen;
 	if (unlikely(extend_tail > rte_pktmbuf_tailroom(m_src))) {
-		plt_dp_err("Not enough tail room");
+		plt_dp_err("Not enough tail room (required: %d, available: %d",
+			   extend_tail, rte_pktmbuf_tailroom(m_src));
 		return -ENOMEM;
 	}
 
-- 
2.7.4

[PATCH 23/25] crypto/cnxk: add aes xcbc and null cipher

[Anoob Joseph]

Add support for AES XCBC and NULL cipher.

Signed-off-by: Anoob Joseph <anoobj@marvell.com>
---
 doc/guides/cryptodevs/cnxk.rst                    |  4 +
 doc/guides/rel_notes/release_22_03.rst            |  2 +
 drivers/common/cnxk/cnxk_security.c               | 48 ++++++++----
 drivers/common/cnxk/roc_ie_on.h                   | 10 +++
 drivers/crypto/cnxk/cn9k_ipsec.c                  | 93 ++++++++++++++++-------
 drivers/crypto/cnxk/cnxk_cryptodev.h              |  2 +-
 drivers/crypto/cnxk/cnxk_cryptodev_capabilities.c | 45 +++++++++++
 drivers/crypto/cnxk/cnxk_ipsec.h                  |  7 ++
 8 files changed, 169 insertions(+), 42 deletions(-)

diff --git a/doc/guides/cryptodevs/cnxk.rst b/doc/guides/cryptodevs/cnxk.rst
index 1239155..6e844f5 100644
--- a/doc/guides/cryptodevs/cnxk.rst
+++ b/doc/guides/cryptodevs/cnxk.rst
@@ -260,6 +260,7 @@ AEAD algorithms
 Cipher algorithms
 +++++++++++++++++
 
+* NULL
 * AES-128/192/256-CBC
 * AES-128/192/256-CTR
 
@@ -270,6 +271,7 @@ Auth algorithms
 * SHA256-128-HMAC
 * SHA384-192-HMAC
 * SHA512-256-HMAC
+* AES-XCBC-96
 
 CN10XX Features supported
 ~~~~~~~~~~~~~~~~~~~~~~~~~
@@ -288,6 +290,7 @@ AEAD algorithms
 Cipher algorithms
 +++++++++++++++++
 
+* NULL
 * AES-128/192/256-CBC
 * AES-128/192/256-CTR
 
@@ -299,3 +302,4 @@ Auth algorithms
 * SHA256-128-HMAC
 * SHA384-192-HMAC
 * SHA512-256-HMAC
+* AES-XCBC-96
diff --git a/doc/guides/rel_notes/release_22_03.rst b/doc/guides/rel_notes/release_22_03.rst
index 4b272e4..e8fec00 100644
--- a/doc/guides/rel_notes/release_22_03.rst
+++ b/doc/guides/rel_notes/release_22_03.rst
@@ -61,6 +61,8 @@ New Features
   * Added SHA384-HMAC support in lookaside protocol (IPsec) for CN9K & CN10K.
   * Added SHA512-HMAC support in lookaside protocol (IPsec) for CN9K & CN10K.
   * Added AES-CTR support in lookaside protocol (IPsec) for CN9K & CN10K.
+  * Added NULL cipher support in lookaside protocol (IPsec) for CN9K & CN10K.
+  * Added AES-XCBC support in lookaside protocol (IPsec) for CN9K & CN10K.
 
 
 Removed Items
diff --git a/drivers/common/cnxk/cnxk_security.c b/drivers/common/cnxk/cnxk_security.c
index 0d4baa9..6ebf084 100644
--- a/drivers/common/cnxk/cnxk_security.c
+++ b/drivers/common/cnxk/cnxk_security.c
@@ -120,6 +120,9 @@ ot_ipsec_sa_common_param_fill(union roc_ot_ipsec_sa_word2 *w2,
 		}
 	} else {
 		switch (cipher_xfrm->cipher.algo) {
+		case RTE_CRYPTO_CIPHER_NULL:
+			w2->s.enc_type = ROC_IE_OT_SA_ENC_NULL;
+			break;
 		case RTE_CRYPTO_CIPHER_AES_CBC:
 			w2->s.enc_type = ROC_IE_OT_SA_ENC_AES_CBC;
 			break;
@@ -146,11 +149,19 @@ ot_ipsec_sa_common_param_fill(union roc_ot_ipsec_sa_word2 *w2,
 		case RTE_CRYPTO_AUTH_SHA512_HMAC:
 			w2->s.auth_type = ROC_IE_OT_SA_AUTH_SHA2_512;
 			break;
+		case RTE_CRYPTO_AUTH_AES_XCBC_MAC:
+			w2->s.auth_type = ROC_IE_OT_SA_AUTH_AES_XCBC_128;
+			break;
 		default:
 			return -ENOTSUP;
 		}
 
-		ipsec_hmac_opad_ipad_gen(auth_xfrm, hmac_opad_ipad);
+		if (auth_xfrm->auth.algo == RTE_CRYPTO_AUTH_AES_XCBC_MAC) {
+			const uint8_t *auth_key = auth_xfrm->auth.key.data;
+			roc_aes_xcbc_key_derive(auth_key, hmac_opad_ipad);
+		} else {
+			ipsec_hmac_opad_ipad_gen(auth_xfrm, hmac_opad_ipad);
+		}
 
 		tmp_key = (uint64_t *)hmac_opad_ipad;
 		for (i = 0;
@@ -174,18 +185,26 @@ ot_ipsec_sa_common_param_fill(union roc_ot_ipsec_sa_word2 *w2,
 	for (i = 0; i < (int)(ROC_CTX_MAX_CKEY_LEN / sizeof(uint64_t)); i++)
 		tmp_key[i] = rte_be_to_cpu_64(tmp_key[i]);
 
-	switch (length) {
-	case ROC_CPT_AES128_KEY_LEN:
-		w2->s.aes_key_len = ROC_IE_SA_AES_KEY_LEN_128;
-		break;
-	case ROC_CPT_AES192_KEY_LEN:
-		w2->s.aes_key_len = ROC_IE_SA_AES_KEY_LEN_192;
-		break;
-	case ROC_CPT_AES256_KEY_LEN:
-		w2->s.aes_key_len = ROC_IE_SA_AES_KEY_LEN_256;
-		break;
-	default:
-		return -EINVAL;
+	/* Set AES key length */
+	if (w2->s.enc_type == ROC_IE_OT_SA_ENC_AES_CBC ||
+	    w2->s.enc_type == ROC_IE_OT_SA_ENC_AES_CCM ||
+	    w2->s.enc_type == ROC_IE_OT_SA_ENC_AES_CTR ||
+	    w2->s.enc_type == ROC_IE_OT_SA_ENC_AES_GCM ||
+	    w2->s.auth_type == ROC_IE_OT_SA_AUTH_AES_GMAC) {
+		switch (length) {
+		case ROC_CPT_AES128_KEY_LEN:
+			w2->s.aes_key_len = ROC_IE_SA_AES_KEY_LEN_128;
+			break;
+		case ROC_CPT_AES192_KEY_LEN:
+			w2->s.aes_key_len = ROC_IE_SA_AES_KEY_LEN_192;
+			break;
+		case ROC_CPT_AES256_KEY_LEN:
+			w2->s.aes_key_len = ROC_IE_SA_AES_KEY_LEN_256;
+			break;
+		default:
+			plt_err("Invalid AES key length");
+			return -EINVAL;
+		}
 	}
 
 	if (ipsec_xfrm->life.packets_soft_limit != 0 ||
@@ -815,6 +834,9 @@ cnxk_ipsec_icvlen_get(enum rte_crypto_cipher_algorithm c_algo,
 	case RTE_CRYPTO_AUTH_SHA512_HMAC:
 		icv = 32;
 		break;
+	case RTE_CRYPTO_AUTH_AES_XCBC_MAC:
+		icv = 12;
+		break;
 	default:
 		break;
 	}
diff --git a/drivers/common/cnxk/roc_ie_on.h b/drivers/common/cnxk/roc_ie_on.h
index 817ef33..cb56a70 100644
--- a/drivers/common/cnxk/roc_ie_on.h
+++ b/drivers/common/cnxk/roc_ie_on.h
@@ -181,6 +181,11 @@ struct roc_ie_on_outb_sa {
 			struct roc_ie_on_ip_template template;
 		} sha1;
 		struct {
+			uint8_t key[16];
+			uint8_t unused[32];
+			struct roc_ie_on_ip_template template;
+		} aes_xcbc;
+		struct {
 			uint8_t hmac_key[64];
 			uint8_t hmac_iv[64];
 			struct roc_ie_on_ip_template template;
@@ -202,6 +207,11 @@ struct roc_ie_on_inb_sa {
 			struct roc_ie_on_traffic_selector selector;
 		} sha1_or_gcm;
 		struct {
+			uint8_t key[16];
+			uint8_t unused[32];
+			struct roc_ie_on_traffic_selector selector;
+		} aes_xcbc;
+		struct {
 			uint8_t hmac_key[64];
 			uint8_t hmac_iv[64];
 			struct roc_ie_on_traffic_selector selector;
diff --git a/drivers/crypto/cnxk/cn9k_ipsec.c b/drivers/crypto/cnxk/cn9k_ipsec.c
index 65f46b2..06f6c20 100644
--- a/drivers/crypto/cnxk/cn9k_ipsec.c
+++ b/drivers/crypto/cnxk/cn9k_ipsec.c
@@ -118,7 +118,7 @@ ipsec_sa_ctl_set(struct rte_security_ipsec_xform *ipsec,
 		 struct roc_ie_on_sa_ctl *ctl)
 {
 	struct rte_crypto_sym_xform *cipher_xform, *auth_xform;
-	int aes_key_len;
+	int aes_key_len = 0;
 
 	if (ipsec->direction == RTE_SECURITY_IPSEC_SA_DIR_EGRESS) {
 		ctl->direction = ROC_IE_SA_DIR_OUTBOUND;
@@ -157,37 +157,33 @@ ipsec_sa_ctl_set(struct rte_security_ipsec_xform *ipsec,
 		return -EINVAL;
 
 	if (crypto_xform->type == RTE_CRYPTO_SYM_XFORM_AEAD) {
-		if (crypto_xform->aead.algo == RTE_CRYPTO_AEAD_AES_GCM) {
+		switch (crypto_xform->aead.algo) {
+		case RTE_CRYPTO_AEAD_AES_GCM:
 			ctl->enc_type = ROC_IE_ON_SA_ENC_AES_GCM;
 			aes_key_len = crypto_xform->aead.key.length;
-		} else {
+			break;
+		default:
+			plt_err("Unsupported AEAD algorithm");
 			return -ENOTSUP;
 		}
-	} else if (cipher_xform->cipher.algo == RTE_CRYPTO_CIPHER_AES_CBC) {
-		ctl->enc_type = ROC_IE_ON_SA_ENC_AES_CBC;
-		aes_key_len = cipher_xform->cipher.key.length;
-	} else if (cipher_xform->cipher.algo == RTE_CRYPTO_CIPHER_AES_CTR) {
-		ctl->enc_type = ROC_IE_ON_SA_ENC_AES_CTR;
-		aes_key_len = cipher_xform->cipher.key.length;
 	} else {
-		return -ENOTSUP;
-	}
-
-	switch (aes_key_len) {
-	case 16:
-		ctl->aes_key_len = ROC_IE_SA_AES_KEY_LEN_128;
-		break;
-	case 24:
-		ctl->aes_key_len = ROC_IE_SA_AES_KEY_LEN_192;
-		break;
-	case 32:
-		ctl->aes_key_len = ROC_IE_SA_AES_KEY_LEN_256;
-		break;
-	default:
-		return -EINVAL;
-	}
+		switch (cipher_xform->cipher.algo) {
+		case RTE_CRYPTO_CIPHER_NULL:
+			ctl->enc_type = ROC_IE_ON_SA_ENC_NULL;
+			break;
+		case RTE_CRYPTO_CIPHER_AES_CBC:
+			ctl->enc_type = ROC_IE_ON_SA_ENC_AES_CBC;
+			aes_key_len = cipher_xform->cipher.key.length;
+			break;
+		case RTE_CRYPTO_CIPHER_AES_CTR:
+			ctl->enc_type = ROC_IE_ON_SA_ENC_AES_CTR;
+			aes_key_len = cipher_xform->cipher.key.length;
+			break;
+		default:
+			plt_err("Unsupported cipher algorithm");
+			return -ENOTSUP;
+		}
 
-	if (crypto_xform->type != RTE_CRYPTO_SYM_XFORM_AEAD) {
 		switch (auth_xform->auth.algo) {
 		case RTE_CRYPTO_AUTH_NULL:
 			ctl->auth_type = ROC_IE_ON_SA_AUTH_NULL;
@@ -217,10 +213,33 @@ ipsec_sa_ctl_set(struct rte_security_ipsec_xform *ipsec,
 			ctl->auth_type = ROC_IE_ON_SA_AUTH_AES_XCBC_128;
 			break;
 		default:
+			plt_err("Unsupported auth algorithm");
 			return -ENOTSUP;
 		}
 	}
 
+	/* Set AES key length */
+	if (ctl->enc_type == ROC_IE_ON_SA_ENC_AES_CBC ||
+	    ctl->enc_type == ROC_IE_ON_SA_ENC_AES_CCM ||
+	    ctl->enc_type == ROC_IE_ON_SA_ENC_AES_CTR ||
+	    ctl->enc_type == ROC_IE_ON_SA_ENC_AES_GCM ||
+	    ctl->auth_type == ROC_IE_ON_SA_AUTH_AES_GMAC) {
+		switch (aes_key_len) {
+		case 16:
+			ctl->aes_key_len = ROC_IE_SA_AES_KEY_LEN_128;
+			break;
+		case 24:
+			ctl->aes_key_len = ROC_IE_SA_AES_KEY_LEN_192;
+			break;
+		case 32:
+			ctl->aes_key_len = ROC_IE_SA_AES_KEY_LEN_256;
+			break;
+		default:
+			plt_err("Invalid AES key length");
+			return -EINVAL;
+		}
+	}
+
 	if (ipsec->options.esn)
 		ctl->esn_en = 1;
 
@@ -267,8 +286,6 @@ fill_ipsec_common_sa(struct rte_security_ipsec_xform *ipsec,
 
 	if (cipher_key_len != 0)
 		memcpy(common_sa->cipher_key, cipher_key, cipher_key_len);
-	else
-		return -EINVAL;
 
 	return 0;
 }
@@ -337,7 +354,13 @@ cn9k_ipsec_outb_sa_create(struct cnxk_cpt_qp *qp,
 			ctx_len = offsetof(struct roc_ie_on_outb_sa,
 					   sha2.template);
 			break;
+		case ROC_IE_ON_SA_AUTH_AES_XCBC_128:
+			template = &out_sa->aes_xcbc.template;
+			ctx_len = offsetof(struct roc_ie_on_outb_sa,
+					   aes_xcbc.template);
+			break;
 		default:
+			plt_err("Unsupported auth algorithm");
 			return -EINVAL;
 		}
 	}
@@ -419,6 +442,9 @@ cn9k_ipsec_outb_sa_create(struct cnxk_cpt_qp *qp,
 		case RTE_CRYPTO_AUTH_SHA512_HMAC:
 			memcpy(out_sa->sha2.hmac_key, auth_key, auth_key_len);
 			break;
+		case RTE_CRYPTO_AUTH_AES_XCBC_MAC:
+			memcpy(out_sa->aes_xcbc.key, auth_key, auth_key_len);
+			break;
 		default:
 			plt_err("Unsupported auth algorithm %u",
 				auth_xform->auth.algo);
@@ -505,6 +531,11 @@ cn9k_ipsec_inb_sa_create(struct cnxk_cpt_qp *qp,
 			ctx_len = offsetof(struct roc_ie_on_inb_sa,
 					   sha2.selector);
 			break;
+		case RTE_CRYPTO_AUTH_AES_XCBC_MAC:
+			memcpy(in_sa->aes_xcbc.key, auth_key, auth_key_len);
+			ctx_len = offsetof(struct roc_ie_on_inb_sa,
+					   aes_xcbc.selector);
+			break;
 		default:
 			plt_err("Unsupported auth algorithm %u",
 				auth_xform->auth.algo);
@@ -597,6 +628,12 @@ cn9k_ipsec_xform_verify(struct rte_security_ipsec_xform *ipsec,
 				plt_err("Transport mode AES-CBC SHA2 HMAC 512 is not supported");
 				return -ENOTSUP;
 			}
+
+			if ((cipher->algo == RTE_CRYPTO_CIPHER_AES_CBC) &&
+			    (auth->algo == RTE_CRYPTO_AUTH_AES_XCBC_MAC)) {
+				plt_err("Transport mode AES-CBC AES-XCBC is not supported");
+				return -ENOTSUP;
+			}
 		}
 	}
 
diff --git a/drivers/crypto/cnxk/cnxk_cryptodev.h b/drivers/crypto/cnxk/cnxk_cryptodev.h
index 4a1e377..16e7572 100644
--- a/drivers/crypto/cnxk/cnxk_cryptodev.h
+++ b/drivers/crypto/cnxk/cnxk_cryptodev.h
@@ -11,7 +11,7 @@
 #include "roc_cpt.h"
 
 #define CNXK_CPT_MAX_CAPS	 34
-#define CNXK_SEC_CRYPTO_MAX_CAPS 9
+#define CNXK_SEC_CRYPTO_MAX_CAPS 11
 #define CNXK_SEC_MAX_CAPS	 5
 #define CNXK_AE_EC_ID_MAX	 8
 /**
diff --git a/drivers/crypto/cnxk/cnxk_cryptodev_capabilities.c b/drivers/crypto/cnxk/cnxk_cryptodev_capabilities.c
index fae433e..a0b2a1f 100644
--- a/drivers/crypto/cnxk/cnxk_cryptodev_capabilities.c
+++ b/drivers/crypto/cnxk/cnxk_cryptodev_capabilities.c
@@ -794,6 +794,26 @@ static const struct rte_cryptodev_capabilities sec_caps_aes[] = {
 			}, }
 		}, }
 	},
+	{	/* AES-XCBC */
+		.op = RTE_CRYPTO_OP_TYPE_SYMMETRIC,
+		{ .sym = {
+			.xform_type = RTE_CRYPTO_SYM_XFORM_AUTH,
+			{.auth = {
+				.algo = RTE_CRYPTO_AUTH_AES_XCBC_MAC,
+				.block_size = 16,
+				.key_size = {
+					.min = 16,
+					.max = 16,
+					.increment = 0
+				},
+				.digest_size = {
+					.min = 12,
+					.max = 12,
+					.increment = 0,
+				},
+			}, }
+		}, }
+	},
 };
 
 static const struct rte_cryptodev_capabilities sec_caps_sha1_sha2[] = {
@@ -879,6 +899,29 @@ static const struct rte_cryptodev_capabilities sec_caps_sha1_sha2[] = {
 	},
 };
 
+static const struct rte_cryptodev_capabilities sec_caps_null[] = {
+	{	/* NULL (CIPHER) */
+		.op = RTE_CRYPTO_OP_TYPE_SYMMETRIC,
+		{.sym = {
+			.xform_type = RTE_CRYPTO_SYM_XFORM_CIPHER,
+			{.cipher = {
+				.algo = RTE_CRYPTO_CIPHER_NULL,
+				.block_size = 1,
+				.key_size = {
+					.min = 0,
+					.max = 0,
+					.increment = 0
+				},
+				.iv_size = {
+					.min = 0,
+					.max = 0,
+					.increment = 0
+				}
+			}, },
+		}, }
+	},
+};
+
 static const struct rte_security_capability sec_caps_templ[] = {
 	{	/* IPsec Lookaside Protocol ESP Tunnel Ingress */
 		.action = RTE_SECURITY_ACTION_TYPE_LOOKASIDE_PROTOCOL,
@@ -1069,6 +1112,8 @@ sec_crypto_caps_populate(struct rte_cryptodev_capabilities cnxk_caps[],
 	else
 		cn9k_sec_crypto_caps_update(cnxk_caps);
 
+	sec_caps_add(cnxk_caps, &cur_pos, sec_caps_null,
+		     RTE_DIM(sec_caps_null));
 	sec_caps_add(cnxk_caps, &cur_pos, caps_end, RTE_DIM(caps_end));
 }
 
diff --git a/drivers/crypto/cnxk/cnxk_ipsec.h b/drivers/crypto/cnxk/cnxk_ipsec.h
index f5a51b5..f50d9fa 100644
--- a/drivers/crypto/cnxk/cnxk_ipsec.h
+++ b/drivers/crypto/cnxk/cnxk_ipsec.h
@@ -20,6 +20,9 @@ struct cnxk_cpt_inst_tmpl {
 static inline int
 ipsec_xform_cipher_verify(struct rte_crypto_sym_xform *crypto_xform)
 {
+	if (crypto_xform->cipher.algo == RTE_CRYPTO_CIPHER_NULL)
+		return 0;
+
 	if (crypto_xform->cipher.algo == RTE_CRYPTO_CIPHER_AES_CBC ||
 	    crypto_xform->cipher.algo == RTE_CRYPTO_CIPHER_AES_CTR) {
 		switch (crypto_xform->cipher.key.length) {
@@ -58,6 +61,10 @@ ipsec_xform_auth_verify(struct rte_crypto_sym_xform *crypto_xform)
 			return 0;
 	}
 
+	if (crypto_xform->auth.algo == RTE_CRYPTO_AUTH_AES_XCBC_MAC &&
+	    keylen == ROC_CPT_AES_XCBC_KEY_LENGTH)
+		return 0;
+
 	return -ENOTSUP;
 }
 
-- 
2.7.4

[PATCH 24/25] crypto/cnxk: add copy and set DF

[Anoob Joseph]

Add support for copy and set DF bit.

Signed-off-by: Anoob Joseph <anoobj@marvell.com>
---
 drivers/crypto/cnxk/cn9k_ipsec.c                  | 7 ++++++-
 drivers/crypto/cnxk/cnxk_cryptodev_capabilities.c | 1 +
 2 files changed, 7 insertions(+), 1 deletion(-)

diff --git a/drivers/crypto/cnxk/cn9k_ipsec.c b/drivers/crypto/cnxk/cn9k_ipsec.c
index 06f6c20..5bc0a50 100644
--- a/drivers/crypto/cnxk/cn9k_ipsec.c
+++ b/drivers/crypto/cnxk/cn9k_ipsec.c
@@ -246,6 +246,8 @@ ipsec_sa_ctl_set(struct rte_security_ipsec_xform *ipsec,
 	if (ipsec->options.udp_encap == 1)
 		ctl->encap_type = ROC_IE_ON_SA_ENCAP_UDP;
 
+	ctl->copy_df = ipsec->options.copy_df;
+
 	ctl->spi = rte_cpu_to_be_32(ipsec->spi);
 
 	rte_io_wmb();
@@ -376,13 +378,16 @@ cn9k_ipsec_outb_sa_create(struct cnxk_cpt_qp *qp,
 
 	if (ipsec->mode == RTE_SECURITY_IPSEC_SA_MODE_TUNNEL) {
 		if (ipsec->tunnel.type == RTE_SECURITY_IPSEC_TUNNEL_IPV4) {
+			uint16_t frag_off = 0;
 			ctx_len += sizeof(template->ip4);
 
 			ip4->version_ihl = RTE_IPV4_VHL_DEF;
 			ip4->time_to_live = ipsec->tunnel.ipv4.ttl;
 			ip4->type_of_service |= (ipsec->tunnel.ipv4.dscp << 2);
 			if (ipsec->tunnel.ipv4.df)
-				ip4->fragment_offset = BIT(14);
+				frag_off |= RTE_IPV4_HDR_DF_FLAG;
+			ip4->fragment_offset = rte_cpu_to_be_16(frag_off);
+
 			memcpy(&ip4->src_addr, &ipsec->tunnel.ipv4.src_ip,
 			       sizeof(struct in_addr));
 			memcpy(&ip4->dst_addr, &ipsec->tunnel.ipv4.dst_ip,
diff --git a/drivers/crypto/cnxk/cnxk_cryptodev_capabilities.c b/drivers/crypto/cnxk/cnxk_cryptodev_capabilities.c
index a0b2a1f..69ee0d9 100644
--- a/drivers/crypto/cnxk/cnxk_cryptodev_capabilities.c
+++ b/drivers/crypto/cnxk/cnxk_cryptodev_capabilities.c
@@ -1121,6 +1121,7 @@ static void
 cnxk_sec_caps_update(struct rte_security_capability *sec_cap)
 {
 	sec_cap->ipsec.options.udp_encap = 1;
+	sec_cap->ipsec.options.copy_df = 1;
 }
 
 static void
-- 
2.7.4

[PATCH 25/25] crypto/cnxk: add aes cmac

[Anoob Joseph]

Add support for AES CMAC auth algorithm.

Signed-off-by: Anoob Joseph <anoobj@marvell.com>
---
 doc/guides/cryptodevs/cnxk.rst                    |  1 +
 doc/guides/cryptodevs/features/cn10k.ini          | 37 +++++++-------
 doc/guides/cryptodevs/features/cn9k.ini           | 37 +++++++-------
 doc/guides/rel_notes/release_22_03.rst            |  1 +
 drivers/common/cnxk/roc_se.h                      |  8 +--
 drivers/crypto/cnxk/cnxk_cryptodev_capabilities.c | 20 ++++++++
 drivers/crypto/cnxk/cnxk_se.h                     | 60 ++++++++++++++---------
 7 files changed, 103 insertions(+), 61 deletions(-)

diff --git a/doc/guides/cryptodevs/cnxk.rst b/doc/guides/cryptodevs/cnxk.rst
index 6e844f5..3c58517 100644
--- a/doc/guides/cryptodevs/cnxk.rst
+++ b/doc/guides/cryptodevs/cnxk.rst
@@ -61,6 +61,7 @@ Hash algorithms:
 * ``RTE_CRYPTO_AUTH_SHA512_HMAC``
 * ``RTE_CRYPTO_AUTH_SNOW3G_UIA2``
 * ``RTE_CRYPTO_AUTH_ZUC_EIA3``
+* ``RTE_CRYPTO_AUTH_AES_CMAC``
 
 AEAD algorithms:
 
diff --git a/doc/guides/cryptodevs/features/cn10k.ini b/doc/guides/cryptodevs/features/cn10k.ini
index ab21d9d..c8193c2 100644
--- a/doc/guides/cryptodevs/features/cn10k.ini
+++ b/doc/guides/cryptodevs/features/cn10k.ini
@@ -41,23 +41,26 @@ ZUC EEA3       = Y
 ; Supported authentication algorithms of 'cn10k' crypto driver.
 ;
 [Auth]
-NULL         = Y
-AES GMAC     = Y
-KASUMI F9    = Y
-MD5          = Y
-MD5 HMAC     = Y
-SHA1         = Y
-SHA1 HMAC    = Y
-SHA224       = Y
-SHA224 HMAC  = Y
-SHA256       = Y
-SHA256 HMAC  = Y
-SHA384       = Y
-SHA384 HMAC  = Y
-SHA512       = Y
-SHA512 HMAC  = Y
-SNOW3G UIA2  = Y
-ZUC EIA3     = Y
+NULL            = Y
+AES GMAC        = Y
+KASUMI F9       = Y
+MD5             = Y
+MD5 HMAC        = Y
+SHA1            = Y
+SHA1 HMAC       = Y
+SHA224          = Y
+SHA224 HMAC     = Y
+SHA256          = Y
+SHA256 HMAC     = Y
+SHA384          = Y
+SHA384 HMAC     = Y
+SHA512          = Y
+SHA512 HMAC     = Y
+SNOW3G UIA2     = Y
+ZUC EIA3        = Y
+AES CMAC (128)  = Y
+AES CMAC (192)  = Y
+AES CMAC (256)  = Y
 
 ;
 ; Supported AEAD algorithms of 'cn10k' crypto driver.
diff --git a/doc/guides/cryptodevs/features/cn9k.ini b/doc/guides/cryptodevs/features/cn9k.ini
index d834659..f215ee0 100644
--- a/doc/guides/cryptodevs/features/cn9k.ini
+++ b/doc/guides/cryptodevs/features/cn9k.ini
@@ -40,23 +40,26 @@ ZUC EEA3       = Y
 ; Supported authentication algorithms of 'cn9k' crypto driver.
 ;
 [Auth]
-NULL         = Y
-AES GMAC     = Y
-KASUMI F9    = Y
-MD5          = Y
-MD5 HMAC     = Y
-SHA1         = Y
-SHA1 HMAC    = Y
-SHA224       = Y
-SHA224 HMAC  = Y
-SHA256       = Y
-SHA256 HMAC  = Y
-SHA384       = Y
-SHA384 HMAC  = Y
-SHA512       = Y
-SHA512 HMAC  = Y
-SNOW3G UIA2  = Y
-ZUC EIA3     = Y
+NULL            = Y
+AES GMAC        = Y
+KASUMI F9       = Y
+MD5             = Y
+MD5 HMAC        = Y
+SHA1            = Y
+SHA1 HMAC       = Y
+SHA224          = Y
+SHA224 HMAC     = Y
+SHA256          = Y
+SHA256 HMAC     = Y
+SHA384          = Y
+SHA384 HMAC     = Y
+SHA512          = Y
+SHA512 HMAC     = Y
+SNOW3G UIA2     = Y
+ZUC EIA3        = Y
+AES CMAC (128)  = Y
+AES CMAC (192)  = Y
+AES CMAC (256)  = Y
 
 ;
 ; Supported AEAD algorithms of 'cn9k' crypto driver.
diff --git a/doc/guides/rel_notes/release_22_03.rst b/doc/guides/rel_notes/release_22_03.rst
index e8fec00..72e758e 100644
--- a/doc/guides/rel_notes/release_22_03.rst
+++ b/doc/guides/rel_notes/release_22_03.rst
@@ -63,6 +63,7 @@ New Features
   * Added AES-CTR support in lookaside protocol (IPsec) for CN9K & CN10K.
   * Added NULL cipher support in lookaside protocol (IPsec) for CN9K & CN10K.
   * Added AES-XCBC support in lookaside protocol (IPsec) for CN9K & CN10K.
+  * Added AES-CMAC support in CN9K & CN10K.
 
 
 Removed Items
diff --git a/drivers/common/cnxk/roc_se.h b/drivers/common/cnxk/roc_se.h
index 253575a..145a182 100644
--- a/drivers/common/cnxk/roc_se.h
+++ b/drivers/common/cnxk/roc_se.h
@@ -11,10 +11,10 @@
 #define ROC_SE_FC_MINOR_OP_DECRYPT    0x1
 #define ROC_SE_FC_MINOR_OP_HMAC_FIRST 0x10
 
-#define ROC_SE_MAJOR_OP_HASH	   0x34
-#define ROC_SE_MAJOR_OP_HMAC	   0x35
-#define ROC_SE_MAJOR_OP_ZUC_SNOW3G 0x37
-#define ROC_SE_MAJOR_OP_KASUMI	   0x38
+#define ROC_SE_MAJOR_OP_HASH   0x34
+#define ROC_SE_MAJOR_OP_HMAC   0x35
+#define ROC_SE_MAJOR_OP_PDCP   0x37
+#define ROC_SE_MAJOR_OP_KASUMI 0x38
 
 #define ROC_SE_MAJOR_OP_MISC		 0x01
 #define ROC_SE_MISC_MINOR_OP_PASSTHROUGH 0x03
diff --git a/drivers/crypto/cnxk/cnxk_cryptodev_capabilities.c b/drivers/crypto/cnxk/cnxk_cryptodev_capabilities.c
index 69ee0d9..457e166 100644
--- a/drivers/crypto/cnxk/cnxk_cryptodev_capabilities.c
+++ b/drivers/crypto/cnxk/cnxk_cryptodev_capabilities.c
@@ -568,6 +568,26 @@ static const struct rte_cryptodev_capabilities caps_aes[] = {
 			}, }
 		}, }
 	},
+	{	/* AES CMAC */
+		.op = RTE_CRYPTO_OP_TYPE_SYMMETRIC,
+		{.sym = {
+			.xform_type = RTE_CRYPTO_SYM_XFORM_AUTH,
+			{.auth = {
+				.algo = RTE_CRYPTO_AUTH_AES_CMAC,
+				.block_size = 16,
+				.key_size = {
+					.min = 16,
+					.max = 32,
+					.increment = 8
+				},
+				.digest_size = {
+					.min = 4,
+					.max = 4,
+					.increment = 0
+				},
+			}, }
+		}, }
+	},
 };
 
 static const struct rte_cryptodev_capabilities caps_kasumi[] = {
diff --git a/drivers/crypto/cnxk/cnxk_se.h b/drivers/crypto/cnxk/cnxk_se.h
index a8cd2c5..e988d57 100644
--- a/drivers/crypto/cnxk/cnxk_se.h
+++ b/drivers/crypto/cnxk/cnxk_se.h
@@ -73,11 +73,15 @@ pdcp_iv_copy(uint8_t *iv_d, uint8_t *iv_s, const uint8_t pdcp_alg_type,
 		for (j = 0; j < 4; j++)
 			iv_temp[j] = iv_s_temp[3 - j];
 		memcpy(iv_d, iv_temp, 16);
-	} else {
+	} else if (pdcp_alg_type == ROC_SE_PDCP_ALG_TYPE_ZUC) {
 		/* ZUC doesn't need a swap */
 		memcpy(iv_d, iv_s, 16);
 		if (pack_iv)
 			cpt_pack_iv(iv_s, iv_d);
+	} else {
+		/* AES-CMAC EIA2, microcode expects 16B zeroized IV */
+		for (j = 0; j < 4; j++)
+			iv_d[j] = 0;
 	}
 }
 
@@ -992,8 +996,8 @@ cpt_dec_hmac_prep(uint32_t flags, uint64_t d_offs, uint64_t d_lens,
 }
 
 static __rte_always_inline int
-cpt_zuc_snow3g_prep(uint32_t req_flags, uint64_t d_offs, uint64_t d_lens,
-		    struct roc_se_fc_params *params, struct cpt_inst_s *inst)
+cpt_pdcp_alg_prep(uint32_t req_flags, uint64_t d_offs, uint64_t d_lens,
+		  struct roc_se_fc_params *params, struct cpt_inst_s *inst)
 {
 	uint32_t size;
 	int32_t inputlen, outputlen;
@@ -1014,33 +1018,43 @@ cpt_zuc_snow3g_prep(uint32_t req_flags, uint64_t d_offs, uint64_t d_lens,
 	mac_len = se_ctx->mac_len;
 	pdcp_alg_type = se_ctx->pdcp_alg_type;
 
-	cpt_inst_w4.s.opcode_major = ROC_SE_MAJOR_OP_ZUC_SNOW3G;
-
+	cpt_inst_w4.s.opcode_major = ROC_SE_MAJOR_OP_PDCP;
 	cpt_inst_w4.s.opcode_minor = se_ctx->template_w4.s.opcode_minor;
 
 	if (flags == 0x1) {
 		iv_s = params->auth_iv_buf;
-		iv_len = params->auth_iv_len;
-
-		if (iv_len == 25) {
-			iv_len -= 2;
-			pack_iv = 1;
-		}
 
 		/*
 		 * Microcode expects offsets in bytes
 		 * TODO: Rounding off
 		 */
 		auth_data_len = ROC_SE_AUTH_DLEN(d_lens);
-
-		/* EIA3 or UIA2 */
 		auth_offset = ROC_SE_AUTH_OFFSET(d_offs);
-		auth_offset = auth_offset / 8;
 
-		/* consider iv len */
-		auth_offset += iv_len;
+		if (se_ctx->pdcp_alg_type != ROC_SE_PDCP_ALG_TYPE_AES_CTR) {
+			iv_len = params->auth_iv_len;
+
+			if (iv_len == 25) {
+				iv_len -= 2;
+				pack_iv = 1;
+			}
+
+			auth_offset = auth_offset / 8;
+
+			/* consider iv len */
+			auth_offset += iv_len;
+
+			inputlen =
+				auth_offset + (RTE_ALIGN(auth_data_len, 8) / 8);
+		} else {
+			iv_len = 16;
+
+			/* consider iv len */
+			auth_offset += iv_len;
+
+			inputlen = auth_offset + auth_data_len;
+		}
 
-		inputlen = auth_offset + (RTE_ALIGN(auth_data_len, 8) / 8);
 		outputlen = mac_len;
 
 		offset_ctrl = rte_cpu_to_be_64((uint64_t)auth_offset);
@@ -1056,7 +1070,6 @@ cpt_zuc_snow3g_prep(uint32_t req_flags, uint64_t d_offs, uint64_t d_lens,
 			pack_iv = 1;
 		}
 
-		/* EEA3 or UEA2 */
 		/*
 		 * Microcode expects offsets in bytes
 		 * TODO: Rounding off
@@ -1589,8 +1602,7 @@ cpt_fc_dec_hmac_prep(uint32_t flags, uint64_t d_offs, uint64_t d_lens,
 	if (likely(fc_type == ROC_SE_FC_GEN)) {
 		ret = cpt_dec_hmac_prep(flags, d_offs, d_lens, fc_params, inst);
 	} else if (fc_type == ROC_SE_PDCP) {
-		ret = cpt_zuc_snow3g_prep(flags, d_offs, d_lens, fc_params,
-					  inst);
+		ret = cpt_pdcp_alg_prep(flags, d_offs, d_lens, fc_params, inst);
 	} else if (fc_type == ROC_SE_KASUMI) {
 		ret = cpt_kasumi_dec_prep(d_offs, d_lens, fc_params, inst);
 	}
@@ -1618,8 +1630,7 @@ cpt_fc_enc_hmac_prep(uint32_t flags, uint64_t d_offs, uint64_t d_lens,
 	if (likely(fc_type == ROC_SE_FC_GEN)) {
 		ret = cpt_enc_hmac_prep(flags, d_offs, d_lens, fc_params, inst);
 	} else if (fc_type == ROC_SE_PDCP) {
-		ret = cpt_zuc_snow3g_prep(flags, d_offs, d_lens, fc_params,
-					  inst);
+		ret = cpt_pdcp_alg_prep(flags, d_offs, d_lens, fc_params, inst);
 	} else if (fc_type == ROC_SE_KASUMI) {
 		ret = cpt_kasumi_enc_prep(flags, d_offs, d_lens, fc_params,
 					  inst);
@@ -1883,8 +1894,11 @@ fill_sess_auth(struct rte_crypto_sym_xform *xform, struct cnxk_se_sess *sess)
 		auth_type = 0;
 		is_null = 1;
 		break;
-	case RTE_CRYPTO_AUTH_AES_XCBC_MAC:
 	case RTE_CRYPTO_AUTH_AES_CMAC:
+		auth_type = ROC_SE_AES_CMAC_EIA2;
+		zsk_flag = ROC_SE_ZS_IA;
+		break;
+	case RTE_CRYPTO_AUTH_AES_XCBC_MAC:
 	case RTE_CRYPTO_AUTH_AES_CBC_MAC:
 		plt_dp_err("Crypto: Unsupported hash algo %u", a_form->algo);
 		return -1;
-- 
2.7.4

[PATCH v2 00/29] New features and improvements in cnxk crypto PMD

[Anoob Joseph]

New features and fixes to cnxk crypto PMDs
- Support for more algorithms in lookaside crypto & protocol
- Support for copy & set DF bit
- Support for CPT CTX update
- Support for security session stats in cn10k

Changes in v2
- Support for copy & set DSCP
- Support for per packet IV in cn9k
- Support for cn10k v1.19 microcode

Ankur Dwivedi (1):
  crypto/cnxk: add security session stats get

Anoob Joseph (20):
  common/cnxk: define minor opcodes for MISC opcode
  common/cnxk: add aes-xcbc key derive
  common/cnxk: fix reset of fields
  common/cnxk: verify input args
  common/cnxk: update completion code
  crypto/cnxk: clear session data before populating
  crypto/cnxk: update max sec crypto caps
  crypto/cnxk: account for CPT CTX updates and flush delays
  crypto/cnxk: use struct sizes for ctx writes
  crypto/cnxk: add skip for unsupported cases
  crypto/cnxk: handle null chained ops
  crypto/cnxk: fix inflight cnt calculation
  crypto/cnxk: use atomics to access CPT res
  crypto/cnxk: add more info on command timeout
  crypto/cnxk: fix extend tail calculation
  crypto/cnxk: add aes xcbc and null cipher
  crypto/cnxk: add copy and set DF
  crypto/cnxk: add aes cmac
  crypto/cnxk: enable copy dscp
  crypto/cnxk: update microcode completion handling

Archana Muniganti (2):
  common/cnxk: add bit fields for params
  crypto/cnxk: add per pkt IV in lookaside IPsec debug mode

Shijith Thotton (1):
  crypto/cnxk: only enable queues that are allocated

Tejasree Kondoj (5):
  crypto/cnxk: add lookaside IPsec AES-CBC-HMAC-SHA256 support
  crypto/cnxk: write CPT CTX through microcode op
  crypto/cnxk: support cnxk lookaside IPsec HMAC-SHA384/512
  crypto/cnxk: add context reload for IV
  crypto/cnxk: support lookaside IPsec AES-CTR

 doc/guides/cryptodevs/cnxk.rst                    |  50 ++++-
 doc/guides/cryptodevs/features/cn10k.ini          |  37 ++--
 doc/guides/cryptodevs/features/cn9k.ini           |  37 ++--
 doc/guides/rel_notes/release_22_03.rst            |  10 +
 drivers/common/cnxk/cnxk_security.c               |  92 ++++++--
 drivers/common/cnxk/hw/cpt.h                      |  15 ++
 drivers/common/cnxk/meson.build                   |   1 +
 drivers/common/cnxk/roc_aes.c                     | 208 ++++++++++++++++++
 drivers/common/cnxk/roc_aes.h                     |  14 ++
 drivers/common/cnxk/roc_api.h                     |   3 +
 drivers/common/cnxk/roc_cpt.c                     |   4 +-
 drivers/common/cnxk/roc_cpt.h                     |  24 +-
 drivers/common/cnxk/roc_ie_on.h                   |  47 +++-
 drivers/common/cnxk/roc_ie_ot.h                   |   4 +-
 drivers/common/cnxk/roc_se.h                      |  14 +-
 drivers/common/cnxk/version.map                   |   1 +
 drivers/crypto/cnxk/cn10k_cryptodev_ops.c         |  91 ++++----
 drivers/crypto/cnxk/cn10k_ipsec.c                 | 232 ++++++++++++++++----
 drivers/crypto/cnxk/cn10k_ipsec.h                 |  26 ++-
 drivers/crypto/cnxk/cn10k_ipsec_la_ops.h          |  28 ++-
 drivers/crypto/cnxk/cn9k_cryptodev_ops.c          |  29 ++-
 drivers/crypto/cnxk/cn9k_ipsec.c                  | 253 +++++++++++++++++-----
 drivers/crypto/cnxk/cn9k_ipsec.h                  |   2 +
 drivers/crypto/cnxk/cn9k_ipsec_la_ops.h           |  20 +-
 drivers/crypto/cnxk/cnxk_cryptodev.h              |   2 +-
 drivers/crypto/cnxk/cnxk_cryptodev_capabilities.c | 158 +++++++++++++-
 drivers/crypto/cnxk/cnxk_cryptodev_ops.c          | 245 +++++++++++++--------
 drivers/crypto/cnxk/cnxk_cryptodev_ops.h          |  17 +-
 drivers/crypto/cnxk/cnxk_cryptodev_sec.c          |   1 +
 drivers/crypto/cnxk/cnxk_ipsec.h                  |  19 +-
 drivers/crypto/cnxk/cnxk_se.h                     |  66 ++++--
 31 files changed, 1348 insertions(+), 402 deletions(-)
 create mode 100644 drivers/common/cnxk/roc_aes.c
 create mode 100644 drivers/common/cnxk/roc_aes.h

-- 
2.7.4

[PATCH v2 01/29] common/cnxk: define minor opcodes for MISC opcode

[Anoob Joseph]

MISC CPT instruction behaves differently based on minor opcode.
Define the missing minor opcodes for MISC major opcode.

Signed-off-by: Aakash Sasidharan <asasidharan@marvell.com>
Signed-off-by: Anoob Joseph <anoobj@marvell.com>
---
 drivers/common/cnxk/roc_se.h | 6 +++++-
 1 file changed, 5 insertions(+), 1 deletion(-)

diff --git a/drivers/common/cnxk/roc_se.h b/drivers/common/cnxk/roc_se.h
index 5be832f..253575a 100644
--- a/drivers/common/cnxk/roc_se.h
+++ b/drivers/common/cnxk/roc_se.h
@@ -15,7 +15,11 @@
 #define ROC_SE_MAJOR_OP_HMAC	   0x35
 #define ROC_SE_MAJOR_OP_ZUC_SNOW3G 0x37
 #define ROC_SE_MAJOR_OP_KASUMI	   0x38
-#define ROC_SE_MAJOR_OP_MISC	   0x01
+
+#define ROC_SE_MAJOR_OP_MISC		 0x01
+#define ROC_SE_MISC_MINOR_OP_PASSTHROUGH 0x03
+#define ROC_SE_MISC_MINOR_OP_DUMMY	 0x04
+#define ROC_SE_MISC_MINOR_OP_HW_SUPPORT	 0x08
 
 #define ROC_SE_MAX_AAD_SIZE 64
 #define ROC_SE_MAX_MAC_LEN  64
-- 
2.7.4

[PATCH v2 02/29] common/cnxk: add aes-xcbc key derive

[Anoob Joseph]

Add support for AES-XCBC key derivation.

Signed-off-by: Anoob Joseph <anoobj@marvell.com>
---
 drivers/common/cnxk/meson.build |   1 +
 drivers/common/cnxk/roc_aes.c   | 208 ++++++++++++++++++++++++++++++++++++++++
 drivers/common/cnxk/roc_aes.h   |  14 +++
 drivers/common/cnxk/roc_api.h   |   3 +
 drivers/common/cnxk/roc_cpt.h   |  24 ++---
 drivers/common/cnxk/version.map |   1 +
 6 files changed, 239 insertions(+), 12 deletions(-)
 create mode 100644 drivers/common/cnxk/roc_aes.c
 create mode 100644 drivers/common/cnxk/roc_aes.h

diff --git a/drivers/common/cnxk/meson.build b/drivers/common/cnxk/meson.build
index 4928f7e..4995cfd 100644
--- a/drivers/common/cnxk/meson.build
+++ b/drivers/common/cnxk/meson.build
@@ -12,6 +12,7 @@ config_flag_fmt = 'RTE_LIBRTE_@0@_COMMON'
 deps = ['eal', 'pci', 'bus_pci', 'mbuf', 'security']
 sources = files(
         'roc_ae.c',
+        'roc_aes.c',
         'roc_ae_fpm_tables.c',
         'roc_bphy.c',
         'roc_bphy_cgx.c',
diff --git a/drivers/common/cnxk/roc_aes.c b/drivers/common/cnxk/roc_aes.c
new file mode 100644
index 0000000..f821c8b
--- /dev/null
+++ b/drivers/common/cnxk/roc_aes.c
@@ -0,0 +1,208 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright (c) 2021 Marvell.
+ */
+
+#include "roc_api.h"
+
+#define KEY_WORD_LEN	 (ROC_CPT_AES_XCBC_KEY_LENGTH / sizeof(uint32_t))
+#define KEY_ROUNDS	 10			/* (Nr+1)*Nb */
+#define KEY_SCHEDULE_LEN ((KEY_ROUNDS + 1) * 4) /* (Nr+1)*Nb words */
+
+/*
+ * AES 128 implementation based on NIST FIPS 197 suitable for LittleEndian
+ * https://nvlpubs.nist.gov/nistpubs/FIPS/NIST.FIPS.197.pdf
+ */
+
+/* Sbox from NIST FIPS 197 */
+static uint8_t Sbox[] = {
+	0x63, 0x7c, 0x77, 0x7b, 0xf2, 0x6b, 0x6f, 0xc5, 0x30, 0x01, 0x67, 0x2b,
+	0xfe, 0xd7, 0xab, 0x76, 0xca, 0x82, 0xc9, 0x7d, 0xfa, 0x59, 0x47, 0xf0,
+	0xad, 0xd4, 0xa2, 0xaf, 0x9c, 0xa4, 0x72, 0xc0, 0xb7, 0xfd, 0x93, 0x26,
+	0x36, 0x3f, 0xf7, 0xcc, 0x34, 0xa5, 0xe5, 0xf1, 0x71, 0xd8, 0x31, 0x15,
+	0x04, 0xc7, 0x23, 0xc3, 0x18, 0x96, 0x05, 0x9a, 0x07, 0x12, 0x80, 0xe2,
+	0xeb, 0x27, 0xb2, 0x75, 0x09, 0x83, 0x2c, 0x1a, 0x1b, 0x6e, 0x5a, 0xa0,
+	0x52, 0x3b, 0xd6, 0xb3, 0x29, 0xe3, 0x2f, 0x84, 0x53, 0xd1, 0x00, 0xed,
+	0x20, 0xfc, 0xb1, 0x5b, 0x6a, 0xcb, 0xbe, 0x39, 0x4a, 0x4c, 0x58, 0xcf,
+	0xd0, 0xef, 0xaa, 0xfb, 0x43, 0x4d, 0x33, 0x85, 0x45, 0xf9, 0x02, 0x7f,
+	0x50, 0x3c, 0x9f, 0xa8, 0x51, 0xa3, 0x40, 0x8f, 0x92, 0x9d, 0x38, 0xf5,
+	0xbc, 0xb6, 0xda, 0x21, 0x10, 0xff, 0xf3, 0xd2, 0xcd, 0x0c, 0x13, 0xec,
+	0x5f, 0x97, 0x44, 0x17, 0xc4, 0xa7, 0x7e, 0x3d, 0x64, 0x5d, 0x19, 0x73,
+	0x60, 0x81, 0x4f, 0xdc, 0x22, 0x2a, 0x90, 0x88, 0x46, 0xee, 0xb8, 0x14,
+	0xde, 0x5e, 0x0b, 0xdb, 0xe0, 0x32, 0x3a, 0x0a, 0x49, 0x06, 0x24, 0x5c,
+	0xc2, 0xd3, 0xac, 0x62, 0x91, 0x95, 0xe4, 0x79, 0xe7, 0xc8, 0x37, 0x6d,
+	0x8d, 0xd5, 0x4e, 0xa9, 0x6c, 0x56, 0xf4, 0xea, 0x65, 0x7a, 0xae, 0x08,
+	0xba, 0x78, 0x25, 0x2e, 0x1c, 0xa6, 0xb4, 0xc6, 0xe8, 0xdd, 0x74, 0x1f,
+	0x4b, 0xbd, 0x8b, 0x8a, 0x70, 0x3e, 0xb5, 0x66, 0x48, 0x03, 0xf6, 0x0e,
+	0x61, 0x35, 0x57, 0xb9, 0x86, 0xc1, 0x1d, 0x9e, 0xe1, 0xf8, 0x98, 0x11,
+	0x69, 0xd9, 0x8e, 0x94, 0x9b, 0x1e, 0x87, 0xe9, 0xce, 0x55, 0x28, 0xdf,
+	0x8c, 0xa1, 0x89, 0x0d, 0xbf, 0xe6, 0x42, 0x68, 0x41, 0x99, 0x2d, 0x0f,
+	0xb0, 0x54, 0xbb, 0x16,
+};
+
+/* Substitute a byte with Sbox[byte]. Do it for a word for 4 bytes */
+static uint32_t
+sub_word(uint32_t word)
+{
+	word = (Sbox[(word >> 24) & 0xFF] << 24) |
+	       (Sbox[(word >> 16) & 0xFF] << 16) |
+	       (Sbox[(word >> 8) & 0xFF] << 8) | Sbox[word & 0xFF];
+	return word;
+}
+
+/* Rotate a word by one byte */
+static uint32_t
+rot_word(uint32_t word)
+{
+	return ((word >> 8) & 0xFFFFFF) | (word << 24);
+}
+
+/*
+ * Multiply with power of 2 and polynomial reduce the result using AES
+ * polynomial
+ */
+static uint8_t
+Xtime(uint8_t byte, uint8_t pow)
+{
+	uint32_t w = byte;
+
+	while (pow) {
+		w = w << 1;
+		if (w >> 8)
+			w ^= 0x11b;
+		pow--;
+	}
+
+	return (uint8_t)w;
+}
+
+/*
+ * Multiply a byte with another number such that the result is polynomial
+ * reduced in the GF8 space
+ */
+static uint8_t
+GF8mul(uint8_t byte, uint32_t mp)
+{
+	uint8_t pow, mul = 0;
+
+	while (mp) {
+		pow = ffs(mp) - 1;
+		mul ^= Xtime(byte, pow);
+		mp ^= (1 << pow);
+	}
+	return mul;
+}
+
+static void
+aes_key_expand(const uint8_t *key, uint32_t *ks)
+{
+	unsigned int i = 4;
+	uint32_t temp;
+
+	/* Skip key in ks */
+	memcpy(ks, key, KEY_WORD_LEN * sizeof(uint32_t));
+
+	while (i < KEY_SCHEDULE_LEN) {
+		temp = ks[i - 1];
+		if ((i & 0x3) == 0) {
+			temp = rot_word(temp);
+			temp = sub_word(temp);
+			temp ^= (uint32_t)GF8mul(1, 1 << ((i >> 2) - 1));
+		}
+		ks[i] = ks[i - 4] ^ temp;
+		i++;
+	}
+}
+
+/* Shift Rows(columns in state in this implementation) */
+static void
+shift_word(uint8_t *sRc, uint8_t c, int count)
+{
+	/* rotate across non-consecutive locations */
+	while (count) {
+		uint8_t t = sRc[c];
+
+		sRc[c] = sRc[0x4 + c];
+		sRc[0x4 + c] = sRc[0x8 + c];
+		sRc[0x8 + c] = sRc[0xc + c];
+		sRc[0xc + c] = t;
+		count--;
+	}
+}
+
+/* Mix Columns(rows in state in this implementation) */
+static void
+mix_columns(uint8_t *sRc)
+{
+	uint8_t new_st[4];
+	int i;
+
+	for (i = 0; i < 4; i++)
+		new_st[i] = GF8mul(sRc[i], 0x2) ^
+			    GF8mul(sRc[(i + 1) & 0x3], 0x3) ^
+			    sRc[(i + 2) & 0x3] ^ sRc[(i + 3) & 0x3];
+	for (i = 0; i < 4; i++)
+		sRc[i] = new_st[i];
+}
+
+static void
+cipher(uint8_t *in, uint8_t *out, uint32_t *ks)
+{
+	uint32_t state[KEY_WORD_LEN];
+	unsigned int i, round;
+
+	memcpy(state, in, sizeof(state));
+
+	/* AddRoundKey(state, w[0, Nb-1]) // See Sec. 5.1.4 */
+	for (i = 0; i < KEY_WORD_LEN; i++)
+		state[i] ^= ks[i];
+
+	for (round = 1; round < KEY_ROUNDS; round++) {
+		/* SubBytes(state) // See Sec. 5.1.1 */
+		for (i = 0; i < KEY_WORD_LEN; i++)
+			state[i] = sub_word(state[i]);
+
+		/* ShiftRows(state) // See Sec. 5.1.2 */
+		for (i = 0; i < KEY_WORD_LEN; i++)
+			shift_word((uint8_t *)state, i, i);
+
+		/* MixColumns(state) // See Sec. 5.1.3 */
+		for (i = 0; i < KEY_WORD_LEN; i++)
+			mix_columns((uint8_t *)&state[i]);
+
+		/* AddRoundKey(state, w[round*Nb, (round+1)*Nb-1]) */
+		for (i = 0; i < KEY_WORD_LEN; i++)
+			state[i] ^= ks[round * 4 + i];
+	}
+
+	/* SubBytes(state) */
+	for (i = 0; i < KEY_WORD_LEN; i++)
+		state[i] = sub_word(state[i]);
+
+	/* ShiftRows(state) */
+	for (i = 0; i < KEY_WORD_LEN; i++)
+		shift_word((uint8_t *)state, i, i);
+
+	/* AddRoundKey(state, w[Nr*Nb, (Nr+1)*Nb-1]) */
+	for (i = 0; i < KEY_WORD_LEN; i++)
+		state[i] ^= ks[KEY_ROUNDS * 4 + i];
+	memcpy(out, state, KEY_WORD_LEN * sizeof(uint32_t));
+}
+
+void
+roc_aes_xcbc_key_derive(const uint8_t *auth_key, uint8_t *derived_key)
+{
+	uint32_t aes_ks[KEY_SCHEDULE_LEN] = {0};
+	uint8_t k1[16] = {[0 ... 15] = 0x01};
+	uint8_t k2[16] = {[0 ... 15] = 0x02};
+	uint8_t k3[16] = {[0 ... 15] = 0x03};
+
+	aes_key_expand(auth_key, aes_ks);
+
+	cipher(k1, derived_key, aes_ks);
+	derived_key += sizeof(k1);
+
+	cipher(k2, derived_key, aes_ks);
+	derived_key += sizeof(k2);
+
+	cipher(k3, derived_key, aes_ks);
+}
diff --git a/drivers/common/cnxk/roc_aes.h b/drivers/common/cnxk/roc_aes.h
new file mode 100644
index 0000000..9540391
--- /dev/null
+++ b/drivers/common/cnxk/roc_aes.h
@@ -0,0 +1,14 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright (c) 2021 Marvell.
+ */
+
+#ifndef _ROC_AES_H_
+#define _ROC_AES_H_
+
+/*
+ * Derive k1, k2, k3 from 128 bit AES key
+ */
+void __roc_api roc_aes_xcbc_key_derive(const uint8_t *auth_key,
+				       uint8_t *derived_key);
+
+#endif /* _ROC_AES_H_ */
diff --git a/drivers/common/cnxk/roc_api.h b/drivers/common/cnxk/roc_api.h
index e7aaa07..cf4d487 100644
--- a/drivers/common/cnxk/roc_api.h
+++ b/drivers/common/cnxk/roc_api.h
@@ -90,6 +90,9 @@
 /* DPI */
 #include "roc_dpi.h"
 
+/* AES */
+#include "roc_aes.h"
+
 /* HASH computation */
 #include "roc_hash.h"
 
diff --git a/drivers/common/cnxk/roc_cpt.h b/drivers/common/cnxk/roc_cpt.h
index 12e6b81..99cb8b2 100644
--- a/drivers/common/cnxk/roc_cpt.h
+++ b/drivers/common/cnxk/roc_cpt.h
@@ -49,18 +49,18 @@
 #define ROC_CPT_AES_CBC_IV_LEN	 16
 #define ROC_CPT_SHA1_HMAC_LEN	 12
 #define ROC_CPT_SHA2_HMAC_LEN	 16
-#define ROC_CPT_AUTH_KEY_LEN_MAX 64
-
-#define ROC_CPT_DES3_KEY_LEN	  24
-#define ROC_CPT_AES128_KEY_LEN	  16
-#define ROC_CPT_AES192_KEY_LEN	  24
-#define ROC_CPT_AES256_KEY_LEN	  32
-#define ROC_CPT_MD5_KEY_LENGTH	  16
-#define ROC_CPT_SHA1_KEY_LENGTH	  20
-#define ROC_CPT_SHA256_KEY_LENGTH 32
-#define ROC_CPT_SHA384_KEY_LENGTH 48
-#define ROC_CPT_SHA512_KEY_LENGTH 64
-#define ROC_CPT_AUTH_KEY_LEN_MAX  64
+
+#define ROC_CPT_DES3_KEY_LEN	    24
+#define ROC_CPT_AES128_KEY_LEN	    16
+#define ROC_CPT_AES192_KEY_LEN	    24
+#define ROC_CPT_AES256_KEY_LEN	    32
+#define ROC_CPT_MD5_KEY_LENGTH	    16
+#define ROC_CPT_SHA1_KEY_LENGTH	    20
+#define ROC_CPT_SHA256_KEY_LENGTH   32
+#define ROC_CPT_SHA384_KEY_LENGTH   48
+#define ROC_CPT_SHA512_KEY_LENGTH   64
+#define ROC_CPT_AES_XCBC_KEY_LENGTH 16
+#define ROC_CPT_AUTH_KEY_LEN_MAX    64
 
 #define ROC_CPT_DES_BLOCK_LENGTH 8
 #define ROC_CPT_AES_BLOCK_LENGTH 16
diff --git a/drivers/common/cnxk/version.map b/drivers/common/cnxk/version.map
index 07c6720..b31e8eb 100644
--- a/drivers/common/cnxk/version.map
+++ b/drivers/common/cnxk/version.map
@@ -26,6 +26,7 @@ INTERNAL {
 	roc_ae_ec_grp_put;
 	roc_ae_fpm_get;
 	roc_ae_fpm_put;
+	roc_aes_xcbc_key_derive;
 	roc_bphy_cgx_dev_fini;
 	roc_bphy_cgx_dev_init;
 	roc_bphy_cgx_fec_set;
-- 
2.7.4

[PATCH v2 03/29] common/cnxk: add bit fields for params

[Anoob Joseph]

From: Archana Muniganti <marchana@marvell.com>

Added new structure with bit fields for params.

Signed-off-by: Archana Muniganti <marchana@marvell.com>
---
 drivers/common/cnxk/roc_ie_on.h  | 30 +++++++++++++++++++++++++++++-
 drivers/crypto/cnxk/cn9k_ipsec.c | 16 +++++++++++++---
 2 files changed, 42 insertions(+), 4 deletions(-)

diff --git a/drivers/common/cnxk/roc_ie_on.h b/drivers/common/cnxk/roc_ie_on.h
index 53591c6..817ef33 100644
--- a/drivers/common/cnxk/roc_ie_on.h
+++ b/drivers/common/cnxk/roc_ie_on.h
@@ -21,7 +21,6 @@ enum roc_ie_on_ucc_ipsec {
 };
 
 /* Helper macros */
-#define ROC_IE_ON_PER_PKT_IV   BIT(11)
 #define ROC_IE_ON_INB_RPTR_HDR 0x8
 
 enum {
@@ -102,6 +101,35 @@ struct roc_ie_on_ip_template {
 	};
 };
 
+union roc_on_ipsec_outb_param1 {
+	uint16_t u16;
+	struct {
+		uint16_t frag_num : 4;
+		uint16_t rsvd_4_6 : 3;
+		uint16_t gre_select : 1;
+		uint16_t dsiv : 1;
+		uint16_t ikev2 : 1;
+		uint16_t min_frag_size : 1;
+		uint16_t per_pkt_iv : 1;
+		uint16_t tfc_pad_enable : 1;
+		uint16_t tfc_dummy_pkt : 1;
+		uint16_t rfc_or_override_mode : 1;
+		uint16_t custom_hdr_or_p99 : 1;
+	} s;
+};
+
+union roc_on_ipsec_inb_param2 {
+	uint16_t u16;
+	struct {
+		uint16_t rsvd_0_10 : 11;
+		uint16_t gre_select : 1;
+		uint16_t ikev2 : 1;
+		uint16_t udp_cksum : 1;
+		uint16_t ctx_addr_sel : 1;
+		uint16_t custom_hdr_or_p99 : 1;
+	} s;
+};
+
 struct roc_ie_on_sa_ctl {
 	uint64_t spi : 32;
 	uint64_t exp_proto_inter_frag : 8;
diff --git a/drivers/crypto/cnxk/cn9k_ipsec.c b/drivers/crypto/cnxk/cn9k_ipsec.c
index a81130b..6455ef9 100644
--- a/drivers/crypto/cnxk/cn9k_ipsec.c
+++ b/drivers/crypto/cnxk/cn9k_ipsec.c
@@ -280,6 +280,7 @@ cn9k_ipsec_outb_sa_create(struct cnxk_cpt_qp *qp,
 	struct rte_crypto_sym_xform *auth_xform = crypto_xform->next;
 	struct roc_ie_on_ip_template *template = NULL;
 	struct roc_cpt *roc_cpt = qp->lf.roc_cpt;
+	union roc_on_ipsec_outb_param1 param1;
 	struct cnxk_cpt_inst_tmpl *inst_tmpl;
 	struct roc_ie_on_outb_sa *out_sa;
 	struct cn9k_sec_session *sess;
@@ -407,8 +408,12 @@ cn9k_ipsec_outb_sa_create(struct cnxk_cpt_qp *qp,
 	w4.u64 = 0;
 	w4.s.opcode_major = ROC_IE_ON_MAJOR_OP_PROCESS_OUTBOUND_IPSEC;
 	w4.s.opcode_minor = ctx_len >> 3;
-	w4.s.param1 = BIT(9);
-	w4.s.param1 |= ROC_IE_ON_PER_PKT_IV;
+
+	param1.u16 = 0;
+	param1.s.ikev2 = 1;
+	param1.s.per_pkt_iv = 1;
+	w4.s.param1 = param1.u16;
+
 	inst_tmpl->w4 = w4.u64;
 
 	w7.u64 = 0;
@@ -428,6 +433,7 @@ cn9k_ipsec_inb_sa_create(struct cnxk_cpt_qp *qp,
 {
 	struct rte_crypto_sym_xform *auth_xform = crypto_xform;
 	struct roc_cpt *roc_cpt = qp->lf.roc_cpt;
+	union roc_on_ipsec_inb_param2 param2;
 	struct cnxk_cpt_inst_tmpl *inst_tmpl;
 	struct roc_ie_on_inb_sa *in_sa;
 	struct cn9k_sec_session *sess;
@@ -478,7 +484,11 @@ cn9k_ipsec_inb_sa_create(struct cnxk_cpt_qp *qp,
 	w4.u64 = 0;
 	w4.s.opcode_major = ROC_IE_ON_MAJOR_OP_PROCESS_INBOUND_IPSEC;
 	w4.s.opcode_minor = ctx_len >> 3;
-	w4.s.param2 = BIT(12);
+
+	param2.u16 = 0;
+	param2.s.ikev2 = 1;
+	w4.s.param2 = param2.u16;
+
 	inst_tmpl->w4 = w4.u64;
 
 	w7.u64 = 0;
-- 
2.7.4

[PATCH v2 04/29] common/cnxk: fix reset of fields

[Anoob Joseph]

Copy DF/DSCP fields would get set based on ipsec_xform in the code
preceding this. Setting it again would cause the options to be reset.

Fixes: 78d03027f2cc ("common/cnxk: add IPsec common code")
Cc: schalla@marvell.com

Signed-off-by: Anoob Joseph <anoobj@marvell.com>
---
 drivers/common/cnxk/cnxk_security.c | 4 ----
 1 file changed, 4 deletions(-)

diff --git a/drivers/common/cnxk/cnxk_security.c b/drivers/common/cnxk/cnxk_security.c
index 30562b4..787138b 100644
--- a/drivers/common/cnxk/cnxk_security.c
+++ b/drivers/common/cnxk/cnxk_security.c
@@ -444,10 +444,6 @@ cnxk_ot_ipsec_outb_sa_fill(struct roc_ot_ipsec_outb_sa *sa,
 		return -EINVAL;
 	}
 
-	/* Default options of DSCP and Flow label/DF */
-	sa->w2.s.dscp_src = ROC_IE_OT_SA_COPY_FROM_SA;
-	sa->w2.s.ipv4_df_src_or_ipv6_flw_lbl_src = ROC_IE_OT_SA_COPY_FROM_SA;
-
 skip_tunnel_info:
 	/* ESN */
 	sa->w0.s.esn_en = !!ipsec_xfrm->options.esn;
-- 
2.7.4

[PATCH v2 05/29] common/cnxk: verify input args

[Anoob Joseph]

Add input arg verification.

Signed-off-by: Anoob Joseph <anoobj@marvell.com>
---
 drivers/common/cnxk/hw/cpt.h  | 2 ++
 drivers/common/cnxk/roc_cpt.c | 4 +++-
 2 files changed, 5 insertions(+), 1 deletion(-)

diff --git a/drivers/common/cnxk/hw/cpt.h b/drivers/common/cnxk/hw/cpt.h
index 919f842..ccc7af4 100644
--- a/drivers/common/cnxk/hw/cpt.h
+++ b/drivers/common/cnxk/hw/cpt.h
@@ -64,6 +64,7 @@ union cpt_lf_ctx_flush {
 	struct {
 		uint64_t cptr : 46;
 		uint64_t inval : 1;
+		uint64_t reserved_47_63 : 17;
 	} s;
 };
 
@@ -71,6 +72,7 @@ union cpt_lf_ctx_reload {
 	uint64_t u;
 	struct {
 		uint64_t cptr : 46;
+		uint64_t reserved_46_63 : 18;
 	} s;
 };
 
diff --git a/drivers/common/cnxk/roc_cpt.c b/drivers/common/cnxk/roc_cpt.c
index 8f8e6d3..1bc7a29 100644
--- a/drivers/common/cnxk/roc_cpt.c
+++ b/drivers/common/cnxk/roc_cpt.c
@@ -681,8 +681,10 @@ roc_cpt_lf_ctx_flush(struct roc_cpt_lf *lf, void *cptr, bool inval)
 {
 	union cpt_lf_ctx_flush reg;
 
-	if (lf == NULL)
+	if (lf == NULL) {
+		plt_err("Could not trigger CTX flush");
 		return -ENOTSUP;
+	}
 
 	reg.u = 0;
 	reg.s.inval = inval;
-- 
2.7.4

[PATCH v2 06/29] common/cnxk: update completion code

[Anoob Joseph]

Update completion code to match v1.19 microcode release.

Signed-off-by: Anoob Joseph <anoobj@marvell.com>
---
 drivers/common/cnxk/roc_ie_ot.h | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/drivers/common/cnxk/roc_ie_ot.h b/drivers/common/cnxk/roc_ie_ot.h
index 5b61902..923656f 100644
--- a/drivers/common/cnxk/roc_ie_ot.h
+++ b/drivers/common/cnxk/roc_ie_ot.h
@@ -43,8 +43,8 @@ enum roc_ie_ot_ucc_ipsec {
 	ROC_IE_OT_UCC_ERR_SA_ESP_BAD_KEYS = 0xc5,
 	ROC_IE_OT_UCC_ERR_SA_AH_BAD_KEYS = 0xc6,
 	ROC_IE_OT_UCC_ERR_SA_BAD_IP = 0xc7,
-	ROC_IE_OT_UCC_ERR_PKT_REPLAY_WINDOW = 0xc8,
-	ROC_IE_OT_UCC_ERR_PKT_IP_FRAG = 0xc9,
+	ROC_IE_OT_UCC_ERR_PKT_IP_FRAG = 0xc8,
+	ROC_IE_OT_UCC_ERR_PKT_REPLAY_WINDOW = 0xc9,
 	ROC_IE_OT_UCC_SUCCESS_SA_SOFTEXP_FIRST = 0xf0,
 	ROC_IE_OT_UCC_SUCCESS_PKT_IP_BADCSUM = 0xf1,
 	ROC_IE_OT_UCC_SUCCESS_SA_SOFTEXP_AGAIN = 0xf2,
-- 
2.7.4

[PATCH v2 07/29] crypto/cnxk: only enable queues that are allocated

[Anoob Joseph]

From: Shijith Thotton <sthotton@marvell.com>

Only enable/disable queue pairs that are allocated during cryptodev
start/stop.

Fixes: 6a95dbc1a291 ("crypto/cnxk: add dev start and dev stop")

Signed-off-by: Shijith Thotton <sthotton@marvell.com>
---
 drivers/crypto/cnxk/cnxk_cryptodev_ops.c | 13 +++++++++++--
 1 file changed, 11 insertions(+), 2 deletions(-)

diff --git a/drivers/crypto/cnxk/cnxk_cryptodev_ops.c b/drivers/crypto/cnxk/cnxk_cryptodev_ops.c
index a2281fb..21ee09f 100644
--- a/drivers/crypto/cnxk/cnxk_cryptodev_ops.c
+++ b/drivers/crypto/cnxk/cnxk_cryptodev_ops.c
@@ -100,8 +100,13 @@ cnxk_cpt_dev_start(struct rte_cryptodev *dev)
 	uint16_t nb_lf = roc_cpt->nb_lf;
 	uint16_t qp_id;
 
-	for (qp_id = 0; qp_id < nb_lf; qp_id++)
+	for (qp_id = 0; qp_id < nb_lf; qp_id++) {
+		/* Application may not setup all queue pair */
+		if (roc_cpt->lf[qp_id] == NULL)
+			continue;
+
 		roc_cpt_iq_enable(roc_cpt->lf[qp_id]);
+	}
 
 	return 0;
 }
@@ -114,8 +119,12 @@ cnxk_cpt_dev_stop(struct rte_cryptodev *dev)
 	uint16_t nb_lf = roc_cpt->nb_lf;
 	uint16_t qp_id;
 
-	for (qp_id = 0; qp_id < nb_lf; qp_id++)
+	for (qp_id = 0; qp_id < nb_lf; qp_id++) {
+		if (roc_cpt->lf[qp_id] == NULL)
+			continue;
+
 		roc_cpt_iq_disable(roc_cpt->lf[qp_id]);
+	}
 }
 
 int
-- 
2.7.4

[PATCH v2 08/29] crypto/cnxk: add lookaside IPsec AES-CBC-HMAC-SHA256 support

[Anoob Joseph]

From: Tejasree Kondoj <ktejasree@marvell.com>

Adding AES-CBC-HMAC-SHA256 support to lookaside IPsec PMD.

Signed-off-by: Tejasree Kondoj <ktejasree@marvell.com>
---
 doc/guides/cryptodevs/cnxk.rst                    | 39 +++++++++++++++++++----
 doc/guides/rel_notes/release_22_03.rst            |  4 +++
 drivers/common/cnxk/cnxk_security.c               | 14 ++++++++
 drivers/crypto/cnxk/cn10k_ipsec.c                 |  3 ++
 drivers/crypto/cnxk/cnxk_cryptodev_capabilities.c | 20 ++++++++++++
 drivers/crypto/cnxk/cnxk_ipsec.h                  |  3 +-
 6 files changed, 75 insertions(+), 8 deletions(-)

diff --git a/doc/guides/cryptodevs/cnxk.rst b/doc/guides/cryptodevs/cnxk.rst
index 23cc823..8c4c4ea 100644
--- a/doc/guides/cryptodevs/cnxk.rst
+++ b/doc/guides/cryptodevs/cnxk.rst
@@ -246,14 +246,27 @@ CN9XX Features supported
 * IPv4
 * IPv6
 * ESP
+* ESN
+* Anti-replay
 * Tunnel mode
 * Transport mode(IPv4)
 * UDP Encapsulation
+
+AEAD algorithms
++++++++++++++++
+
 * AES-128/192/256-GCM
-* AES-128/192/256-CBC-SHA1-HMAC
-* AES-128/192/256-CBC-SHA256-128-HMAC
-* ESN
-* Anti-replay
+
+Cipher algorithms
++++++++++++++++++
+
+* AES-128/192/256-CBC
+
+Auth algorithms
++++++++++++++++
+
+* SHA1-HMAC
+* SHA256-128-HMAC
 
 CN10XX Features supported
 ~~~~~~~~~~~~~~~~~~~~~~~~~
@@ -263,6 +276,20 @@ CN10XX Features supported
 * Tunnel mode
 * Transport mode
 * UDP Encapsulation
+
+AEAD algorithms
++++++++++++++++
+
 * AES-128/192/256-GCM
-* AES-128/192/256-CBC-NULL
-* AES-128/192/256-CBC-SHA1-HMAC
+
+Cipher algorithms
++++++++++++++++++
+
+* AES-128/192/256-CBC
+
+Auth algorithms
++++++++++++++++
+
+* NULL
+* SHA1-HMAC
+* SHA256-128-HMAC
diff --git a/doc/guides/rel_notes/release_22_03.rst b/doc/guides/rel_notes/release_22_03.rst
index 6d99d1e..1639b0e 100644
--- a/doc/guides/rel_notes/release_22_03.rst
+++ b/doc/guides/rel_notes/release_22_03.rst
@@ -55,6 +55,10 @@ New Features
      Also, make sure to start the actual text at the margin.
      =======================================================
 
+* **Updated Marvell cnxk crypto PMD.**
+
+  * Added SHA256-HMAC support in lookaside protocol (IPsec) for CN10K.
+
 
 Removed Items
 -------------
diff --git a/drivers/common/cnxk/cnxk_security.c b/drivers/common/cnxk/cnxk_security.c
index 787138b..f39bc1e 100644
--- a/drivers/common/cnxk/cnxk_security.c
+++ b/drivers/common/cnxk/cnxk_security.c
@@ -32,6 +32,10 @@ ipsec_hmac_opad_ipad_gen(struct rte_crypto_sym_xform *auth_xform,
 		roc_hash_sha1_gen(opad, (uint32_t *)&hmac_opad_ipad[0]);
 		roc_hash_sha1_gen(ipad, (uint32_t *)&hmac_opad_ipad[24]);
 		break;
+	case RTE_CRYPTO_AUTH_SHA256_HMAC:
+		roc_hash_sha256_gen(opad, (uint32_t *)&hmac_opad_ipad[0]);
+		roc_hash_sha256_gen(ipad, (uint32_t *)&hmac_opad_ipad[64]);
+		break;
 	default:
 		break;
 	}
@@ -129,6 +133,16 @@ ot_ipsec_sa_common_param_fill(union roc_ot_ipsec_sa_word2 *w2,
 			     i++)
 				tmp_key[i] = rte_be_to_cpu_64(tmp_key[i]);
 			break;
+		case RTE_CRYPTO_AUTH_SHA256_HMAC:
+			w2->s.auth_type = ROC_IE_OT_SA_AUTH_SHA2_256;
+			ipsec_hmac_opad_ipad_gen(auth_xfrm, hmac_opad_ipad);
+
+			tmp_key = (uint64_t *)hmac_opad_ipad;
+			for (i = 0; i < (int)(ROC_CTX_MAX_OPAD_IPAD_LEN /
+					      sizeof(uint64_t));
+			     i++)
+				tmp_key[i] = rte_be_to_cpu_64(tmp_key[i]);
+			break;
 		default:
 			return -ENOTSUP;
 		}
diff --git a/drivers/crypto/cnxk/cn10k_ipsec.c b/drivers/crypto/cnxk/cn10k_ipsec.c
index 27df1dc..93eab1b 100644
--- a/drivers/crypto/cnxk/cn10k_ipsec.c
+++ b/drivers/crypto/cnxk/cn10k_ipsec.c
@@ -65,6 +65,9 @@ cn10k_ipsec_outb_sa_create(struct roc_cpt *roc_cpt,
 		if (crypto_xfrm->type == RTE_CRYPTO_SYM_XFORM_AEAD) {
 			sa->iv_offset = crypto_xfrm->aead.iv.offset;
 			sa->iv_length = crypto_xfrm->aead.iv.length;
+		} else {
+			sa->iv_offset = crypto_xfrm->cipher.iv.offset;
+			sa->iv_length = crypto_xfrm->cipher.iv.length;
 		}
 	}
 #else
diff --git a/drivers/crypto/cnxk/cnxk_cryptodev_capabilities.c b/drivers/crypto/cnxk/cnxk_cryptodev_capabilities.c
index 59b63ed..7d22626 100644
--- a/drivers/crypto/cnxk/cnxk_cryptodev_capabilities.c
+++ b/drivers/crypto/cnxk/cnxk_cryptodev_capabilities.c
@@ -797,6 +797,26 @@ static const struct rte_cryptodev_capabilities sec_caps_sha1_sha2[] = {
 			}, }
 		}, }
 	},
+	{	/* SHA256 HMAC */
+		.op = RTE_CRYPTO_OP_TYPE_SYMMETRIC,
+		{.sym = {
+			.xform_type = RTE_CRYPTO_SYM_XFORM_AUTH,
+			{.auth = {
+				.algo = RTE_CRYPTO_AUTH_SHA256_HMAC,
+				.block_size = 64,
+				.key_size = {
+					.min = 1,
+					.max = 1024,
+					.increment = 1
+				},
+				.digest_size = {
+					.min = 16,
+					.max = 16,
+					.increment = 0
+				},
+			}, }
+		}, }
+	},
 };
 
 static const struct rte_security_capability sec_caps_templ[] = {
diff --git a/drivers/crypto/cnxk/cnxk_ipsec.h b/drivers/crypto/cnxk/cnxk_ipsec.h
index dddb414..f4a1012 100644
--- a/drivers/crypto/cnxk/cnxk_ipsec.h
+++ b/drivers/crypto/cnxk/cnxk_ipsec.h
@@ -46,8 +46,7 @@ ipsec_xform_auth_verify(struct rte_crypto_sym_xform *crypto_xform)
 	if (crypto_xform->auth.algo == RTE_CRYPTO_AUTH_SHA1_HMAC) {
 		if (keylen >= 20 && keylen <= 64)
 			return 0;
-	} else if (roc_model_is_cn9k() &&
-		   (crypto_xform->auth.algo == RTE_CRYPTO_AUTH_SHA256_HMAC)) {
+	} else if (crypto_xform->auth.algo == RTE_CRYPTO_AUTH_SHA256_HMAC) {
 		if (keylen >= 32 && keylen <= 64)
 			return 0;
 	}
-- 
2.7.4

[PATCH v2 09/29] crypto/cnxk: clear session data before populating

[Anoob Joseph]

Clear session data before populating fields to not have garbage data.

Signed-off-by: Anoob Joseph <anoobj@marvell.com>
---
 drivers/crypto/cnxk/cn10k_ipsec.c | 2 ++
 1 file changed, 2 insertions(+)

diff --git a/drivers/crypto/cnxk/cn10k_ipsec.c b/drivers/crypto/cnxk/cn10k_ipsec.c
index 93eab1b..1bd127e 100644
--- a/drivers/crypto/cnxk/cn10k_ipsec.c
+++ b/drivers/crypto/cnxk/cn10k_ipsec.c
@@ -130,6 +130,8 @@ cn10k_ipsec_inb_sa_create(struct roc_cpt *roc_cpt,
 	sa = &sess->sa;
 	in_sa = &sa->in_sa;
 
+	memset(in_sa, 0, sizeof(struct roc_ot_ipsec_inb_sa));
+
 	/* Translate security parameters to SA */
 	ret = cnxk_ot_ipsec_inb_sa_fill(in_sa, ipsec_xfrm, crypto_xfrm);
 	if (ret)
-- 
2.7.4

[PATCH v2 10/29] crypto/cnxk: update max sec crypto caps

[Anoob Joseph]

Update the macro to include newly added ciphers. Updated the functions
populating caps to throw error when max is exceeded.

Signed-off-by: Anoob Joseph <anoobj@marvell.com>
---
 drivers/crypto/cnxk/cnxk_cryptodev.h              | 2 +-
 drivers/crypto/cnxk/cnxk_cryptodev_capabilities.c | 8 ++++++--
 2 files changed, 7 insertions(+), 3 deletions(-)

diff --git a/drivers/crypto/cnxk/cnxk_cryptodev.h b/drivers/crypto/cnxk/cnxk_cryptodev.h
index cfb9d29..2e0f467 100644
--- a/drivers/crypto/cnxk/cnxk_cryptodev.h
+++ b/drivers/crypto/cnxk/cnxk_cryptodev.h
@@ -11,7 +11,7 @@
 #include "roc_cpt.h"
 
 #define CNXK_CPT_MAX_CAPS	 34
-#define CNXK_SEC_CRYPTO_MAX_CAPS 4
+#define CNXK_SEC_CRYPTO_MAX_CAPS 6
 #define CNXK_SEC_MAX_CAPS	 5
 #define CNXK_AE_EC_ID_MAX	 8
 /**
diff --git a/drivers/crypto/cnxk/cnxk_cryptodev_capabilities.c b/drivers/crypto/cnxk/cnxk_cryptodev_capabilities.c
index 7d22626..8305341 100644
--- a/drivers/crypto/cnxk/cnxk_cryptodev_capabilities.c
+++ b/drivers/crypto/cnxk/cnxk_cryptodev_capabilities.c
@@ -943,8 +943,10 @@ static void
 sec_caps_add(struct rte_cryptodev_capabilities cnxk_caps[], int *cur_pos,
 	     const struct rte_cryptodev_capabilities *caps, int nb_caps)
 {
-	if (*cur_pos + nb_caps > CNXK_SEC_CRYPTO_MAX_CAPS)
+	if (*cur_pos + nb_caps > CNXK_SEC_CRYPTO_MAX_CAPS) {
+		rte_panic("Could not add sec crypto caps");
 		return;
+	}
 
 	memcpy(&cnxk_caps[*cur_pos], caps, nb_caps * sizeof(caps[0]));
 	*cur_pos += nb_caps;
@@ -957,8 +959,10 @@ cn10k_sec_crypto_caps_update(struct rte_cryptodev_capabilities cnxk_caps[],
 	const struct rte_cryptodev_capabilities *cap;
 	unsigned int i;
 
-	if ((CNXK_CPT_MAX_CAPS - *cur_pos) < 1)
+	if ((CNXK_SEC_CRYPTO_MAX_CAPS - *cur_pos) < 1) {
+		rte_panic("Could not add sec crypto caps");
 		return;
+	}
 
 	/* NULL auth */
 	for (i = 0; i < RTE_DIM(caps_null); i++) {
-- 
2.7.4

[PATCH v2 11/29] crypto/cnxk: write CPT CTX through microcode op

[Anoob Joseph]

From: Tejasree Kondoj <ktejasree@marvell.com>

Adding support to write CPT CTX through microcode op(SET_CTX) for
cn10k lookaside PMD.

Signed-off-by: Tejasree Kondoj <ktejasree@marvell.com>
---
 drivers/crypto/cnxk/cn10k_ipsec.c | 121 ++++++++++++++++++++++++++++----------
 1 file changed, 89 insertions(+), 32 deletions(-)

diff --git a/drivers/crypto/cnxk/cn10k_ipsec.c b/drivers/crypto/cnxk/cn10k_ipsec.c
index 1bd127e..a11a6b7 100644
--- a/drivers/crypto/cnxk/cn10k_ipsec.c
+++ b/drivers/crypto/cnxk/cn10k_ipsec.c
@@ -2,18 +2,19 @@
  * Copyright(C) 2021 Marvell.
  */
 
-#include <rte_malloc.h>
 #include <cryptodev_pmd.h>
 #include <rte_esp.h>
 #include <rte_ip.h>
+#include <rte_malloc.h>
 #include <rte_security.h>
 #include <rte_security_driver.h>
 #include <rte_udp.h>
 
+#include "cn10k_ipsec.h"
 #include "cnxk_cryptodev.h"
+#include "cnxk_cryptodev_ops.h"
 #include "cnxk_ipsec.h"
 #include "cnxk_security.h"
-#include "cn10k_ipsec.h"
 
 #include "roc_api.h"
 
@@ -32,36 +33,46 @@ ipsec_cpt_inst_w7_get(struct roc_cpt *roc_cpt, void *sa)
 }
 
 static int
-cn10k_ipsec_outb_sa_create(struct roc_cpt *roc_cpt,
+cn10k_ipsec_outb_sa_create(struct roc_cpt *roc_cpt, struct roc_cpt_lf *lf,
 			   struct rte_security_ipsec_xform *ipsec_xfrm,
 			   struct rte_crypto_sym_xform *crypto_xfrm,
 			   struct rte_security_session *sec_sess)
 {
 	union roc_ot_ipsec_outb_param1 param1;
-	struct roc_ot_ipsec_outb_sa *out_sa;
+	struct roc_ot_ipsec_outb_sa *sa_dptr;
 	struct cnxk_ipsec_outb_rlens rlens;
 	struct cn10k_sec_session *sess;
 	struct cn10k_ipsec_sa *sa;
 	union cpt_inst_w4 inst_w4;
-	int ret;
+	void *out_sa;
+	int ret = 0;
 
 	sess = get_sec_session_private_data(sec_sess);
 	sa = &sess->sa;
 	out_sa = &sa->out_sa;
 
-	memset(out_sa, 0, sizeof(struct roc_ot_ipsec_outb_sa));
+	/* Allocate memory to be used as dptr for CPT ucode WRITE_SA op */
+	sa_dptr = plt_zmalloc(ROC_NIX_INL_OT_IPSEC_OUTB_HW_SZ, 0);
+	if (sa_dptr == NULL) {
+		plt_err("Couldn't allocate memory for SA dptr");
+		return -ENOMEM;
+	}
+
+	memset(sa_dptr, 0, sizeof(struct roc_ot_ipsec_outb_sa));
 
 	/* Translate security parameters to SA */
-	ret = cnxk_ot_ipsec_outb_sa_fill(out_sa, ipsec_xfrm, crypto_xfrm);
-	if (ret)
-		return ret;
+	ret = cnxk_ot_ipsec_outb_sa_fill(sa_dptr, ipsec_xfrm, crypto_xfrm);
+	if (ret) {
+		plt_err("Could not fill outbound session parameters");
+		goto sa_dptr_free;
+	}
 
 	sa->inst.w7 = ipsec_cpt_inst_w7_get(roc_cpt, sa);
 
 #ifdef LA_IPSEC_DEBUG
 	/* Use IV from application in debug mode */
 	if (ipsec_xfrm->options.iv_gen_disable == 1) {
-		out_sa->w2.s.iv_src = ROC_IE_OT_SA_IV_SRC_FROM_SA;
+		sa_dptr->w2.s.iv_src = ROC_IE_OT_SA_IV_SRC_FROM_SA;
 		if (crypto_xfrm->type == RTE_CRYPTO_SYM_XFORM_AEAD) {
 			sa->iv_offset = crypto_xfrm->aead.iv.offset;
 			sa->iv_length = crypto_xfrm->aead.iv.length;
@@ -73,14 +84,15 @@ cn10k_ipsec_outb_sa_create(struct roc_cpt *roc_cpt,
 #else
 	if (ipsec_xfrm->options.iv_gen_disable != 0) {
 		plt_err("Application provided IV not supported");
-		return -ENOTSUP;
+		ret = -ENOTSUP;
+		goto sa_dptr_free;
 	}
 #endif
 
 	/* Get Rlen calculation data */
 	ret = cnxk_ipsec_outb_rlens_get(&rlens, ipsec_xfrm, crypto_xfrm);
 	if (ret)
-		return ret;
+		goto sa_dptr_free;
 
 	sa->max_extended_len = rlens.max_extended_len;
 
@@ -110,37 +122,61 @@ cn10k_ipsec_outb_sa_create(struct roc_cpt *roc_cpt,
 
 	sa->inst.w4 = inst_w4.u64;
 
-	return 0;
+	memset(out_sa, 0, sizeof(struct roc_ot_ipsec_outb_sa));
+
+	/* Copy word0 from sa_dptr to populate ctx_push_sz ctx_size fields */
+	memcpy(out_sa, sa_dptr, 8);
+
+	/* Write session using microcode opcode */
+	ret = roc_cpt_ctx_write(lf, sa_dptr, out_sa,
+				ROC_NIX_INL_OT_IPSEC_OUTB_HW_SZ);
+	if (ret) {
+		plt_err("Could not write outbound session to hardware");
+		goto sa_dptr_free;
+	}
+
+	/* Trigger CTX flush to write dirty data back to DRAM */
+	roc_cpt_lf_ctx_flush(lf, out_sa, false);
+
+sa_dptr_free:
+	plt_free(sa_dptr);
+
+	return ret;
 }
 
 static int
-cn10k_ipsec_inb_sa_create(struct roc_cpt *roc_cpt,
+cn10k_ipsec_inb_sa_create(struct roc_cpt *roc_cpt, struct roc_cpt_lf *lf,
 			  struct rte_security_ipsec_xform *ipsec_xfrm,
 			  struct rte_crypto_sym_xform *crypto_xfrm,
 			  struct rte_security_session *sec_sess)
 {
 	union roc_ot_ipsec_inb_param1 param1;
-	struct roc_ot_ipsec_inb_sa *in_sa;
+	struct roc_ot_ipsec_inb_sa *sa_dptr;
 	struct cn10k_sec_session *sess;
 	struct cn10k_ipsec_sa *sa;
 	union cpt_inst_w4 inst_w4;
-	int ret;
+	void *in_sa;
+	int ret = 0;
 
 	sess = get_sec_session_private_data(sec_sess);
 	sa = &sess->sa;
 	in_sa = &sa->in_sa;
 
-	memset(in_sa, 0, sizeof(struct roc_ot_ipsec_inb_sa));
-
-	/* Translate security parameters to SA */
-	ret = cnxk_ot_ipsec_inb_sa_fill(in_sa, ipsec_xfrm, crypto_xfrm);
-	if (ret)
-		return ret;
+	/* Allocate memory to be used as dptr for CPT ucode WRITE_SA op */
+	sa_dptr = plt_zmalloc(ROC_NIX_INL_OT_IPSEC_INB_HW_SZ, 0);
+	if (sa_dptr == NULL) {
+		plt_err("Couldn't allocate memory for SA dptr");
+		return -ENOMEM;
+	}
 
-	/* TODO add support for antireplay */
-	sa->in_sa.w0.s.ar_win = 0;
+	memset(sa_dptr, 0, sizeof(struct roc_ot_ipsec_inb_sa));
 
-	/* TODO add support for udp encap */
+	/* Translate security parameters to SA */
+	ret = cnxk_ot_ipsec_inb_sa_fill(sa_dptr, ipsec_xfrm, crypto_xfrm);
+	if (ret) {
+		plt_err("Could not fill inbound session parameters");
+		goto sa_dptr_free;
+	}
 
 	sa->inst.w7 = ipsec_cpt_inst_w7_get(roc_cpt, sa);
 
@@ -173,7 +209,26 @@ cn10k_ipsec_inb_sa_create(struct roc_cpt *roc_cpt,
 
 	sa->inst.w4 = inst_w4.u64;
 
-	return 0;
+	memset(in_sa, 0, sizeof(struct roc_ot_ipsec_inb_sa));
+
+	/* Copy word0 from sa_dptr to populate ctx_push_sz ctx_size fields */
+	memcpy(in_sa, sa_dptr, 8);
+
+	/* Write session using microcode opcode */
+	ret = roc_cpt_ctx_write(lf, sa_dptr, in_sa,
+				ROC_NIX_INL_OT_IPSEC_INB_HW_SZ);
+	if (ret) {
+		plt_err("Could not write inbound session to hardware");
+		goto sa_dptr_free;
+	}
+
+	/* Trigger CTX flush to write dirty data back to DRAM */
+	roc_cpt_lf_ctx_flush(lf, in_sa, false);
+
+sa_dptr_free:
+	plt_free(sa_dptr);
+
+	return ret;
 }
 
 static int
@@ -185,12 +240,11 @@ cn10k_ipsec_session_create(void *dev,
 	struct rte_cryptodev *crypto_dev = dev;
 	struct roc_cpt *roc_cpt;
 	struct cnxk_cpt_vf *vf;
+	struct cnxk_cpt_qp *qp;
 	int ret;
 
-	vf = crypto_dev->data->dev_private;
-	roc_cpt = &vf->cpt;
-
-	if (crypto_dev->data->queue_pairs[0] == NULL) {
+	qp = crypto_dev->data->queue_pairs[0];
+	if (qp == NULL) {
 		plt_err("Setup cpt queue pair before creating security session");
 		return -EPERM;
 	}
@@ -199,11 +253,14 @@ cn10k_ipsec_session_create(void *dev,
 	if (ret)
 		return ret;
 
+	vf = crypto_dev->data->dev_private;
+	roc_cpt = &vf->cpt;
+
 	if (ipsec_xfrm->direction == RTE_SECURITY_IPSEC_SA_DIR_INGRESS)
-		return cn10k_ipsec_inb_sa_create(roc_cpt, ipsec_xfrm,
+		return cn10k_ipsec_inb_sa_create(roc_cpt, &qp->lf, ipsec_xfrm,
 						 crypto_xfrm, sess);
 	else
-		return cn10k_ipsec_outb_sa_create(roc_cpt, ipsec_xfrm,
+		return cn10k_ipsec_outb_sa_create(roc_cpt, &qp->lf, ipsec_xfrm,
 						  crypto_xfrm, sess);
 }
 
-- 
2.7.4

[PATCH v2 12/29] crypto/cnxk: support cnxk lookaside IPsec HMAC-SHA384/512

[Anoob Joseph]

From: Tejasree Kondoj <ktejasree@marvell.com>

Adding HMAC-SHA384/512 support to cnxk lookaside IPsec.

Signed-off-by: Tejasree Kondoj <ktejasree@marvell.com>
---
 doc/guides/cryptodevs/cnxk.rst                    |  4 ++
 doc/guides/rel_notes/release_22_03.rst            |  2 +
 drivers/common/cnxk/cnxk_security.c               | 36 +++++++++------
 drivers/crypto/cnxk/cn9k_ipsec.c                  | 55 ++++++++++++++++++-----
 drivers/crypto/cnxk/cnxk_cryptodev.h              |  2 +-
 drivers/crypto/cnxk/cnxk_cryptodev_capabilities.c | 40 +++++++++++++++++
 drivers/crypto/cnxk/cnxk_ipsec.h                  |  6 +++
 7 files changed, 118 insertions(+), 27 deletions(-)

diff --git a/doc/guides/cryptodevs/cnxk.rst b/doc/guides/cryptodevs/cnxk.rst
index 8c4c4ea..c49a779 100644
--- a/doc/guides/cryptodevs/cnxk.rst
+++ b/doc/guides/cryptodevs/cnxk.rst
@@ -267,6 +267,8 @@ Auth algorithms
 
 * SHA1-HMAC
 * SHA256-128-HMAC
+* SHA384-192-HMAC
+* SHA512-256-HMAC
 
 CN10XX Features supported
 ~~~~~~~~~~~~~~~~~~~~~~~~~
@@ -293,3 +295,5 @@ Auth algorithms
 * NULL
 * SHA1-HMAC
 * SHA256-128-HMAC
+* SHA384-192-HMAC
+* SHA512-256-HMAC
diff --git a/doc/guides/rel_notes/release_22_03.rst b/doc/guides/rel_notes/release_22_03.rst
index 1639b0e..8df9092 100644
--- a/doc/guides/rel_notes/release_22_03.rst
+++ b/doc/guides/rel_notes/release_22_03.rst
@@ -58,6 +58,8 @@ New Features
 * **Updated Marvell cnxk crypto PMD.**
 
   * Added SHA256-HMAC support in lookaside protocol (IPsec) for CN10K.
+  * Added SHA384-HMAC support in lookaside protocol (IPsec) for CN9K & CN10K.
+  * Added SHA512-HMAC support in lookaside protocol (IPsec) for CN9K & CN10K.
 
 
 Removed Items
diff --git a/drivers/common/cnxk/cnxk_security.c b/drivers/common/cnxk/cnxk_security.c
index f39bc1e..1c86f82 100644
--- a/drivers/common/cnxk/cnxk_security.c
+++ b/drivers/common/cnxk/cnxk_security.c
@@ -36,6 +36,14 @@ ipsec_hmac_opad_ipad_gen(struct rte_crypto_sym_xform *auth_xform,
 		roc_hash_sha256_gen(opad, (uint32_t *)&hmac_opad_ipad[0]);
 		roc_hash_sha256_gen(ipad, (uint32_t *)&hmac_opad_ipad[64]);
 		break;
+	case RTE_CRYPTO_AUTH_SHA384_HMAC:
+		roc_hash_sha512_gen(opad, (uint64_t *)&hmac_opad_ipad[0], 384);
+		roc_hash_sha512_gen(ipad, (uint64_t *)&hmac_opad_ipad[64], 384);
+		break;
+	case RTE_CRYPTO_AUTH_SHA512_HMAC:
+		roc_hash_sha512_gen(opad, (uint64_t *)&hmac_opad_ipad[0], 512);
+		roc_hash_sha512_gen(ipad, (uint64_t *)&hmac_opad_ipad[64], 512);
+		break;
 	default:
 		break;
 	}
@@ -125,28 +133,28 @@ ot_ipsec_sa_common_param_fill(union roc_ot_ipsec_sa_word2 *w2,
 			break;
 		case RTE_CRYPTO_AUTH_SHA1_HMAC:
 			w2->s.auth_type = ROC_IE_OT_SA_AUTH_SHA1;
-			ipsec_hmac_opad_ipad_gen(auth_xfrm, hmac_opad_ipad);
-
-			tmp_key = (uint64_t *)hmac_opad_ipad;
-			for (i = 0; i < (int)(ROC_CTX_MAX_OPAD_IPAD_LEN /
-					      sizeof(uint64_t));
-			     i++)
-				tmp_key[i] = rte_be_to_cpu_64(tmp_key[i]);
 			break;
 		case RTE_CRYPTO_AUTH_SHA256_HMAC:
 			w2->s.auth_type = ROC_IE_OT_SA_AUTH_SHA2_256;
-			ipsec_hmac_opad_ipad_gen(auth_xfrm, hmac_opad_ipad);
-
-			tmp_key = (uint64_t *)hmac_opad_ipad;
-			for (i = 0; i < (int)(ROC_CTX_MAX_OPAD_IPAD_LEN /
-					      sizeof(uint64_t));
-			     i++)
-				tmp_key[i] = rte_be_to_cpu_64(tmp_key[i]);
+			break;
+		case RTE_CRYPTO_AUTH_SHA384_HMAC:
+			w2->s.auth_type = ROC_IE_OT_SA_AUTH_SHA2_384;
+			break;
+		case RTE_CRYPTO_AUTH_SHA512_HMAC:
+			w2->s.auth_type = ROC_IE_OT_SA_AUTH_SHA2_512;
 			break;
 		default:
 			return -ENOTSUP;
 		}
 
+		ipsec_hmac_opad_ipad_gen(auth_xfrm, hmac_opad_ipad);
+
+		tmp_key = (uint64_t *)hmac_opad_ipad;
+		for (i = 0;
+		     i < (int)(ROC_CTX_MAX_OPAD_IPAD_LEN / sizeof(uint64_t));
+		     i++)
+			tmp_key[i] = rte_be_to_cpu_64(tmp_key[i]);
+
 		key = cipher_xfrm->cipher.key.data;
 		length = cipher_xfrm->cipher.key.length;
 	}
diff --git a/drivers/crypto/cnxk/cn9k_ipsec.c b/drivers/crypto/cnxk/cn9k_ipsec.c
index 6455ef9..395b0d5 100644
--- a/drivers/crypto/cnxk/cn9k_ipsec.c
+++ b/drivers/crypto/cnxk/cn9k_ipsec.c
@@ -321,14 +321,23 @@ cn9k_ipsec_outb_sa_create(struct cnxk_cpt_qp *qp,
 	    ctl->auth_type == ROC_IE_ON_SA_AUTH_NULL) {
 		template = &out_sa->aes_gcm.template;
 		ctx_len = offsetof(struct roc_ie_on_outb_sa, aes_gcm.template);
-	} else if (ctl->auth_type == ROC_IE_ON_SA_AUTH_SHA1) {
-		template = &out_sa->sha1.template;
-		ctx_len = offsetof(struct roc_ie_on_outb_sa, sha1.template);
-	} else if (ctl->auth_type == ROC_IE_ON_SA_AUTH_SHA2_256) {
-		template = &out_sa->sha2.template;
-		ctx_len = offsetof(struct roc_ie_on_outb_sa, sha2.template);
 	} else {
-		return -EINVAL;
+		switch (ctl->auth_type) {
+		case ROC_IE_ON_SA_AUTH_SHA1:
+			template = &out_sa->sha1.template;
+			ctx_len = offsetof(struct roc_ie_on_outb_sa,
+					   sha1.template);
+			break;
+		case ROC_IE_ON_SA_AUTH_SHA2_256:
+		case ROC_IE_ON_SA_AUTH_SHA2_384:
+		case ROC_IE_ON_SA_AUTH_SHA2_512:
+			template = &out_sa->sha2.template;
+			ctx_len = offsetof(struct roc_ie_on_outb_sa,
+					   sha2.template);
+			break;
+		default:
+			return -EINVAL;
+		}
 	}
 
 	ip4 = (struct rte_ipv4_hdr *)&template->ip4.ipv4_hdr;
@@ -397,10 +406,22 @@ cn9k_ipsec_outb_sa_create(struct cnxk_cpt_qp *qp,
 		auth_key = auth_xform->auth.key.data;
 		auth_key_len = auth_xform->auth.key.length;
 
-		if (auth_xform->auth.algo == RTE_CRYPTO_AUTH_SHA1_HMAC)
+		switch (auth_xform->auth.algo) {
+		case RTE_CRYPTO_AUTH_NULL:
+			break;
+		case RTE_CRYPTO_AUTH_SHA1_HMAC:
 			memcpy(out_sa->sha1.hmac_key, auth_key, auth_key_len);
-		else if (auth_xform->auth.algo == RTE_CRYPTO_AUTH_SHA256_HMAC)
+			break;
+		case RTE_CRYPTO_AUTH_SHA256_HMAC:
+		case RTE_CRYPTO_AUTH_SHA384_HMAC:
+		case RTE_CRYPTO_AUTH_SHA512_HMAC:
 			memcpy(out_sa->sha2.hmac_key, auth_key, auth_key_len);
+			break;
+		default:
+			plt_err("Unsupported auth algorithm %u",
+				auth_xform->auth.algo);
+			return -ENOTSUP;
+		}
 	}
 
 	inst_tmpl = &sa->inst;
@@ -466,16 +487,26 @@ cn9k_ipsec_inb_sa_create(struct cnxk_cpt_qp *qp,
 		auth_key = auth_xform->auth.key.data;
 		auth_key_len = auth_xform->auth.key.length;
 
-		if (auth_xform->auth.algo == RTE_CRYPTO_AUTH_SHA1_HMAC) {
+		switch (auth_xform->auth.algo) {
+		case RTE_CRYPTO_AUTH_NULL:
+			break;
+		case RTE_CRYPTO_AUTH_SHA1_HMAC:
 			memcpy(in_sa->sha1_or_gcm.hmac_key, auth_key,
 			       auth_key_len);
 			ctx_len = offsetof(struct roc_ie_on_inb_sa,
 					   sha1_or_gcm.selector);
-		} else if (auth_xform->auth.algo ==
-			   RTE_CRYPTO_AUTH_SHA256_HMAC) {
+			break;
+		case RTE_CRYPTO_AUTH_SHA256_HMAC:
+		case RTE_CRYPTO_AUTH_SHA384_HMAC:
+		case RTE_CRYPTO_AUTH_SHA512_HMAC:
 			memcpy(in_sa->sha2.hmac_key, auth_key, auth_key_len);
 			ctx_len = offsetof(struct roc_ie_on_inb_sa,
 					   sha2.selector);
+			break;
+		default:
+			plt_err("Unsupported auth algorithm %u",
+				auth_xform->auth.algo);
+			return -ENOTSUP;
 		}
 	}
 
diff --git a/drivers/crypto/cnxk/cnxk_cryptodev.h b/drivers/crypto/cnxk/cnxk_cryptodev.h
index 2e0f467..f701c26 100644
--- a/drivers/crypto/cnxk/cnxk_cryptodev.h
+++ b/drivers/crypto/cnxk/cnxk_cryptodev.h
@@ -11,7 +11,7 @@
 #include "roc_cpt.h"
 
 #define CNXK_CPT_MAX_CAPS	 34
-#define CNXK_SEC_CRYPTO_MAX_CAPS 6
+#define CNXK_SEC_CRYPTO_MAX_CAPS 8
 #define CNXK_SEC_MAX_CAPS	 5
 #define CNXK_AE_EC_ID_MAX	 8
 /**
diff --git a/drivers/crypto/cnxk/cnxk_cryptodev_capabilities.c b/drivers/crypto/cnxk/cnxk_cryptodev_capabilities.c
index 8305341..9a55474 100644
--- a/drivers/crypto/cnxk/cnxk_cryptodev_capabilities.c
+++ b/drivers/crypto/cnxk/cnxk_cryptodev_capabilities.c
@@ -817,6 +817,46 @@ static const struct rte_cryptodev_capabilities sec_caps_sha1_sha2[] = {
 			}, }
 		}, }
 	},
+	{	/* SHA384 HMAC */
+		.op = RTE_CRYPTO_OP_TYPE_SYMMETRIC,
+		{.sym = {
+			.xform_type = RTE_CRYPTO_SYM_XFORM_AUTH,
+			{.auth = {
+				.algo = RTE_CRYPTO_AUTH_SHA384_HMAC,
+				.block_size = 64,
+				.key_size = {
+					.min = 48,
+					.max = 48,
+					.increment = 0
+				},
+				.digest_size = {
+					.min = 24,
+					.max = 24,
+					.increment = 0
+					},
+			}, }
+		}, }
+	},
+	{	/* SHA512 HMAC */
+		.op = RTE_CRYPTO_OP_TYPE_SYMMETRIC,
+		{.sym = {
+			.xform_type = RTE_CRYPTO_SYM_XFORM_AUTH,
+			{.auth = {
+				.algo = RTE_CRYPTO_AUTH_SHA512_HMAC,
+				.block_size = 128,
+				.key_size = {
+					.min = 64,
+					.max = 64,
+					.increment = 0
+				},
+				.digest_size = {
+					.min = 32,
+					.max = 32,
+					.increment = 0
+				},
+			}, }
+		}, }
+	},
 };
 
 static const struct rte_security_capability sec_caps_templ[] = {
diff --git a/drivers/crypto/cnxk/cnxk_ipsec.h b/drivers/crypto/cnxk/cnxk_ipsec.h
index f4a1012..426eaa8 100644
--- a/drivers/crypto/cnxk/cnxk_ipsec.h
+++ b/drivers/crypto/cnxk/cnxk_ipsec.h
@@ -49,6 +49,12 @@ ipsec_xform_auth_verify(struct rte_crypto_sym_xform *crypto_xform)
 	} else if (crypto_xform->auth.algo == RTE_CRYPTO_AUTH_SHA256_HMAC) {
 		if (keylen >= 32 && keylen <= 64)
 			return 0;
+	} else if (crypto_xform->auth.algo == RTE_CRYPTO_AUTH_SHA384_HMAC) {
+		if (keylen == 48)
+			return 0;
+	} else if (crypto_xform->auth.algo == RTE_CRYPTO_AUTH_SHA512_HMAC) {
+		if (keylen == 64)
+			return 0;
 	}
 
 	return -ENOTSUP;
-- 
2.7.4

[PATCH v2 13/29] crypto/cnxk: account for CPT CTX updates and flush delays

[Anoob Joseph]

CPT CTX write with microcode would require CPT flush to complete to have
DRAM updated with the SA. Since datapath requires SA direction field,
introduce a new flag for the same.

Session destroy path is also updated to clear sa.valid bit using CTX
reload operation.

Session is updated with marker to differentiate s/w immutable and s/w
mutable portions.

Signed-off-by: Anoob Joseph <anoobj@marvell.com>
---
 drivers/crypto/cnxk/cn10k_cryptodev_ops.c |  4 +--
 drivers/crypto/cnxk/cn10k_ipsec.c         | 60 ++++++++++++++++++++++++-------
 drivers/crypto/cnxk/cn10k_ipsec.h         | 27 +++++++++-----
 drivers/crypto/cnxk/cn10k_ipsec_la_ops.h  | 18 +++++-----
 4 files changed, 77 insertions(+), 32 deletions(-)

diff --git a/drivers/crypto/cnxk/cn10k_cryptodev_ops.c b/drivers/crypto/cnxk/cn10k_cryptodev_ops.c
index d25a17c..7617bdc 100644
--- a/drivers/crypto/cnxk/cn10k_cryptodev_ops.c
+++ b/drivers/crypto/cnxk/cn10k_cryptodev_ops.c
@@ -53,7 +53,6 @@ cpt_sec_inst_fill(struct rte_crypto_op *op, struct cn10k_sec_session *sess,
 		  struct cpt_inflight_req *infl_req, struct cpt_inst_s *inst)
 {
 	struct rte_crypto_sym_op *sym_op = op->sym;
-	union roc_ot_ipsec_sa_word2 *w2;
 	struct cn10k_ipsec_sa *sa;
 	int ret;
 
@@ -68,9 +67,8 @@ cpt_sec_inst_fill(struct rte_crypto_op *op, struct cn10k_sec_session *sess,
 	}
 
 	sa = &sess->sa;
-	w2 = (union roc_ot_ipsec_sa_word2 *)&sa->in_sa.w2;
 
-	if (w2->s.dir == ROC_IE_SA_DIR_OUTBOUND)
+	if (sa->is_outbound)
 		ret = process_outb_sa(op, sa, inst);
 	else {
 		infl_req->op_flags |= CPT_OP_FLAGS_IPSEC_DIR_INBOUND;
diff --git a/drivers/crypto/cnxk/cn10k_ipsec.c b/drivers/crypto/cnxk/cn10k_ipsec.c
index a11a6b7..b4acbac 100644
--- a/drivers/crypto/cnxk/cn10k_ipsec.c
+++ b/drivers/crypto/cnxk/cn10k_ipsec.c
@@ -67,7 +67,7 @@ cn10k_ipsec_outb_sa_create(struct roc_cpt *roc_cpt, struct roc_cpt_lf *lf,
 		goto sa_dptr_free;
 	}
 
-	sa->inst.w7 = ipsec_cpt_inst_w7_get(roc_cpt, sa);
+	sa->inst.w7 = ipsec_cpt_inst_w7_get(roc_cpt, out_sa);
 
 #ifdef LA_IPSEC_DEBUG
 	/* Use IV from application in debug mode */
@@ -89,6 +89,8 @@ cn10k_ipsec_outb_sa_create(struct roc_cpt *roc_cpt, struct roc_cpt_lf *lf,
 	}
 #endif
 
+	sa->is_outbound = true;
+
 	/* Get Rlen calculation data */
 	ret = cnxk_ipsec_outb_rlens_get(&rlens, ipsec_xfrm, crypto_xfrm);
 	if (ret)
@@ -127,6 +129,8 @@ cn10k_ipsec_outb_sa_create(struct roc_cpt *roc_cpt, struct roc_cpt_lf *lf,
 	/* Copy word0 from sa_dptr to populate ctx_push_sz ctx_size fields */
 	memcpy(out_sa, sa_dptr, 8);
 
+	plt_atomic_thread_fence(__ATOMIC_SEQ_CST);
+
 	/* Write session using microcode opcode */
 	ret = roc_cpt_ctx_write(lf, sa_dptr, out_sa,
 				ROC_NIX_INL_OT_IPSEC_OUTB_HW_SZ);
@@ -135,9 +139,11 @@ cn10k_ipsec_outb_sa_create(struct roc_cpt *roc_cpt, struct roc_cpt_lf *lf,
 		goto sa_dptr_free;
 	}
 
-	/* Trigger CTX flush to write dirty data back to DRAM */
+	/* Trigger CTX flush so that data is written back to DRAM */
 	roc_cpt_lf_ctx_flush(lf, out_sa, false);
 
+	plt_atomic_thread_fence(__ATOMIC_SEQ_CST);
+
 sa_dptr_free:
 	plt_free(sa_dptr);
 
@@ -178,7 +184,8 @@ cn10k_ipsec_inb_sa_create(struct roc_cpt *roc_cpt, struct roc_cpt_lf *lf,
 		goto sa_dptr_free;
 	}
 
-	sa->inst.w7 = ipsec_cpt_inst_w7_get(roc_cpt, sa);
+	sa->is_outbound = false;
+	sa->inst.w7 = ipsec_cpt_inst_w7_get(roc_cpt, in_sa);
 
 	/* pre-populate CPT INST word 4 */
 	inst_w4.u64 = 0;
@@ -214,6 +221,8 @@ cn10k_ipsec_inb_sa_create(struct roc_cpt *roc_cpt, struct roc_cpt_lf *lf,
 	/* Copy word0 from sa_dptr to populate ctx_push_sz ctx_size fields */
 	memcpy(in_sa, sa_dptr, 8);
 
+	plt_atomic_thread_fence(__ATOMIC_SEQ_CST);
+
 	/* Write session using microcode opcode */
 	ret = roc_cpt_ctx_write(lf, sa_dptr, in_sa,
 				ROC_NIX_INL_OT_IPSEC_INB_HW_SZ);
@@ -222,9 +231,11 @@ cn10k_ipsec_inb_sa_create(struct roc_cpt *roc_cpt, struct roc_cpt_lf *lf,
 		goto sa_dptr_free;
 	}
 
-	/* Trigger CTX flush to write dirty data back to DRAM */
+	/* Trigger CTX flush so that data is written back to DRAM */
 	roc_cpt_lf_ctx_flush(lf, in_sa, false);
 
+	plt_atomic_thread_fence(__ATOMIC_SEQ_CST);
+
 sa_dptr_free:
 	plt_free(sa_dptr);
 
@@ -300,21 +311,46 @@ cn10k_sec_session_create(void *device, struct rte_security_session_conf *conf,
 }
 
 static int
-cn10k_sec_session_destroy(void *device __rte_unused,
-			  struct rte_security_session *sess)
+cn10k_sec_session_destroy(void *dev, struct rte_security_session *sec_sess)
 {
-	struct cn10k_sec_session *priv;
+	struct rte_cryptodev *crypto_dev = dev;
+	union roc_ot_ipsec_sa_word2 *w2;
+	struct cn10k_sec_session *sess;
 	struct rte_mempool *sess_mp;
+	struct cn10k_ipsec_sa *sa;
+	struct cnxk_cpt_qp *qp;
+	struct roc_cpt_lf *lf;
 
-	priv = get_sec_session_private_data(sess);
+	sess = get_sec_session_private_data(sec_sess);
+	if (sess == NULL)
+		return 0;
 
-	if (priv == NULL)
+	qp = crypto_dev->data->queue_pairs[0];
+	if (qp == NULL)
 		return 0;
 
-	sess_mp = rte_mempool_from_obj(priv);
+	lf = &qp->lf;
 
-	set_sec_session_private_data(sess, NULL);
-	rte_mempool_put(sess_mp, priv);
+	sa = &sess->sa;
+
+	/* Trigger CTX flush to write dirty data back to DRAM */
+	roc_cpt_lf_ctx_flush(lf, &sa->in_sa, false);
+
+	/* Wait for 1 ms so that flush is complete */
+	rte_delay_ms(1);
+
+	w2 = (union roc_ot_ipsec_sa_word2 *)&sa->in_sa.w2;
+	w2->s.valid = 0;
+
+	plt_atomic_thread_fence(__ATOMIC_SEQ_CST);
+
+	/* Trigger CTX reload to fetch new data from DRAM */
+	roc_cpt_lf_ctx_reload(lf, &sa->in_sa);
+
+	sess_mp = rte_mempool_from_obj(sess);
+
+	set_sec_session_private_data(sec_sess, NULL);
+	rte_mempool_put(sess_mp, sess);
 
 	return 0;
 }
diff --git a/drivers/crypto/cnxk/cn10k_ipsec.h b/drivers/crypto/cnxk/cn10k_ipsec.h
index 86cd248..cc7ca19 100644
--- a/drivers/crypto/cnxk/cn10k_ipsec.h
+++ b/drivers/crypto/cnxk/cn10k_ipsec.h
@@ -7,28 +7,37 @@
 
 #include <rte_security.h>
 
+#include "roc_api.h"
+
 #include "cnxk_ipsec.h"
 
-#define CN10K_IPSEC_SA_CTX_HDR_SIZE 1
+typedef void *CN10K_SA_CONTEXT_MARKER[0];
 
 struct cn10k_ipsec_sa {
-	union {
-		/** Inbound SA */
-		struct roc_ot_ipsec_inb_sa in_sa;
-		/** Outbound SA */
-		struct roc_ot_ipsec_outb_sa out_sa;
-	};
 	/** Pre-populated CPT inst words */
 	struct cnxk_cpt_inst_tmpl inst;
 	uint16_t max_extended_len;
 	uint16_t iv_offset;
 	uint8_t iv_length;
 	bool ip_csum_enable;
-};
+	bool is_outbound;
+
+	/**
+	 * End of SW mutable area
+	 */
+	CN10K_SA_CONTEXT_MARKER sw_area_end __rte_aligned(ROC_ALIGN);
+
+	union {
+		/** Inbound SA */
+		struct roc_ot_ipsec_inb_sa in_sa;
+		/** Outbound SA */
+		struct roc_ot_ipsec_outb_sa out_sa;
+	};
+} __rte_aligned(ROC_ALIGN);
 
 struct cn10k_sec_session {
 	struct cn10k_ipsec_sa sa;
-} __rte_cache_aligned;
+} __rte_aligned(ROC_ALIGN);
 
 void cn10k_sec_ops_override(void);
 
diff --git a/drivers/crypto/cnxk/cn10k_ipsec_la_ops.h b/drivers/crypto/cnxk/cn10k_ipsec_la_ops.h
index 881fbd1..cab6a50 100644
--- a/drivers/crypto/cnxk/cn10k_ipsec_la_ops.h
+++ b/drivers/crypto/cnxk/cn10k_ipsec_la_ops.h
@@ -54,6 +54,7 @@ process_outb_sa(struct rte_crypto_op *cop, struct cn10k_ipsec_sa *sess,
 	struct rte_crypto_sym_op *sym_op = cop->sym;
 	struct rte_mbuf *m_src = sym_op->m_src;
 	uint64_t inst_w4_u64 = sess->inst.w4;
+	uint64_t dptr;
 
 	if (unlikely(rte_pktmbuf_tailroom(m_src) < sess->max_extended_len)) {
 		plt_dp_err("Not enough tail room");
@@ -76,10 +77,10 @@ process_outb_sa(struct rte_crypto_op *cop, struct cn10k_ipsec_sa *sess,
 		inst_w4_u64 &= ~BIT_ULL(32);
 
 	/* Prepare CPT instruction */
-	inst->w4.u64 = inst_w4_u64;
-	inst->w4.s.dlen = rte_pktmbuf_pkt_len(m_src);
-	inst->dptr = rte_pktmbuf_iova(m_src);
-	inst->rptr = inst->dptr;
+	inst->w4.u64 = inst_w4_u64 | rte_pktmbuf_pkt_len(m_src);
+	dptr = rte_pktmbuf_iova(m_src);
+	inst->dptr = dptr;
+	inst->rptr = dptr;
 
 	return 0;
 }
@@ -90,12 +91,13 @@ process_inb_sa(struct rte_crypto_op *cop, struct cn10k_ipsec_sa *sa,
 {
 	struct rte_crypto_sym_op *sym_op = cop->sym;
 	struct rte_mbuf *m_src = sym_op->m_src;
+	uint64_t dptr;
 
 	/* Prepare CPT instruction */
-	inst->w4.u64 = sa->inst.w4;
-	inst->w4.s.dlen = rte_pktmbuf_pkt_len(m_src);
-	inst->dptr = rte_pktmbuf_iova(m_src);
-	inst->rptr = inst->dptr;
+	inst->w4.u64 = sa->inst.w4 | rte_pktmbuf_pkt_len(m_src);
+	dptr = rte_pktmbuf_iova(m_src);
+	inst->dptr = dptr;
+	inst->rptr = dptr;
 
 	return 0;
 }
-- 
2.7.4

[PATCH v2 14/29] crypto/cnxk: use struct sizes for ctx writes

[Anoob Joseph]

CTX writes only require the lengths are 8B aligned. Use the struct size
directly.

Signed-off-by: Anoob Joseph <anoobj@marvell.com>
---
 drivers/crypto/cnxk/cn10k_ipsec.c | 12 ++++--------
 1 file changed, 4 insertions(+), 8 deletions(-)

diff --git a/drivers/crypto/cnxk/cn10k_ipsec.c b/drivers/crypto/cnxk/cn10k_ipsec.c
index b4acbac..0832b53 100644
--- a/drivers/crypto/cnxk/cn10k_ipsec.c
+++ b/drivers/crypto/cnxk/cn10k_ipsec.c
@@ -52,14 +52,12 @@ cn10k_ipsec_outb_sa_create(struct roc_cpt *roc_cpt, struct roc_cpt_lf *lf,
 	out_sa = &sa->out_sa;
 
 	/* Allocate memory to be used as dptr for CPT ucode WRITE_SA op */
-	sa_dptr = plt_zmalloc(ROC_NIX_INL_OT_IPSEC_OUTB_HW_SZ, 0);
+	sa_dptr = plt_zmalloc(sizeof(struct roc_ot_ipsec_outb_sa), 8);
 	if (sa_dptr == NULL) {
 		plt_err("Couldn't allocate memory for SA dptr");
 		return -ENOMEM;
 	}
 
-	memset(sa_dptr, 0, sizeof(struct roc_ot_ipsec_outb_sa));
-
 	/* Translate security parameters to SA */
 	ret = cnxk_ot_ipsec_outb_sa_fill(sa_dptr, ipsec_xfrm, crypto_xfrm);
 	if (ret) {
@@ -133,7 +131,7 @@ cn10k_ipsec_outb_sa_create(struct roc_cpt *roc_cpt, struct roc_cpt_lf *lf,
 
 	/* Write session using microcode opcode */
 	ret = roc_cpt_ctx_write(lf, sa_dptr, out_sa,
-				ROC_NIX_INL_OT_IPSEC_OUTB_HW_SZ);
+				sizeof(struct roc_ot_ipsec_outb_sa));
 	if (ret) {
 		plt_err("Could not write outbound session to hardware");
 		goto sa_dptr_free;
@@ -169,14 +167,12 @@ cn10k_ipsec_inb_sa_create(struct roc_cpt *roc_cpt, struct roc_cpt_lf *lf,
 	in_sa = &sa->in_sa;
 
 	/* Allocate memory to be used as dptr for CPT ucode WRITE_SA op */
-	sa_dptr = plt_zmalloc(ROC_NIX_INL_OT_IPSEC_INB_HW_SZ, 0);
+	sa_dptr = plt_zmalloc(sizeof(struct roc_ot_ipsec_inb_sa), 8);
 	if (sa_dptr == NULL) {
 		plt_err("Couldn't allocate memory for SA dptr");
 		return -ENOMEM;
 	}
 
-	memset(sa_dptr, 0, sizeof(struct roc_ot_ipsec_inb_sa));
-
 	/* Translate security parameters to SA */
 	ret = cnxk_ot_ipsec_inb_sa_fill(sa_dptr, ipsec_xfrm, crypto_xfrm);
 	if (ret) {
@@ -225,7 +221,7 @@ cn10k_ipsec_inb_sa_create(struct roc_cpt *roc_cpt, struct roc_cpt_lf *lf,
 
 	/* Write session using microcode opcode */
 	ret = roc_cpt_ctx_write(lf, sa_dptr, in_sa,
-				ROC_NIX_INL_OT_IPSEC_INB_HW_SZ);
+				sizeof(struct roc_ot_ipsec_inb_sa));
 	if (ret) {
 		plt_err("Could not write inbound session to hardware");
 		goto sa_dptr_free;
-- 
2.7.4

[PATCH v2 15/29] crypto/cnxk: add security session stats get

[Anoob Joseph]

From: Ankur Dwivedi <adwivedi@marvell.com>

Adds the security session stats get op for cn10k.

Signed-off-by: Ankur Dwivedi <adwivedi@marvell.com>
---
 drivers/crypto/cnxk/cn10k_ipsec.c                 | 55 +++++++++++++++++++++++
 drivers/crypto/cnxk/cnxk_cryptodev_capabilities.c |  1 +
 drivers/crypto/cnxk/cnxk_cryptodev_sec.c          |  1 +
 3 files changed, 57 insertions(+)

diff --git a/drivers/crypto/cnxk/cn10k_ipsec.c b/drivers/crypto/cnxk/cn10k_ipsec.c
index 0832b53..a93c211 100644
--- a/drivers/crypto/cnxk/cn10k_ipsec.c
+++ b/drivers/crypto/cnxk/cn10k_ipsec.c
@@ -122,6 +122,12 @@ cn10k_ipsec_outb_sa_create(struct roc_cpt *roc_cpt, struct roc_cpt_lf *lf,
 
 	sa->inst.w4 = inst_w4.u64;
 
+	if (ipsec_xfrm->options.stats == 1) {
+		/* Enable mib counters */
+		sa_dptr->w0.s.count_mib_bytes = 1;
+		sa_dptr->w0.s.count_mib_pkts = 1;
+	}
+
 	memset(out_sa, 0, sizeof(struct roc_ot_ipsec_outb_sa));
 
 	/* Copy word0 from sa_dptr to populate ctx_push_sz ctx_size fields */
@@ -212,6 +218,12 @@ cn10k_ipsec_inb_sa_create(struct roc_cpt *roc_cpt, struct roc_cpt_lf *lf,
 
 	sa->inst.w4 = inst_w4.u64;
 
+	if (ipsec_xfrm->options.stats == 1) {
+		/* Enable mib counters */
+		sa_dptr->w0.s.count_mib_bytes = 1;
+		sa_dptr->w0.s.count_mib_pkts = 1;
+	}
+
 	memset(in_sa, 0, sizeof(struct roc_ot_ipsec_inb_sa));
 
 	/* Copy word0 from sa_dptr to populate ctx_push_sz ctx_size fields */
@@ -357,6 +369,48 @@ cn10k_sec_session_get_size(void *device __rte_unused)
 	return sizeof(struct cn10k_sec_session);
 }
 
+static int
+cn10k_sec_session_stats_get(void *device, struct rte_security_session *sess,
+			    struct rte_security_stats *stats)
+{
+	struct rte_cryptodev *crypto_dev = device;
+	struct roc_ot_ipsec_outb_sa *out_sa;
+	struct roc_ot_ipsec_inb_sa *in_sa;
+	union roc_ot_ipsec_sa_word2 *w2;
+	struct cn10k_sec_session *priv;
+	struct cn10k_ipsec_sa *sa;
+	struct cnxk_cpt_qp *qp;
+
+	priv = get_sec_session_private_data(sess);
+	if (priv == NULL)
+		return -EINVAL;
+
+	qp = crypto_dev->data->queue_pairs[0];
+	if (qp == NULL)
+		return -EINVAL;
+
+	sa = &priv->sa;
+	w2 = (union roc_ot_ipsec_sa_word2 *)&sa->in_sa.w2;
+
+	stats->protocol = RTE_SECURITY_PROTOCOL_IPSEC;
+
+	if (w2->s.dir == ROC_IE_SA_DIR_OUTBOUND) {
+		out_sa = &sa->out_sa;
+		roc_cpt_lf_ctx_flush(&qp->lf, out_sa, false);
+		rte_delay_ms(1);
+		stats->ipsec.opackets = out_sa->ctx.mib_pkts;
+		stats->ipsec.obytes = out_sa->ctx.mib_octs;
+	} else {
+		in_sa = &sa->in_sa;
+		roc_cpt_lf_ctx_flush(&qp->lf, in_sa, false);
+		rte_delay_ms(1);
+		stats->ipsec.ipackets = in_sa->ctx.mib_pkts;
+		stats->ipsec.ibytes = in_sa->ctx.mib_octs;
+	}
+
+	return 0;
+}
+
 /* Update platform specific security ops */
 void
 cn10k_sec_ops_override(void)
@@ -365,4 +419,5 @@ cn10k_sec_ops_override(void)
 	cnxk_sec_ops.session_create = cn10k_sec_session_create;
 	cnxk_sec_ops.session_destroy = cn10k_sec_session_destroy;
 	cnxk_sec_ops.session_get_size = cn10k_sec_session_get_size;
+	cnxk_sec_ops.session_stats_get = cn10k_sec_session_stats_get;
 }
diff --git a/drivers/crypto/cnxk/cnxk_cryptodev_capabilities.c b/drivers/crypto/cnxk/cnxk_cryptodev_capabilities.c
index 9a55474..0fdd91a 100644
--- a/drivers/crypto/cnxk/cnxk_cryptodev_capabilities.c
+++ b/drivers/crypto/cnxk/cnxk_cryptodev_capabilities.c
@@ -1073,6 +1073,7 @@ cn10k_sec_caps_update(struct rte_security_capability *sec_cap)
 	}
 	sec_cap->ipsec.options.ip_csum_enable = 1;
 	sec_cap->ipsec.options.l4_csum_enable = 1;
+	sec_cap->ipsec.options.stats = 1;
 }
 
 static void
diff --git a/drivers/crypto/cnxk/cnxk_cryptodev_sec.c b/drivers/crypto/cnxk/cnxk_cryptodev_sec.c
index 2021d5c..e5a5d2d 100644
--- a/drivers/crypto/cnxk/cnxk_cryptodev_sec.c
+++ b/drivers/crypto/cnxk/cnxk_cryptodev_sec.c
@@ -15,6 +15,7 @@ struct rte_security_ops cnxk_sec_ops = {
 	.session_create = NULL,
 	.session_destroy = NULL,
 	.session_get_size = NULL,
+	.session_stats_get = NULL,
 	.set_pkt_metadata = NULL,
 	.get_userdata = NULL,
 	.capabilities_get = cnxk_crypto_sec_capabilities_get
-- 
2.7.4

[PATCH v2 16/29] crypto/cnxk: add skip for unsupported cases

[Anoob Joseph]

Add skip for transport mode tests that are not supported. Also,
updated the transport mode path to configure IP version as v4.

Signed-off-by: Anoob Joseph <anoobj@marvell.com>
---
 drivers/crypto/cnxk/cn9k_ipsec.c | 53 +++++++++++++++++++++++++++++++++++-----
 1 file changed, 47 insertions(+), 6 deletions(-)

diff --git a/drivers/crypto/cnxk/cn9k_ipsec.c b/drivers/crypto/cnxk/cn9k_ipsec.c
index 395b0d5..c27845c 100644
--- a/drivers/crypto/cnxk/cn9k_ipsec.c
+++ b/drivers/crypto/cnxk/cn9k_ipsec.c
@@ -141,11 +141,10 @@ ipsec_sa_ctl_set(struct rte_security_ipsec_xform *ipsec,
 			return -EINVAL;
 	}
 
-	ctl->inner_ip_ver = ctl->outer_ip_ver;
-
-	if (ipsec->mode == RTE_SECURITY_IPSEC_SA_MODE_TRANSPORT)
+	if (ipsec->mode == RTE_SECURITY_IPSEC_SA_MODE_TRANSPORT) {
 		ctl->ipsec_mode = ROC_IE_SA_MODE_TRANSPORT;
-	else if (ipsec->mode == RTE_SECURITY_IPSEC_SA_MODE_TUNNEL)
+		ctl->outer_ip_ver = ROC_IE_SA_IP_VERSION_4;
+	} else if (ipsec->mode == RTE_SECURITY_IPSEC_SA_MODE_TUNNEL)
 		ctl->ipsec_mode = ROC_IE_SA_MODE_TUNNEL;
 	else
 		return -EINVAL;
@@ -548,7 +547,8 @@ cn9k_ipsec_inb_sa_create(struct cnxk_cpt_qp *qp,
 }
 
 static inline int
-cn9k_ipsec_xform_verify(struct rte_security_ipsec_xform *ipsec)
+cn9k_ipsec_xform_verify(struct rte_security_ipsec_xform *ipsec,
+			struct rte_crypto_sym_xform *crypto)
 {
 	if (ipsec->life.bytes_hard_limit != 0 ||
 	    ipsec->life.bytes_soft_limit != 0 ||
@@ -556,6 +556,47 @@ cn9k_ipsec_xform_verify(struct rte_security_ipsec_xform *ipsec)
 	    ipsec->life.packets_soft_limit != 0)
 		return -ENOTSUP;
 
+	if (ipsec->mode == RTE_SECURITY_IPSEC_SA_MODE_TRANSPORT) {
+		enum rte_crypto_sym_xform_type type = crypto->type;
+
+		if (type == RTE_CRYPTO_SYM_XFORM_AEAD) {
+			if ((crypto->aead.algo == RTE_CRYPTO_AEAD_AES_GCM) &&
+			    (crypto->aead.key.length == 32)) {
+				plt_err("Transport mode AES-256-GCM is not supported");
+				return -ENOTSUP;
+			}
+		} else {
+			struct rte_crypto_cipher_xform *cipher;
+			struct rte_crypto_auth_xform *auth;
+
+			if (crypto->type == RTE_CRYPTO_SYM_XFORM_CIPHER) {
+				cipher = &crypto->cipher;
+				auth = &crypto->next->auth;
+			} else {
+				cipher = &crypto->next->cipher;
+				auth = &crypto->auth;
+			}
+
+			if ((cipher->algo == RTE_CRYPTO_CIPHER_AES_CBC) &&
+			    (auth->algo == RTE_CRYPTO_AUTH_SHA256_HMAC)) {
+				plt_err("Transport mode AES-CBC SHA2 HMAC 256 is not supported");
+				return -ENOTSUP;
+			}
+
+			if ((cipher->algo == RTE_CRYPTO_CIPHER_AES_CBC) &&
+			    (auth->algo == RTE_CRYPTO_AUTH_SHA384_HMAC)) {
+				plt_err("Transport mode AES-CBC SHA2 HMAC 384 is not supported");
+				return -ENOTSUP;
+			}
+
+			if ((cipher->algo == RTE_CRYPTO_CIPHER_AES_CBC) &&
+			    (auth->algo == RTE_CRYPTO_AUTH_SHA512_HMAC)) {
+				plt_err("Transport mode AES-CBC SHA2 HMAC 512 is not supported");
+				return -ENOTSUP;
+			}
+		}
+	}
+
 	return 0;
 }
 
@@ -580,7 +621,7 @@ cn9k_ipsec_session_create(void *dev,
 	if (ret)
 		return ret;
 
-	ret = cn9k_ipsec_xform_verify(ipsec_xform);
+	ret = cn9k_ipsec_xform_verify(ipsec_xform, crypto_xform);
 	if (ret)
 		return ret;
 
-- 
2.7.4

[PATCH v2 17/29] crypto/cnxk: add context reload for IV

[Anoob Joseph]

From: Tejasree Kondoj <ktejasree@marvell.com>

Adding context reload in datapath for IV in debug mode.

Signed-off-by: Tejasree Kondoj <ktejasree@marvell.com>
---
 drivers/crypto/cnxk/cn10k_cryptodev_ops.c |  7 ++++---
 drivers/crypto/cnxk/cn10k_ipsec_la_ops.h  | 10 ++++++++--
 2 files changed, 12 insertions(+), 5 deletions(-)

diff --git a/drivers/crypto/cnxk/cn10k_cryptodev_ops.c b/drivers/crypto/cnxk/cn10k_cryptodev_ops.c
index 7617bdc..638268e 100644
--- a/drivers/crypto/cnxk/cn10k_cryptodev_ops.c
+++ b/drivers/crypto/cnxk/cn10k_cryptodev_ops.c
@@ -49,7 +49,8 @@ cn10k_cpt_sym_temp_sess_create(struct cnxk_cpt_qp *qp, struct rte_crypto_op *op)
 }
 
 static __rte_always_inline int __rte_hot
-cpt_sec_inst_fill(struct rte_crypto_op *op, struct cn10k_sec_session *sess,
+cpt_sec_inst_fill(struct cnxk_cpt_qp *qp, struct rte_crypto_op *op,
+		  struct cn10k_sec_session *sess,
 		  struct cpt_inflight_req *infl_req, struct cpt_inst_s *inst)
 {
 	struct rte_crypto_sym_op *sym_op = op->sym;
@@ -69,7 +70,7 @@ cpt_sec_inst_fill(struct rte_crypto_op *op, struct cn10k_sec_session *sess,
 	sa = &sess->sa;
 
 	if (sa->is_outbound)
-		ret = process_outb_sa(op, sa, inst);
+		ret = process_outb_sa(&qp->lf, op, sa, inst);
 	else {
 		infl_req->op_flags |= CPT_OP_FLAGS_IPSEC_DIR_INBOUND;
 		ret = process_inb_sa(op, sa, inst);
@@ -122,7 +123,7 @@ cn10k_cpt_fill_inst(struct cnxk_cpt_qp *qp, struct rte_crypto_op *ops[],
 		if (op->sess_type == RTE_CRYPTO_OP_SECURITY_SESSION) {
 			sec_sess = get_sec_session_private_data(
 				sym_op->sec_session);
-			ret = cpt_sec_inst_fill(op, sec_sess, infl_req,
+			ret = cpt_sec_inst_fill(qp, op, sec_sess, infl_req,
 						&inst[0]);
 			if (unlikely(ret))
 				return 0;
diff --git a/drivers/crypto/cnxk/cn10k_ipsec_la_ops.h b/drivers/crypto/cnxk/cn10k_ipsec_la_ops.h
index cab6a50..f2d8122 100644
--- a/drivers/crypto/cnxk/cn10k_ipsec_la_ops.h
+++ b/drivers/crypto/cnxk/cn10k_ipsec_la_ops.h
@@ -48,8 +48,8 @@ ipsec_po_sa_aes_gcm_iv_set(struct cn10k_ipsec_sa *sess,
 }
 
 static __rte_always_inline int
-process_outb_sa(struct rte_crypto_op *cop, struct cn10k_ipsec_sa *sess,
-		struct cpt_inst_s *inst)
+process_outb_sa(struct roc_cpt_lf *lf, struct rte_crypto_op *cop,
+		struct cn10k_ipsec_sa *sess, struct cpt_inst_s *inst)
 {
 	struct rte_crypto_sym_op *sym_op = cop->sym;
 	struct rte_mbuf *m_src = sym_op->m_src;
@@ -61,6 +61,8 @@ process_outb_sa(struct rte_crypto_op *cop, struct cn10k_ipsec_sa *sess,
 		return -ENOMEM;
 	}
 
+	RTE_SET_USED(lf);
+
 #ifdef LA_IPSEC_DEBUG
 	if (sess->out_sa.w2.s.iv_src == ROC_IE_OT_SA_IV_SRC_FROM_SA) {
 		if (sess->out_sa.w2.s.enc_type == ROC_IE_OT_SA_ENC_AES_GCM)
@@ -68,6 +70,10 @@ process_outb_sa(struct rte_crypto_op *cop, struct cn10k_ipsec_sa *sess,
 		else
 			ipsec_po_sa_iv_set(sess, cop);
 	}
+
+	/* Trigger CTX reload to fetch new data from DRAM */
+	roc_cpt_lf_ctx_reload(lf, &sess->out_sa);
+	rte_delay_ms(1);
 #endif
 
 	if (m_src->ol_flags & RTE_MBUF_F_TX_IP_CKSUM)
-- 
2.7.4

[PATCH v2 18/29] crypto/cnxk: handle null chained ops

[Anoob Joseph]

Verification doesn't cover cases when NULL auth/cipher is provided as a
chain. Removed the separate function for verification and added a
replacement function which calls the appropriate downstream functions.

Signed-off-by: Anoob Joseph <anoobj@marvell.com>
---
 drivers/crypto/cnxk/cnxk_cryptodev_ops.c | 189 ++++++++++++++++---------------
 drivers/crypto/cnxk/cnxk_cryptodev_ops.h |  10 --
 drivers/crypto/cnxk/cnxk_se.h            |   6 +
 3 files changed, 102 insertions(+), 103 deletions(-)

diff --git a/drivers/crypto/cnxk/cnxk_cryptodev_ops.c b/drivers/crypto/cnxk/cnxk_cryptodev_ops.c
index 21ee09f..b02f070 100644
--- a/drivers/crypto/cnxk/cnxk_cryptodev_ops.c
+++ b/drivers/crypto/cnxk/cnxk_cryptodev_ops.c
@@ -418,84 +418,121 @@ cnxk_cpt_sym_session_get_size(struct rte_cryptodev *dev __rte_unused)
 }
 
 static int
-sym_xform_verify(struct rte_crypto_sym_xform *xform)
+cnxk_sess_fill(struct rte_crypto_sym_xform *xform, struct cnxk_se_sess *sess)
 {
-	if (xform->type == RTE_CRYPTO_SYM_XFORM_AUTH &&
-	    xform->auth.algo == RTE_CRYPTO_AUTH_NULL &&
-	    xform->auth.op == RTE_CRYPTO_AUTH_OP_VERIFY)
-		return -ENOTSUP;
+	struct rte_crypto_sym_xform *aead_xfrm = NULL;
+	struct rte_crypto_sym_xform *c_xfrm = NULL;
+	struct rte_crypto_sym_xform *a_xfrm = NULL;
+	bool ciph_then_auth;
 
-	if (xform->type == RTE_CRYPTO_SYM_XFORM_CIPHER && xform->next == NULL)
-		return CNXK_CPT_CIPHER;
+	if (xform == NULL)
+		return -EINVAL;
 
-	if (xform->type == RTE_CRYPTO_SYM_XFORM_AUTH && xform->next == NULL)
-		return CNXK_CPT_AUTH;
+	if (xform->type == RTE_CRYPTO_SYM_XFORM_CIPHER) {
+		c_xfrm = xform;
+		a_xfrm = xform->next;
+		ciph_then_auth = true;
+	} else if (xform->type == RTE_CRYPTO_SYM_XFORM_AUTH) {
+		c_xfrm = xform->next;
+		a_xfrm = xform;
+		ciph_then_auth = false;
+	} else {
+		aead_xfrm = xform;
+	}
 
-	if (xform->type == RTE_CRYPTO_SYM_XFORM_AEAD && xform->next == NULL)
-		return CNXK_CPT_AEAD;
+	if (c_xfrm != NULL && c_xfrm->type != RTE_CRYPTO_SYM_XFORM_CIPHER) {
+		plt_dp_err("Invalid type in cipher xform");
+		return -EINVAL;
+	}
 
-	if (xform->next == NULL)
-		return -EIO;
+	if (a_xfrm != NULL && a_xfrm->type != RTE_CRYPTO_SYM_XFORM_AUTH) {
+		plt_dp_err("Invalid type in auth xform");
+		return -EINVAL;
+	}
+
+	if (aead_xfrm != NULL && aead_xfrm->type != RTE_CRYPTO_SYM_XFORM_AEAD) {
+		plt_dp_err("Invalid type in AEAD xform");
+		return -EINVAL;
+	}
 
-	if (xform->type == RTE_CRYPTO_SYM_XFORM_CIPHER &&
-	    xform->cipher.algo == RTE_CRYPTO_CIPHER_3DES_CBC &&
-	    xform->next->type == RTE_CRYPTO_SYM_XFORM_AUTH &&
-	    xform->next->auth.algo == RTE_CRYPTO_AUTH_SHA1)
+	if ((c_xfrm == NULL || c_xfrm->cipher.algo == RTE_CRYPTO_CIPHER_NULL) &&
+	    a_xfrm != NULL && a_xfrm->auth.algo == RTE_CRYPTO_AUTH_NULL &&
+	    a_xfrm->auth.op == RTE_CRYPTO_AUTH_OP_VERIFY) {
+		plt_dp_err("Null cipher + null auth verify is not supported");
 		return -ENOTSUP;
+	}
+
+	/* Cipher only */
+	if (c_xfrm != NULL &&
+	    (a_xfrm == NULL || a_xfrm->auth.algo == RTE_CRYPTO_AUTH_NULL)) {
+		if (fill_sess_cipher(c_xfrm, sess))
+			return -ENOTSUP;
+		else
+			return 0;
+	}
+
+	/* Auth only */
+	if (a_xfrm != NULL &&
+	    (c_xfrm == NULL || c_xfrm->cipher.algo == RTE_CRYPTO_CIPHER_NULL)) {
+		if (fill_sess_auth(a_xfrm, sess))
+			return -ENOTSUP;
+		else
+			return 0;
+	}
+
+	/* AEAD */
+	if (aead_xfrm != NULL) {
+		if (fill_sess_aead(aead_xfrm, sess))
+			return -ENOTSUP;
+		else
+			return 0;
+	}
+
+	/* Chained ops */
+	if (c_xfrm == NULL || a_xfrm == NULL) {
+		plt_dp_err("Invalid xforms");
+		return -EINVAL;
+	}
 
-	if (xform->type == RTE_CRYPTO_SYM_XFORM_AUTH &&
-	    xform->auth.algo == RTE_CRYPTO_AUTH_SHA1 &&
-	    xform->next->type == RTE_CRYPTO_SYM_XFORM_CIPHER &&
-	    xform->next->cipher.algo == RTE_CRYPTO_CIPHER_3DES_CBC)
+	if (c_xfrm->cipher.algo == RTE_CRYPTO_CIPHER_3DES_CBC &&
+	    a_xfrm->auth.algo == RTE_CRYPTO_AUTH_SHA1) {
+		plt_dp_err("3DES-CBC + SHA1 is not supported");
 		return -ENOTSUP;
+	}
 
-	if (xform->type == RTE_CRYPTO_SYM_XFORM_CIPHER &&
-	    xform->cipher.op == RTE_CRYPTO_CIPHER_OP_ENCRYPT &&
-	    xform->next->type == RTE_CRYPTO_SYM_XFORM_AUTH &&
-	    xform->next->auth.op == RTE_CRYPTO_AUTH_OP_GENERATE)
-		return CNXK_CPT_CIPHER_ENC_AUTH_GEN;
-
-	if (xform->type == RTE_CRYPTO_SYM_XFORM_AUTH &&
-	    xform->auth.op == RTE_CRYPTO_AUTH_OP_VERIFY &&
-	    xform->next->type == RTE_CRYPTO_SYM_XFORM_CIPHER &&
-	    xform->next->cipher.op == RTE_CRYPTO_CIPHER_OP_DECRYPT)
-		return CNXK_CPT_AUTH_VRFY_CIPHER_DEC;
-
-	if (xform->type == RTE_CRYPTO_SYM_XFORM_AUTH &&
-	    xform->auth.op == RTE_CRYPTO_AUTH_OP_GENERATE &&
-	    xform->next->type == RTE_CRYPTO_SYM_XFORM_CIPHER &&
-	    xform->next->cipher.op == RTE_CRYPTO_CIPHER_OP_ENCRYPT) {
-		switch (xform->auth.algo) {
-		case RTE_CRYPTO_AUTH_SHA1_HMAC:
-			switch (xform->next->cipher.algo) {
-			case RTE_CRYPTO_CIPHER_AES_CBC:
-				return CNXK_CPT_AUTH_GEN_CIPHER_ENC;
-			default:
-				return -ENOTSUP;
-			}
-		default:
+	/* Cipher then auth */
+	if (ciph_then_auth) {
+		if (fill_sess_cipher(c_xfrm, sess))
 			return -ENOTSUP;
-		}
+		if (fill_sess_auth(a_xfrm, sess))
+			return -ENOTSUP;
+		else
+			return 0;
 	}
 
-	if (xform->type == RTE_CRYPTO_SYM_XFORM_CIPHER &&
-	    xform->cipher.op == RTE_CRYPTO_CIPHER_OP_DECRYPT &&
-	    xform->next->type == RTE_CRYPTO_SYM_XFORM_AUTH &&
-	    xform->next->auth.op == RTE_CRYPTO_AUTH_OP_VERIFY) {
-		switch (xform->cipher.algo) {
-		case RTE_CRYPTO_CIPHER_AES_CBC:
-			switch (xform->next->auth.algo) {
-			case RTE_CRYPTO_AUTH_SHA1_HMAC:
-				return CNXK_CPT_CIPHER_DEC_AUTH_VRFY;
+	/* else */
+
+	if (c_xfrm->cipher.op == RTE_CRYPTO_CIPHER_OP_ENCRYPT) {
+		switch (a_xfrm->auth.algo) {
+		case RTE_CRYPTO_AUTH_SHA1_HMAC:
+			switch (c_xfrm->cipher.algo) {
+			case RTE_CRYPTO_CIPHER_AES_CBC:
+				break;
 			default:
 				return -ENOTSUP;
 			}
+			break;
 		default:
 			return -ENOTSUP;
 		}
 	}
 
-	return -ENOTSUP;
+	if (fill_sess_auth(a_xfrm, sess))
+		return -ENOTSUP;
+	if (fill_sess_cipher(c_xfrm, sess))
+		return -ENOTSUP;
+	else
+		return 0;
 }
 
 static uint64_t
@@ -524,10 +561,6 @@ sym_session_configure(struct roc_cpt *roc_cpt, int driver_id,
 	void *priv;
 	int ret;
 
-	ret = sym_xform_verify(xform);
-	if (unlikely(ret < 0))
-		return ret;
-
 	if (unlikely(rte_mempool_get(pool, &priv))) {
 		plt_dp_err("Could not allocate session private data");
 		return -ENOMEM;
@@ -537,37 +570,7 @@ sym_session_configure(struct roc_cpt *roc_cpt, int driver_id,
 
 	sess_priv = priv;
 
-	switch (ret) {
-	case CNXK_CPT_CIPHER:
-		ret = fill_sess_cipher(xform, sess_priv);
-		break;
-	case CNXK_CPT_AUTH:
-		if (xform->auth.algo == RTE_CRYPTO_AUTH_AES_GMAC)
-			ret = fill_sess_gmac(xform, sess_priv);
-		else
-			ret = fill_sess_auth(xform, sess_priv);
-		break;
-	case CNXK_CPT_AEAD:
-		ret = fill_sess_aead(xform, sess_priv);
-		break;
-	case CNXK_CPT_CIPHER_ENC_AUTH_GEN:
-	case CNXK_CPT_CIPHER_DEC_AUTH_VRFY:
-		ret = fill_sess_cipher(xform, sess_priv);
-		if (ret < 0)
-			break;
-		ret = fill_sess_auth(xform->next, sess_priv);
-		break;
-	case CNXK_CPT_AUTH_VRFY_CIPHER_DEC:
-	case CNXK_CPT_AUTH_GEN_CIPHER_ENC:
-		ret = fill_sess_auth(xform, sess_priv);
-		if (ret < 0)
-			break;
-		ret = fill_sess_cipher(xform->next, sess_priv);
-		break;
-	default:
-		ret = -1;
-	}
-
+	ret = cnxk_sess_fill(xform, sess_priv);
 	if (ret)
 		goto priv_put;
 
@@ -592,7 +595,7 @@ sym_session_configure(struct roc_cpt *roc_cpt, int driver_id,
 priv_put:
 	rte_mempool_put(pool, priv);
 
-	return -ENOTSUP;
+	return ret;
 }
 
 int
diff --git a/drivers/crypto/cnxk/cnxk_cryptodev_ops.h b/drivers/crypto/cnxk/cnxk_cryptodev_ops.h
index 0d36365..ca363bb 100644
--- a/drivers/crypto/cnxk/cnxk_cryptodev_ops.h
+++ b/drivers/crypto/cnxk/cnxk_cryptodev_ops.h
@@ -30,16 +30,6 @@ struct cpt_qp_meta_info {
 	int mlen;
 };
 
-enum sym_xform_type {
-	CNXK_CPT_CIPHER = 1,
-	CNXK_CPT_AUTH,
-	CNXK_CPT_AEAD,
-	CNXK_CPT_CIPHER_ENC_AUTH_GEN,
-	CNXK_CPT_AUTH_VRFY_CIPHER_DEC,
-	CNXK_CPT_AUTH_GEN_CIPHER_ENC,
-	CNXK_CPT_CIPHER_DEC_AUTH_VRFY
-};
-
 #define CPT_OP_FLAGS_METABUF	       (1 << 1)
 #define CPT_OP_FLAGS_AUTH_VERIFY       (1 << 0)
 #define CPT_OP_FLAGS_IPSEC_DIR_INBOUND (1 << 2)
diff --git a/drivers/crypto/cnxk/cnxk_se.h b/drivers/crypto/cnxk/cnxk_se.h
index 37237de..a8cd2c5 100644
--- a/drivers/crypto/cnxk/cnxk_se.h
+++ b/drivers/crypto/cnxk/cnxk_se.h
@@ -36,6 +36,9 @@ struct cnxk_se_sess {
 	struct roc_se_ctx roc_se_ctx;
 } __rte_cache_aligned;
 
+static __rte_always_inline int
+fill_sess_gmac(struct rte_crypto_sym_xform *xform, struct cnxk_se_sess *sess);
+
 static inline void
 cpt_pack_iv(uint8_t *iv_src, uint8_t *iv_dst)
 {
@@ -1808,6 +1811,9 @@ fill_sess_auth(struct rte_crypto_sym_xform *xform, struct cnxk_se_sess *sess)
 	roc_se_auth_type auth_type = 0; /* NULL Auth type */
 	uint8_t zsk_flag = 0, aes_gcm = 0, is_null = 0;
 
+	if (xform->auth.algo == RTE_CRYPTO_AUTH_AES_GMAC)
+		return fill_sess_gmac(xform, sess);
+
 	if (xform->next != NULL &&
 	    xform->next->type == RTE_CRYPTO_SYM_XFORM_CIPHER &&
 	    xform->next->cipher.op == RTE_CRYPTO_CIPHER_OP_ENCRYPT) {
-- 
2.7.4

[PATCH v2 19/29] crypto/cnxk: fix inflight cnt calculation

[Anoob Joseph]

Inflight count calculation is updated to cover wrap around cases where
head can become smaller than tail.


Reported-by: Kiran Kumar K <kirankumark@marvell.com>
Signed-off-by: Anoob Joseph <anoobj@marvell.com>
---
 drivers/crypto/cnxk/cnxk_cryptodev_ops.h | 6 +++++-
 1 file changed, 5 insertions(+), 1 deletion(-)

diff --git a/drivers/crypto/cnxk/cnxk_cryptodev_ops.h b/drivers/crypto/cnxk/cnxk_cryptodev_ops.h
index ca363bb..0336ae1 100644
--- a/drivers/crypto/cnxk/cnxk_cryptodev_ops.h
+++ b/drivers/crypto/cnxk/cnxk_cryptodev_ops.h
@@ -156,7 +156,11 @@ pending_queue_retreat(uint64_t *index, const uint64_t mask, uint64_t nb_entry)
 static __rte_always_inline uint64_t
 pending_queue_infl_cnt(uint64_t head, uint64_t tail, const uint64_t mask)
 {
-	return (head - tail) & mask;
+	/*
+	 * Mask is nb_desc - 1. Add nb_desc to head and mask to account for
+	 * cases when tail > head, which happens during wrap around.
+	 */
+	return ((head + mask + 1) - tail) & mask;
 }
 
 static __rte_always_inline uint64_t
-- 
2.7.4

[PATCH v2 20/29] crypto/cnxk: use atomics to access CPT res

[Anoob Joseph]

The memory would be updated by hardware. Use atomics to read the same.

Signed-off-by: Anoob Joseph <anoobj@marvell.com>
---
 drivers/common/cnxk/hw/cpt.h              |  2 ++
 drivers/crypto/cnxk/cn10k_cryptodev_ops.c | 24 ++++++++++++++++--------
 drivers/crypto/cnxk/cn9k_cryptodev_ops.c  | 28 +++++++++++++++++++---------
 3 files changed, 37 insertions(+), 17 deletions(-)

diff --git a/drivers/common/cnxk/hw/cpt.h b/drivers/common/cnxk/hw/cpt.h
index ccc7af4..412dd76 100644
--- a/drivers/common/cnxk/hw/cpt.h
+++ b/drivers/common/cnxk/hw/cpt.h
@@ -215,6 +215,8 @@ union cpt_res_s {
 
 		uint64_t reserved_64_127;
 	} cn9k;
+
+	uint64_t u64[2];
 };
 
 /* [CN10K, .) */
diff --git a/drivers/crypto/cnxk/cn10k_cryptodev_ops.c b/drivers/crypto/cnxk/cn10k_cryptodev_ops.c
index 638268e..f8240e1 100644
--- a/drivers/crypto/cnxk/cn10k_cryptodev_ops.c
+++ b/drivers/crypto/cnxk/cn10k_cryptodev_ops.c
@@ -111,6 +111,10 @@ cn10k_cpt_fill_inst(struct cnxk_cpt_qp *qp, struct rte_crypto_op *ops[],
 	uint64_t w7;
 	int ret;
 
+	const union cpt_res_s res = {
+		.cn10k.compcode = CPT_COMP_NOT_DONE,
+	};
+
 	op = ops[0];
 
 	inst[0].w0.u64 = 0;
@@ -174,7 +178,7 @@ cn10k_cpt_fill_inst(struct cnxk_cpt_qp *qp, struct rte_crypto_op *ops[],
 	}
 
 	inst[0].res_addr = (uint64_t)&infl_req->res;
-	infl_req->res.cn10k.compcode = CPT_COMP_NOT_DONE;
+	__atomic_store_n(&infl_req->res.u64[0], res.u64[0], __ATOMIC_RELAXED);
 	infl_req->cop = op;
 
 	inst[0].w7.u64 = w7;
@@ -395,9 +399,9 @@ cn10k_cpt_sec_ucc_process(struct rte_crypto_op *cop,
 static inline void
 cn10k_cpt_dequeue_post_process(struct cnxk_cpt_qp *qp,
 			       struct rte_crypto_op *cop,
-			       struct cpt_inflight_req *infl_req)
+			       struct cpt_inflight_req *infl_req,
+			       struct cpt_cn10k_res_s *res)
 {
-	struct cpt_cn10k_res_s *res = (struct cpt_cn10k_res_s *)&infl_req->res;
 	const uint8_t uc_compcode = res->uc_compcode;
 	const uint8_t compcode = res->compcode;
 	unsigned int sz;
@@ -495,12 +499,15 @@ cn10k_cpt_crypto_adapter_dequeue(uintptr_t get_work1)
 	struct cpt_inflight_req *infl_req;
 	struct rte_crypto_op *cop;
 	struct cnxk_cpt_qp *qp;
+	union cpt_res_s res;
 
 	infl_req = (struct cpt_inflight_req *)(get_work1);
 	cop = infl_req->cop;
 	qp = infl_req->qp;
 
-	cn10k_cpt_dequeue_post_process(qp, infl_req->cop, infl_req);
+	res.u64[0] = __atomic_load_n(&infl_req->res.u64[0], __ATOMIC_RELAXED);
+
+	cn10k_cpt_dequeue_post_process(qp, infl_req->cop, infl_req, &res.cn10k);
 
 	if (unlikely(infl_req->op_flags & CPT_OP_FLAGS_METABUF))
 		rte_mempool_put(qp->meta_info.pool, infl_req->mdata);
@@ -515,9 +522,9 @@ cn10k_cpt_dequeue_burst(void *qptr, struct rte_crypto_op **ops, uint16_t nb_ops)
 	struct cpt_inflight_req *infl_req;
 	struct cnxk_cpt_qp *qp = qptr;
 	struct pending_queue *pend_q;
-	struct cpt_cn10k_res_s *res;
 	uint64_t infl_cnt, pq_tail;
 	struct rte_crypto_op *cop;
+	union cpt_res_s res;
 	int i;
 
 	pend_q = &qp->pend_q;
@@ -534,9 +541,10 @@ cn10k_cpt_dequeue_burst(void *qptr, struct rte_crypto_op **ops, uint16_t nb_ops)
 	for (i = 0; i < nb_ops; i++) {
 		infl_req = &pend_q->req_queue[pq_tail];
 
-		res = (struct cpt_cn10k_res_s *)&infl_req->res;
+		res.u64[0] = __atomic_load_n(&infl_req->res.u64[0],
+					     __ATOMIC_RELAXED);
 
-		if (unlikely(res->compcode == CPT_COMP_NOT_DONE)) {
+		if (unlikely(res.cn10k.compcode == CPT_COMP_NOT_DONE)) {
 			if (unlikely(rte_get_timer_cycles() >
 				     pend_q->time_out)) {
 				plt_err("Request timed out");
@@ -553,7 +561,7 @@ cn10k_cpt_dequeue_burst(void *qptr, struct rte_crypto_op **ops, uint16_t nb_ops)
 
 		ops[i] = cop;
 
-		cn10k_cpt_dequeue_post_process(qp, cop, infl_req);
+		cn10k_cpt_dequeue_post_process(qp, cop, infl_req, &res.cn10k);
 
 		if (unlikely(infl_req->op_flags & CPT_OP_FLAGS_METABUF))
 			rte_mempool_put(qp->meta_info.pool, infl_req->mdata);
diff --git a/drivers/crypto/cnxk/cn9k_cryptodev_ops.c b/drivers/crypto/cnxk/cn9k_cryptodev_ops.c
index 449208d..cf80d47 100644
--- a/drivers/crypto/cnxk/cn9k_cryptodev_ops.c
+++ b/drivers/crypto/cnxk/cn9k_cryptodev_ops.c
@@ -221,6 +221,10 @@ cn9k_cpt_enqueue_burst(void *qptr, struct rte_crypto_op **ops, uint16_t nb_ops)
 	uint64_t head;
 	int ret;
 
+	const union cpt_res_s res = {
+		.cn10k.compcode = CPT_COMP_NOT_DONE,
+	};
+
 	pend_q = &qp->pend_q;
 
 	const uint64_t lmt_base = qp->lf.lmt_base;
@@ -274,10 +278,12 @@ cn9k_cpt_enqueue_burst(void *qptr, struct rte_crypto_op **ops, uint16_t nb_ops)
 		infl_req_1->op_flags = 0;
 		infl_req_2->op_flags = 0;
 
-		infl_req_1->res.cn9k.compcode = CPT_COMP_NOT_DONE;
+		__atomic_store_n(&infl_req_1->res.u64[0], res.u64[0],
+				 __ATOMIC_RELAXED);
 		inst[0].res_addr = (uint64_t)&infl_req_1->res;
 
-		infl_req_2->res.cn9k.compcode = CPT_COMP_NOT_DONE;
+		__atomic_store_n(&infl_req_2->res.u64[0], res.u64[0],
+				 __ATOMIC_RELAXED);
 		inst[1].res_addr = (uint64_t)&infl_req_2->res;
 
 		ret = cn9k_cpt_inst_prep(qp, op_1, infl_req_1, &inst[0]);
@@ -410,9 +416,9 @@ cn9k_cpt_sec_post_process(struct rte_crypto_op *cop,
 
 static inline void
 cn9k_cpt_dequeue_post_process(struct cnxk_cpt_qp *qp, struct rte_crypto_op *cop,
-			      struct cpt_inflight_req *infl_req)
+			      struct cpt_inflight_req *infl_req,
+			      struct cpt_cn9k_res_s *res)
 {
-	struct cpt_cn9k_res_s *res = (struct cpt_cn9k_res_s *)&infl_req->res;
 	unsigned int sz;
 
 	if (likely(res->compcode == CPT_COMP_GOOD)) {
@@ -492,12 +498,15 @@ cn9k_cpt_crypto_adapter_dequeue(uintptr_t get_work1)
 	struct cpt_inflight_req *infl_req;
 	struct rte_crypto_op *cop;
 	struct cnxk_cpt_qp *qp;
+	union cpt_res_s res;
 
 	infl_req = (struct cpt_inflight_req *)(get_work1);
 	cop = infl_req->cop;
 	qp = infl_req->qp;
 
-	cn9k_cpt_dequeue_post_process(qp, infl_req->cop, infl_req);
+	res.u64[0] = __atomic_load_n(&infl_req->res.u64[0], __ATOMIC_RELAXED);
+
+	cn9k_cpt_dequeue_post_process(qp, infl_req->cop, infl_req, &res.cn9k);
 
 	if (unlikely(infl_req->op_flags & CPT_OP_FLAGS_METABUF))
 		rte_mempool_put(qp->meta_info.pool, infl_req->mdata);
@@ -512,9 +521,9 @@ cn9k_cpt_dequeue_burst(void *qptr, struct rte_crypto_op **ops, uint16_t nb_ops)
 	struct cpt_inflight_req *infl_req;
 	struct cnxk_cpt_qp *qp = qptr;
 	struct pending_queue *pend_q;
-	struct cpt_cn9k_res_s *res;
 	uint64_t infl_cnt, pq_tail;
 	struct rte_crypto_op *cop;
+	union cpt_res_s res;
 	int i;
 
 	pend_q = &qp->pend_q;
@@ -531,9 +540,10 @@ cn9k_cpt_dequeue_burst(void *qptr, struct rte_crypto_op **ops, uint16_t nb_ops)
 	for (i = 0; i < nb_ops; i++) {
 		infl_req = &pend_q->req_queue[pq_tail];
 
-		res = (struct cpt_cn9k_res_s *)&infl_req->res;
+		res.u64[0] = __atomic_load_n(&infl_req->res.u64[0],
+					     __ATOMIC_RELAXED);
 
-		if (unlikely(res->compcode == CPT_COMP_NOT_DONE)) {
+		if (unlikely(res.cn9k.compcode == CPT_COMP_NOT_DONE)) {
 			if (unlikely(rte_get_timer_cycles() >
 				     pend_q->time_out)) {
 				plt_err("Request timed out");
@@ -550,7 +560,7 @@ cn9k_cpt_dequeue_burst(void *qptr, struct rte_crypto_op **ops, uint16_t nb_ops)
 
 		ops[i] = cop;
 
-		cn9k_cpt_dequeue_post_process(qp, cop, infl_req);
+		cn9k_cpt_dequeue_post_process(qp, cop, infl_req, &res.cn9k);
 
 		if (unlikely(infl_req->op_flags & CPT_OP_FLAGS_METABUF))
 			rte_mempool_put(qp->meta_info.pool, infl_req->mdata);
-- 
2.7.4

[PATCH v2 21/29] crypto/cnxk: add more info on command timeout

[Anoob Joseph]

Print more info when command timeout happens. Print software and
hardware queue information.

Signed-off-by: Anoob Joseph <anoobj@marvell.com>
Signed-off-by: Tejasree Kondoj <ktejasree@marvell.com>
---
 drivers/common/cnxk/hw/cpt.h              | 11 ++++++++
 drivers/crypto/cnxk/cn10k_cryptodev_ops.c |  1 +
 drivers/crypto/cnxk/cn9k_cryptodev_ops.c  |  1 +
 drivers/crypto/cnxk/cnxk_cryptodev_ops.c  | 43 +++++++++++++++++++++++++++++++
 drivers/crypto/cnxk/cnxk_cryptodev_ops.h  |  1 +
 5 files changed, 57 insertions(+)

diff --git a/drivers/common/cnxk/hw/cpt.h b/drivers/common/cnxk/hw/cpt.h
index 412dd76..4d9c896 100644
--- a/drivers/common/cnxk/hw/cpt.h
+++ b/drivers/common/cnxk/hw/cpt.h
@@ -91,6 +91,17 @@ union cpt_lf_inprog {
 	} s;
 };
 
+union cpt_lf_q_inst_ptr {
+	uint64_t u;
+	struct cpt_lf_q_inst_ptr_s {
+		uint64_t dq_ptr : 20;
+		uint64_t reserved_20_31 : 12;
+		uint64_t nq_ptr : 20;
+		uint64_t reserved_52_62 : 11;
+		uint64_t xq_xor : 1;
+	} s;
+};
+
 union cpt_lf_q_base {
 	uint64_t u;
 	struct cpt_lf_q_base_s {
diff --git a/drivers/crypto/cnxk/cn10k_cryptodev_ops.c b/drivers/crypto/cnxk/cn10k_cryptodev_ops.c
index f8240e1..1905ea3 100644
--- a/drivers/crypto/cnxk/cn10k_cryptodev_ops.c
+++ b/drivers/crypto/cnxk/cn10k_cryptodev_ops.c
@@ -548,6 +548,7 @@ cn10k_cpt_dequeue_burst(void *qptr, struct rte_crypto_op **ops, uint16_t nb_ops)
 			if (unlikely(rte_get_timer_cycles() >
 				     pend_q->time_out)) {
 				plt_err("Request timed out");
+				cnxk_cpt_dump_on_err(qp);
 				pend_q->time_out = rte_get_timer_cycles() +
 						   DEFAULT_COMMAND_TIMEOUT *
 							   rte_get_timer_hz();
diff --git a/drivers/crypto/cnxk/cn9k_cryptodev_ops.c b/drivers/crypto/cnxk/cn9k_cryptodev_ops.c
index cf80d47..ac1953b 100644
--- a/drivers/crypto/cnxk/cn9k_cryptodev_ops.c
+++ b/drivers/crypto/cnxk/cn9k_cryptodev_ops.c
@@ -547,6 +547,7 @@ cn9k_cpt_dequeue_burst(void *qptr, struct rte_crypto_op **ops, uint16_t nb_ops)
 			if (unlikely(rte_get_timer_cycles() >
 				     pend_q->time_out)) {
 				plt_err("Request timed out");
+				cnxk_cpt_dump_on_err(qp);
 				pend_q->time_out = rte_get_timer_cycles() +
 						   DEFAULT_COMMAND_TIMEOUT *
 							   rte_get_timer_hz();
diff --git a/drivers/crypto/cnxk/cnxk_cryptodev_ops.c b/drivers/crypto/cnxk/cnxk_cryptodev_ops.c
index b02f070..f63dcdd 100644
--- a/drivers/crypto/cnxk/cnxk_cryptodev_ops.c
+++ b/drivers/crypto/cnxk/cnxk_cryptodev_ops.c
@@ -703,3 +703,46 @@ cnxk_ae_session_cfg(struct rte_cryptodev *dev,
 
 	return 0;
 }
+
+void
+cnxk_cpt_dump_on_err(struct cnxk_cpt_qp *qp)
+{
+	struct pending_queue *pend_q = &qp->pend_q;
+	uint64_t inflight, enq_ptr, deq_ptr, insts;
+	union cpt_lf_q_inst_ptr inst_ptr;
+	union cpt_lf_inprog lf_inprog;
+
+	plt_print("Lcore ID: %d, LF/QP ID: %d", rte_lcore_id(), qp->lf.lf_id);
+	plt_print("");
+	plt_print("S/w pending queue:");
+	plt_print("\tHead: %ld", pend_q->head);
+	plt_print("\tTail: %ld", pend_q->tail);
+	plt_print("\tMask: 0x%lx", pend_q->pq_mask);
+	plt_print("\tInflight count: %ld",
+		  pending_queue_infl_cnt(pend_q->head, pend_q->tail,
+					 pend_q->pq_mask));
+
+	plt_print("");
+	plt_print("H/w pending queue:");
+
+	lf_inprog.u = plt_read64(qp->lf.rbase + CPT_LF_INPROG);
+	inflight = lf_inprog.s.inflight;
+	plt_print("\tInflight in engines: %ld", inflight);
+
+	inst_ptr.u = plt_read64(qp->lf.rbase + CPT_LF_Q_INST_PTR);
+
+	enq_ptr = inst_ptr.s.nq_ptr;
+	deq_ptr = inst_ptr.s.dq_ptr;
+
+	if (enq_ptr >= deq_ptr)
+		insts = enq_ptr - deq_ptr;
+	else
+		insts = (enq_ptr + pend_q->pq_mask + 1 + 320 + 40) - deq_ptr;
+
+	plt_print("\tNQ ptr: 0x%lx", enq_ptr);
+	plt_print("\tDQ ptr: 0x%lx", deq_ptr);
+	plt_print("Insts waiting in CPT: %ld", insts);
+
+	plt_print("");
+	roc_cpt_afs_print(qp->lf.roc_cpt);
+}
diff --git a/drivers/crypto/cnxk/cnxk_cryptodev_ops.h b/drivers/crypto/cnxk/cnxk_cryptodev_ops.h
index 0336ae1..e521f07 100644
--- a/drivers/crypto/cnxk/cnxk_cryptodev_ops.h
+++ b/drivers/crypto/cnxk/cnxk_cryptodev_ops.h
@@ -122,6 +122,7 @@ int cnxk_ae_session_cfg(struct rte_cryptodev *dev,
 			struct rte_crypto_asym_xform *xform,
 			struct rte_cryptodev_asym_session *sess,
 			struct rte_mempool *pool);
+void cnxk_cpt_dump_on_err(struct cnxk_cpt_qp *qp);
 
 static inline union rte_event_crypto_metadata *
 cnxk_event_crypto_mdata_get(struct rte_crypto_op *op)
-- 
2.7.4

[PATCH v2 22/29] crypto/cnxk: support lookaside IPsec AES-CTR

[Anoob Joseph]

From: Tejasree Kondoj <ktejasree@marvell.com>

Adding AES-CTR support to cnxk CPT in
lookaside IPsec mode.

Signed-off-by: Tejasree Kondoj <ktejasree@marvell.com>
---
 doc/guides/cryptodevs/cnxk.rst                    |  2 ++
 doc/guides/rel_notes/release_22_03.rst            |  1 +
 drivers/common/cnxk/cnxk_security.c               |  6 ++++++
 drivers/crypto/cnxk/cn9k_ipsec.c                  |  3 +++
 drivers/crypto/cnxk/cnxk_cryptodev.h              |  2 +-
 drivers/crypto/cnxk/cnxk_cryptodev_capabilities.c | 20 ++++++++++++++++++++
 drivers/crypto/cnxk/cnxk_ipsec.h                  |  3 ++-
 7 files changed, 35 insertions(+), 2 deletions(-)

diff --git a/doc/guides/cryptodevs/cnxk.rst b/doc/guides/cryptodevs/cnxk.rst
index c49a779..1239155 100644
--- a/doc/guides/cryptodevs/cnxk.rst
+++ b/doc/guides/cryptodevs/cnxk.rst
@@ -261,6 +261,7 @@ Cipher algorithms
 +++++++++++++++++
 
 * AES-128/192/256-CBC
+* AES-128/192/256-CTR
 
 Auth algorithms
 +++++++++++++++
@@ -288,6 +289,7 @@ Cipher algorithms
 +++++++++++++++++
 
 * AES-128/192/256-CBC
+* AES-128/192/256-CTR
 
 Auth algorithms
 +++++++++++++++
diff --git a/doc/guides/rel_notes/release_22_03.rst b/doc/guides/rel_notes/release_22_03.rst
index 8df9092..4b272e4 100644
--- a/doc/guides/rel_notes/release_22_03.rst
+++ b/doc/guides/rel_notes/release_22_03.rst
@@ -60,6 +60,7 @@ New Features
   * Added SHA256-HMAC support in lookaside protocol (IPsec) for CN10K.
   * Added SHA384-HMAC support in lookaside protocol (IPsec) for CN9K & CN10K.
   * Added SHA512-HMAC support in lookaside protocol (IPsec) for CN9K & CN10K.
+  * Added AES-CTR support in lookaside protocol (IPsec) for CN9K & CN10K.
 
 
 Removed Items
diff --git a/drivers/common/cnxk/cnxk_security.c b/drivers/common/cnxk/cnxk_security.c
index 1c86f82..0d4baa9 100644
--- a/drivers/common/cnxk/cnxk_security.c
+++ b/drivers/common/cnxk/cnxk_security.c
@@ -123,6 +123,9 @@ ot_ipsec_sa_common_param_fill(union roc_ot_ipsec_sa_word2 *w2,
 		case RTE_CRYPTO_CIPHER_AES_CBC:
 			w2->s.enc_type = ROC_IE_OT_SA_ENC_AES_CBC;
 			break;
+		case RTE_CRYPTO_CIPHER_AES_CTR:
+			w2->s.enc_type = ROC_IE_OT_SA_ENC_AES_CTR;
+			break;
 		default:
 			return -ENOTSUP;
 		}
@@ -630,6 +633,9 @@ onf_ipsec_sa_common_param_fill(struct roc_ie_onf_sa_ctl *ctl, uint8_t *salt,
 		case RTE_CRYPTO_CIPHER_AES_CBC:
 			ctl->enc_type = ROC_IE_ON_SA_ENC_AES_CBC;
 			break;
+		case RTE_CRYPTO_CIPHER_AES_CTR:
+			ctl->enc_type = ROC_IE_ON_SA_ENC_AES_CTR;
+			break;
 		default:
 			return -ENOTSUP;
 		}
diff --git a/drivers/crypto/cnxk/cn9k_ipsec.c b/drivers/crypto/cnxk/cn9k_ipsec.c
index c27845c..1e2269c 100644
--- a/drivers/crypto/cnxk/cn9k_ipsec.c
+++ b/drivers/crypto/cnxk/cn9k_ipsec.c
@@ -166,6 +166,9 @@ ipsec_sa_ctl_set(struct rte_security_ipsec_xform *ipsec,
 	} else if (cipher_xform->cipher.algo == RTE_CRYPTO_CIPHER_AES_CBC) {
 		ctl->enc_type = ROC_IE_ON_SA_ENC_AES_CBC;
 		aes_key_len = cipher_xform->cipher.key.length;
+	} else if (cipher_xform->cipher.algo == RTE_CRYPTO_CIPHER_AES_CTR) {
+		ctl->enc_type = ROC_IE_ON_SA_ENC_AES_CTR;
+		aes_key_len = cipher_xform->cipher.key.length;
 	} else {
 		return -ENOTSUP;
 	}
diff --git a/drivers/crypto/cnxk/cnxk_cryptodev.h b/drivers/crypto/cnxk/cnxk_cryptodev.h
index f701c26..4a1e377 100644
--- a/drivers/crypto/cnxk/cnxk_cryptodev.h
+++ b/drivers/crypto/cnxk/cnxk_cryptodev.h
@@ -11,7 +11,7 @@
 #include "roc_cpt.h"
 
 #define CNXK_CPT_MAX_CAPS	 34
-#define CNXK_SEC_CRYPTO_MAX_CAPS 8
+#define CNXK_SEC_CRYPTO_MAX_CAPS 9
 #define CNXK_SEC_MAX_CAPS	 5
 #define CNXK_AE_EC_ID_MAX	 8
 /**
diff --git a/drivers/crypto/cnxk/cnxk_cryptodev_capabilities.c b/drivers/crypto/cnxk/cnxk_cryptodev_capabilities.c
index 0fdd91a..fae433e 100644
--- a/drivers/crypto/cnxk/cnxk_cryptodev_capabilities.c
+++ b/drivers/crypto/cnxk/cnxk_cryptodev_capabilities.c
@@ -754,6 +754,26 @@ static const struct rte_cryptodev_capabilities sec_caps_aes[] = {
 			}, }
 		}, }
 	},
+	{	/* AES CTR */
+		.op = RTE_CRYPTO_OP_TYPE_SYMMETRIC,
+		{.sym = {
+			.xform_type = RTE_CRYPTO_SYM_XFORM_CIPHER,
+			{.cipher = {
+				.algo = RTE_CRYPTO_CIPHER_AES_CTR,
+				.block_size = 16,
+				.key_size = {
+					.min = 16,
+					.max = 32,
+					.increment = 8
+				},
+				.iv_size = {
+					.min = 12,
+					.max = 16,
+					.increment = 4
+				}
+			}, }
+		}, }
+	},
 	{	/* AES CBC */
 		.op = RTE_CRYPTO_OP_TYPE_SYMMETRIC,
 		{.sym = {
diff --git a/drivers/crypto/cnxk/cnxk_ipsec.h b/drivers/crypto/cnxk/cnxk_ipsec.h
index 426eaa8..f5a51b5 100644
--- a/drivers/crypto/cnxk/cnxk_ipsec.h
+++ b/drivers/crypto/cnxk/cnxk_ipsec.h
@@ -20,7 +20,8 @@ struct cnxk_cpt_inst_tmpl {
 static inline int
 ipsec_xform_cipher_verify(struct rte_crypto_sym_xform *crypto_xform)
 {
-	if (crypto_xform->cipher.algo == RTE_CRYPTO_CIPHER_AES_CBC) {
+	if (crypto_xform->cipher.algo == RTE_CRYPTO_CIPHER_AES_CBC ||
+	    crypto_xform->cipher.algo == RTE_CRYPTO_CIPHER_AES_CTR) {
 		switch (crypto_xform->cipher.key.length) {
 		case 16:
 		case 24:
-- 
2.7.4

[PATCH v2 23/29] crypto/cnxk: fix extend tail calculation

[Anoob Joseph]

If the packet size to be incremented after IPsec processing is less
than size of hdr (size incremented before submitting), then extend_tail
can become negative. Allow negative values for the variable.

Fixes: 67a87e89561c ("crypto/cnxk: add cn9k lookaside IPsec datapath")
Cc: marchana@marvell.com

Signed-off-by: Anoob Joseph <anoobj@marvell.com>
---
 drivers/crypto/cnxk/cn9k_ipsec_la_ops.h | 6 ++++--
 1 file changed, 4 insertions(+), 2 deletions(-)

diff --git a/drivers/crypto/cnxk/cn9k_ipsec_la_ops.h b/drivers/crypto/cnxk/cn9k_ipsec_la_ops.h
index 2dc8913..2b0261e 100644
--- a/drivers/crypto/cnxk/cn9k_ipsec_la_ops.h
+++ b/drivers/crypto/cnxk/cn9k_ipsec_la_ops.h
@@ -77,9 +77,10 @@ process_outb_sa(struct rte_crypto_op *cop, struct cn9k_ipsec_sa *sa,
 	const unsigned int hdr_len = sizeof(struct roc_ie_on_outb_hdr);
 	struct rte_crypto_sym_op *sym_op = cop->sym;
 	struct rte_mbuf *m_src = sym_op->m_src;
-	uint32_t dlen, rlen, extend_tail;
 	struct roc_ie_on_outb_sa *out_sa;
 	struct roc_ie_on_outb_hdr *hdr;
+	uint32_t dlen, rlen;
+	int32_t extend_tail;
 
 	out_sa = &sa->out_sa;
 
@@ -88,7 +89,8 @@ process_outb_sa(struct rte_crypto_op *cop, struct cn9k_ipsec_sa *sa,
 
 	extend_tail = rlen - dlen;
 	if (unlikely(extend_tail > rte_pktmbuf_tailroom(m_src))) {
-		plt_dp_err("Not enough tail room");
+		plt_dp_err("Not enough tail room (required: %d, available: %d",
+			   extend_tail, rte_pktmbuf_tailroom(m_src));
 		return -ENOMEM;
 	}
 
-- 
2.7.4

[PATCH v2 24/29] crypto/cnxk: add aes xcbc and null cipher

[Anoob Joseph]

Add support for AES XCBC and NULL cipher.

Signed-off-by: Anoob Joseph <anoobj@marvell.com>
---
 doc/guides/cryptodevs/cnxk.rst                    |  4 +
 doc/guides/rel_notes/release_22_03.rst            |  2 +
 drivers/common/cnxk/cnxk_security.c               | 48 ++++++++----
 drivers/common/cnxk/roc_ie_on.h                   | 10 +++
 drivers/crypto/cnxk/cn9k_ipsec.c                  | 93 ++++++++++++++++-------
 drivers/crypto/cnxk/cnxk_cryptodev.h              |  2 +-
 drivers/crypto/cnxk/cnxk_cryptodev_capabilities.c | 45 +++++++++++
 drivers/crypto/cnxk/cnxk_ipsec.h                  |  7 ++
 8 files changed, 169 insertions(+), 42 deletions(-)

diff --git a/doc/guides/cryptodevs/cnxk.rst b/doc/guides/cryptodevs/cnxk.rst
index 1239155..6e844f5 100644
--- a/doc/guides/cryptodevs/cnxk.rst
+++ b/doc/guides/cryptodevs/cnxk.rst
@@ -260,6 +260,7 @@ AEAD algorithms
 Cipher algorithms
 +++++++++++++++++
 
+* NULL
 * AES-128/192/256-CBC
 * AES-128/192/256-CTR
 
@@ -270,6 +271,7 @@ Auth algorithms
 * SHA256-128-HMAC
 * SHA384-192-HMAC
 * SHA512-256-HMAC
+* AES-XCBC-96
 
 CN10XX Features supported
 ~~~~~~~~~~~~~~~~~~~~~~~~~
@@ -288,6 +290,7 @@ AEAD algorithms
 Cipher algorithms
 +++++++++++++++++
 
+* NULL
 * AES-128/192/256-CBC
 * AES-128/192/256-CTR
 
@@ -299,3 +302,4 @@ Auth algorithms
 * SHA256-128-HMAC
 * SHA384-192-HMAC
 * SHA512-256-HMAC
+* AES-XCBC-96
diff --git a/doc/guides/rel_notes/release_22_03.rst b/doc/guides/rel_notes/release_22_03.rst
index 4b272e4..e8fec00 100644
--- a/doc/guides/rel_notes/release_22_03.rst
+++ b/doc/guides/rel_notes/release_22_03.rst
@@ -61,6 +61,8 @@ New Features
   * Added SHA384-HMAC support in lookaside protocol (IPsec) for CN9K & CN10K.
   * Added SHA512-HMAC support in lookaside protocol (IPsec) for CN9K & CN10K.
   * Added AES-CTR support in lookaside protocol (IPsec) for CN9K & CN10K.
+  * Added NULL cipher support in lookaside protocol (IPsec) for CN9K & CN10K.
+  * Added AES-XCBC support in lookaside protocol (IPsec) for CN9K & CN10K.
 
 
 Removed Items
diff --git a/drivers/common/cnxk/cnxk_security.c b/drivers/common/cnxk/cnxk_security.c
index 0d4baa9..6ebf084 100644
--- a/drivers/common/cnxk/cnxk_security.c
+++ b/drivers/common/cnxk/cnxk_security.c
@@ -120,6 +120,9 @@ ot_ipsec_sa_common_param_fill(union roc_ot_ipsec_sa_word2 *w2,
 		}
 	} else {
 		switch (cipher_xfrm->cipher.algo) {
+		case RTE_CRYPTO_CIPHER_NULL:
+			w2->s.enc_type = ROC_IE_OT_SA_ENC_NULL;
+			break;
 		case RTE_CRYPTO_CIPHER_AES_CBC:
 			w2->s.enc_type = ROC_IE_OT_SA_ENC_AES_CBC;
 			break;
@@ -146,11 +149,19 @@ ot_ipsec_sa_common_param_fill(union roc_ot_ipsec_sa_word2 *w2,
 		case RTE_CRYPTO_AUTH_SHA512_HMAC:
 			w2->s.auth_type = ROC_IE_OT_SA_AUTH_SHA2_512;
 			break;
+		case RTE_CRYPTO_AUTH_AES_XCBC_MAC:
+			w2->s.auth_type = ROC_IE_OT_SA_AUTH_AES_XCBC_128;
+			break;
 		default:
 			return -ENOTSUP;
 		}
 
-		ipsec_hmac_opad_ipad_gen(auth_xfrm, hmac_opad_ipad);
+		if (auth_xfrm->auth.algo == RTE_CRYPTO_AUTH_AES_XCBC_MAC) {
+			const uint8_t *auth_key = auth_xfrm->auth.key.data;
+			roc_aes_xcbc_key_derive(auth_key, hmac_opad_ipad);
+		} else {
+			ipsec_hmac_opad_ipad_gen(auth_xfrm, hmac_opad_ipad);
+		}
 
 		tmp_key = (uint64_t *)hmac_opad_ipad;
 		for (i = 0;
@@ -174,18 +185,26 @@ ot_ipsec_sa_common_param_fill(union roc_ot_ipsec_sa_word2 *w2,
 	for (i = 0; i < (int)(ROC_CTX_MAX_CKEY_LEN / sizeof(uint64_t)); i++)
 		tmp_key[i] = rte_be_to_cpu_64(tmp_key[i]);
 
-	switch (length) {
-	case ROC_CPT_AES128_KEY_LEN:
-		w2->s.aes_key_len = ROC_IE_SA_AES_KEY_LEN_128;
-		break;
-	case ROC_CPT_AES192_KEY_LEN:
-		w2->s.aes_key_len = ROC_IE_SA_AES_KEY_LEN_192;
-		break;
-	case ROC_CPT_AES256_KEY_LEN:
-		w2->s.aes_key_len = ROC_IE_SA_AES_KEY_LEN_256;
-		break;
-	default:
-		return -EINVAL;
+	/* Set AES key length */
+	if (w2->s.enc_type == ROC_IE_OT_SA_ENC_AES_CBC ||
+	    w2->s.enc_type == ROC_IE_OT_SA_ENC_AES_CCM ||
+	    w2->s.enc_type == ROC_IE_OT_SA_ENC_AES_CTR ||
+	    w2->s.enc_type == ROC_IE_OT_SA_ENC_AES_GCM ||
+	    w2->s.auth_type == ROC_IE_OT_SA_AUTH_AES_GMAC) {
+		switch (length) {
+		case ROC_CPT_AES128_KEY_LEN:
+			w2->s.aes_key_len = ROC_IE_SA_AES_KEY_LEN_128;
+			break;
+		case ROC_CPT_AES192_KEY_LEN:
+			w2->s.aes_key_len = ROC_IE_SA_AES_KEY_LEN_192;
+			break;
+		case ROC_CPT_AES256_KEY_LEN:
+			w2->s.aes_key_len = ROC_IE_SA_AES_KEY_LEN_256;
+			break;
+		default:
+			plt_err("Invalid AES key length");
+			return -EINVAL;
+		}
 	}
 
 	if (ipsec_xfrm->life.packets_soft_limit != 0 ||
@@ -815,6 +834,9 @@ cnxk_ipsec_icvlen_get(enum rte_crypto_cipher_algorithm c_algo,
 	case RTE_CRYPTO_AUTH_SHA512_HMAC:
 		icv = 32;
 		break;
+	case RTE_CRYPTO_AUTH_AES_XCBC_MAC:
+		icv = 12;
+		break;
 	default:
 		break;
 	}
diff --git a/drivers/common/cnxk/roc_ie_on.h b/drivers/common/cnxk/roc_ie_on.h
index 817ef33..cb56a70 100644
--- a/drivers/common/cnxk/roc_ie_on.h
+++ b/drivers/common/cnxk/roc_ie_on.h
@@ -181,6 +181,11 @@ struct roc_ie_on_outb_sa {
 			struct roc_ie_on_ip_template template;
 		} sha1;
 		struct {
+			uint8_t key[16];
+			uint8_t unused[32];
+			struct roc_ie_on_ip_template template;
+		} aes_xcbc;
+		struct {
 			uint8_t hmac_key[64];
 			uint8_t hmac_iv[64];
 			struct roc_ie_on_ip_template template;
@@ -202,6 +207,11 @@ struct roc_ie_on_inb_sa {
 			struct roc_ie_on_traffic_selector selector;
 		} sha1_or_gcm;
 		struct {
+			uint8_t key[16];
+			uint8_t unused[32];
+			struct roc_ie_on_traffic_selector selector;
+		} aes_xcbc;
+		struct {
 			uint8_t hmac_key[64];
 			uint8_t hmac_iv[64];
 			struct roc_ie_on_traffic_selector selector;
diff --git a/drivers/crypto/cnxk/cn9k_ipsec.c b/drivers/crypto/cnxk/cn9k_ipsec.c
index 1e2269c..c9f5825 100644
--- a/drivers/crypto/cnxk/cn9k_ipsec.c
+++ b/drivers/crypto/cnxk/cn9k_ipsec.c
@@ -118,7 +118,7 @@ ipsec_sa_ctl_set(struct rte_security_ipsec_xform *ipsec,
 		 struct roc_ie_on_sa_ctl *ctl)
 {
 	struct rte_crypto_sym_xform *cipher_xform, *auth_xform;
-	int aes_key_len;
+	int aes_key_len = 0;
 
 	if (ipsec->direction == RTE_SECURITY_IPSEC_SA_DIR_EGRESS) {
 		ctl->direction = ROC_IE_SA_DIR_OUTBOUND;
@@ -157,37 +157,33 @@ ipsec_sa_ctl_set(struct rte_security_ipsec_xform *ipsec,
 		return -EINVAL;
 
 	if (crypto_xform->type == RTE_CRYPTO_SYM_XFORM_AEAD) {
-		if (crypto_xform->aead.algo == RTE_CRYPTO_AEAD_AES_GCM) {
+		switch (crypto_xform->aead.algo) {
+		case RTE_CRYPTO_AEAD_AES_GCM:
 			ctl->enc_type = ROC_IE_ON_SA_ENC_AES_GCM;
 			aes_key_len = crypto_xform->aead.key.length;
-		} else {
+			break;
+		default:
+			plt_err("Unsupported AEAD algorithm");
 			return -ENOTSUP;
 		}
-	} else if (cipher_xform->cipher.algo == RTE_CRYPTO_CIPHER_AES_CBC) {
-		ctl->enc_type = ROC_IE_ON_SA_ENC_AES_CBC;
-		aes_key_len = cipher_xform->cipher.key.length;
-	} else if (cipher_xform->cipher.algo == RTE_CRYPTO_CIPHER_AES_CTR) {
-		ctl->enc_type = ROC_IE_ON_SA_ENC_AES_CTR;
-		aes_key_len = cipher_xform->cipher.key.length;
 	} else {
-		return -ENOTSUP;
-	}
-
-	switch (aes_key_len) {
-	case 16:
-		ctl->aes_key_len = ROC_IE_SA_AES_KEY_LEN_128;
-		break;
-	case 24:
-		ctl->aes_key_len = ROC_IE_SA_AES_KEY_LEN_192;
-		break;
-	case 32:
-		ctl->aes_key_len = ROC_IE_SA_AES_KEY_LEN_256;
-		break;
-	default:
-		return -EINVAL;
-	}
+		switch (cipher_xform->cipher.algo) {
+		case RTE_CRYPTO_CIPHER_NULL:
+			ctl->enc_type = ROC_IE_ON_SA_ENC_NULL;
+			break;
+		case RTE_CRYPTO_CIPHER_AES_CBC:
+			ctl->enc_type = ROC_IE_ON_SA_ENC_AES_CBC;
+			aes_key_len = cipher_xform->cipher.key.length;
+			break;
+		case RTE_CRYPTO_CIPHER_AES_CTR:
+			ctl->enc_type = ROC_IE_ON_SA_ENC_AES_CTR;
+			aes_key_len = cipher_xform->cipher.key.length;
+			break;
+		default:
+			plt_err("Unsupported cipher algorithm");
+			return -ENOTSUP;
+		}
 
-	if (crypto_xform->type != RTE_CRYPTO_SYM_XFORM_AEAD) {
 		switch (auth_xform->auth.algo) {
 		case RTE_CRYPTO_AUTH_NULL:
 			ctl->auth_type = ROC_IE_ON_SA_AUTH_NULL;
@@ -217,10 +213,33 @@ ipsec_sa_ctl_set(struct rte_security_ipsec_xform *ipsec,
 			ctl->auth_type = ROC_IE_ON_SA_AUTH_AES_XCBC_128;
 			break;
 		default:
+			plt_err("Unsupported auth algorithm");
 			return -ENOTSUP;
 		}
 	}
 
+	/* Set AES key length */
+	if (ctl->enc_type == ROC_IE_ON_SA_ENC_AES_CBC ||
+	    ctl->enc_type == ROC_IE_ON_SA_ENC_AES_CCM ||
+	    ctl->enc_type == ROC_IE_ON_SA_ENC_AES_CTR ||
+	    ctl->enc_type == ROC_IE_ON_SA_ENC_AES_GCM ||
+	    ctl->auth_type == ROC_IE_ON_SA_AUTH_AES_GMAC) {
+		switch (aes_key_len) {
+		case 16:
+			ctl->aes_key_len = ROC_IE_SA_AES_KEY_LEN_128;
+			break;
+		case 24:
+			ctl->aes_key_len = ROC_IE_SA_AES_KEY_LEN_192;
+			break;
+		case 32:
+			ctl->aes_key_len = ROC_IE_SA_AES_KEY_LEN_256;
+			break;
+		default:
+			plt_err("Invalid AES key length");
+			return -EINVAL;
+		}
+	}
+
 	if (ipsec->options.esn)
 		ctl->esn_en = 1;
 
@@ -267,8 +286,6 @@ fill_ipsec_common_sa(struct rte_security_ipsec_xform *ipsec,
 
 	if (cipher_key_len != 0)
 		memcpy(common_sa->cipher_key, cipher_key, cipher_key_len);
-	else
-		return -EINVAL;
 
 	return 0;
 }
@@ -337,7 +354,13 @@ cn9k_ipsec_outb_sa_create(struct cnxk_cpt_qp *qp,
 			ctx_len = offsetof(struct roc_ie_on_outb_sa,
 					   sha2.template);
 			break;
+		case ROC_IE_ON_SA_AUTH_AES_XCBC_128:
+			template = &out_sa->aes_xcbc.template;
+			ctx_len = offsetof(struct roc_ie_on_outb_sa,
+					   aes_xcbc.template);
+			break;
 		default:
+			plt_err("Unsupported auth algorithm");
 			return -EINVAL;
 		}
 	}
@@ -419,6 +442,9 @@ cn9k_ipsec_outb_sa_create(struct cnxk_cpt_qp *qp,
 		case RTE_CRYPTO_AUTH_SHA512_HMAC:
 			memcpy(out_sa->sha2.hmac_key, auth_key, auth_key_len);
 			break;
+		case RTE_CRYPTO_AUTH_AES_XCBC_MAC:
+			memcpy(out_sa->aes_xcbc.key, auth_key, auth_key_len);
+			break;
 		default:
 			plt_err("Unsupported auth algorithm %u",
 				auth_xform->auth.algo);
@@ -505,6 +531,11 @@ cn9k_ipsec_inb_sa_create(struct cnxk_cpt_qp *qp,
 			ctx_len = offsetof(struct roc_ie_on_inb_sa,
 					   sha2.selector);
 			break;
+		case RTE_CRYPTO_AUTH_AES_XCBC_MAC:
+			memcpy(in_sa->aes_xcbc.key, auth_key, auth_key_len);
+			ctx_len = offsetof(struct roc_ie_on_inb_sa,
+					   aes_xcbc.selector);
+			break;
 		default:
 			plt_err("Unsupported auth algorithm %u",
 				auth_xform->auth.algo);
@@ -597,6 +628,12 @@ cn9k_ipsec_xform_verify(struct rte_security_ipsec_xform *ipsec,
 				plt_err("Transport mode AES-CBC SHA2 HMAC 512 is not supported");
 				return -ENOTSUP;
 			}
+
+			if ((cipher->algo == RTE_CRYPTO_CIPHER_AES_CBC) &&
+			    (auth->algo == RTE_CRYPTO_AUTH_AES_XCBC_MAC)) {
+				plt_err("Transport mode AES-CBC AES-XCBC is not supported");
+				return -ENOTSUP;
+			}
 		}
 	}
 
diff --git a/drivers/crypto/cnxk/cnxk_cryptodev.h b/drivers/crypto/cnxk/cnxk_cryptodev.h
index 4a1e377..16e7572 100644
--- a/drivers/crypto/cnxk/cnxk_cryptodev.h
+++ b/drivers/crypto/cnxk/cnxk_cryptodev.h
@@ -11,7 +11,7 @@
 #include "roc_cpt.h"
 
 #define CNXK_CPT_MAX_CAPS	 34
-#define CNXK_SEC_CRYPTO_MAX_CAPS 9
+#define CNXK_SEC_CRYPTO_MAX_CAPS 11
 #define CNXK_SEC_MAX_CAPS	 5
 #define CNXK_AE_EC_ID_MAX	 8
 /**
diff --git a/drivers/crypto/cnxk/cnxk_cryptodev_capabilities.c b/drivers/crypto/cnxk/cnxk_cryptodev_capabilities.c
index fae433e..a0b2a1f 100644
--- a/drivers/crypto/cnxk/cnxk_cryptodev_capabilities.c
+++ b/drivers/crypto/cnxk/cnxk_cryptodev_capabilities.c
@@ -794,6 +794,26 @@ static const struct rte_cryptodev_capabilities sec_caps_aes[] = {
 			}, }
 		}, }
 	},
+	{	/* AES-XCBC */
+		.op = RTE_CRYPTO_OP_TYPE_SYMMETRIC,
+		{ .sym = {
+			.xform_type = RTE_CRYPTO_SYM_XFORM_AUTH,
+			{.auth = {
+				.algo = RTE_CRYPTO_AUTH_AES_XCBC_MAC,
+				.block_size = 16,
+				.key_size = {
+					.min = 16,
+					.max = 16,
+					.increment = 0
+				},
+				.digest_size = {
+					.min = 12,
+					.max = 12,
+					.increment = 0,
+				},
+			}, }
+		}, }
+	},
 };
 
 static const struct rte_cryptodev_capabilities sec_caps_sha1_sha2[] = {
@@ -879,6 +899,29 @@ static const struct rte_cryptodev_capabilities sec_caps_sha1_sha2[] = {
 	},
 };
 
+static const struct rte_cryptodev_capabilities sec_caps_null[] = {
+	{	/* NULL (CIPHER) */
+		.op = RTE_CRYPTO_OP_TYPE_SYMMETRIC,
+		{.sym = {
+			.xform_type = RTE_CRYPTO_SYM_XFORM_CIPHER,
+			{.cipher = {
+				.algo = RTE_CRYPTO_CIPHER_NULL,
+				.block_size = 1,
+				.key_size = {
+					.min = 0,
+					.max = 0,
+					.increment = 0
+				},
+				.iv_size = {
+					.min = 0,
+					.max = 0,
+					.increment = 0
+				}
+			}, },
+		}, }
+	},
+};
+
 static const struct rte_security_capability sec_caps_templ[] = {
 	{	/* IPsec Lookaside Protocol ESP Tunnel Ingress */
 		.action = RTE_SECURITY_ACTION_TYPE_LOOKASIDE_PROTOCOL,
@@ -1069,6 +1112,8 @@ sec_crypto_caps_populate(struct rte_cryptodev_capabilities cnxk_caps[],
 	else
 		cn9k_sec_crypto_caps_update(cnxk_caps);
 
+	sec_caps_add(cnxk_caps, &cur_pos, sec_caps_null,
+		     RTE_DIM(sec_caps_null));
 	sec_caps_add(cnxk_caps, &cur_pos, caps_end, RTE_DIM(caps_end));
 }
 
diff --git a/drivers/crypto/cnxk/cnxk_ipsec.h b/drivers/crypto/cnxk/cnxk_ipsec.h
index f5a51b5..f50d9fa 100644
--- a/drivers/crypto/cnxk/cnxk_ipsec.h
+++ b/drivers/crypto/cnxk/cnxk_ipsec.h
@@ -20,6 +20,9 @@ struct cnxk_cpt_inst_tmpl {
 static inline int
 ipsec_xform_cipher_verify(struct rte_crypto_sym_xform *crypto_xform)
 {
+	if (crypto_xform->cipher.algo == RTE_CRYPTO_CIPHER_NULL)
+		return 0;
+
 	if (crypto_xform->cipher.algo == RTE_CRYPTO_CIPHER_AES_CBC ||
 	    crypto_xform->cipher.algo == RTE_CRYPTO_CIPHER_AES_CTR) {
 		switch (crypto_xform->cipher.key.length) {
@@ -58,6 +61,10 @@ ipsec_xform_auth_verify(struct rte_crypto_sym_xform *crypto_xform)
 			return 0;
 	}
 
+	if (crypto_xform->auth.algo == RTE_CRYPTO_AUTH_AES_XCBC_MAC &&
+	    keylen == ROC_CPT_AES_XCBC_KEY_LENGTH)
+		return 0;
+
 	return -ENOTSUP;
 }
 
-- 
2.7.4

[PATCH v2 25/29] crypto/cnxk: add copy and set DF

[Anoob Joseph]

Add support for copy and set DF bit.

Signed-off-by: Anoob Joseph <anoobj@marvell.com>
---
 drivers/crypto/cnxk/cn9k_ipsec.c                  | 7 ++++++-
 drivers/crypto/cnxk/cnxk_cryptodev_capabilities.c | 1 +
 2 files changed, 7 insertions(+), 1 deletion(-)

diff --git a/drivers/crypto/cnxk/cn9k_ipsec.c b/drivers/crypto/cnxk/cn9k_ipsec.c
index c9f5825..62b9c26 100644
--- a/drivers/crypto/cnxk/cn9k_ipsec.c
+++ b/drivers/crypto/cnxk/cn9k_ipsec.c
@@ -246,6 +246,8 @@ ipsec_sa_ctl_set(struct rte_security_ipsec_xform *ipsec,
 	if (ipsec->options.udp_encap == 1)
 		ctl->encap_type = ROC_IE_ON_SA_ENCAP_UDP;
 
+	ctl->copy_df = ipsec->options.copy_df;
+
 	ctl->spi = rte_cpu_to_be_32(ipsec->spi);
 
 	rte_io_wmb();
@@ -376,13 +378,16 @@ cn9k_ipsec_outb_sa_create(struct cnxk_cpt_qp *qp,
 
 	if (ipsec->mode == RTE_SECURITY_IPSEC_SA_MODE_TUNNEL) {
 		if (ipsec->tunnel.type == RTE_SECURITY_IPSEC_TUNNEL_IPV4) {
+			uint16_t frag_off = 0;
 			ctx_len += sizeof(template->ip4);
 
 			ip4->version_ihl = RTE_IPV4_VHL_DEF;
 			ip4->time_to_live = ipsec->tunnel.ipv4.ttl;
 			ip4->type_of_service |= (ipsec->tunnel.ipv4.dscp << 2);
 			if (ipsec->tunnel.ipv4.df)
-				ip4->fragment_offset = BIT(14);
+				frag_off |= RTE_IPV4_HDR_DF_FLAG;
+			ip4->fragment_offset = rte_cpu_to_be_16(frag_off);
+
 			memcpy(&ip4->src_addr, &ipsec->tunnel.ipv4.src_ip,
 			       sizeof(struct in_addr));
 			memcpy(&ip4->dst_addr, &ipsec->tunnel.ipv4.dst_ip,
diff --git a/drivers/crypto/cnxk/cnxk_cryptodev_capabilities.c b/drivers/crypto/cnxk/cnxk_cryptodev_capabilities.c
index a0b2a1f..69ee0d9 100644
--- a/drivers/crypto/cnxk/cnxk_cryptodev_capabilities.c
+++ b/drivers/crypto/cnxk/cnxk_cryptodev_capabilities.c
@@ -1121,6 +1121,7 @@ static void
 cnxk_sec_caps_update(struct rte_security_capability *sec_cap)
 {
 	sec_cap->ipsec.options.udp_encap = 1;
+	sec_cap->ipsec.options.copy_df = 1;
 }
 
 static void
-- 
2.7.4

[PATCH v2 26/29] crypto/cnxk: add aes cmac

[Anoob Joseph]

Add support for AES CMAC auth algorithm.

Signed-off-by: Anoob Joseph <anoobj@marvell.com>
---
 doc/guides/cryptodevs/cnxk.rst                    |  1 +
 doc/guides/cryptodevs/features/cn10k.ini          | 37 +++++++-------
 doc/guides/cryptodevs/features/cn9k.ini           | 37 +++++++-------
 doc/guides/rel_notes/release_22_03.rst            |  1 +
 drivers/common/cnxk/roc_se.h                      |  8 +--
 drivers/crypto/cnxk/cnxk_cryptodev_capabilities.c | 20 ++++++++
 drivers/crypto/cnxk/cnxk_se.h                     | 60 ++++++++++++++---------
 7 files changed, 103 insertions(+), 61 deletions(-)

diff --git a/doc/guides/cryptodevs/cnxk.rst b/doc/guides/cryptodevs/cnxk.rst
index 6e844f5..3c58517 100644
--- a/doc/guides/cryptodevs/cnxk.rst
+++ b/doc/guides/cryptodevs/cnxk.rst
@@ -61,6 +61,7 @@ Hash algorithms:
 * ``RTE_CRYPTO_AUTH_SHA512_HMAC``
 * ``RTE_CRYPTO_AUTH_SNOW3G_UIA2``
 * ``RTE_CRYPTO_AUTH_ZUC_EIA3``
+* ``RTE_CRYPTO_AUTH_AES_CMAC``
 
 AEAD algorithms:
 
diff --git a/doc/guides/cryptodevs/features/cn10k.ini b/doc/guides/cryptodevs/features/cn10k.ini
index ab21d9d..c8193c2 100644
--- a/doc/guides/cryptodevs/features/cn10k.ini
+++ b/doc/guides/cryptodevs/features/cn10k.ini
@@ -41,23 +41,26 @@ ZUC EEA3       = Y
 ; Supported authentication algorithms of 'cn10k' crypto driver.
 ;
 [Auth]
-NULL         = Y
-AES GMAC     = Y
-KASUMI F9    = Y
-MD5          = Y
-MD5 HMAC     = Y
-SHA1         = Y
-SHA1 HMAC    = Y
-SHA224       = Y
-SHA224 HMAC  = Y
-SHA256       = Y
-SHA256 HMAC  = Y
-SHA384       = Y
-SHA384 HMAC  = Y
-SHA512       = Y
-SHA512 HMAC  = Y
-SNOW3G UIA2  = Y
-ZUC EIA3     = Y
+NULL            = Y
+AES GMAC        = Y
+KASUMI F9       = Y
+MD5             = Y
+MD5 HMAC        = Y
+SHA1            = Y
+SHA1 HMAC       = Y
+SHA224          = Y
+SHA224 HMAC     = Y
+SHA256          = Y
+SHA256 HMAC     = Y
+SHA384          = Y
+SHA384 HMAC     = Y
+SHA512          = Y
+SHA512 HMAC     = Y
+SNOW3G UIA2     = Y
+ZUC EIA3        = Y
+AES CMAC (128)  = Y
+AES CMAC (192)  = Y
+AES CMAC (256)  = Y
 
 ;
 ; Supported AEAD algorithms of 'cn10k' crypto driver.
diff --git a/doc/guides/cryptodevs/features/cn9k.ini b/doc/guides/cryptodevs/features/cn9k.ini
index d834659..f215ee0 100644
--- a/doc/guides/cryptodevs/features/cn9k.ini
+++ b/doc/guides/cryptodevs/features/cn9k.ini
@@ -40,23 +40,26 @@ ZUC EEA3       = Y
 ; Supported authentication algorithms of 'cn9k' crypto driver.
 ;
 [Auth]
-NULL         = Y
-AES GMAC     = Y
-KASUMI F9    = Y
-MD5          = Y
-MD5 HMAC     = Y
-SHA1         = Y
-SHA1 HMAC    = Y
-SHA224       = Y
-SHA224 HMAC  = Y
-SHA256       = Y
-SHA256 HMAC  = Y
-SHA384       = Y
-SHA384 HMAC  = Y
-SHA512       = Y
-SHA512 HMAC  = Y
-SNOW3G UIA2  = Y
-ZUC EIA3     = Y
+NULL            = Y
+AES GMAC        = Y
+KASUMI F9       = Y
+MD5             = Y
+MD5 HMAC        = Y
+SHA1            = Y
+SHA1 HMAC       = Y
+SHA224          = Y
+SHA224 HMAC     = Y
+SHA256          = Y
+SHA256 HMAC     = Y
+SHA384          = Y
+SHA384 HMAC     = Y
+SHA512          = Y
+SHA512 HMAC     = Y
+SNOW3G UIA2     = Y
+ZUC EIA3        = Y
+AES CMAC (128)  = Y
+AES CMAC (192)  = Y
+AES CMAC (256)  = Y
 
 ;
 ; Supported AEAD algorithms of 'cn9k' crypto driver.
diff --git a/doc/guides/rel_notes/release_22_03.rst b/doc/guides/rel_notes/release_22_03.rst
index e8fec00..72e758e 100644
--- a/doc/guides/rel_notes/release_22_03.rst
+++ b/doc/guides/rel_notes/release_22_03.rst
@@ -63,6 +63,7 @@ New Features
   * Added AES-CTR support in lookaside protocol (IPsec) for CN9K & CN10K.
   * Added NULL cipher support in lookaside protocol (IPsec) for CN9K & CN10K.
   * Added AES-XCBC support in lookaside protocol (IPsec) for CN9K & CN10K.
+  * Added AES-CMAC support in CN9K & CN10K.
 
 
 Removed Items
diff --git a/drivers/common/cnxk/roc_se.h b/drivers/common/cnxk/roc_se.h
index 253575a..145a182 100644
--- a/drivers/common/cnxk/roc_se.h
+++ b/drivers/common/cnxk/roc_se.h
@@ -11,10 +11,10 @@
 #define ROC_SE_FC_MINOR_OP_DECRYPT    0x1
 #define ROC_SE_FC_MINOR_OP_HMAC_FIRST 0x10
 
-#define ROC_SE_MAJOR_OP_HASH	   0x34
-#define ROC_SE_MAJOR_OP_HMAC	   0x35
-#define ROC_SE_MAJOR_OP_ZUC_SNOW3G 0x37
-#define ROC_SE_MAJOR_OP_KASUMI	   0x38
+#define ROC_SE_MAJOR_OP_HASH   0x34
+#define ROC_SE_MAJOR_OP_HMAC   0x35
+#define ROC_SE_MAJOR_OP_PDCP   0x37
+#define ROC_SE_MAJOR_OP_KASUMI 0x38
 
 #define ROC_SE_MAJOR_OP_MISC		 0x01
 #define ROC_SE_MISC_MINOR_OP_PASSTHROUGH 0x03
diff --git a/drivers/crypto/cnxk/cnxk_cryptodev_capabilities.c b/drivers/crypto/cnxk/cnxk_cryptodev_capabilities.c
index 69ee0d9..457e166 100644
--- a/drivers/crypto/cnxk/cnxk_cryptodev_capabilities.c
+++ b/drivers/crypto/cnxk/cnxk_cryptodev_capabilities.c
@@ -568,6 +568,26 @@ static const struct rte_cryptodev_capabilities caps_aes[] = {
 			}, }
 		}, }
 	},
+	{	/* AES CMAC */
+		.op = RTE_CRYPTO_OP_TYPE_SYMMETRIC,
+		{.sym = {
+			.xform_type = RTE_CRYPTO_SYM_XFORM_AUTH,
+			{.auth = {
+				.algo = RTE_CRYPTO_AUTH_AES_CMAC,
+				.block_size = 16,
+				.key_size = {
+					.min = 16,
+					.max = 32,
+					.increment = 8
+				},
+				.digest_size = {
+					.min = 4,
+					.max = 4,
+					.increment = 0
+				},
+			}, }
+		}, }
+	},
 };
 
 static const struct rte_cryptodev_capabilities caps_kasumi[] = {
diff --git a/drivers/crypto/cnxk/cnxk_se.h b/drivers/crypto/cnxk/cnxk_se.h
index a8cd2c5..e988d57 100644
--- a/drivers/crypto/cnxk/cnxk_se.h
+++ b/drivers/crypto/cnxk/cnxk_se.h
@@ -73,11 +73,15 @@ pdcp_iv_copy(uint8_t *iv_d, uint8_t *iv_s, const uint8_t pdcp_alg_type,
 		for (j = 0; j < 4; j++)
 			iv_temp[j] = iv_s_temp[3 - j];
 		memcpy(iv_d, iv_temp, 16);
-	} else {
+	} else if (pdcp_alg_type == ROC_SE_PDCP_ALG_TYPE_ZUC) {
 		/* ZUC doesn't need a swap */
 		memcpy(iv_d, iv_s, 16);
 		if (pack_iv)
 			cpt_pack_iv(iv_s, iv_d);
+	} else {
+		/* AES-CMAC EIA2, microcode expects 16B zeroized IV */
+		for (j = 0; j < 4; j++)
+			iv_d[j] = 0;
 	}
 }
 
@@ -992,8 +996,8 @@ cpt_dec_hmac_prep(uint32_t flags, uint64_t d_offs, uint64_t d_lens,
 }
 
 static __rte_always_inline int
-cpt_zuc_snow3g_prep(uint32_t req_flags, uint64_t d_offs, uint64_t d_lens,
-		    struct roc_se_fc_params *params, struct cpt_inst_s *inst)
+cpt_pdcp_alg_prep(uint32_t req_flags, uint64_t d_offs, uint64_t d_lens,
+		  struct roc_se_fc_params *params, struct cpt_inst_s *inst)
 {
 	uint32_t size;
 	int32_t inputlen, outputlen;
@@ -1014,33 +1018,43 @@ cpt_zuc_snow3g_prep(uint32_t req_flags, uint64_t d_offs, uint64_t d_lens,
 	mac_len = se_ctx->mac_len;
 	pdcp_alg_type = se_ctx->pdcp_alg_type;
 
-	cpt_inst_w4.s.opcode_major = ROC_SE_MAJOR_OP_ZUC_SNOW3G;
-
+	cpt_inst_w4.s.opcode_major = ROC_SE_MAJOR_OP_PDCP;
 	cpt_inst_w4.s.opcode_minor = se_ctx->template_w4.s.opcode_minor;
 
 	if (flags == 0x1) {
 		iv_s = params->auth_iv_buf;
-		iv_len = params->auth_iv_len;
-
-		if (iv_len == 25) {
-			iv_len -= 2;
-			pack_iv = 1;
-		}
 
 		/*
 		 * Microcode expects offsets in bytes
 		 * TODO: Rounding off
 		 */
 		auth_data_len = ROC_SE_AUTH_DLEN(d_lens);
-
-		/* EIA3 or UIA2 */
 		auth_offset = ROC_SE_AUTH_OFFSET(d_offs);
-		auth_offset = auth_offset / 8;
 
-		/* consider iv len */
-		auth_offset += iv_len;
+		if (se_ctx->pdcp_alg_type != ROC_SE_PDCP_ALG_TYPE_AES_CTR) {
+			iv_len = params->auth_iv_len;
+
+			if (iv_len == 25) {
+				iv_len -= 2;
+				pack_iv = 1;
+			}
+
+			auth_offset = auth_offset / 8;
+
+			/* consider iv len */
+			auth_offset += iv_len;
+
+			inputlen =
+				auth_offset + (RTE_ALIGN(auth_data_len, 8) / 8);
+		} else {
+			iv_len = 16;
+
+			/* consider iv len */
+			auth_offset += iv_len;
+
+			inputlen = auth_offset + auth_data_len;
+		}
 
-		inputlen = auth_offset + (RTE_ALIGN(auth_data_len, 8) / 8);
 		outputlen = mac_len;
 
 		offset_ctrl = rte_cpu_to_be_64((uint64_t)auth_offset);
@@ -1056,7 +1070,6 @@ cpt_zuc_snow3g_prep(uint32_t req_flags, uint64_t d_offs, uint64_t d_lens,
 			pack_iv = 1;
 		}
 
-		/* EEA3 or UEA2 */
 		/*
 		 * Microcode expects offsets in bytes
 		 * TODO: Rounding off
@@ -1589,8 +1602,7 @@ cpt_fc_dec_hmac_prep(uint32_t flags, uint64_t d_offs, uint64_t d_lens,
 	if (likely(fc_type == ROC_SE_FC_GEN)) {
 		ret = cpt_dec_hmac_prep(flags, d_offs, d_lens, fc_params, inst);
 	} else if (fc_type == ROC_SE_PDCP) {
-		ret = cpt_zuc_snow3g_prep(flags, d_offs, d_lens, fc_params,
-					  inst);
+		ret = cpt_pdcp_alg_prep(flags, d_offs, d_lens, fc_params, inst);
 	} else if (fc_type == ROC_SE_KASUMI) {
 		ret = cpt_kasumi_dec_prep(d_offs, d_lens, fc_params, inst);
 	}
@@ -1618,8 +1630,7 @@ cpt_fc_enc_hmac_prep(uint32_t flags, uint64_t d_offs, uint64_t d_lens,
 	if (likely(fc_type == ROC_SE_FC_GEN)) {
 		ret = cpt_enc_hmac_prep(flags, d_offs, d_lens, fc_params, inst);
 	} else if (fc_type == ROC_SE_PDCP) {
-		ret = cpt_zuc_snow3g_prep(flags, d_offs, d_lens, fc_params,
-					  inst);
+		ret = cpt_pdcp_alg_prep(flags, d_offs, d_lens, fc_params, inst);
 	} else if (fc_type == ROC_SE_KASUMI) {
 		ret = cpt_kasumi_enc_prep(flags, d_offs, d_lens, fc_params,
 					  inst);
@@ -1883,8 +1894,11 @@ fill_sess_auth(struct rte_crypto_sym_xform *xform, struct cnxk_se_sess *sess)
 		auth_type = 0;
 		is_null = 1;
 		break;
-	case RTE_CRYPTO_AUTH_AES_XCBC_MAC:
 	case RTE_CRYPTO_AUTH_AES_CMAC:
+		auth_type = ROC_SE_AES_CMAC_EIA2;
+		zsk_flag = ROC_SE_ZS_IA;
+		break;
+	case RTE_CRYPTO_AUTH_AES_XCBC_MAC:
 	case RTE_CRYPTO_AUTH_AES_CBC_MAC:
 		plt_dp_err("Crypto: Unsupported hash algo %u", a_form->algo);
 		return -1;
-- 
2.7.4

[PATCH v2 27/29] crypto/cnxk: add per pkt IV in lookaside IPsec debug mode

[Anoob Joseph]

From: Archana Muniganti <marchana@marvell.com>

For cn9k, use HW GEN IV as default and add per pkt IV
in lookaside IPsec debug mode. Debug mode helps to verify
lookaside PMD using known outbound vectors in lookaside
autotest.

Signed-off-by: Archana Muniganti <marchana@marvell.com>
---
 drivers/common/cnxk/roc_ie_on.h                   |  7 +++++
 drivers/crypto/cnxk/cn9k_ipsec.c                  | 34 +++++++++++++++++------
 drivers/crypto/cnxk/cn9k_ipsec.h                  |  2 ++
 drivers/crypto/cnxk/cn9k_ipsec_la_ops.h           | 14 +++++++---
 drivers/crypto/cnxk/cnxk_cryptodev_capabilities.c |  2 ++
 5 files changed, 47 insertions(+), 12 deletions(-)

diff --git a/drivers/common/cnxk/roc_ie_on.h b/drivers/common/cnxk/roc_ie_on.h
index cb56a70..aaad872 100644
--- a/drivers/common/cnxk/roc_ie_on.h
+++ b/drivers/common/cnxk/roc_ie_on.h
@@ -22,6 +22,8 @@ enum roc_ie_on_ucc_ipsec {
 
 /* Helper macros */
 #define ROC_IE_ON_INB_RPTR_HDR 0x8
+#define ROC_IE_ON_MAX_IV_LEN   16
+#define ROC_IE_ON_PER_PKT_IV   BIT(43)
 
 enum {
 	ROC_IE_ON_SA_ENC_NULL = 0,
@@ -55,6 +57,11 @@ enum {
 	ROC_IE_ON_SA_ENCAP_UDP = 1,
 };
 
+enum {
+	ROC_IE_ON_IV_SRC_HW_GEN_DEFAULT = 0,
+	ROC_IE_ON_IV_SRC_FROM_DPTR = 1,
+};
+
 struct roc_ie_on_outb_hdr {
 	uint32_t ip_id;
 	uint32_t seq;
diff --git a/drivers/crypto/cnxk/cn9k_ipsec.c b/drivers/crypto/cnxk/cn9k_ipsec.c
index 62b9c26..9f876f7 100644
--- a/drivers/crypto/cnxk/cn9k_ipsec.c
+++ b/drivers/crypto/cnxk/cn9k_ipsec.c
@@ -426,13 +426,7 @@ cn9k_ipsec_outb_sa_create(struct cnxk_cpt_qp *qp,
 
 	ctx_len += RTE_ALIGN_CEIL(ctx_len, 8);
 
-	if (crypto_xform->type == RTE_CRYPTO_SYM_XFORM_AEAD) {
-		sa->cipher_iv_off = crypto_xform->aead.iv.offset;
-		sa->cipher_iv_len = crypto_xform->aead.iv.length;
-	} else {
-		sa->cipher_iv_off = crypto_xform->cipher.iv.offset;
-		sa->cipher_iv_len = crypto_xform->cipher.iv.length;
-
+	if (crypto_xform->type != RTE_CRYPTO_SYM_XFORM_AEAD) {
 		auth_key = auth_xform->auth.key.data;
 		auth_key_len = auth_xform->auth.key.length;
 
@@ -465,7 +459,31 @@ cn9k_ipsec_outb_sa_create(struct cnxk_cpt_qp *qp,
 
 	param1.u16 = 0;
 	param1.s.ikev2 = 1;
-	param1.s.per_pkt_iv = 1;
+
+	sa->custom_hdr_len = sizeof(struct roc_ie_on_outb_hdr) -
+			     ROC_IE_ON_MAX_IV_LEN;
+
+#ifdef LA_IPSEC_DEBUG
+	/* Use IV from application in debug mode */
+	if (ipsec->options.iv_gen_disable == 1) {
+		param1.s.per_pkt_iv = ROC_IE_ON_IV_SRC_FROM_DPTR;
+		sa->custom_hdr_len = sizeof(struct roc_ie_on_outb_hdr);
+
+		if (crypto_xform->type == RTE_CRYPTO_SYM_XFORM_AEAD) {
+			sa->cipher_iv_off = crypto_xform->aead.iv.offset;
+			sa->cipher_iv_len = crypto_xform->aead.iv.length;
+		} else {
+			sa->cipher_iv_off = crypto_xform->cipher.iv.offset;
+			sa->cipher_iv_len = crypto_xform->cipher.iv.length;
+		}
+	}
+#else
+	if (ipsec->options.iv_gen_disable != 0) {
+		plt_err("Application provided IV is not supported");
+		return -ENOTSUP;
+	}
+#endif
+
 	w4.s.param1 = param1.u16;
 
 	inst_tmpl->w4 = w4.u64;
diff --git a/drivers/crypto/cnxk/cn9k_ipsec.h b/drivers/crypto/cnxk/cn9k_ipsec.h
index fc440d5..f3acad5 100644
--- a/drivers/crypto/cnxk/cn9k_ipsec.h
+++ b/drivers/crypto/cnxk/cn9k_ipsec.h
@@ -24,6 +24,8 @@ struct cn9k_ipsec_sa {
 	uint16_t cipher_iv_off;
 	/** Cipher IV length in bytes */
 	uint8_t cipher_iv_len;
+	/** Outbound custom header length */
+	uint8_t custom_hdr_len;
 	/** Response length calculation data */
 	struct cnxk_ipsec_outb_rlens rlens;
 	/** Outbound IP-ID */
diff --git a/drivers/crypto/cnxk/cn9k_ipsec_la_ops.h b/drivers/crypto/cnxk/cn9k_ipsec_la_ops.h
index 2b0261e..9a1e217 100644
--- a/drivers/crypto/cnxk/cn9k_ipsec_la_ops.h
+++ b/drivers/crypto/cnxk/cn9k_ipsec_la_ops.h
@@ -74,7 +74,7 @@ static __rte_always_inline int
 process_outb_sa(struct rte_crypto_op *cop, struct cn9k_ipsec_sa *sa,
 		struct cpt_inst_s *inst)
 {
-	const unsigned int hdr_len = sizeof(struct roc_ie_on_outb_hdr);
+	const unsigned int hdr_len = sa->custom_hdr_len;
 	struct rte_crypto_sym_op *sym_op = cop->sym;
 	struct rte_mbuf *m_src = sym_op->m_src;
 	struct roc_ie_on_outb_sa *out_sa;
@@ -103,9 +103,15 @@ process_outb_sa(struct rte_crypto_op *cop, struct cn9k_ipsec_sa *sa,
 		return -ENOMEM;
 	}
 
-	memcpy(&hdr->iv[0],
-	       rte_crypto_op_ctod_offset(cop, uint8_t *, sa->cipher_iv_off),
-	       sa->cipher_iv_len);
+#ifdef LA_IPSEC_DEBUG
+	if (sa->inst.w4 & ROC_IE_ON_PER_PKT_IV) {
+		memcpy(&hdr->iv[0],
+		       rte_crypto_op_ctod_offset(cop, uint8_t *,
+						 sa->cipher_iv_off),
+		       sa->cipher_iv_len);
+	}
+#endif
+
 	hdr->seq = rte_cpu_to_be_32(sa->seq_lo);
 	hdr->ip_id = rte_cpu_to_be_32(sa->ip_id);
 
diff --git a/drivers/crypto/cnxk/cnxk_cryptodev_capabilities.c b/drivers/crypto/cnxk/cnxk_cryptodev_capabilities.c
index 457e166..f79e4d7 100644
--- a/drivers/crypto/cnxk/cnxk_cryptodev_capabilities.c
+++ b/drivers/crypto/cnxk/cnxk_cryptodev_capabilities.c
@@ -1166,7 +1166,9 @@ static void
 cn9k_sec_caps_update(struct rte_security_capability *sec_cap)
 {
 	if (sec_cap->ipsec.direction == RTE_SECURITY_IPSEC_SA_DIR_EGRESS) {
+#ifdef LA_IPSEC_DEBUG
 		sec_cap->ipsec.options.iv_gen_disable = 1;
+#endif
 	}
 }
 
-- 
2.7.4

[PATCH v2 28/29] crypto/cnxk: enable copy dscp

[Anoob Joseph]

Copy DSCP is supported. Enable it in capabilities.

Signed-off-by: Anoob Joseph <anoobj@marvell.com>
---
 drivers/crypto/cnxk/cnxk_cryptodev_capabilities.c | 1 +
 1 file changed, 1 insertion(+)

diff --git a/drivers/crypto/cnxk/cnxk_cryptodev_capabilities.c b/drivers/crypto/cnxk/cnxk_cryptodev_capabilities.c
index f79e4d7..f8c007e 100644
--- a/drivers/crypto/cnxk/cnxk_cryptodev_capabilities.c
+++ b/drivers/crypto/cnxk/cnxk_cryptodev_capabilities.c
@@ -1142,6 +1142,7 @@ cnxk_sec_caps_update(struct rte_security_capability *sec_cap)
 {
 	sec_cap->ipsec.options.udp_encap = 1;
 	sec_cap->ipsec.options.copy_df = 1;
+	sec_cap->ipsec.options.copy_dscp = 1;
 }
 
 static void
-- 
2.7.4

[PATCH v2 29/29] crypto/cnxk: update microcode completion handling

[Anoob Joseph]

Update microcode completion code handling to update the required mbuf &
crypto op flags. IP checksum good case is now reported by specific
microcode completion code.

Signed-off-by: Anoob Joseph <anoobj@marvell.com>
---
 drivers/crypto/cnxk/cn10k_cryptodev_ops.c | 59 ++++++++++---------------------
 drivers/crypto/cnxk/cn10k_ipsec.c         |  1 -
 drivers/crypto/cnxk/cn10k_ipsec.h         |  1 -
 3 files changed, 18 insertions(+), 43 deletions(-)

diff --git a/drivers/crypto/cnxk/cn10k_cryptodev_ops.c b/drivers/crypto/cnxk/cn10k_cryptodev_ops.c
index 1905ea3..d217bbf 100644
--- a/drivers/crypto/cnxk/cn10k_cryptodev_ops.c
+++ b/drivers/crypto/cnxk/cn10k_cryptodev_ops.c
@@ -50,8 +50,7 @@ cn10k_cpt_sym_temp_sess_create(struct cnxk_cpt_qp *qp, struct rte_crypto_op *op)
 
 static __rte_always_inline int __rte_hot
 cpt_sec_inst_fill(struct cnxk_cpt_qp *qp, struct rte_crypto_op *op,
-		  struct cn10k_sec_session *sess,
-		  struct cpt_inflight_req *infl_req, struct cpt_inst_s *inst)
+		  struct cn10k_sec_session *sess, struct cpt_inst_s *inst)
 {
 	struct rte_crypto_sym_op *sym_op = op->sym;
 	struct cn10k_ipsec_sa *sa;
@@ -71,10 +70,8 @@ cpt_sec_inst_fill(struct cnxk_cpt_qp *qp, struct rte_crypto_op *op,
 
 	if (sa->is_outbound)
 		ret = process_outb_sa(&qp->lf, op, sa, inst);
-	else {
-		infl_req->op_flags |= CPT_OP_FLAGS_IPSEC_DIR_INBOUND;
+	else
 		ret = process_inb_sa(op, sa, inst);
-	}
 
 	return ret;
 }
@@ -127,8 +124,7 @@ cn10k_cpt_fill_inst(struct cnxk_cpt_qp *qp, struct rte_crypto_op *ops[],
 		if (op->sess_type == RTE_CRYPTO_OP_SECURITY_SESSION) {
 			sec_sess = get_sec_session_private_data(
 				sym_op->sec_session);
-			ret = cpt_sec_inst_fill(qp, op, sec_sess, infl_req,
-						&inst[0]);
+			ret = cpt_sec_inst_fill(qp, op, sec_sess, &inst[0]);
 			if (unlikely(ret))
 				return 0;
 			w7 = sec_sess->sa.inst.w7;
@@ -346,52 +342,34 @@ static inline void
 cn10k_cpt_sec_post_process(struct rte_crypto_op *cop,
 			   struct cpt_cn10k_res_s *res)
 {
-	struct rte_mbuf *m = cop->sym->m_src;
+	struct rte_mbuf *mbuf = cop->sym->m_src;
 	const uint16_t m_len = res->rlen;
 
-	m->data_len = m_len;
-	m->pkt_len = m_len;
-}
-
-static inline void
-cn10k_cpt_sec_ucc_process(struct rte_crypto_op *cop,
-			  struct cpt_inflight_req *infl_req,
-			  const uint8_t uc_compcode)
-{
-	struct cn10k_sec_session *sess;
-	struct cn10k_ipsec_sa *sa;
-	struct rte_mbuf *mbuf;
-
-	if (uc_compcode == ROC_IE_OT_UCC_SUCCESS_SA_SOFTEXP_FIRST)
-		cop->aux_flags = RTE_CRYPTO_OP_AUX_FLAGS_IPSEC_SOFT_EXPIRY;
-
-	if (!(infl_req->op_flags & CPT_OP_FLAGS_IPSEC_DIR_INBOUND))
-		return;
-
-	sess = get_sec_session_private_data(cop->sym->sec_session);
-	sa = &sess->sa;
+	mbuf->data_len = m_len;
+	mbuf->pkt_len = m_len;
 
-	mbuf = cop->sym->m_src;
-
-	switch (uc_compcode) {
+	switch (res->uc_compcode) {
 	case ROC_IE_OT_UCC_SUCCESS:
-		if (sa->ip_csum_enable)
-			mbuf->ol_flags |= RTE_MBUF_F_RX_IP_CKSUM_GOOD;
 		break;
 	case ROC_IE_OT_UCC_SUCCESS_PKT_IP_BADCSUM:
 		mbuf->ol_flags |= RTE_MBUF_F_RX_IP_CKSUM_BAD;
 		break;
 	case ROC_IE_OT_UCC_SUCCESS_PKT_L4_GOODCSUM:
-		mbuf->ol_flags |= RTE_MBUF_F_RX_L4_CKSUM_GOOD;
-		if (sa->ip_csum_enable)
-			mbuf->ol_flags |= RTE_MBUF_F_RX_IP_CKSUM_GOOD;
+		mbuf->ol_flags |= RTE_MBUF_F_RX_L4_CKSUM_GOOD |
+				  RTE_MBUF_F_RX_IP_CKSUM_GOOD;
 		break;
 	case ROC_IE_OT_UCC_SUCCESS_PKT_L4_BADCSUM:
-		mbuf->ol_flags |= RTE_MBUF_F_RX_L4_CKSUM_BAD;
-		if (sa->ip_csum_enable)
-			mbuf->ol_flags |= RTE_MBUF_F_RX_IP_CKSUM_GOOD;
+		mbuf->ol_flags |= RTE_MBUF_F_RX_L4_CKSUM_BAD |
+				  RTE_MBUF_F_RX_IP_CKSUM_GOOD;
+		break;
+	case ROC_IE_OT_UCC_SUCCESS_PKT_IP_GOODCSUM:
+		mbuf->ol_flags |= RTE_MBUF_F_RX_IP_CKSUM_GOOD;
+		break;
+	case ROC_IE_OT_UCC_SUCCESS_SA_SOFTEXP_FIRST:
+		cop->aux_flags = RTE_CRYPTO_OP_AUX_FLAGS_IPSEC_SOFT_EXPIRY;
 		break;
 	default:
+		plt_dp_err("Success with unknown microcode completion code");
 		break;
 	}
 }
@@ -412,7 +390,6 @@ cn10k_cpt_dequeue_post_process(struct cnxk_cpt_qp *qp,
 	    cop->sess_type == RTE_CRYPTO_OP_SECURITY_SESSION) {
 		if (likely(compcode == CPT_COMP_WARN)) {
 			/* Success with additional info */
-			cn10k_cpt_sec_ucc_process(cop, infl_req, uc_compcode);
 			cn10k_cpt_sec_post_process(cop, res);
 		} else {
 			cop->status = RTE_CRYPTO_OP_STATUS_ERROR;
diff --git a/drivers/crypto/cnxk/cn10k_ipsec.c b/drivers/crypto/cnxk/cn10k_ipsec.c
index a93c211..7f4ccaf 100644
--- a/drivers/crypto/cnxk/cn10k_ipsec.c
+++ b/drivers/crypto/cnxk/cn10k_ipsec.c
@@ -201,7 +201,6 @@ cn10k_ipsec_inb_sa_create(struct roc_cpt *roc_cpt, struct roc_cpt_lf *lf,
 	if (ipsec_xfrm->options.ip_csum_enable) {
 		param1.s.ip_csum_disable =
 			ROC_IE_OT_SA_INNER_PKT_IP_CSUM_ENABLE;
-		sa->ip_csum_enable = true;
 	}
 
 	/* Disable L4 checksum verification by default */
diff --git a/drivers/crypto/cnxk/cn10k_ipsec.h b/drivers/crypto/cnxk/cn10k_ipsec.h
index cc7ca19..647a71c 100644
--- a/drivers/crypto/cnxk/cn10k_ipsec.h
+++ b/drivers/crypto/cnxk/cn10k_ipsec.h
@@ -19,7 +19,6 @@ struct cn10k_ipsec_sa {
 	uint16_t max_extended_len;
 	uint16_t iv_offset;
 	uint8_t iv_length;
-	bool ip_csum_enable;
 	bool is_outbound;
 
 	/**
-- 
2.7.4

[PATCH v3 00/29] New features and improvements in cnxk crypto PMD

[Anoob Joseph]

New features and fixes to cnxk crypto PMDs
- Support for more algorithms in lookaside crypto & protocol
- Support for copy & set DF bit
- Support for CPT CTX update
- Support for security session stats in cn10k

Changes in v3
- Fixed build error from CI

Changes in v2
- Support for copy & set DSCP
- Support for per packet IV in cn9k
- Support for cn10k v1.19 microcode

Ankur Dwivedi (1):
  crypto/cnxk: add security session stats get

Anoob Joseph (20):
  common/cnxk: define minor opcodes for MISC opcode
  common/cnxk: add aes-xcbc key derive
  common/cnxk: fix reset of fields
  common/cnxk: verify input args
  common/cnxk: update completion code
  crypto/cnxk: clear session data before populating
  crypto/cnxk: update max sec crypto caps
  crypto/cnxk: account for CPT CTX updates and flush delays
  crypto/cnxk: use struct sizes for ctx writes
  crypto/cnxk: add skip for unsupported cases
  crypto/cnxk: handle null chained ops
  crypto/cnxk: fix inflight cnt calculation
  crypto/cnxk: use atomics to access CPT res
  crypto/cnxk: add more info on command timeout
  crypto/cnxk: fix extend tail calculation
  crypto/cnxk: add aes xcbc and null cipher
  crypto/cnxk: add copy and set DF
  crypto/cnxk: add aes cmac
  crypto/cnxk: enable copy dscp
  crypto/cnxk: update microcode completion handling

Archana Muniganti (2):
  common/cnxk: add bit fields for params
  crypto/cnxk: add per pkt IV in lookaside IPsec debug mode

Shijith Thotton (1):
  crypto/cnxk: only enable queues that are allocated

Tejasree Kondoj (5):
  crypto/cnxk: add lookaside IPsec AES-CBC-HMAC-SHA256 support
  crypto/cnxk: write CPT CTX through microcode op
  crypto/cnxk: support cnxk lookaside IPsec HMAC-SHA384/512
  crypto/cnxk: add context reload for IV
  crypto/cnxk: support lookaside IPsec AES-CTR

 doc/guides/cryptodevs/cnxk.rst                    |  50 ++++-
 doc/guides/cryptodevs/features/cn10k.ini          |  37 ++--
 doc/guides/cryptodevs/features/cn9k.ini           |  37 ++--
 doc/guides/rel_notes/release_22_03.rst            |  10 +
 drivers/common/cnxk/cnxk_security.c               |  92 ++++++--
 drivers/common/cnxk/hw/cpt.h                      |  15 ++
 drivers/common/cnxk/meson.build                   |   1 +
 drivers/common/cnxk/roc_aes.c                     | 208 ++++++++++++++++++
 drivers/common/cnxk/roc_aes.h                     |  14 ++
 drivers/common/cnxk/roc_api.h                     |   3 +
 drivers/common/cnxk/roc_cpt.c                     |   4 +-
 drivers/common/cnxk/roc_cpt.h                     |  24 +-
 drivers/common/cnxk/roc_ie_on.h                   |  47 +++-
 drivers/common/cnxk/roc_ie_ot.h                   |   4 +-
 drivers/common/cnxk/roc_se.h                      |  14 +-
 drivers/common/cnxk/version.map                   |   1 +
 drivers/crypto/cnxk/cn10k_cryptodev_ops.c         |  91 ++++----
 drivers/crypto/cnxk/cn10k_ipsec.c                 | 232 ++++++++++++++++----
 drivers/crypto/cnxk/cn10k_ipsec.h                 |  26 ++-
 drivers/crypto/cnxk/cn10k_ipsec_la_ops.h          |  28 ++-
 drivers/crypto/cnxk/cn9k_cryptodev_ops.c          |  29 ++-
 drivers/crypto/cnxk/cn9k_ipsec.c                  | 253 +++++++++++++++++-----
 drivers/crypto/cnxk/cn9k_ipsec.h                  |   2 +
 drivers/crypto/cnxk/cn9k_ipsec_la_ops.h           |  20 +-
 drivers/crypto/cnxk/cnxk_cryptodev.h              |   2 +-
 drivers/crypto/cnxk/cnxk_cryptodev_capabilities.c | 158 +++++++++++++-
 drivers/crypto/cnxk/cnxk_cryptodev_ops.c          | 245 +++++++++++++--------
 drivers/crypto/cnxk/cnxk_cryptodev_ops.h          |  17 +-
 drivers/crypto/cnxk/cnxk_cryptodev_sec.c          |   1 +
 drivers/crypto/cnxk/cnxk_ipsec.h                  |  19 +-
 drivers/crypto/cnxk/cnxk_se.h                     |  66 ++++--
 31 files changed, 1348 insertions(+), 402 deletions(-)
 create mode 100644 drivers/common/cnxk/roc_aes.c
 create mode 100644 drivers/common/cnxk/roc_aes.h

-- 
2.7.4

[PATCH v3 01/29] common/cnxk: define minor opcodes for MISC opcode

[Anoob Joseph]

MISC CPT instruction behaves differently based on minor opcode.
Define the missing minor opcodes for MISC major opcode.

Signed-off-by: Aakash Sasidharan <asasidharan@marvell.com>
Signed-off-by: Anoob Joseph <anoobj@marvell.com>
---
 drivers/common/cnxk/roc_se.h | 6 +++++-
 1 file changed, 5 insertions(+), 1 deletion(-)

diff --git a/drivers/common/cnxk/roc_se.h b/drivers/common/cnxk/roc_se.h
index 5be832f..253575a 100644
--- a/drivers/common/cnxk/roc_se.h
+++ b/drivers/common/cnxk/roc_se.h
@@ -15,7 +15,11 @@
 #define ROC_SE_MAJOR_OP_HMAC	   0x35
 #define ROC_SE_MAJOR_OP_ZUC_SNOW3G 0x37
 #define ROC_SE_MAJOR_OP_KASUMI	   0x38
-#define ROC_SE_MAJOR_OP_MISC	   0x01
+
+#define ROC_SE_MAJOR_OP_MISC		 0x01
+#define ROC_SE_MISC_MINOR_OP_PASSTHROUGH 0x03
+#define ROC_SE_MISC_MINOR_OP_DUMMY	 0x04
+#define ROC_SE_MISC_MINOR_OP_HW_SUPPORT	 0x08
 
 #define ROC_SE_MAX_AAD_SIZE 64
 #define ROC_SE_MAX_MAC_LEN  64
-- 
2.7.4

[PATCH v3 02/29] common/cnxk: add aes-xcbc key derive

[Anoob Joseph]

Add support for AES-XCBC key derivation.


Signed-off-by: Anoob Joseph <anoobj@marvell.com>
---
 drivers/common/cnxk/meson.build |   1 +
 drivers/common/cnxk/roc_aes.c   | 208 ++++++++++++++++++++++++++++++++++++++++
 drivers/common/cnxk/roc_aes.h   |  14 +++
 drivers/common/cnxk/roc_api.h   |   3 +
 drivers/common/cnxk/roc_cpt.h   |  24 ++---
 drivers/common/cnxk/version.map |   1 +
 6 files changed, 239 insertions(+), 12 deletions(-)
 create mode 100644 drivers/common/cnxk/roc_aes.c
 create mode 100644 drivers/common/cnxk/roc_aes.h

diff --git a/drivers/common/cnxk/meson.build b/drivers/common/cnxk/meson.build
index 4928f7e..4995cfd 100644
--- a/drivers/common/cnxk/meson.build
+++ b/drivers/common/cnxk/meson.build
@@ -12,6 +12,7 @@ config_flag_fmt = 'RTE_LIBRTE_@0@_COMMON'
 deps = ['eal', 'pci', 'bus_pci', 'mbuf', 'security']
 sources = files(
         'roc_ae.c',
+        'roc_aes.c',
         'roc_ae_fpm_tables.c',
         'roc_bphy.c',
         'roc_bphy_cgx.c',
diff --git a/drivers/common/cnxk/roc_aes.c b/drivers/common/cnxk/roc_aes.c
new file mode 100644
index 0000000..f821c8b
--- /dev/null
+++ b/drivers/common/cnxk/roc_aes.c
@@ -0,0 +1,208 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright (c) 2021 Marvell.
+ */
+
+#include "roc_api.h"
+
+#define KEY_WORD_LEN	 (ROC_CPT_AES_XCBC_KEY_LENGTH / sizeof(uint32_t))
+#define KEY_ROUNDS	 10			/* (Nr+1)*Nb */
+#define KEY_SCHEDULE_LEN ((KEY_ROUNDS + 1) * 4) /* (Nr+1)*Nb words */
+
+/*
+ * AES 128 implementation based on NIST FIPS 197 suitable for LittleEndian
+ * https://nvlpubs.nist.gov/nistpubs/FIPS/NIST.FIPS.197.pdf
+ */
+
+/* Sbox from NIST FIPS 197 */
+static uint8_t Sbox[] = {
+	0x63, 0x7c, 0x77, 0x7b, 0xf2, 0x6b, 0x6f, 0xc5, 0x30, 0x01, 0x67, 0x2b,
+	0xfe, 0xd7, 0xab, 0x76, 0xca, 0x82, 0xc9, 0x7d, 0xfa, 0x59, 0x47, 0xf0,
+	0xad, 0xd4, 0xa2, 0xaf, 0x9c, 0xa4, 0x72, 0xc0, 0xb7, 0xfd, 0x93, 0x26,
+	0x36, 0x3f, 0xf7, 0xcc, 0x34, 0xa5, 0xe5, 0xf1, 0x71, 0xd8, 0x31, 0x15,
+	0x04, 0xc7, 0x23, 0xc3, 0x18, 0x96, 0x05, 0x9a, 0x07, 0x12, 0x80, 0xe2,
+	0xeb, 0x27, 0xb2, 0x75, 0x09, 0x83, 0x2c, 0x1a, 0x1b, 0x6e, 0x5a, 0xa0,
+	0x52, 0x3b, 0xd6, 0xb3, 0x29, 0xe3, 0x2f, 0x84, 0x53, 0xd1, 0x00, 0xed,
+	0x20, 0xfc, 0xb1, 0x5b, 0x6a, 0xcb, 0xbe, 0x39, 0x4a, 0x4c, 0x58, 0xcf,
+	0xd0, 0xef, 0xaa, 0xfb, 0x43, 0x4d, 0x33, 0x85, 0x45, 0xf9, 0x02, 0x7f,
+	0x50, 0x3c, 0x9f, 0xa8, 0x51, 0xa3, 0x40, 0x8f, 0x92, 0x9d, 0x38, 0xf5,
+	0xbc, 0xb6, 0xda, 0x21, 0x10, 0xff, 0xf3, 0xd2, 0xcd, 0x0c, 0x13, 0xec,
+	0x5f, 0x97, 0x44, 0x17, 0xc4, 0xa7, 0x7e, 0x3d, 0x64, 0x5d, 0x19, 0x73,
+	0x60, 0x81, 0x4f, 0xdc, 0x22, 0x2a, 0x90, 0x88, 0x46, 0xee, 0xb8, 0x14,
+	0xde, 0x5e, 0x0b, 0xdb, 0xe0, 0x32, 0x3a, 0x0a, 0x49, 0x06, 0x24, 0x5c,
+	0xc2, 0xd3, 0xac, 0x62, 0x91, 0x95, 0xe4, 0x79, 0xe7, 0xc8, 0x37, 0x6d,
+	0x8d, 0xd5, 0x4e, 0xa9, 0x6c, 0x56, 0xf4, 0xea, 0x65, 0x7a, 0xae, 0x08,
+	0xba, 0x78, 0x25, 0x2e, 0x1c, 0xa6, 0xb4, 0xc6, 0xe8, 0xdd, 0x74, 0x1f,
+	0x4b, 0xbd, 0x8b, 0x8a, 0x70, 0x3e, 0xb5, 0x66, 0x48, 0x03, 0xf6, 0x0e,
+	0x61, 0x35, 0x57, 0xb9, 0x86, 0xc1, 0x1d, 0x9e, 0xe1, 0xf8, 0x98, 0x11,
+	0x69, 0xd9, 0x8e, 0x94, 0x9b, 0x1e, 0x87, 0xe9, 0xce, 0x55, 0x28, 0xdf,
+	0x8c, 0xa1, 0x89, 0x0d, 0xbf, 0xe6, 0x42, 0x68, 0x41, 0x99, 0x2d, 0x0f,
+	0xb0, 0x54, 0xbb, 0x16,
+};
+
+/* Substitute a byte with Sbox[byte]. Do it for a word for 4 bytes */
+static uint32_t
+sub_word(uint32_t word)
+{
+	word = (Sbox[(word >> 24) & 0xFF] << 24) |
+	       (Sbox[(word >> 16) & 0xFF] << 16) |
+	       (Sbox[(word >> 8) & 0xFF] << 8) | Sbox[word & 0xFF];
+	return word;
+}
+
+/* Rotate a word by one byte */
+static uint32_t
+rot_word(uint32_t word)
+{
+	return ((word >> 8) & 0xFFFFFF) | (word << 24);
+}
+
+/*
+ * Multiply with power of 2 and polynomial reduce the result using AES
+ * polynomial
+ */
+static uint8_t
+Xtime(uint8_t byte, uint8_t pow)
+{
+	uint32_t w = byte;
+
+	while (pow) {
+		w = w << 1;
+		if (w >> 8)
+			w ^= 0x11b;
+		pow--;
+	}
+
+	return (uint8_t)w;
+}
+
+/*
+ * Multiply a byte with another number such that the result is polynomial
+ * reduced in the GF8 space
+ */
+static uint8_t
+GF8mul(uint8_t byte, uint32_t mp)
+{
+	uint8_t pow, mul = 0;
+
+	while (mp) {
+		pow = ffs(mp) - 1;
+		mul ^= Xtime(byte, pow);
+		mp ^= (1 << pow);
+	}
+	return mul;
+}
+
+static void
+aes_key_expand(const uint8_t *key, uint32_t *ks)
+{
+	unsigned int i = 4;
+	uint32_t temp;
+
+	/* Skip key in ks */
+	memcpy(ks, key, KEY_WORD_LEN * sizeof(uint32_t));
+
+	while (i < KEY_SCHEDULE_LEN) {
+		temp = ks[i - 1];
+		if ((i & 0x3) == 0) {
+			temp = rot_word(temp);
+			temp = sub_word(temp);
+			temp ^= (uint32_t)GF8mul(1, 1 << ((i >> 2) - 1));
+		}
+		ks[i] = ks[i - 4] ^ temp;
+		i++;
+	}
+}
+
+/* Shift Rows(columns in state in this implementation) */
+static void
+shift_word(uint8_t *sRc, uint8_t c, int count)
+{
+	/* rotate across non-consecutive locations */
+	while (count) {
+		uint8_t t = sRc[c];
+
+		sRc[c] = sRc[0x4 + c];
+		sRc[0x4 + c] = sRc[0x8 + c];
+		sRc[0x8 + c] = sRc[0xc + c];
+		sRc[0xc + c] = t;
+		count--;
+	}
+}
+
+/* Mix Columns(rows in state in this implementation) */
+static void
+mix_columns(uint8_t *sRc)
+{
+	uint8_t new_st[4];
+	int i;
+
+	for (i = 0; i < 4; i++)
+		new_st[i] = GF8mul(sRc[i], 0x2) ^
+			    GF8mul(sRc[(i + 1) & 0x3], 0x3) ^
+			    sRc[(i + 2) & 0x3] ^ sRc[(i + 3) & 0x3];
+	for (i = 0; i < 4; i++)
+		sRc[i] = new_st[i];
+}
+
+static void
+cipher(uint8_t *in, uint8_t *out, uint32_t *ks)
+{
+	uint32_t state[KEY_WORD_LEN];
+	unsigned int i, round;
+
+	memcpy(state, in, sizeof(state));
+
+	/* AddRoundKey(state, w[0, Nb-1]) // See Sec. 5.1.4 */
+	for (i = 0; i < KEY_WORD_LEN; i++)
+		state[i] ^= ks[i];
+
+	for (round = 1; round < KEY_ROUNDS; round++) {
+		/* SubBytes(state) // See Sec. 5.1.1 */
+		for (i = 0; i < KEY_WORD_LEN; i++)
+			state[i] = sub_word(state[i]);
+
+		/* ShiftRows(state) // See Sec. 5.1.2 */
+		for (i = 0; i < KEY_WORD_LEN; i++)
+			shift_word((uint8_t *)state, i, i);
+
+		/* MixColumns(state) // See Sec. 5.1.3 */
+		for (i = 0; i < KEY_WORD_LEN; i++)
+			mix_columns((uint8_t *)&state[i]);
+
+		/* AddRoundKey(state, w[round*Nb, (round+1)*Nb-1]) */
+		for (i = 0; i < KEY_WORD_LEN; i++)
+			state[i] ^= ks[round * 4 + i];
+	}
+
+	/* SubBytes(state) */
+	for (i = 0; i < KEY_WORD_LEN; i++)
+		state[i] = sub_word(state[i]);
+
+	/* ShiftRows(state) */
+	for (i = 0; i < KEY_WORD_LEN; i++)
+		shift_word((uint8_t *)state, i, i);
+
+	/* AddRoundKey(state, w[Nr*Nb, (Nr+1)*Nb-1]) */
+	for (i = 0; i < KEY_WORD_LEN; i++)
+		state[i] ^= ks[KEY_ROUNDS * 4 + i];
+	memcpy(out, state, KEY_WORD_LEN * sizeof(uint32_t));
+}
+
+void
+roc_aes_xcbc_key_derive(const uint8_t *auth_key, uint8_t *derived_key)
+{
+	uint32_t aes_ks[KEY_SCHEDULE_LEN] = {0};
+	uint8_t k1[16] = {[0 ... 15] = 0x01};
+	uint8_t k2[16] = {[0 ... 15] = 0x02};
+	uint8_t k3[16] = {[0 ... 15] = 0x03};
+
+	aes_key_expand(auth_key, aes_ks);
+
+	cipher(k1, derived_key, aes_ks);
+	derived_key += sizeof(k1);
+
+	cipher(k2, derived_key, aes_ks);
+	derived_key += sizeof(k2);
+
+	cipher(k3, derived_key, aes_ks);
+}
diff --git a/drivers/common/cnxk/roc_aes.h b/drivers/common/cnxk/roc_aes.h
new file mode 100644
index 0000000..9540391
--- /dev/null
+++ b/drivers/common/cnxk/roc_aes.h
@@ -0,0 +1,14 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright (c) 2021 Marvell.
+ */
+
+#ifndef _ROC_AES_H_
+#define _ROC_AES_H_
+
+/*
+ * Derive k1, k2, k3 from 128 bit AES key
+ */
+void __roc_api roc_aes_xcbc_key_derive(const uint8_t *auth_key,
+				       uint8_t *derived_key);
+
+#endif /* _ROC_AES_H_ */
diff --git a/drivers/common/cnxk/roc_api.h b/drivers/common/cnxk/roc_api.h
index e7aaa07..cf4d487 100644
--- a/drivers/common/cnxk/roc_api.h
+++ b/drivers/common/cnxk/roc_api.h
@@ -90,6 +90,9 @@
 /* DPI */
 #include "roc_dpi.h"
 
+/* AES */
+#include "roc_aes.h"
+
 /* HASH computation */
 #include "roc_hash.h"
 
diff --git a/drivers/common/cnxk/roc_cpt.h b/drivers/common/cnxk/roc_cpt.h
index 12e6b81..99cb8b2 100644
--- a/drivers/common/cnxk/roc_cpt.h
+++ b/drivers/common/cnxk/roc_cpt.h
@@ -49,18 +49,18 @@
 #define ROC_CPT_AES_CBC_IV_LEN	 16
 #define ROC_CPT_SHA1_HMAC_LEN	 12
 #define ROC_CPT_SHA2_HMAC_LEN	 16
-#define ROC_CPT_AUTH_KEY_LEN_MAX 64
-
-#define ROC_CPT_DES3_KEY_LEN	  24
-#define ROC_CPT_AES128_KEY_LEN	  16
-#define ROC_CPT_AES192_KEY_LEN	  24
-#define ROC_CPT_AES256_KEY_LEN	  32
-#define ROC_CPT_MD5_KEY_LENGTH	  16
-#define ROC_CPT_SHA1_KEY_LENGTH	  20
-#define ROC_CPT_SHA256_KEY_LENGTH 32
-#define ROC_CPT_SHA384_KEY_LENGTH 48
-#define ROC_CPT_SHA512_KEY_LENGTH 64
-#define ROC_CPT_AUTH_KEY_LEN_MAX  64
+
+#define ROC_CPT_DES3_KEY_LEN	    24
+#define ROC_CPT_AES128_KEY_LEN	    16
+#define ROC_CPT_AES192_KEY_LEN	    24
+#define ROC_CPT_AES256_KEY_LEN	    32
+#define ROC_CPT_MD5_KEY_LENGTH	    16
+#define ROC_CPT_SHA1_KEY_LENGTH	    20
+#define ROC_CPT_SHA256_KEY_LENGTH   32
+#define ROC_CPT_SHA384_KEY_LENGTH   48
+#define ROC_CPT_SHA512_KEY_LENGTH   64
+#define ROC_CPT_AES_XCBC_KEY_LENGTH 16
+#define ROC_CPT_AUTH_KEY_LEN_MAX    64
 
 #define ROC_CPT_DES_BLOCK_LENGTH 8
 #define ROC_CPT_AES_BLOCK_LENGTH 16
diff --git a/drivers/common/cnxk/version.map b/drivers/common/cnxk/version.map
index 07c6720..b31e8eb 100644
--- a/drivers/common/cnxk/version.map
+++ b/drivers/common/cnxk/version.map
@@ -26,6 +26,7 @@ INTERNAL {
 	roc_ae_ec_grp_put;
 	roc_ae_fpm_get;
 	roc_ae_fpm_put;
+	roc_aes_xcbc_key_derive;
 	roc_bphy_cgx_dev_fini;
 	roc_bphy_cgx_dev_init;
 	roc_bphy_cgx_fec_set;
-- 
2.7.4

[PATCH v3 03/29] common/cnxk: add bit fields for params

[Anoob Joseph]

From: Archana Muniganti <marchana@marvell.com>

Added new structure with bit fields for params.


Signed-off-by: Archana Muniganti <marchana@marvell.com>
---
 drivers/common/cnxk/roc_ie_on.h  | 30 +++++++++++++++++++++++++++++-
 drivers/crypto/cnxk/cn9k_ipsec.c | 16 +++++++++++++---
 2 files changed, 42 insertions(+), 4 deletions(-)

diff --git a/drivers/common/cnxk/roc_ie_on.h b/drivers/common/cnxk/roc_ie_on.h
index 53591c6..817ef33 100644
--- a/drivers/common/cnxk/roc_ie_on.h
+++ b/drivers/common/cnxk/roc_ie_on.h
@@ -21,7 +21,6 @@ enum roc_ie_on_ucc_ipsec {
 };
 
 /* Helper macros */
-#define ROC_IE_ON_PER_PKT_IV   BIT(11)
 #define ROC_IE_ON_INB_RPTR_HDR 0x8
 
 enum {
@@ -102,6 +101,35 @@ struct roc_ie_on_ip_template {
 	};
 };
 
+union roc_on_ipsec_outb_param1 {
+	uint16_t u16;
+	struct {
+		uint16_t frag_num : 4;
+		uint16_t rsvd_4_6 : 3;
+		uint16_t gre_select : 1;
+		uint16_t dsiv : 1;
+		uint16_t ikev2 : 1;
+		uint16_t min_frag_size : 1;
+		uint16_t per_pkt_iv : 1;
+		uint16_t tfc_pad_enable : 1;
+		uint16_t tfc_dummy_pkt : 1;
+		uint16_t rfc_or_override_mode : 1;
+		uint16_t custom_hdr_or_p99 : 1;
+	} s;
+};
+
+union roc_on_ipsec_inb_param2 {
+	uint16_t u16;
+	struct {
+		uint16_t rsvd_0_10 : 11;
+		uint16_t gre_select : 1;
+		uint16_t ikev2 : 1;
+		uint16_t udp_cksum : 1;
+		uint16_t ctx_addr_sel : 1;
+		uint16_t custom_hdr_or_p99 : 1;
+	} s;
+};
+
 struct roc_ie_on_sa_ctl {
 	uint64_t spi : 32;
 	uint64_t exp_proto_inter_frag : 8;
diff --git a/drivers/crypto/cnxk/cn9k_ipsec.c b/drivers/crypto/cnxk/cn9k_ipsec.c
index a81130b..6455ef9 100644
--- a/drivers/crypto/cnxk/cn9k_ipsec.c
+++ b/drivers/crypto/cnxk/cn9k_ipsec.c
@@ -280,6 +280,7 @@ cn9k_ipsec_outb_sa_create(struct cnxk_cpt_qp *qp,
 	struct rte_crypto_sym_xform *auth_xform = crypto_xform->next;
 	struct roc_ie_on_ip_template *template = NULL;
 	struct roc_cpt *roc_cpt = qp->lf.roc_cpt;
+	union roc_on_ipsec_outb_param1 param1;
 	struct cnxk_cpt_inst_tmpl *inst_tmpl;
 	struct roc_ie_on_outb_sa *out_sa;
 	struct cn9k_sec_session *sess;
@@ -407,8 +408,12 @@ cn9k_ipsec_outb_sa_create(struct cnxk_cpt_qp *qp,
 	w4.u64 = 0;
 	w4.s.opcode_major = ROC_IE_ON_MAJOR_OP_PROCESS_OUTBOUND_IPSEC;
 	w4.s.opcode_minor = ctx_len >> 3;
-	w4.s.param1 = BIT(9);
-	w4.s.param1 |= ROC_IE_ON_PER_PKT_IV;
+
+	param1.u16 = 0;
+	param1.s.ikev2 = 1;
+	param1.s.per_pkt_iv = 1;
+	w4.s.param1 = param1.u16;
+
 	inst_tmpl->w4 = w4.u64;
 
 	w7.u64 = 0;
@@ -428,6 +433,7 @@ cn9k_ipsec_inb_sa_create(struct cnxk_cpt_qp *qp,
 {
 	struct rte_crypto_sym_xform *auth_xform = crypto_xform;
 	struct roc_cpt *roc_cpt = qp->lf.roc_cpt;
+	union roc_on_ipsec_inb_param2 param2;
 	struct cnxk_cpt_inst_tmpl *inst_tmpl;
 	struct roc_ie_on_inb_sa *in_sa;
 	struct cn9k_sec_session *sess;
@@ -478,7 +484,11 @@ cn9k_ipsec_inb_sa_create(struct cnxk_cpt_qp *qp,
 	w4.u64 = 0;
 	w4.s.opcode_major = ROC_IE_ON_MAJOR_OP_PROCESS_INBOUND_IPSEC;
 	w4.s.opcode_minor = ctx_len >> 3;
-	w4.s.param2 = BIT(12);
+
+	param2.u16 = 0;
+	param2.s.ikev2 = 1;
+	w4.s.param2 = param2.u16;
+
 	inst_tmpl->w4 = w4.u64;
 
 	w7.u64 = 0;
-- 
2.7.4

[PATCH v3 04/29] common/cnxk: fix reset of fields

[Anoob Joseph]

Copy DF/DSCP fields would get set based on ipsec_xform in the code
preceding this. Setting it again would cause the options to be reset.

Fixes: 78d03027f2cc ("common/cnxk: add IPsec common code")
Cc: schalla@marvell.com

Signed-off-by: Anoob Joseph <anoobj@marvell.com>
---
 drivers/common/cnxk/cnxk_security.c | 4 ----
 1 file changed, 4 deletions(-)

diff --git a/drivers/common/cnxk/cnxk_security.c b/drivers/common/cnxk/cnxk_security.c
index 30562b4..787138b 100644
--- a/drivers/common/cnxk/cnxk_security.c
+++ b/drivers/common/cnxk/cnxk_security.c
@@ -444,10 +444,6 @@ cnxk_ot_ipsec_outb_sa_fill(struct roc_ot_ipsec_outb_sa *sa,
 		return -EINVAL;
 	}
 
-	/* Default options of DSCP and Flow label/DF */
-	sa->w2.s.dscp_src = ROC_IE_OT_SA_COPY_FROM_SA;
-	sa->w2.s.ipv4_df_src_or_ipv6_flw_lbl_src = ROC_IE_OT_SA_COPY_FROM_SA;
-
 skip_tunnel_info:
 	/* ESN */
 	sa->w0.s.esn_en = !!ipsec_xfrm->options.esn;
-- 
2.7.4

[PATCH v3 05/29] common/cnxk: verify input args

[Anoob Joseph]

Add input arg verification.

Signed-off-by: Anoob Joseph <anoobj@marvell.com>
---
 drivers/common/cnxk/hw/cpt.h  | 2 ++
 drivers/common/cnxk/roc_cpt.c | 4 +++-
 2 files changed, 5 insertions(+), 1 deletion(-)

diff --git a/drivers/common/cnxk/hw/cpt.h b/drivers/common/cnxk/hw/cpt.h
index 919f842..ccc7af4 100644
--- a/drivers/common/cnxk/hw/cpt.h
+++ b/drivers/common/cnxk/hw/cpt.h
@@ -64,6 +64,7 @@ union cpt_lf_ctx_flush {
 	struct {
 		uint64_t cptr : 46;
 		uint64_t inval : 1;
+		uint64_t reserved_47_63 : 17;
 	} s;
 };
 
@@ -71,6 +72,7 @@ union cpt_lf_ctx_reload {
 	uint64_t u;
 	struct {
 		uint64_t cptr : 46;
+		uint64_t reserved_46_63 : 18;
 	} s;
 };
 
diff --git a/drivers/common/cnxk/roc_cpt.c b/drivers/common/cnxk/roc_cpt.c
index 8f8e6d3..1bc7a29 100644
--- a/drivers/common/cnxk/roc_cpt.c
+++ b/drivers/common/cnxk/roc_cpt.c
@@ -681,8 +681,10 @@ roc_cpt_lf_ctx_flush(struct roc_cpt_lf *lf, void *cptr, bool inval)
 {
 	union cpt_lf_ctx_flush reg;
 
-	if (lf == NULL)
+	if (lf == NULL) {
+		plt_err("Could not trigger CTX flush");
 		return -ENOTSUP;
+	}
 
 	reg.u = 0;
 	reg.s.inval = inval;
-- 
2.7.4

[PATCH v3 06/29] common/cnxk: update completion code

[Anoob Joseph]

Update completion code to match v1.19 microcode release.

Signed-off-by: Anoob Joseph <anoobj@marvell.com>
---
 drivers/common/cnxk/roc_ie_ot.h | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/drivers/common/cnxk/roc_ie_ot.h b/drivers/common/cnxk/roc_ie_ot.h
index 5b61902..923656f 100644
--- a/drivers/common/cnxk/roc_ie_ot.h
+++ b/drivers/common/cnxk/roc_ie_ot.h
@@ -43,8 +43,8 @@ enum roc_ie_ot_ucc_ipsec {
 	ROC_IE_OT_UCC_ERR_SA_ESP_BAD_KEYS = 0xc5,
 	ROC_IE_OT_UCC_ERR_SA_AH_BAD_KEYS = 0xc6,
 	ROC_IE_OT_UCC_ERR_SA_BAD_IP = 0xc7,
-	ROC_IE_OT_UCC_ERR_PKT_REPLAY_WINDOW = 0xc8,
-	ROC_IE_OT_UCC_ERR_PKT_IP_FRAG = 0xc9,
+	ROC_IE_OT_UCC_ERR_PKT_IP_FRAG = 0xc8,
+	ROC_IE_OT_UCC_ERR_PKT_REPLAY_WINDOW = 0xc9,
 	ROC_IE_OT_UCC_SUCCESS_SA_SOFTEXP_FIRST = 0xf0,
 	ROC_IE_OT_UCC_SUCCESS_PKT_IP_BADCSUM = 0xf1,
 	ROC_IE_OT_UCC_SUCCESS_SA_SOFTEXP_AGAIN = 0xf2,
-- 
2.7.4

[PATCH v3 07/29] crypto/cnxk: only enable queues that are allocated

[Anoob Joseph]

From: Shijith Thotton <sthotton@marvell.com>

Only enable/disable queue pairs that are allocated during cryptodev
start/stop.

Fixes: 6a95dbc1a291 ("crypto/cnxk: add dev start and dev stop")

Signed-off-by: Shijith Thotton <sthotton@marvell.com>
---
 drivers/crypto/cnxk/cnxk_cryptodev_ops.c | 13 +++++++++++--
 1 file changed, 11 insertions(+), 2 deletions(-)

diff --git a/drivers/crypto/cnxk/cnxk_cryptodev_ops.c b/drivers/crypto/cnxk/cnxk_cryptodev_ops.c
index a2281fb..21ee09f 100644
--- a/drivers/crypto/cnxk/cnxk_cryptodev_ops.c
+++ b/drivers/crypto/cnxk/cnxk_cryptodev_ops.c
@@ -100,8 +100,13 @@ cnxk_cpt_dev_start(struct rte_cryptodev *dev)
 	uint16_t nb_lf = roc_cpt->nb_lf;
 	uint16_t qp_id;
 
-	for (qp_id = 0; qp_id < nb_lf; qp_id++)
+	for (qp_id = 0; qp_id < nb_lf; qp_id++) {
+		/* Application may not setup all queue pair */
+		if (roc_cpt->lf[qp_id] == NULL)
+			continue;
+
 		roc_cpt_iq_enable(roc_cpt->lf[qp_id]);
+	}
 
 	return 0;
 }
@@ -114,8 +119,12 @@ cnxk_cpt_dev_stop(struct rte_cryptodev *dev)
 	uint16_t nb_lf = roc_cpt->nb_lf;
 	uint16_t qp_id;
 
-	for (qp_id = 0; qp_id < nb_lf; qp_id++)
+	for (qp_id = 0; qp_id < nb_lf; qp_id++) {
+		if (roc_cpt->lf[qp_id] == NULL)
+			continue;
+
 		roc_cpt_iq_disable(roc_cpt->lf[qp_id]);
+	}
 }
 
 int
-- 
2.7.4

[PATCH v3 08/29] crypto/cnxk: add lookaside IPsec AES-CBC-HMAC-SHA256 support

[Anoob Joseph]

From: Tejasree Kondoj <ktejasree@marvell.com>

Adding AES-CBC-HMAC-SHA256 support to lookaside IPsec PMD.

Signed-off-by: Tejasree Kondoj <ktejasree@marvell.com>
---
 doc/guides/cryptodevs/cnxk.rst                    | 39 +++++++++++++++++++----
 doc/guides/rel_notes/release_22_03.rst            |  4 +++
 drivers/common/cnxk/cnxk_security.c               | 14 ++++++++
 drivers/crypto/cnxk/cn10k_ipsec.c                 |  3 ++
 drivers/crypto/cnxk/cnxk_cryptodev_capabilities.c | 20 ++++++++++++
 drivers/crypto/cnxk/cnxk_ipsec.h                  |  3 +-
 6 files changed, 75 insertions(+), 8 deletions(-)

diff --git a/doc/guides/cryptodevs/cnxk.rst b/doc/guides/cryptodevs/cnxk.rst
index 23cc823..8c4c4ea 100644
--- a/doc/guides/cryptodevs/cnxk.rst
+++ b/doc/guides/cryptodevs/cnxk.rst
@@ -246,14 +246,27 @@ CN9XX Features supported
 * IPv4
 * IPv6
 * ESP
+* ESN
+* Anti-replay
 * Tunnel mode
 * Transport mode(IPv4)
 * UDP Encapsulation
+
+AEAD algorithms
++++++++++++++++
+
 * AES-128/192/256-GCM
-* AES-128/192/256-CBC-SHA1-HMAC
-* AES-128/192/256-CBC-SHA256-128-HMAC
-* ESN
-* Anti-replay
+
+Cipher algorithms
++++++++++++++++++
+
+* AES-128/192/256-CBC
+
+Auth algorithms
++++++++++++++++
+
+* SHA1-HMAC
+* SHA256-128-HMAC
 
 CN10XX Features supported
 ~~~~~~~~~~~~~~~~~~~~~~~~~
@@ -263,6 +276,20 @@ CN10XX Features supported
 * Tunnel mode
 * Transport mode
 * UDP Encapsulation
+
+AEAD algorithms
++++++++++++++++
+
 * AES-128/192/256-GCM
-* AES-128/192/256-CBC-NULL
-* AES-128/192/256-CBC-SHA1-HMAC
+
+Cipher algorithms
++++++++++++++++++
+
+* AES-128/192/256-CBC
+
+Auth algorithms
++++++++++++++++
+
+* NULL
+* SHA1-HMAC
+* SHA256-128-HMAC
diff --git a/doc/guides/rel_notes/release_22_03.rst b/doc/guides/rel_notes/release_22_03.rst
index 6d99d1e..1639b0e 100644
--- a/doc/guides/rel_notes/release_22_03.rst
+++ b/doc/guides/rel_notes/release_22_03.rst
@@ -55,6 +55,10 @@ New Features
      Also, make sure to start the actual text at the margin.
      =======================================================
 
+* **Updated Marvell cnxk crypto PMD.**
+
+  * Added SHA256-HMAC support in lookaside protocol (IPsec) for CN10K.
+
 
 Removed Items
 -------------
diff --git a/drivers/common/cnxk/cnxk_security.c b/drivers/common/cnxk/cnxk_security.c
index 787138b..f39bc1e 100644
--- a/drivers/common/cnxk/cnxk_security.c
+++ b/drivers/common/cnxk/cnxk_security.c
@@ -32,6 +32,10 @@ ipsec_hmac_opad_ipad_gen(struct rte_crypto_sym_xform *auth_xform,
 		roc_hash_sha1_gen(opad, (uint32_t *)&hmac_opad_ipad[0]);
 		roc_hash_sha1_gen(ipad, (uint32_t *)&hmac_opad_ipad[24]);
 		break;
+	case RTE_CRYPTO_AUTH_SHA256_HMAC:
+		roc_hash_sha256_gen(opad, (uint32_t *)&hmac_opad_ipad[0]);
+		roc_hash_sha256_gen(ipad, (uint32_t *)&hmac_opad_ipad[64]);
+		break;
 	default:
 		break;
 	}
@@ -129,6 +133,16 @@ ot_ipsec_sa_common_param_fill(union roc_ot_ipsec_sa_word2 *w2,
 			     i++)
 				tmp_key[i] = rte_be_to_cpu_64(tmp_key[i]);
 			break;
+		case RTE_CRYPTO_AUTH_SHA256_HMAC:
+			w2->s.auth_type = ROC_IE_OT_SA_AUTH_SHA2_256;
+			ipsec_hmac_opad_ipad_gen(auth_xfrm, hmac_opad_ipad);
+
+			tmp_key = (uint64_t *)hmac_opad_ipad;
+			for (i = 0; i < (int)(ROC_CTX_MAX_OPAD_IPAD_LEN /
+					      sizeof(uint64_t));
+			     i++)
+				tmp_key[i] = rte_be_to_cpu_64(tmp_key[i]);
+			break;
 		default:
 			return -ENOTSUP;
 		}
diff --git a/drivers/crypto/cnxk/cn10k_ipsec.c b/drivers/crypto/cnxk/cn10k_ipsec.c
index 27df1dc..93eab1b 100644
--- a/drivers/crypto/cnxk/cn10k_ipsec.c
+++ b/drivers/crypto/cnxk/cn10k_ipsec.c
@@ -65,6 +65,9 @@ cn10k_ipsec_outb_sa_create(struct roc_cpt *roc_cpt,
 		if (crypto_xfrm->type == RTE_CRYPTO_SYM_XFORM_AEAD) {
 			sa->iv_offset = crypto_xfrm->aead.iv.offset;
 			sa->iv_length = crypto_xfrm->aead.iv.length;
+		} else {
+			sa->iv_offset = crypto_xfrm->cipher.iv.offset;
+			sa->iv_length = crypto_xfrm->cipher.iv.length;
 		}
 	}
 #else
diff --git a/drivers/crypto/cnxk/cnxk_cryptodev_capabilities.c b/drivers/crypto/cnxk/cnxk_cryptodev_capabilities.c
index 59b63ed..7d22626 100644
--- a/drivers/crypto/cnxk/cnxk_cryptodev_capabilities.c
+++ b/drivers/crypto/cnxk/cnxk_cryptodev_capabilities.c
@@ -797,6 +797,26 @@ static const struct rte_cryptodev_capabilities sec_caps_sha1_sha2[] = {
 			}, }
 		}, }
 	},
+	{	/* SHA256 HMAC */
+		.op = RTE_CRYPTO_OP_TYPE_SYMMETRIC,
+		{.sym = {
+			.xform_type = RTE_CRYPTO_SYM_XFORM_AUTH,
+			{.auth = {
+				.algo = RTE_CRYPTO_AUTH_SHA256_HMAC,
+				.block_size = 64,
+				.key_size = {
+					.min = 1,
+					.max = 1024,
+					.increment = 1
+				},
+				.digest_size = {
+					.min = 16,
+					.max = 16,
+					.increment = 0
+				},
+			}, }
+		}, }
+	},
 };
 
 static const struct rte_security_capability sec_caps_templ[] = {
diff --git a/drivers/crypto/cnxk/cnxk_ipsec.h b/drivers/crypto/cnxk/cnxk_ipsec.h
index dddb414..f4a1012 100644
--- a/drivers/crypto/cnxk/cnxk_ipsec.h
+++ b/drivers/crypto/cnxk/cnxk_ipsec.h
@@ -46,8 +46,7 @@ ipsec_xform_auth_verify(struct rte_crypto_sym_xform *crypto_xform)
 	if (crypto_xform->auth.algo == RTE_CRYPTO_AUTH_SHA1_HMAC) {
 		if (keylen >= 20 && keylen <= 64)
 			return 0;
-	} else if (roc_model_is_cn9k() &&
-		   (crypto_xform->auth.algo == RTE_CRYPTO_AUTH_SHA256_HMAC)) {
+	} else if (crypto_xform->auth.algo == RTE_CRYPTO_AUTH_SHA256_HMAC) {
 		if (keylen >= 32 && keylen <= 64)
 			return 0;
 	}
-- 
2.7.4

[PATCH v3 09/29] crypto/cnxk: clear session data before populating

[Anoob Joseph]

Clear session data before populating fields to not have garbage data.

Signed-off-by: Anoob Joseph <anoobj@marvell.com>
---
 drivers/crypto/cnxk/cn10k_ipsec.c | 2 ++
 1 file changed, 2 insertions(+)

diff --git a/drivers/crypto/cnxk/cn10k_ipsec.c b/drivers/crypto/cnxk/cn10k_ipsec.c
index 93eab1b..1bd127e 100644
--- a/drivers/crypto/cnxk/cn10k_ipsec.c
+++ b/drivers/crypto/cnxk/cn10k_ipsec.c
@@ -130,6 +130,8 @@ cn10k_ipsec_inb_sa_create(struct roc_cpt *roc_cpt,
 	sa = &sess->sa;
 	in_sa = &sa->in_sa;
 
+	memset(in_sa, 0, sizeof(struct roc_ot_ipsec_inb_sa));
+
 	/* Translate security parameters to SA */
 	ret = cnxk_ot_ipsec_inb_sa_fill(in_sa, ipsec_xfrm, crypto_xfrm);
 	if (ret)
-- 
2.7.4

[PATCH v3 10/29] crypto/cnxk: update max sec crypto caps

[Anoob Joseph]

Update the macro to include newly added ciphers. Updated the functions
populating caps to throw error when max is exceeded.

Signed-off-by: Anoob Joseph <anoobj@marvell.com>
---
 drivers/crypto/cnxk/cnxk_cryptodev.h              | 2 +-
 drivers/crypto/cnxk/cnxk_cryptodev_capabilities.c | 8 ++++++--
 2 files changed, 7 insertions(+), 3 deletions(-)

diff --git a/drivers/crypto/cnxk/cnxk_cryptodev.h b/drivers/crypto/cnxk/cnxk_cryptodev.h
index cfb9d29..2e0f467 100644
--- a/drivers/crypto/cnxk/cnxk_cryptodev.h
+++ b/drivers/crypto/cnxk/cnxk_cryptodev.h
@@ -11,7 +11,7 @@
 #include "roc_cpt.h"
 
 #define CNXK_CPT_MAX_CAPS	 34
-#define CNXK_SEC_CRYPTO_MAX_CAPS 4
+#define CNXK_SEC_CRYPTO_MAX_CAPS 6
 #define CNXK_SEC_MAX_CAPS	 5
 #define CNXK_AE_EC_ID_MAX	 8
 /**
diff --git a/drivers/crypto/cnxk/cnxk_cryptodev_capabilities.c b/drivers/crypto/cnxk/cnxk_cryptodev_capabilities.c
index 7d22626..8305341 100644
--- a/drivers/crypto/cnxk/cnxk_cryptodev_capabilities.c
+++ b/drivers/crypto/cnxk/cnxk_cryptodev_capabilities.c
@@ -943,8 +943,10 @@ static void
 sec_caps_add(struct rte_cryptodev_capabilities cnxk_caps[], int *cur_pos,
 	     const struct rte_cryptodev_capabilities *caps, int nb_caps)
 {
-	if (*cur_pos + nb_caps > CNXK_SEC_CRYPTO_MAX_CAPS)
+	if (*cur_pos + nb_caps > CNXK_SEC_CRYPTO_MAX_CAPS) {
+		rte_panic("Could not add sec crypto caps");
 		return;
+	}
 
 	memcpy(&cnxk_caps[*cur_pos], caps, nb_caps * sizeof(caps[0]));
 	*cur_pos += nb_caps;
@@ -957,8 +959,10 @@ cn10k_sec_crypto_caps_update(struct rte_cryptodev_capabilities cnxk_caps[],
 	const struct rte_cryptodev_capabilities *cap;
 	unsigned int i;
 
-	if ((CNXK_CPT_MAX_CAPS - *cur_pos) < 1)
+	if ((CNXK_SEC_CRYPTO_MAX_CAPS - *cur_pos) < 1) {
+		rte_panic("Could not add sec crypto caps");
 		return;
+	}
 
 	/* NULL auth */
 	for (i = 0; i < RTE_DIM(caps_null); i++) {
-- 
2.7.4

[PATCH v3 11/29] crypto/cnxk: write CPT CTX through microcode op

[Anoob Joseph]

From: Tejasree Kondoj <ktejasree@marvell.com>

Adding support to write CPT CTX through microcode op(SET_CTX) for
cn10k lookaside PMD.

Signed-off-by: Tejasree Kondoj <ktejasree@marvell.com>
---
 drivers/crypto/cnxk/cn10k_ipsec.c | 121 ++++++++++++++++++++++++++++----------
 1 file changed, 89 insertions(+), 32 deletions(-)

diff --git a/drivers/crypto/cnxk/cn10k_ipsec.c b/drivers/crypto/cnxk/cn10k_ipsec.c
index 1bd127e..a11a6b7 100644
--- a/drivers/crypto/cnxk/cn10k_ipsec.c
+++ b/drivers/crypto/cnxk/cn10k_ipsec.c
@@ -2,18 +2,19 @@
  * Copyright(C) 2021 Marvell.
  */
 
-#include <rte_malloc.h>
 #include <cryptodev_pmd.h>
 #include <rte_esp.h>
 #include <rte_ip.h>
+#include <rte_malloc.h>
 #include <rte_security.h>
 #include <rte_security_driver.h>
 #include <rte_udp.h>
 
+#include "cn10k_ipsec.h"
 #include "cnxk_cryptodev.h"
+#include "cnxk_cryptodev_ops.h"
 #include "cnxk_ipsec.h"
 #include "cnxk_security.h"
-#include "cn10k_ipsec.h"
 
 #include "roc_api.h"
 
@@ -32,36 +33,46 @@ ipsec_cpt_inst_w7_get(struct roc_cpt *roc_cpt, void *sa)
 }
 
 static int
-cn10k_ipsec_outb_sa_create(struct roc_cpt *roc_cpt,
+cn10k_ipsec_outb_sa_create(struct roc_cpt *roc_cpt, struct roc_cpt_lf *lf,
 			   struct rte_security_ipsec_xform *ipsec_xfrm,
 			   struct rte_crypto_sym_xform *crypto_xfrm,
 			   struct rte_security_session *sec_sess)
 {
 	union roc_ot_ipsec_outb_param1 param1;
-	struct roc_ot_ipsec_outb_sa *out_sa;
+	struct roc_ot_ipsec_outb_sa *sa_dptr;
 	struct cnxk_ipsec_outb_rlens rlens;
 	struct cn10k_sec_session *sess;
 	struct cn10k_ipsec_sa *sa;
 	union cpt_inst_w4 inst_w4;
-	int ret;
+	void *out_sa;
+	int ret = 0;
 
 	sess = get_sec_session_private_data(sec_sess);
 	sa = &sess->sa;
 	out_sa = &sa->out_sa;
 
-	memset(out_sa, 0, sizeof(struct roc_ot_ipsec_outb_sa));
+	/* Allocate memory to be used as dptr for CPT ucode WRITE_SA op */
+	sa_dptr = plt_zmalloc(ROC_NIX_INL_OT_IPSEC_OUTB_HW_SZ, 0);
+	if (sa_dptr == NULL) {
+		plt_err("Couldn't allocate memory for SA dptr");
+		return -ENOMEM;
+	}
+
+	memset(sa_dptr, 0, sizeof(struct roc_ot_ipsec_outb_sa));
 
 	/* Translate security parameters to SA */
-	ret = cnxk_ot_ipsec_outb_sa_fill(out_sa, ipsec_xfrm, crypto_xfrm);
-	if (ret)
-		return ret;
+	ret = cnxk_ot_ipsec_outb_sa_fill(sa_dptr, ipsec_xfrm, crypto_xfrm);
+	if (ret) {
+		plt_err("Could not fill outbound session parameters");
+		goto sa_dptr_free;
+	}
 
 	sa->inst.w7 = ipsec_cpt_inst_w7_get(roc_cpt, sa);
 
 #ifdef LA_IPSEC_DEBUG
 	/* Use IV from application in debug mode */
 	if (ipsec_xfrm->options.iv_gen_disable == 1) {
-		out_sa->w2.s.iv_src = ROC_IE_OT_SA_IV_SRC_FROM_SA;
+		sa_dptr->w2.s.iv_src = ROC_IE_OT_SA_IV_SRC_FROM_SA;
 		if (crypto_xfrm->type == RTE_CRYPTO_SYM_XFORM_AEAD) {
 			sa->iv_offset = crypto_xfrm->aead.iv.offset;
 			sa->iv_length = crypto_xfrm->aead.iv.length;
@@ -73,14 +84,15 @@ cn10k_ipsec_outb_sa_create(struct roc_cpt *roc_cpt,
 #else
 	if (ipsec_xfrm->options.iv_gen_disable != 0) {
 		plt_err("Application provided IV not supported");
-		return -ENOTSUP;
+		ret = -ENOTSUP;
+		goto sa_dptr_free;
 	}
 #endif
 
 	/* Get Rlen calculation data */
 	ret = cnxk_ipsec_outb_rlens_get(&rlens, ipsec_xfrm, crypto_xfrm);
 	if (ret)
-		return ret;
+		goto sa_dptr_free;
 
 	sa->max_extended_len = rlens.max_extended_len;
 
@@ -110,37 +122,61 @@ cn10k_ipsec_outb_sa_create(struct roc_cpt *roc_cpt,
 
 	sa->inst.w4 = inst_w4.u64;
 
-	return 0;
+	memset(out_sa, 0, sizeof(struct roc_ot_ipsec_outb_sa));
+
+	/* Copy word0 from sa_dptr to populate ctx_push_sz ctx_size fields */
+	memcpy(out_sa, sa_dptr, 8);
+
+	/* Write session using microcode opcode */
+	ret = roc_cpt_ctx_write(lf, sa_dptr, out_sa,
+				ROC_NIX_INL_OT_IPSEC_OUTB_HW_SZ);
+	if (ret) {
+		plt_err("Could not write outbound session to hardware");
+		goto sa_dptr_free;
+	}
+
+	/* Trigger CTX flush to write dirty data back to DRAM */
+	roc_cpt_lf_ctx_flush(lf, out_sa, false);
+
+sa_dptr_free:
+	plt_free(sa_dptr);
+
+	return ret;
 }
 
 static int
-cn10k_ipsec_inb_sa_create(struct roc_cpt *roc_cpt,
+cn10k_ipsec_inb_sa_create(struct roc_cpt *roc_cpt, struct roc_cpt_lf *lf,
 			  struct rte_security_ipsec_xform *ipsec_xfrm,
 			  struct rte_crypto_sym_xform *crypto_xfrm,
 			  struct rte_security_session *sec_sess)
 {
 	union roc_ot_ipsec_inb_param1 param1;
-	struct roc_ot_ipsec_inb_sa *in_sa;
+	struct roc_ot_ipsec_inb_sa *sa_dptr;
 	struct cn10k_sec_session *sess;
 	struct cn10k_ipsec_sa *sa;
 	union cpt_inst_w4 inst_w4;
-	int ret;
+	void *in_sa;
+	int ret = 0;
 
 	sess = get_sec_session_private_data(sec_sess);
 	sa = &sess->sa;
 	in_sa = &sa->in_sa;
 
-	memset(in_sa, 0, sizeof(struct roc_ot_ipsec_inb_sa));
-
-	/* Translate security parameters to SA */
-	ret = cnxk_ot_ipsec_inb_sa_fill(in_sa, ipsec_xfrm, crypto_xfrm);
-	if (ret)
-		return ret;
+	/* Allocate memory to be used as dptr for CPT ucode WRITE_SA op */
+	sa_dptr = plt_zmalloc(ROC_NIX_INL_OT_IPSEC_INB_HW_SZ, 0);
+	if (sa_dptr == NULL) {
+		plt_err("Couldn't allocate memory for SA dptr");
+		return -ENOMEM;
+	}
 
-	/* TODO add support for antireplay */
-	sa->in_sa.w0.s.ar_win = 0;
+	memset(sa_dptr, 0, sizeof(struct roc_ot_ipsec_inb_sa));
 
-	/* TODO add support for udp encap */
+	/* Translate security parameters to SA */
+	ret = cnxk_ot_ipsec_inb_sa_fill(sa_dptr, ipsec_xfrm, crypto_xfrm);
+	if (ret) {
+		plt_err("Could not fill inbound session parameters");
+		goto sa_dptr_free;
+	}
 
 	sa->inst.w7 = ipsec_cpt_inst_w7_get(roc_cpt, sa);
 
@@ -173,7 +209,26 @@ cn10k_ipsec_inb_sa_create(struct roc_cpt *roc_cpt,
 
 	sa->inst.w4 = inst_w4.u64;
 
-	return 0;
+	memset(in_sa, 0, sizeof(struct roc_ot_ipsec_inb_sa));
+
+	/* Copy word0 from sa_dptr to populate ctx_push_sz ctx_size fields */
+	memcpy(in_sa, sa_dptr, 8);
+
+	/* Write session using microcode opcode */
+	ret = roc_cpt_ctx_write(lf, sa_dptr, in_sa,
+				ROC_NIX_INL_OT_IPSEC_INB_HW_SZ);
+	if (ret) {
+		plt_err("Could not write inbound session to hardware");
+		goto sa_dptr_free;
+	}
+
+	/* Trigger CTX flush to write dirty data back to DRAM */
+	roc_cpt_lf_ctx_flush(lf, in_sa, false);
+
+sa_dptr_free:
+	plt_free(sa_dptr);
+
+	return ret;
 }
 
 static int
@@ -185,12 +240,11 @@ cn10k_ipsec_session_create(void *dev,
 	struct rte_cryptodev *crypto_dev = dev;
 	struct roc_cpt *roc_cpt;
 	struct cnxk_cpt_vf *vf;
+	struct cnxk_cpt_qp *qp;
 	int ret;
 
-	vf = crypto_dev->data->dev_private;
-	roc_cpt = &vf->cpt;
-
-	if (crypto_dev->data->queue_pairs[0] == NULL) {
+	qp = crypto_dev->data->queue_pairs[0];
+	if (qp == NULL) {
 		plt_err("Setup cpt queue pair before creating security session");
 		return -EPERM;
 	}
@@ -199,11 +253,14 @@ cn10k_ipsec_session_create(void *dev,
 	if (ret)
 		return ret;
 
+	vf = crypto_dev->data->dev_private;
+	roc_cpt = &vf->cpt;
+
 	if (ipsec_xfrm->direction == RTE_SECURITY_IPSEC_SA_DIR_INGRESS)
-		return cn10k_ipsec_inb_sa_create(roc_cpt, ipsec_xfrm,
+		return cn10k_ipsec_inb_sa_create(roc_cpt, &qp->lf, ipsec_xfrm,
 						 crypto_xfrm, sess);
 	else
-		return cn10k_ipsec_outb_sa_create(roc_cpt, ipsec_xfrm,
+		return cn10k_ipsec_outb_sa_create(roc_cpt, &qp->lf, ipsec_xfrm,
 						  crypto_xfrm, sess);
 }
 
-- 
2.7.4

[PATCH v3 12/29] crypto/cnxk: support cnxk lookaside IPsec HMAC-SHA384/512

[Anoob Joseph]

From: Tejasree Kondoj <ktejasree@marvell.com>

Adding HMAC-SHA384/512 support to cnxk lookaside IPsec.

Signed-off-by: Tejasree Kondoj <ktejasree@marvell.com>
---
 doc/guides/cryptodevs/cnxk.rst                    |  4 ++
 doc/guides/rel_notes/release_22_03.rst            |  2 +
 drivers/common/cnxk/cnxk_security.c               | 36 +++++++++------
 drivers/crypto/cnxk/cn9k_ipsec.c                  | 55 ++++++++++++++++++-----
 drivers/crypto/cnxk/cnxk_cryptodev.h              |  2 +-
 drivers/crypto/cnxk/cnxk_cryptodev_capabilities.c | 40 +++++++++++++++++
 drivers/crypto/cnxk/cnxk_ipsec.h                  |  6 +++
 7 files changed, 118 insertions(+), 27 deletions(-)

diff --git a/doc/guides/cryptodevs/cnxk.rst b/doc/guides/cryptodevs/cnxk.rst
index 8c4c4ea..c49a779 100644
--- a/doc/guides/cryptodevs/cnxk.rst
+++ b/doc/guides/cryptodevs/cnxk.rst
@@ -267,6 +267,8 @@ Auth algorithms
 
 * SHA1-HMAC
 * SHA256-128-HMAC
+* SHA384-192-HMAC
+* SHA512-256-HMAC
 
 CN10XX Features supported
 ~~~~~~~~~~~~~~~~~~~~~~~~~
@@ -293,3 +295,5 @@ Auth algorithms
 * NULL
 * SHA1-HMAC
 * SHA256-128-HMAC
+* SHA384-192-HMAC
+* SHA512-256-HMAC
diff --git a/doc/guides/rel_notes/release_22_03.rst b/doc/guides/rel_notes/release_22_03.rst
index 1639b0e..8df9092 100644
--- a/doc/guides/rel_notes/release_22_03.rst
+++ b/doc/guides/rel_notes/release_22_03.rst
@@ -58,6 +58,8 @@ New Features
 * **Updated Marvell cnxk crypto PMD.**
 
   * Added SHA256-HMAC support in lookaside protocol (IPsec) for CN10K.
+  * Added SHA384-HMAC support in lookaside protocol (IPsec) for CN9K & CN10K.
+  * Added SHA512-HMAC support in lookaside protocol (IPsec) for CN9K & CN10K.
 
 
 Removed Items
diff --git a/drivers/common/cnxk/cnxk_security.c b/drivers/common/cnxk/cnxk_security.c
index f39bc1e..1c86f82 100644
--- a/drivers/common/cnxk/cnxk_security.c
+++ b/drivers/common/cnxk/cnxk_security.c
@@ -36,6 +36,14 @@ ipsec_hmac_opad_ipad_gen(struct rte_crypto_sym_xform *auth_xform,
 		roc_hash_sha256_gen(opad, (uint32_t *)&hmac_opad_ipad[0]);
 		roc_hash_sha256_gen(ipad, (uint32_t *)&hmac_opad_ipad[64]);
 		break;
+	case RTE_CRYPTO_AUTH_SHA384_HMAC:
+		roc_hash_sha512_gen(opad, (uint64_t *)&hmac_opad_ipad[0], 384);
+		roc_hash_sha512_gen(ipad, (uint64_t *)&hmac_opad_ipad[64], 384);
+		break;
+	case RTE_CRYPTO_AUTH_SHA512_HMAC:
+		roc_hash_sha512_gen(opad, (uint64_t *)&hmac_opad_ipad[0], 512);
+		roc_hash_sha512_gen(ipad, (uint64_t *)&hmac_opad_ipad[64], 512);
+		break;
 	default:
 		break;
 	}
@@ -125,28 +133,28 @@ ot_ipsec_sa_common_param_fill(union roc_ot_ipsec_sa_word2 *w2,
 			break;
 		case RTE_CRYPTO_AUTH_SHA1_HMAC:
 			w2->s.auth_type = ROC_IE_OT_SA_AUTH_SHA1;
-			ipsec_hmac_opad_ipad_gen(auth_xfrm, hmac_opad_ipad);
-
-			tmp_key = (uint64_t *)hmac_opad_ipad;
-			for (i = 0; i < (int)(ROC_CTX_MAX_OPAD_IPAD_LEN /
-					      sizeof(uint64_t));
-			     i++)
-				tmp_key[i] = rte_be_to_cpu_64(tmp_key[i]);
 			break;
 		case RTE_CRYPTO_AUTH_SHA256_HMAC:
 			w2->s.auth_type = ROC_IE_OT_SA_AUTH_SHA2_256;
-			ipsec_hmac_opad_ipad_gen(auth_xfrm, hmac_opad_ipad);
-
-			tmp_key = (uint64_t *)hmac_opad_ipad;
-			for (i = 0; i < (int)(ROC_CTX_MAX_OPAD_IPAD_LEN /
-					      sizeof(uint64_t));
-			     i++)
-				tmp_key[i] = rte_be_to_cpu_64(tmp_key[i]);
+			break;
+		case RTE_CRYPTO_AUTH_SHA384_HMAC:
+			w2->s.auth_type = ROC_IE_OT_SA_AUTH_SHA2_384;
+			break;
+		case RTE_CRYPTO_AUTH_SHA512_HMAC:
+			w2->s.auth_type = ROC_IE_OT_SA_AUTH_SHA2_512;
 			break;
 		default:
 			return -ENOTSUP;
 		}
 
+		ipsec_hmac_opad_ipad_gen(auth_xfrm, hmac_opad_ipad);
+
+		tmp_key = (uint64_t *)hmac_opad_ipad;
+		for (i = 0;
+		     i < (int)(ROC_CTX_MAX_OPAD_IPAD_LEN / sizeof(uint64_t));
+		     i++)
+			tmp_key[i] = rte_be_to_cpu_64(tmp_key[i]);
+
 		key = cipher_xfrm->cipher.key.data;
 		length = cipher_xfrm->cipher.key.length;
 	}
diff --git a/drivers/crypto/cnxk/cn9k_ipsec.c b/drivers/crypto/cnxk/cn9k_ipsec.c
index 6455ef9..395b0d5 100644
--- a/drivers/crypto/cnxk/cn9k_ipsec.c
+++ b/drivers/crypto/cnxk/cn9k_ipsec.c
@@ -321,14 +321,23 @@ cn9k_ipsec_outb_sa_create(struct cnxk_cpt_qp *qp,
 	    ctl->auth_type == ROC_IE_ON_SA_AUTH_NULL) {
 		template = &out_sa->aes_gcm.template;
 		ctx_len = offsetof(struct roc_ie_on_outb_sa, aes_gcm.template);
-	} else if (ctl->auth_type == ROC_IE_ON_SA_AUTH_SHA1) {
-		template = &out_sa->sha1.template;
-		ctx_len = offsetof(struct roc_ie_on_outb_sa, sha1.template);
-	} else if (ctl->auth_type == ROC_IE_ON_SA_AUTH_SHA2_256) {
-		template = &out_sa->sha2.template;
-		ctx_len = offsetof(struct roc_ie_on_outb_sa, sha2.template);
 	} else {
-		return -EINVAL;
+		switch (ctl->auth_type) {
+		case ROC_IE_ON_SA_AUTH_SHA1:
+			template = &out_sa->sha1.template;
+			ctx_len = offsetof(struct roc_ie_on_outb_sa,
+					   sha1.template);
+			break;
+		case ROC_IE_ON_SA_AUTH_SHA2_256:
+		case ROC_IE_ON_SA_AUTH_SHA2_384:
+		case ROC_IE_ON_SA_AUTH_SHA2_512:
+			template = &out_sa->sha2.template;
+			ctx_len = offsetof(struct roc_ie_on_outb_sa,
+					   sha2.template);
+			break;
+		default:
+			return -EINVAL;
+		}
 	}
 
 	ip4 = (struct rte_ipv4_hdr *)&template->ip4.ipv4_hdr;
@@ -397,10 +406,22 @@ cn9k_ipsec_outb_sa_create(struct cnxk_cpt_qp *qp,
 		auth_key = auth_xform->auth.key.data;
 		auth_key_len = auth_xform->auth.key.length;
 
-		if (auth_xform->auth.algo == RTE_CRYPTO_AUTH_SHA1_HMAC)
+		switch (auth_xform->auth.algo) {
+		case RTE_CRYPTO_AUTH_NULL:
+			break;
+		case RTE_CRYPTO_AUTH_SHA1_HMAC:
 			memcpy(out_sa->sha1.hmac_key, auth_key, auth_key_len);
-		else if (auth_xform->auth.algo == RTE_CRYPTO_AUTH_SHA256_HMAC)
+			break;
+		case RTE_CRYPTO_AUTH_SHA256_HMAC:
+		case RTE_CRYPTO_AUTH_SHA384_HMAC:
+		case RTE_CRYPTO_AUTH_SHA512_HMAC:
 			memcpy(out_sa->sha2.hmac_key, auth_key, auth_key_len);
+			break;
+		default:
+			plt_err("Unsupported auth algorithm %u",
+				auth_xform->auth.algo);
+			return -ENOTSUP;
+		}
 	}
 
 	inst_tmpl = &sa->inst;
@@ -466,16 +487,26 @@ cn9k_ipsec_inb_sa_create(struct cnxk_cpt_qp *qp,
 		auth_key = auth_xform->auth.key.data;
 		auth_key_len = auth_xform->auth.key.length;
 
-		if (auth_xform->auth.algo == RTE_CRYPTO_AUTH_SHA1_HMAC) {
+		switch (auth_xform->auth.algo) {
+		case RTE_CRYPTO_AUTH_NULL:
+			break;
+		case RTE_CRYPTO_AUTH_SHA1_HMAC:
 			memcpy(in_sa->sha1_or_gcm.hmac_key, auth_key,
 			       auth_key_len);
 			ctx_len = offsetof(struct roc_ie_on_inb_sa,
 					   sha1_or_gcm.selector);
-		} else if (auth_xform->auth.algo ==
-			   RTE_CRYPTO_AUTH_SHA256_HMAC) {
+			break;
+		case RTE_CRYPTO_AUTH_SHA256_HMAC:
+		case RTE_CRYPTO_AUTH_SHA384_HMAC:
+		case RTE_CRYPTO_AUTH_SHA512_HMAC:
 			memcpy(in_sa->sha2.hmac_key, auth_key, auth_key_len);
 			ctx_len = offsetof(struct roc_ie_on_inb_sa,
 					   sha2.selector);
+			break;
+		default:
+			plt_err("Unsupported auth algorithm %u",
+				auth_xform->auth.algo);
+			return -ENOTSUP;
 		}
 	}
 
diff --git a/drivers/crypto/cnxk/cnxk_cryptodev.h b/drivers/crypto/cnxk/cnxk_cryptodev.h
index 2e0f467..f701c26 100644
--- a/drivers/crypto/cnxk/cnxk_cryptodev.h
+++ b/drivers/crypto/cnxk/cnxk_cryptodev.h
@@ -11,7 +11,7 @@
 #include "roc_cpt.h"
 
 #define CNXK_CPT_MAX_CAPS	 34
-#define CNXK_SEC_CRYPTO_MAX_CAPS 6
+#define CNXK_SEC_CRYPTO_MAX_CAPS 8
 #define CNXK_SEC_MAX_CAPS	 5
 #define CNXK_AE_EC_ID_MAX	 8
 /**
diff --git a/drivers/crypto/cnxk/cnxk_cryptodev_capabilities.c b/drivers/crypto/cnxk/cnxk_cryptodev_capabilities.c
index 8305341..9a55474 100644
--- a/drivers/crypto/cnxk/cnxk_cryptodev_capabilities.c
+++ b/drivers/crypto/cnxk/cnxk_cryptodev_capabilities.c
@@ -817,6 +817,46 @@ static const struct rte_cryptodev_capabilities sec_caps_sha1_sha2[] = {
 			}, }
 		}, }
 	},
+	{	/* SHA384 HMAC */
+		.op = RTE_CRYPTO_OP_TYPE_SYMMETRIC,
+		{.sym = {
+			.xform_type = RTE_CRYPTO_SYM_XFORM_AUTH,
+			{.auth = {
+				.algo = RTE_CRYPTO_AUTH_SHA384_HMAC,
+				.block_size = 64,
+				.key_size = {
+					.min = 48,
+					.max = 48,
+					.increment = 0
+				},
+				.digest_size = {
+					.min = 24,
+					.max = 24,
+					.increment = 0
+					},
+			}, }
+		}, }
+	},
+	{	/* SHA512 HMAC */
+		.op = RTE_CRYPTO_OP_TYPE_SYMMETRIC,
+		{.sym = {
+			.xform_type = RTE_CRYPTO_SYM_XFORM_AUTH,
+			{.auth = {
+				.algo = RTE_CRYPTO_AUTH_SHA512_HMAC,
+				.block_size = 128,
+				.key_size = {
+					.min = 64,
+					.max = 64,
+					.increment = 0
+				},
+				.digest_size = {
+					.min = 32,
+					.max = 32,
+					.increment = 0
+				},
+			}, }
+		}, }
+	},
 };
 
 static const struct rte_security_capability sec_caps_templ[] = {
diff --git a/drivers/crypto/cnxk/cnxk_ipsec.h b/drivers/crypto/cnxk/cnxk_ipsec.h
index f4a1012..426eaa8 100644
--- a/drivers/crypto/cnxk/cnxk_ipsec.h
+++ b/drivers/crypto/cnxk/cnxk_ipsec.h
@@ -49,6 +49,12 @@ ipsec_xform_auth_verify(struct rte_crypto_sym_xform *crypto_xform)
 	} else if (crypto_xform->auth.algo == RTE_CRYPTO_AUTH_SHA256_HMAC) {
 		if (keylen >= 32 && keylen <= 64)
 			return 0;
+	} else if (crypto_xform->auth.algo == RTE_CRYPTO_AUTH_SHA384_HMAC) {
+		if (keylen == 48)
+			return 0;
+	} else if (crypto_xform->auth.algo == RTE_CRYPTO_AUTH_SHA512_HMAC) {
+		if (keylen == 64)
+			return 0;
 	}
 
 	return -ENOTSUP;
-- 
2.7.4

[PATCH v3 13/29] crypto/cnxk: account for CPT CTX updates and flush delays

[Anoob Joseph]

CPT CTX write with microcode would require CPT flush to complete to have
DRAM updated with the SA. Since datapath requires SA direction field,
introduce a new flag for the same.

Session destroy path is also updated to clear sa.valid bit using CTX
reload operation.

Session is updated with marker to differentiate s/w immutable and s/w
mutable portions.

Signed-off-by: Anoob Joseph <anoobj@marvell.com>
---
 drivers/crypto/cnxk/cn10k_cryptodev_ops.c |  4 +--
 drivers/crypto/cnxk/cn10k_ipsec.c         | 60 ++++++++++++++++++++++++-------
 drivers/crypto/cnxk/cn10k_ipsec.h         | 27 +++++++++-----
 drivers/crypto/cnxk/cn10k_ipsec_la_ops.h  | 18 +++++-----
 4 files changed, 77 insertions(+), 32 deletions(-)

diff --git a/drivers/crypto/cnxk/cn10k_cryptodev_ops.c b/drivers/crypto/cnxk/cn10k_cryptodev_ops.c
index d25a17c..7617bdc 100644
--- a/drivers/crypto/cnxk/cn10k_cryptodev_ops.c
+++ b/drivers/crypto/cnxk/cn10k_cryptodev_ops.c
@@ -53,7 +53,6 @@ cpt_sec_inst_fill(struct rte_crypto_op *op, struct cn10k_sec_session *sess,
 		  struct cpt_inflight_req *infl_req, struct cpt_inst_s *inst)
 {
 	struct rte_crypto_sym_op *sym_op = op->sym;
-	union roc_ot_ipsec_sa_word2 *w2;
 	struct cn10k_ipsec_sa *sa;
 	int ret;
 
@@ -68,9 +67,8 @@ cpt_sec_inst_fill(struct rte_crypto_op *op, struct cn10k_sec_session *sess,
 	}
 
 	sa = &sess->sa;
-	w2 = (union roc_ot_ipsec_sa_word2 *)&sa->in_sa.w2;
 
-	if (w2->s.dir == ROC_IE_SA_DIR_OUTBOUND)
+	if (sa->is_outbound)
 		ret = process_outb_sa(op, sa, inst);
 	else {
 		infl_req->op_flags |= CPT_OP_FLAGS_IPSEC_DIR_INBOUND;
diff --git a/drivers/crypto/cnxk/cn10k_ipsec.c b/drivers/crypto/cnxk/cn10k_ipsec.c
index a11a6b7..b4acbac 100644
--- a/drivers/crypto/cnxk/cn10k_ipsec.c
+++ b/drivers/crypto/cnxk/cn10k_ipsec.c
@@ -67,7 +67,7 @@ cn10k_ipsec_outb_sa_create(struct roc_cpt *roc_cpt, struct roc_cpt_lf *lf,
 		goto sa_dptr_free;
 	}
 
-	sa->inst.w7 = ipsec_cpt_inst_w7_get(roc_cpt, sa);
+	sa->inst.w7 = ipsec_cpt_inst_w7_get(roc_cpt, out_sa);
 
 #ifdef LA_IPSEC_DEBUG
 	/* Use IV from application in debug mode */
@@ -89,6 +89,8 @@ cn10k_ipsec_outb_sa_create(struct roc_cpt *roc_cpt, struct roc_cpt_lf *lf,
 	}
 #endif
 
+	sa->is_outbound = true;
+
 	/* Get Rlen calculation data */
 	ret = cnxk_ipsec_outb_rlens_get(&rlens, ipsec_xfrm, crypto_xfrm);
 	if (ret)
@@ -127,6 +129,8 @@ cn10k_ipsec_outb_sa_create(struct roc_cpt *roc_cpt, struct roc_cpt_lf *lf,
 	/* Copy word0 from sa_dptr to populate ctx_push_sz ctx_size fields */
 	memcpy(out_sa, sa_dptr, 8);
 
+	plt_atomic_thread_fence(__ATOMIC_SEQ_CST);
+
 	/* Write session using microcode opcode */
 	ret = roc_cpt_ctx_write(lf, sa_dptr, out_sa,
 				ROC_NIX_INL_OT_IPSEC_OUTB_HW_SZ);
@@ -135,9 +139,11 @@ cn10k_ipsec_outb_sa_create(struct roc_cpt *roc_cpt, struct roc_cpt_lf *lf,
 		goto sa_dptr_free;
 	}
 
-	/* Trigger CTX flush to write dirty data back to DRAM */
+	/* Trigger CTX flush so that data is written back to DRAM */
 	roc_cpt_lf_ctx_flush(lf, out_sa, false);
 
+	plt_atomic_thread_fence(__ATOMIC_SEQ_CST);
+
 sa_dptr_free:
 	plt_free(sa_dptr);
 
@@ -178,7 +184,8 @@ cn10k_ipsec_inb_sa_create(struct roc_cpt *roc_cpt, struct roc_cpt_lf *lf,
 		goto sa_dptr_free;
 	}
 
-	sa->inst.w7 = ipsec_cpt_inst_w7_get(roc_cpt, sa);
+	sa->is_outbound = false;
+	sa->inst.w7 = ipsec_cpt_inst_w7_get(roc_cpt, in_sa);
 
 	/* pre-populate CPT INST word 4 */
 	inst_w4.u64 = 0;
@@ -214,6 +221,8 @@ cn10k_ipsec_inb_sa_create(struct roc_cpt *roc_cpt, struct roc_cpt_lf *lf,
 	/* Copy word0 from sa_dptr to populate ctx_push_sz ctx_size fields */
 	memcpy(in_sa, sa_dptr, 8);
 
+	plt_atomic_thread_fence(__ATOMIC_SEQ_CST);
+
 	/* Write session using microcode opcode */
 	ret = roc_cpt_ctx_write(lf, sa_dptr, in_sa,
 				ROC_NIX_INL_OT_IPSEC_INB_HW_SZ);
@@ -222,9 +231,11 @@ cn10k_ipsec_inb_sa_create(struct roc_cpt *roc_cpt, struct roc_cpt_lf *lf,
 		goto sa_dptr_free;
 	}
 
-	/* Trigger CTX flush to write dirty data back to DRAM */
+	/* Trigger CTX flush so that data is written back to DRAM */
 	roc_cpt_lf_ctx_flush(lf, in_sa, false);
 
+	plt_atomic_thread_fence(__ATOMIC_SEQ_CST);
+
 sa_dptr_free:
 	plt_free(sa_dptr);
 
@@ -300,21 +311,46 @@ cn10k_sec_session_create(void *device, struct rte_security_session_conf *conf,
 }
 
 static int
-cn10k_sec_session_destroy(void *device __rte_unused,
-			  struct rte_security_session *sess)
+cn10k_sec_session_destroy(void *dev, struct rte_security_session *sec_sess)
 {
-	struct cn10k_sec_session *priv;
+	struct rte_cryptodev *crypto_dev = dev;
+	union roc_ot_ipsec_sa_word2 *w2;
+	struct cn10k_sec_session *sess;
 	struct rte_mempool *sess_mp;
+	struct cn10k_ipsec_sa *sa;
+	struct cnxk_cpt_qp *qp;
+	struct roc_cpt_lf *lf;
 
-	priv = get_sec_session_private_data(sess);
+	sess = get_sec_session_private_data(sec_sess);
+	if (sess == NULL)
+		return 0;
 
-	if (priv == NULL)
+	qp = crypto_dev->data->queue_pairs[0];
+	if (qp == NULL)
 		return 0;
 
-	sess_mp = rte_mempool_from_obj(priv);
+	lf = &qp->lf;
 
-	set_sec_session_private_data(sess, NULL);
-	rte_mempool_put(sess_mp, priv);
+	sa = &sess->sa;
+
+	/* Trigger CTX flush to write dirty data back to DRAM */
+	roc_cpt_lf_ctx_flush(lf, &sa->in_sa, false);
+
+	/* Wait for 1 ms so that flush is complete */
+	rte_delay_ms(1);
+
+	w2 = (union roc_ot_ipsec_sa_word2 *)&sa->in_sa.w2;
+	w2->s.valid = 0;
+
+	plt_atomic_thread_fence(__ATOMIC_SEQ_CST);
+
+	/* Trigger CTX reload to fetch new data from DRAM */
+	roc_cpt_lf_ctx_reload(lf, &sa->in_sa);
+
+	sess_mp = rte_mempool_from_obj(sess);
+
+	set_sec_session_private_data(sec_sess, NULL);
+	rte_mempool_put(sess_mp, sess);
 
 	return 0;
 }
diff --git a/drivers/crypto/cnxk/cn10k_ipsec.h b/drivers/crypto/cnxk/cn10k_ipsec.h
index 86cd248..cc7ca19 100644
--- a/drivers/crypto/cnxk/cn10k_ipsec.h
+++ b/drivers/crypto/cnxk/cn10k_ipsec.h
@@ -7,28 +7,37 @@
 
 #include <rte_security.h>
 
+#include "roc_api.h"
+
 #include "cnxk_ipsec.h"
 
-#define CN10K_IPSEC_SA_CTX_HDR_SIZE 1
+typedef void *CN10K_SA_CONTEXT_MARKER[0];
 
 struct cn10k_ipsec_sa {
-	union {
-		/** Inbound SA */
-		struct roc_ot_ipsec_inb_sa in_sa;
-		/** Outbound SA */
-		struct roc_ot_ipsec_outb_sa out_sa;
-	};
 	/** Pre-populated CPT inst words */
 	struct cnxk_cpt_inst_tmpl inst;
 	uint16_t max_extended_len;
 	uint16_t iv_offset;
 	uint8_t iv_length;
 	bool ip_csum_enable;
-};
+	bool is_outbound;
+
+	/**
+	 * End of SW mutable area
+	 */
+	CN10K_SA_CONTEXT_MARKER sw_area_end __rte_aligned(ROC_ALIGN);
+
+	union {
+		/** Inbound SA */
+		struct roc_ot_ipsec_inb_sa in_sa;
+		/** Outbound SA */
+		struct roc_ot_ipsec_outb_sa out_sa;
+	};
+} __rte_aligned(ROC_ALIGN);
 
 struct cn10k_sec_session {
 	struct cn10k_ipsec_sa sa;
-} __rte_cache_aligned;
+} __rte_aligned(ROC_ALIGN);
 
 void cn10k_sec_ops_override(void);
 
diff --git a/drivers/crypto/cnxk/cn10k_ipsec_la_ops.h b/drivers/crypto/cnxk/cn10k_ipsec_la_ops.h
index 881fbd1..cab6a50 100644
--- a/drivers/crypto/cnxk/cn10k_ipsec_la_ops.h
+++ b/drivers/crypto/cnxk/cn10k_ipsec_la_ops.h
@@ -54,6 +54,7 @@ process_outb_sa(struct rte_crypto_op *cop, struct cn10k_ipsec_sa *sess,
 	struct rte_crypto_sym_op *sym_op = cop->sym;
 	struct rte_mbuf *m_src = sym_op->m_src;
 	uint64_t inst_w4_u64 = sess->inst.w4;
+	uint64_t dptr;
 
 	if (unlikely(rte_pktmbuf_tailroom(m_src) < sess->max_extended_len)) {
 		plt_dp_err("Not enough tail room");
@@ -76,10 +77,10 @@ process_outb_sa(struct rte_crypto_op *cop, struct cn10k_ipsec_sa *sess,
 		inst_w4_u64 &= ~BIT_ULL(32);
 
 	/* Prepare CPT instruction */
-	inst->w4.u64 = inst_w4_u64;
-	inst->w4.s.dlen = rte_pktmbuf_pkt_len(m_src);
-	inst->dptr = rte_pktmbuf_iova(m_src);
-	inst->rptr = inst->dptr;
+	inst->w4.u64 = inst_w4_u64 | rte_pktmbuf_pkt_len(m_src);
+	dptr = rte_pktmbuf_iova(m_src);
+	inst->dptr = dptr;
+	inst->rptr = dptr;
 
 	return 0;
 }
@@ -90,12 +91,13 @@ process_inb_sa(struct rte_crypto_op *cop, struct cn10k_ipsec_sa *sa,
 {
 	struct rte_crypto_sym_op *sym_op = cop->sym;
 	struct rte_mbuf *m_src = sym_op->m_src;
+	uint64_t dptr;
 
 	/* Prepare CPT instruction */
-	inst->w4.u64 = sa->inst.w4;
-	inst->w4.s.dlen = rte_pktmbuf_pkt_len(m_src);
-	inst->dptr = rte_pktmbuf_iova(m_src);
-	inst->rptr = inst->dptr;
+	inst->w4.u64 = sa->inst.w4 | rte_pktmbuf_pkt_len(m_src);
+	dptr = rte_pktmbuf_iova(m_src);
+	inst->dptr = dptr;
+	inst->rptr = dptr;
 
 	return 0;
 }
-- 
2.7.4

[PATCH v3 14/29] crypto/cnxk: use struct sizes for ctx writes

[Anoob Joseph]

CTX writes only require the lengths are 8B aligned. Use the struct size
directly.

Signed-off-by: Anoob Joseph <anoobj@marvell.com>
---
 drivers/crypto/cnxk/cn10k_ipsec.c | 12 ++++--------
 1 file changed, 4 insertions(+), 8 deletions(-)

diff --git a/drivers/crypto/cnxk/cn10k_ipsec.c b/drivers/crypto/cnxk/cn10k_ipsec.c
index b4acbac..0832b53 100644
--- a/drivers/crypto/cnxk/cn10k_ipsec.c
+++ b/drivers/crypto/cnxk/cn10k_ipsec.c
@@ -52,14 +52,12 @@ cn10k_ipsec_outb_sa_create(struct roc_cpt *roc_cpt, struct roc_cpt_lf *lf,
 	out_sa = &sa->out_sa;
 
 	/* Allocate memory to be used as dptr for CPT ucode WRITE_SA op */
-	sa_dptr = plt_zmalloc(ROC_NIX_INL_OT_IPSEC_OUTB_HW_SZ, 0);
+	sa_dptr = plt_zmalloc(sizeof(struct roc_ot_ipsec_outb_sa), 8);
 	if (sa_dptr == NULL) {
 		plt_err("Couldn't allocate memory for SA dptr");
 		return -ENOMEM;
 	}
 
-	memset(sa_dptr, 0, sizeof(struct roc_ot_ipsec_outb_sa));
-
 	/* Translate security parameters to SA */
 	ret = cnxk_ot_ipsec_outb_sa_fill(sa_dptr, ipsec_xfrm, crypto_xfrm);
 	if (ret) {
@@ -133,7 +131,7 @@ cn10k_ipsec_outb_sa_create(struct roc_cpt *roc_cpt, struct roc_cpt_lf *lf,
 
 	/* Write session using microcode opcode */
 	ret = roc_cpt_ctx_write(lf, sa_dptr, out_sa,
-				ROC_NIX_INL_OT_IPSEC_OUTB_HW_SZ);
+				sizeof(struct roc_ot_ipsec_outb_sa));
 	if (ret) {
 		plt_err("Could not write outbound session to hardware");
 		goto sa_dptr_free;
@@ -169,14 +167,12 @@ cn10k_ipsec_inb_sa_create(struct roc_cpt *roc_cpt, struct roc_cpt_lf *lf,
 	in_sa = &sa->in_sa;
 
 	/* Allocate memory to be used as dptr for CPT ucode WRITE_SA op */
-	sa_dptr = plt_zmalloc(ROC_NIX_INL_OT_IPSEC_INB_HW_SZ, 0);
+	sa_dptr = plt_zmalloc(sizeof(struct roc_ot_ipsec_inb_sa), 8);
 	if (sa_dptr == NULL) {
 		plt_err("Couldn't allocate memory for SA dptr");
 		return -ENOMEM;
 	}
 
-	memset(sa_dptr, 0, sizeof(struct roc_ot_ipsec_inb_sa));
-
 	/* Translate security parameters to SA */
 	ret = cnxk_ot_ipsec_inb_sa_fill(sa_dptr, ipsec_xfrm, crypto_xfrm);
 	if (ret) {
@@ -225,7 +221,7 @@ cn10k_ipsec_inb_sa_create(struct roc_cpt *roc_cpt, struct roc_cpt_lf *lf,
 
 	/* Write session using microcode opcode */
 	ret = roc_cpt_ctx_write(lf, sa_dptr, in_sa,
-				ROC_NIX_INL_OT_IPSEC_INB_HW_SZ);
+				sizeof(struct roc_ot_ipsec_inb_sa));
 	if (ret) {
 		plt_err("Could not write inbound session to hardware");
 		goto sa_dptr_free;
-- 
2.7.4

[PATCH v3 15/29] crypto/cnxk: add security session stats get

[Anoob Joseph]

From: Ankur Dwivedi <adwivedi@marvell.com>

Adds the security session stats get op for cn10k.


Signed-off-by: Ankur Dwivedi <adwivedi@marvell.com>
---
 drivers/crypto/cnxk/cn10k_ipsec.c                 | 55 +++++++++++++++++++++++
 drivers/crypto/cnxk/cnxk_cryptodev_capabilities.c |  1 +
 drivers/crypto/cnxk/cnxk_cryptodev_sec.c          |  1 +
 3 files changed, 57 insertions(+)

diff --git a/drivers/crypto/cnxk/cn10k_ipsec.c b/drivers/crypto/cnxk/cn10k_ipsec.c
index 0832b53..a93c211 100644
--- a/drivers/crypto/cnxk/cn10k_ipsec.c
+++ b/drivers/crypto/cnxk/cn10k_ipsec.c
@@ -122,6 +122,12 @@ cn10k_ipsec_outb_sa_create(struct roc_cpt *roc_cpt, struct roc_cpt_lf *lf,
 
 	sa->inst.w4 = inst_w4.u64;
 
+	if (ipsec_xfrm->options.stats == 1) {
+		/* Enable mib counters */
+		sa_dptr->w0.s.count_mib_bytes = 1;
+		sa_dptr->w0.s.count_mib_pkts = 1;
+	}
+
 	memset(out_sa, 0, sizeof(struct roc_ot_ipsec_outb_sa));
 
 	/* Copy word0 from sa_dptr to populate ctx_push_sz ctx_size fields */
@@ -212,6 +218,12 @@ cn10k_ipsec_inb_sa_create(struct roc_cpt *roc_cpt, struct roc_cpt_lf *lf,
 
 	sa->inst.w4 = inst_w4.u64;
 
+	if (ipsec_xfrm->options.stats == 1) {
+		/* Enable mib counters */
+		sa_dptr->w0.s.count_mib_bytes = 1;
+		sa_dptr->w0.s.count_mib_pkts = 1;
+	}
+
 	memset(in_sa, 0, sizeof(struct roc_ot_ipsec_inb_sa));
 
 	/* Copy word0 from sa_dptr to populate ctx_push_sz ctx_size fields */
@@ -357,6 +369,48 @@ cn10k_sec_session_get_size(void *device __rte_unused)
 	return sizeof(struct cn10k_sec_session);
 }
 
+static int
+cn10k_sec_session_stats_get(void *device, struct rte_security_session *sess,
+			    struct rte_security_stats *stats)
+{
+	struct rte_cryptodev *crypto_dev = device;
+	struct roc_ot_ipsec_outb_sa *out_sa;
+	struct roc_ot_ipsec_inb_sa *in_sa;
+	union roc_ot_ipsec_sa_word2 *w2;
+	struct cn10k_sec_session *priv;
+	struct cn10k_ipsec_sa *sa;
+	struct cnxk_cpt_qp *qp;
+
+	priv = get_sec_session_private_data(sess);
+	if (priv == NULL)
+		return -EINVAL;
+
+	qp = crypto_dev->data->queue_pairs[0];
+	if (qp == NULL)
+		return -EINVAL;
+
+	sa = &priv->sa;
+	w2 = (union roc_ot_ipsec_sa_word2 *)&sa->in_sa.w2;
+
+	stats->protocol = RTE_SECURITY_PROTOCOL_IPSEC;
+
+	if (w2->s.dir == ROC_IE_SA_DIR_OUTBOUND) {
+		out_sa = &sa->out_sa;
+		roc_cpt_lf_ctx_flush(&qp->lf, out_sa, false);
+		rte_delay_ms(1);
+		stats->ipsec.opackets = out_sa->ctx.mib_pkts;
+		stats->ipsec.obytes = out_sa->ctx.mib_octs;
+	} else {
+		in_sa = &sa->in_sa;
+		roc_cpt_lf_ctx_flush(&qp->lf, in_sa, false);
+		rte_delay_ms(1);
+		stats->ipsec.ipackets = in_sa->ctx.mib_pkts;
+		stats->ipsec.ibytes = in_sa->ctx.mib_octs;
+	}
+
+	return 0;
+}
+
 /* Update platform specific security ops */
 void
 cn10k_sec_ops_override(void)
@@ -365,4 +419,5 @@ cn10k_sec_ops_override(void)
 	cnxk_sec_ops.session_create = cn10k_sec_session_create;
 	cnxk_sec_ops.session_destroy = cn10k_sec_session_destroy;
 	cnxk_sec_ops.session_get_size = cn10k_sec_session_get_size;
+	cnxk_sec_ops.session_stats_get = cn10k_sec_session_stats_get;
 }
diff --git a/drivers/crypto/cnxk/cnxk_cryptodev_capabilities.c b/drivers/crypto/cnxk/cnxk_cryptodev_capabilities.c
index 9a55474..0fdd91a 100644
--- a/drivers/crypto/cnxk/cnxk_cryptodev_capabilities.c
+++ b/drivers/crypto/cnxk/cnxk_cryptodev_capabilities.c
@@ -1073,6 +1073,7 @@ cn10k_sec_caps_update(struct rte_security_capability *sec_cap)
 	}
 	sec_cap->ipsec.options.ip_csum_enable = 1;
 	sec_cap->ipsec.options.l4_csum_enable = 1;
+	sec_cap->ipsec.options.stats = 1;
 }
 
 static void
diff --git a/drivers/crypto/cnxk/cnxk_cryptodev_sec.c b/drivers/crypto/cnxk/cnxk_cryptodev_sec.c
index 2021d5c..e5a5d2d 100644
--- a/drivers/crypto/cnxk/cnxk_cryptodev_sec.c
+++ b/drivers/crypto/cnxk/cnxk_cryptodev_sec.c
@@ -15,6 +15,7 @@ struct rte_security_ops cnxk_sec_ops = {
 	.session_create = NULL,
 	.session_destroy = NULL,
 	.session_get_size = NULL,
+	.session_stats_get = NULL,
 	.set_pkt_metadata = NULL,
 	.get_userdata = NULL,
 	.capabilities_get = cnxk_crypto_sec_capabilities_get
-- 
2.7.4

[PATCH v3 16/29] crypto/cnxk: add skip for unsupported cases

[Anoob Joseph]

Add skip for transport mode tests that are not supported. Also,
updated the transport mode path to configure IP version as v4.

Signed-off-by: Anoob Joseph <anoobj@marvell.com>
---
 drivers/crypto/cnxk/cn9k_ipsec.c | 53 +++++++++++++++++++++++++++++++++++-----
 1 file changed, 47 insertions(+), 6 deletions(-)

diff --git a/drivers/crypto/cnxk/cn9k_ipsec.c b/drivers/crypto/cnxk/cn9k_ipsec.c
index 395b0d5..c27845c 100644
--- a/drivers/crypto/cnxk/cn9k_ipsec.c
+++ b/drivers/crypto/cnxk/cn9k_ipsec.c
@@ -141,11 +141,10 @@ ipsec_sa_ctl_set(struct rte_security_ipsec_xform *ipsec,
 			return -EINVAL;
 	}
 
-	ctl->inner_ip_ver = ctl->outer_ip_ver;
-
-	if (ipsec->mode == RTE_SECURITY_IPSEC_SA_MODE_TRANSPORT)
+	if (ipsec->mode == RTE_SECURITY_IPSEC_SA_MODE_TRANSPORT) {
 		ctl->ipsec_mode = ROC_IE_SA_MODE_TRANSPORT;
-	else if (ipsec->mode == RTE_SECURITY_IPSEC_SA_MODE_TUNNEL)
+		ctl->outer_ip_ver = ROC_IE_SA_IP_VERSION_4;
+	} else if (ipsec->mode == RTE_SECURITY_IPSEC_SA_MODE_TUNNEL)
 		ctl->ipsec_mode = ROC_IE_SA_MODE_TUNNEL;
 	else
 		return -EINVAL;
@@ -548,7 +547,8 @@ cn9k_ipsec_inb_sa_create(struct cnxk_cpt_qp *qp,
 }
 
 static inline int
-cn9k_ipsec_xform_verify(struct rte_security_ipsec_xform *ipsec)
+cn9k_ipsec_xform_verify(struct rte_security_ipsec_xform *ipsec,
+			struct rte_crypto_sym_xform *crypto)
 {
 	if (ipsec->life.bytes_hard_limit != 0 ||
 	    ipsec->life.bytes_soft_limit != 0 ||
@@ -556,6 +556,47 @@ cn9k_ipsec_xform_verify(struct rte_security_ipsec_xform *ipsec)
 	    ipsec->life.packets_soft_limit != 0)
 		return -ENOTSUP;
 
+	if (ipsec->mode == RTE_SECURITY_IPSEC_SA_MODE_TRANSPORT) {
+		enum rte_crypto_sym_xform_type type = crypto->type;
+
+		if (type == RTE_CRYPTO_SYM_XFORM_AEAD) {
+			if ((crypto->aead.algo == RTE_CRYPTO_AEAD_AES_GCM) &&
+			    (crypto->aead.key.length == 32)) {
+				plt_err("Transport mode AES-256-GCM is not supported");
+				return -ENOTSUP;
+			}
+		} else {
+			struct rte_crypto_cipher_xform *cipher;
+			struct rte_crypto_auth_xform *auth;
+
+			if (crypto->type == RTE_CRYPTO_SYM_XFORM_CIPHER) {
+				cipher = &crypto->cipher;
+				auth = &crypto->next->auth;
+			} else {
+				cipher = &crypto->next->cipher;
+				auth = &crypto->auth;
+			}
+
+			if ((cipher->algo == RTE_CRYPTO_CIPHER_AES_CBC) &&
+			    (auth->algo == RTE_CRYPTO_AUTH_SHA256_HMAC)) {
+				plt_err("Transport mode AES-CBC SHA2 HMAC 256 is not supported");
+				return -ENOTSUP;
+			}
+
+			if ((cipher->algo == RTE_CRYPTO_CIPHER_AES_CBC) &&
+			    (auth->algo == RTE_CRYPTO_AUTH_SHA384_HMAC)) {
+				plt_err("Transport mode AES-CBC SHA2 HMAC 384 is not supported");
+				return -ENOTSUP;
+			}
+
+			if ((cipher->algo == RTE_CRYPTO_CIPHER_AES_CBC) &&
+			    (auth->algo == RTE_CRYPTO_AUTH_SHA512_HMAC)) {
+				plt_err("Transport mode AES-CBC SHA2 HMAC 512 is not supported");
+				return -ENOTSUP;
+			}
+		}
+	}
+
 	return 0;
 }
 
@@ -580,7 +621,7 @@ cn9k_ipsec_session_create(void *dev,
 	if (ret)
 		return ret;
 
-	ret = cn9k_ipsec_xform_verify(ipsec_xform);
+	ret = cn9k_ipsec_xform_verify(ipsec_xform, crypto_xform);
 	if (ret)
 		return ret;
 
-- 
2.7.4

[PATCH v3 17/29] crypto/cnxk: add context reload for IV

[Anoob Joseph]

From: Tejasree Kondoj <ktejasree@marvell.com>

Adding context reload in datapath for IV in debug mode.

Signed-off-by: Tejasree Kondoj <ktejasree@marvell.com>
---
 drivers/crypto/cnxk/cn10k_cryptodev_ops.c |  7 ++++---
 drivers/crypto/cnxk/cn10k_ipsec_la_ops.h  | 10 ++++++++--
 2 files changed, 12 insertions(+), 5 deletions(-)

diff --git a/drivers/crypto/cnxk/cn10k_cryptodev_ops.c b/drivers/crypto/cnxk/cn10k_cryptodev_ops.c
index 7617bdc..638268e 100644
--- a/drivers/crypto/cnxk/cn10k_cryptodev_ops.c
+++ b/drivers/crypto/cnxk/cn10k_cryptodev_ops.c
@@ -49,7 +49,8 @@ cn10k_cpt_sym_temp_sess_create(struct cnxk_cpt_qp *qp, struct rte_crypto_op *op)
 }
 
 static __rte_always_inline int __rte_hot
-cpt_sec_inst_fill(struct rte_crypto_op *op, struct cn10k_sec_session *sess,
+cpt_sec_inst_fill(struct cnxk_cpt_qp *qp, struct rte_crypto_op *op,
+		  struct cn10k_sec_session *sess,
 		  struct cpt_inflight_req *infl_req, struct cpt_inst_s *inst)
 {
 	struct rte_crypto_sym_op *sym_op = op->sym;
@@ -69,7 +70,7 @@ cpt_sec_inst_fill(struct rte_crypto_op *op, struct cn10k_sec_session *sess,
 	sa = &sess->sa;
 
 	if (sa->is_outbound)
-		ret = process_outb_sa(op, sa, inst);
+		ret = process_outb_sa(&qp->lf, op, sa, inst);
 	else {
 		infl_req->op_flags |= CPT_OP_FLAGS_IPSEC_DIR_INBOUND;
 		ret = process_inb_sa(op, sa, inst);
@@ -122,7 +123,7 @@ cn10k_cpt_fill_inst(struct cnxk_cpt_qp *qp, struct rte_crypto_op *ops[],
 		if (op->sess_type == RTE_CRYPTO_OP_SECURITY_SESSION) {
 			sec_sess = get_sec_session_private_data(
 				sym_op->sec_session);
-			ret = cpt_sec_inst_fill(op, sec_sess, infl_req,
+			ret = cpt_sec_inst_fill(qp, op, sec_sess, infl_req,
 						&inst[0]);
 			if (unlikely(ret))
 				return 0;
diff --git a/drivers/crypto/cnxk/cn10k_ipsec_la_ops.h b/drivers/crypto/cnxk/cn10k_ipsec_la_ops.h
index cab6a50..f2d8122 100644
--- a/drivers/crypto/cnxk/cn10k_ipsec_la_ops.h
+++ b/drivers/crypto/cnxk/cn10k_ipsec_la_ops.h
@@ -48,8 +48,8 @@ ipsec_po_sa_aes_gcm_iv_set(struct cn10k_ipsec_sa *sess,
 }
 
 static __rte_always_inline int
-process_outb_sa(struct rte_crypto_op *cop, struct cn10k_ipsec_sa *sess,
-		struct cpt_inst_s *inst)
+process_outb_sa(struct roc_cpt_lf *lf, struct rte_crypto_op *cop,
+		struct cn10k_ipsec_sa *sess, struct cpt_inst_s *inst)
 {
 	struct rte_crypto_sym_op *sym_op = cop->sym;
 	struct rte_mbuf *m_src = sym_op->m_src;
@@ -61,6 +61,8 @@ process_outb_sa(struct rte_crypto_op *cop, struct cn10k_ipsec_sa *sess,
 		return -ENOMEM;
 	}
 
+	RTE_SET_USED(lf);
+
 #ifdef LA_IPSEC_DEBUG
 	if (sess->out_sa.w2.s.iv_src == ROC_IE_OT_SA_IV_SRC_FROM_SA) {
 		if (sess->out_sa.w2.s.enc_type == ROC_IE_OT_SA_ENC_AES_GCM)
@@ -68,6 +70,10 @@ process_outb_sa(struct rte_crypto_op *cop, struct cn10k_ipsec_sa *sess,
 		else
 			ipsec_po_sa_iv_set(sess, cop);
 	}
+
+	/* Trigger CTX reload to fetch new data from DRAM */
+	roc_cpt_lf_ctx_reload(lf, &sess->out_sa);
+	rte_delay_ms(1);
 #endif
 
 	if (m_src->ol_flags & RTE_MBUF_F_TX_IP_CKSUM)
-- 
2.7.4

[PATCH v3 18/29] crypto/cnxk: handle null chained ops

[Anoob Joseph]

Verification doesn't cover cases when NULL auth/cipher is provided as a
chain. Removed the separate function for verification and added a
replacement function which calls the appropriate downstream functions.

Signed-off-by: Anoob Joseph <anoobj@marvell.com>
---
 drivers/crypto/cnxk/cnxk_cryptodev_ops.c | 189 ++++++++++++++++---------------
 drivers/crypto/cnxk/cnxk_cryptodev_ops.h |  10 --
 drivers/crypto/cnxk/cnxk_se.h            |   6 +
 3 files changed, 102 insertions(+), 103 deletions(-)

diff --git a/drivers/crypto/cnxk/cnxk_cryptodev_ops.c b/drivers/crypto/cnxk/cnxk_cryptodev_ops.c
index 21ee09f..7953a08 100644
--- a/drivers/crypto/cnxk/cnxk_cryptodev_ops.c
+++ b/drivers/crypto/cnxk/cnxk_cryptodev_ops.c
@@ -418,84 +418,121 @@ cnxk_cpt_sym_session_get_size(struct rte_cryptodev *dev __rte_unused)
 }
 
 static int
-sym_xform_verify(struct rte_crypto_sym_xform *xform)
+cnxk_sess_fill(struct rte_crypto_sym_xform *xform, struct cnxk_se_sess *sess)
 {
-	if (xform->type == RTE_CRYPTO_SYM_XFORM_AUTH &&
-	    xform->auth.algo == RTE_CRYPTO_AUTH_NULL &&
-	    xform->auth.op == RTE_CRYPTO_AUTH_OP_VERIFY)
-		return -ENOTSUP;
+	struct rte_crypto_sym_xform *aead_xfrm = NULL;
+	struct rte_crypto_sym_xform *c_xfrm = NULL;
+	struct rte_crypto_sym_xform *a_xfrm = NULL;
+	bool ciph_then_auth = false;
 
-	if (xform->type == RTE_CRYPTO_SYM_XFORM_CIPHER && xform->next == NULL)
-		return CNXK_CPT_CIPHER;
+	if (xform == NULL)
+		return -EINVAL;
 
-	if (xform->type == RTE_CRYPTO_SYM_XFORM_AUTH && xform->next == NULL)
-		return CNXK_CPT_AUTH;
+	if (xform->type == RTE_CRYPTO_SYM_XFORM_CIPHER) {
+		c_xfrm = xform;
+		a_xfrm = xform->next;
+		ciph_then_auth = true;
+	} else if (xform->type == RTE_CRYPTO_SYM_XFORM_AUTH) {
+		c_xfrm = xform->next;
+		a_xfrm = xform;
+		ciph_then_auth = false;
+	} else {
+		aead_xfrm = xform;
+	}
 
-	if (xform->type == RTE_CRYPTO_SYM_XFORM_AEAD && xform->next == NULL)
-		return CNXK_CPT_AEAD;
+	if (c_xfrm != NULL && c_xfrm->type != RTE_CRYPTO_SYM_XFORM_CIPHER) {
+		plt_dp_err("Invalid type in cipher xform");
+		return -EINVAL;
+	}
 
-	if (xform->next == NULL)
-		return -EIO;
+	if (a_xfrm != NULL && a_xfrm->type != RTE_CRYPTO_SYM_XFORM_AUTH) {
+		plt_dp_err("Invalid type in auth xform");
+		return -EINVAL;
+	}
+
+	if (aead_xfrm != NULL && aead_xfrm->type != RTE_CRYPTO_SYM_XFORM_AEAD) {
+		plt_dp_err("Invalid type in AEAD xform");
+		return -EINVAL;
+	}
 
-	if (xform->type == RTE_CRYPTO_SYM_XFORM_CIPHER &&
-	    xform->cipher.algo == RTE_CRYPTO_CIPHER_3DES_CBC &&
-	    xform->next->type == RTE_CRYPTO_SYM_XFORM_AUTH &&
-	    xform->next->auth.algo == RTE_CRYPTO_AUTH_SHA1)
+	if ((c_xfrm == NULL || c_xfrm->cipher.algo == RTE_CRYPTO_CIPHER_NULL) &&
+	    a_xfrm != NULL && a_xfrm->auth.algo == RTE_CRYPTO_AUTH_NULL &&
+	    a_xfrm->auth.op == RTE_CRYPTO_AUTH_OP_VERIFY) {
+		plt_dp_err("Null cipher + null auth verify is not supported");
 		return -ENOTSUP;
+	}
+
+	/* Cipher only */
+	if (c_xfrm != NULL &&
+	    (a_xfrm == NULL || a_xfrm->auth.algo == RTE_CRYPTO_AUTH_NULL)) {
+		if (fill_sess_cipher(c_xfrm, sess))
+			return -ENOTSUP;
+		else
+			return 0;
+	}
+
+	/* Auth only */
+	if (a_xfrm != NULL &&
+	    (c_xfrm == NULL || c_xfrm->cipher.algo == RTE_CRYPTO_CIPHER_NULL)) {
+		if (fill_sess_auth(a_xfrm, sess))
+			return -ENOTSUP;
+		else
+			return 0;
+	}
+
+	/* AEAD */
+	if (aead_xfrm != NULL) {
+		if (fill_sess_aead(aead_xfrm, sess))
+			return -ENOTSUP;
+		else
+			return 0;
+	}
+
+	/* Chained ops */
+	if (c_xfrm == NULL || a_xfrm == NULL) {
+		plt_dp_err("Invalid xforms");
+		return -EINVAL;
+	}
 
-	if (xform->type == RTE_CRYPTO_SYM_XFORM_AUTH &&
-	    xform->auth.algo == RTE_CRYPTO_AUTH_SHA1 &&
-	    xform->next->type == RTE_CRYPTO_SYM_XFORM_CIPHER &&
-	    xform->next->cipher.algo == RTE_CRYPTO_CIPHER_3DES_CBC)
+	if (c_xfrm->cipher.algo == RTE_CRYPTO_CIPHER_3DES_CBC &&
+	    a_xfrm->auth.algo == RTE_CRYPTO_AUTH_SHA1) {
+		plt_dp_err("3DES-CBC + SHA1 is not supported");
 		return -ENOTSUP;
+	}
 
-	if (xform->type == RTE_CRYPTO_SYM_XFORM_CIPHER &&
-	    xform->cipher.op == RTE_CRYPTO_CIPHER_OP_ENCRYPT &&
-	    xform->next->type == RTE_CRYPTO_SYM_XFORM_AUTH &&
-	    xform->next->auth.op == RTE_CRYPTO_AUTH_OP_GENERATE)
-		return CNXK_CPT_CIPHER_ENC_AUTH_GEN;
-
-	if (xform->type == RTE_CRYPTO_SYM_XFORM_AUTH &&
-	    xform->auth.op == RTE_CRYPTO_AUTH_OP_VERIFY &&
-	    xform->next->type == RTE_CRYPTO_SYM_XFORM_CIPHER &&
-	    xform->next->cipher.op == RTE_CRYPTO_CIPHER_OP_DECRYPT)
-		return CNXK_CPT_AUTH_VRFY_CIPHER_DEC;
-
-	if (xform->type == RTE_CRYPTO_SYM_XFORM_AUTH &&
-	    xform->auth.op == RTE_CRYPTO_AUTH_OP_GENERATE &&
-	    xform->next->type == RTE_CRYPTO_SYM_XFORM_CIPHER &&
-	    xform->next->cipher.op == RTE_CRYPTO_CIPHER_OP_ENCRYPT) {
-		switch (xform->auth.algo) {
-		case RTE_CRYPTO_AUTH_SHA1_HMAC:
-			switch (xform->next->cipher.algo) {
-			case RTE_CRYPTO_CIPHER_AES_CBC:
-				return CNXK_CPT_AUTH_GEN_CIPHER_ENC;
-			default:
-				return -ENOTSUP;
-			}
-		default:
+	/* Cipher then auth */
+	if (ciph_then_auth) {
+		if (fill_sess_cipher(c_xfrm, sess))
 			return -ENOTSUP;
-		}
+		if (fill_sess_auth(a_xfrm, sess))
+			return -ENOTSUP;
+		else
+			return 0;
 	}
 
-	if (xform->type == RTE_CRYPTO_SYM_XFORM_CIPHER &&
-	    xform->cipher.op == RTE_CRYPTO_CIPHER_OP_DECRYPT &&
-	    xform->next->type == RTE_CRYPTO_SYM_XFORM_AUTH &&
-	    xform->next->auth.op == RTE_CRYPTO_AUTH_OP_VERIFY) {
-		switch (xform->cipher.algo) {
-		case RTE_CRYPTO_CIPHER_AES_CBC:
-			switch (xform->next->auth.algo) {
-			case RTE_CRYPTO_AUTH_SHA1_HMAC:
-				return CNXK_CPT_CIPHER_DEC_AUTH_VRFY;
+	/* else */
+
+	if (c_xfrm->cipher.op == RTE_CRYPTO_CIPHER_OP_ENCRYPT) {
+		switch (a_xfrm->auth.algo) {
+		case RTE_CRYPTO_AUTH_SHA1_HMAC:
+			switch (c_xfrm->cipher.algo) {
+			case RTE_CRYPTO_CIPHER_AES_CBC:
+				break;
 			default:
 				return -ENOTSUP;
 			}
+			break;
 		default:
 			return -ENOTSUP;
 		}
 	}
 
-	return -ENOTSUP;
+	if (fill_sess_auth(a_xfrm, sess))
+		return -ENOTSUP;
+	if (fill_sess_cipher(c_xfrm, sess))
+		return -ENOTSUP;
+	else
+		return 0;
 }
 
 static uint64_t
@@ -524,10 +561,6 @@ sym_session_configure(struct roc_cpt *roc_cpt, int driver_id,
 	void *priv;
 	int ret;
 
-	ret = sym_xform_verify(xform);
-	if (unlikely(ret < 0))
-		return ret;
-
 	if (unlikely(rte_mempool_get(pool, &priv))) {
 		plt_dp_err("Could not allocate session private data");
 		return -ENOMEM;
@@ -537,37 +570,7 @@ sym_session_configure(struct roc_cpt *roc_cpt, int driver_id,
 
 	sess_priv = priv;
 
-	switch (ret) {
-	case CNXK_CPT_CIPHER:
-		ret = fill_sess_cipher(xform, sess_priv);
-		break;
-	case CNXK_CPT_AUTH:
-		if (xform->auth.algo == RTE_CRYPTO_AUTH_AES_GMAC)
-			ret = fill_sess_gmac(xform, sess_priv);
-		else
-			ret = fill_sess_auth(xform, sess_priv);
-		break;
-	case CNXK_CPT_AEAD:
-		ret = fill_sess_aead(xform, sess_priv);
-		break;
-	case CNXK_CPT_CIPHER_ENC_AUTH_GEN:
-	case CNXK_CPT_CIPHER_DEC_AUTH_VRFY:
-		ret = fill_sess_cipher(xform, sess_priv);
-		if (ret < 0)
-			break;
-		ret = fill_sess_auth(xform->next, sess_priv);
-		break;
-	case CNXK_CPT_AUTH_VRFY_CIPHER_DEC:
-	case CNXK_CPT_AUTH_GEN_CIPHER_ENC:
-		ret = fill_sess_auth(xform, sess_priv);
-		if (ret < 0)
-			break;
-		ret = fill_sess_cipher(xform->next, sess_priv);
-		break;
-	default:
-		ret = -1;
-	}
-
+	ret = cnxk_sess_fill(xform, sess_priv);
 	if (ret)
 		goto priv_put;
 
@@ -592,7 +595,7 @@ sym_session_configure(struct roc_cpt *roc_cpt, int driver_id,
 priv_put:
 	rte_mempool_put(pool, priv);
 
-	return -ENOTSUP;
+	return ret;
 }
 
 int
diff --git a/drivers/crypto/cnxk/cnxk_cryptodev_ops.h b/drivers/crypto/cnxk/cnxk_cryptodev_ops.h
index 0d36365..ca363bb 100644
--- a/drivers/crypto/cnxk/cnxk_cryptodev_ops.h
+++ b/drivers/crypto/cnxk/cnxk_cryptodev_ops.h
@@ -30,16 +30,6 @@ struct cpt_qp_meta_info {
 	int mlen;
 };
 
-enum sym_xform_type {
-	CNXK_CPT_CIPHER = 1,
-	CNXK_CPT_AUTH,
-	CNXK_CPT_AEAD,
-	CNXK_CPT_CIPHER_ENC_AUTH_GEN,
-	CNXK_CPT_AUTH_VRFY_CIPHER_DEC,
-	CNXK_CPT_AUTH_GEN_CIPHER_ENC,
-	CNXK_CPT_CIPHER_DEC_AUTH_VRFY
-};
-
 #define CPT_OP_FLAGS_METABUF	       (1 << 1)
 #define CPT_OP_FLAGS_AUTH_VERIFY       (1 << 0)
 #define CPT_OP_FLAGS_IPSEC_DIR_INBOUND (1 << 2)
diff --git a/drivers/crypto/cnxk/cnxk_se.h b/drivers/crypto/cnxk/cnxk_se.h
index 37237de..a8cd2c5 100644
--- a/drivers/crypto/cnxk/cnxk_se.h
+++ b/drivers/crypto/cnxk/cnxk_se.h
@@ -36,6 +36,9 @@ struct cnxk_se_sess {
 	struct roc_se_ctx roc_se_ctx;
 } __rte_cache_aligned;
 
+static __rte_always_inline int
+fill_sess_gmac(struct rte_crypto_sym_xform *xform, struct cnxk_se_sess *sess);
+
 static inline void
 cpt_pack_iv(uint8_t *iv_src, uint8_t *iv_dst)
 {
@@ -1808,6 +1811,9 @@ fill_sess_auth(struct rte_crypto_sym_xform *xform, struct cnxk_se_sess *sess)
 	roc_se_auth_type auth_type = 0; /* NULL Auth type */
 	uint8_t zsk_flag = 0, aes_gcm = 0, is_null = 0;
 
+	if (xform->auth.algo == RTE_CRYPTO_AUTH_AES_GMAC)
+		return fill_sess_gmac(xform, sess);
+
 	if (xform->next != NULL &&
 	    xform->next->type == RTE_CRYPTO_SYM_XFORM_CIPHER &&
 	    xform->next->cipher.op == RTE_CRYPTO_CIPHER_OP_ENCRYPT) {
-- 
2.7.4

[PATCH v3 19/29] crypto/cnxk: fix inflight cnt calculation

[Anoob Joseph]

Inflight count calculation is updated to cover wrap around cases where
head can become smaller than tail.


Reported-by: Kiran Kumar K <kirankumark@marvell.com>
Signed-off-by: Anoob Joseph <anoobj@marvell.com>
---
 drivers/crypto/cnxk/cnxk_cryptodev_ops.h | 6 +++++-
 1 file changed, 5 insertions(+), 1 deletion(-)

diff --git a/drivers/crypto/cnxk/cnxk_cryptodev_ops.h b/drivers/crypto/cnxk/cnxk_cryptodev_ops.h
index ca363bb..0336ae1 100644
--- a/drivers/crypto/cnxk/cnxk_cryptodev_ops.h
+++ b/drivers/crypto/cnxk/cnxk_cryptodev_ops.h
@@ -156,7 +156,11 @@ pending_queue_retreat(uint64_t *index, const uint64_t mask, uint64_t nb_entry)
 static __rte_always_inline uint64_t
 pending_queue_infl_cnt(uint64_t head, uint64_t tail, const uint64_t mask)
 {
-	return (head - tail) & mask;
+	/*
+	 * Mask is nb_desc - 1. Add nb_desc to head and mask to account for
+	 * cases when tail > head, which happens during wrap around.
+	 */
+	return ((head + mask + 1) - tail) & mask;
 }
 
 static __rte_always_inline uint64_t
-- 
2.7.4

[PATCH v3 20/29] crypto/cnxk: use atomics to access CPT res

[Anoob Joseph]

The memory would be updated by hardware. Use atomics to read the same.

Signed-off-by: Anoob Joseph <anoobj@marvell.com>
---
 drivers/common/cnxk/hw/cpt.h              |  2 ++
 drivers/crypto/cnxk/cn10k_cryptodev_ops.c | 24 ++++++++++++++++--------
 drivers/crypto/cnxk/cn9k_cryptodev_ops.c  | 28 +++++++++++++++++++---------
 3 files changed, 37 insertions(+), 17 deletions(-)

diff --git a/drivers/common/cnxk/hw/cpt.h b/drivers/common/cnxk/hw/cpt.h
index ccc7af4..412dd76 100644
--- a/drivers/common/cnxk/hw/cpt.h
+++ b/drivers/common/cnxk/hw/cpt.h
@@ -215,6 +215,8 @@ union cpt_res_s {
 
 		uint64_t reserved_64_127;
 	} cn9k;
+
+	uint64_t u64[2];
 };
 
 /* [CN10K, .) */
diff --git a/drivers/crypto/cnxk/cn10k_cryptodev_ops.c b/drivers/crypto/cnxk/cn10k_cryptodev_ops.c
index 638268e..f8240e1 100644
--- a/drivers/crypto/cnxk/cn10k_cryptodev_ops.c
+++ b/drivers/crypto/cnxk/cn10k_cryptodev_ops.c
@@ -111,6 +111,10 @@ cn10k_cpt_fill_inst(struct cnxk_cpt_qp *qp, struct rte_crypto_op *ops[],
 	uint64_t w7;
 	int ret;
 
+	const union cpt_res_s res = {
+		.cn10k.compcode = CPT_COMP_NOT_DONE,
+	};
+
 	op = ops[0];
 
 	inst[0].w0.u64 = 0;
@@ -174,7 +178,7 @@ cn10k_cpt_fill_inst(struct cnxk_cpt_qp *qp, struct rte_crypto_op *ops[],
 	}
 
 	inst[0].res_addr = (uint64_t)&infl_req->res;
-	infl_req->res.cn10k.compcode = CPT_COMP_NOT_DONE;
+	__atomic_store_n(&infl_req->res.u64[0], res.u64[0], __ATOMIC_RELAXED);
 	infl_req->cop = op;
 
 	inst[0].w7.u64 = w7;
@@ -395,9 +399,9 @@ cn10k_cpt_sec_ucc_process(struct rte_crypto_op *cop,
 static inline void
 cn10k_cpt_dequeue_post_process(struct cnxk_cpt_qp *qp,
 			       struct rte_crypto_op *cop,
-			       struct cpt_inflight_req *infl_req)
+			       struct cpt_inflight_req *infl_req,
+			       struct cpt_cn10k_res_s *res)
 {
-	struct cpt_cn10k_res_s *res = (struct cpt_cn10k_res_s *)&infl_req->res;
 	const uint8_t uc_compcode = res->uc_compcode;
 	const uint8_t compcode = res->compcode;
 	unsigned int sz;
@@ -495,12 +499,15 @@ cn10k_cpt_crypto_adapter_dequeue(uintptr_t get_work1)
 	struct cpt_inflight_req *infl_req;
 	struct rte_crypto_op *cop;
 	struct cnxk_cpt_qp *qp;
+	union cpt_res_s res;
 
 	infl_req = (struct cpt_inflight_req *)(get_work1);
 	cop = infl_req->cop;
 	qp = infl_req->qp;
 
-	cn10k_cpt_dequeue_post_process(qp, infl_req->cop, infl_req);
+	res.u64[0] = __atomic_load_n(&infl_req->res.u64[0], __ATOMIC_RELAXED);
+
+	cn10k_cpt_dequeue_post_process(qp, infl_req->cop, infl_req, &res.cn10k);
 
 	if (unlikely(infl_req->op_flags & CPT_OP_FLAGS_METABUF))
 		rte_mempool_put(qp->meta_info.pool, infl_req->mdata);
@@ -515,9 +522,9 @@ cn10k_cpt_dequeue_burst(void *qptr, struct rte_crypto_op **ops, uint16_t nb_ops)
 	struct cpt_inflight_req *infl_req;
 	struct cnxk_cpt_qp *qp = qptr;
 	struct pending_queue *pend_q;
-	struct cpt_cn10k_res_s *res;
 	uint64_t infl_cnt, pq_tail;
 	struct rte_crypto_op *cop;
+	union cpt_res_s res;
 	int i;
 
 	pend_q = &qp->pend_q;
@@ -534,9 +541,10 @@ cn10k_cpt_dequeue_burst(void *qptr, struct rte_crypto_op **ops, uint16_t nb_ops)
 	for (i = 0; i < nb_ops; i++) {
 		infl_req = &pend_q->req_queue[pq_tail];
 
-		res = (struct cpt_cn10k_res_s *)&infl_req->res;
+		res.u64[0] = __atomic_load_n(&infl_req->res.u64[0],
+					     __ATOMIC_RELAXED);
 
-		if (unlikely(res->compcode == CPT_COMP_NOT_DONE)) {
+		if (unlikely(res.cn10k.compcode == CPT_COMP_NOT_DONE)) {
 			if (unlikely(rte_get_timer_cycles() >
 				     pend_q->time_out)) {
 				plt_err("Request timed out");
@@ -553,7 +561,7 @@ cn10k_cpt_dequeue_burst(void *qptr, struct rte_crypto_op **ops, uint16_t nb_ops)
 
 		ops[i] = cop;
 
-		cn10k_cpt_dequeue_post_process(qp, cop, infl_req);
+		cn10k_cpt_dequeue_post_process(qp, cop, infl_req, &res.cn10k);
 
 		if (unlikely(infl_req->op_flags & CPT_OP_FLAGS_METABUF))
 			rte_mempool_put(qp->meta_info.pool, infl_req->mdata);
diff --git a/drivers/crypto/cnxk/cn9k_cryptodev_ops.c b/drivers/crypto/cnxk/cn9k_cryptodev_ops.c
index 449208d..cf80d47 100644
--- a/drivers/crypto/cnxk/cn9k_cryptodev_ops.c
+++ b/drivers/crypto/cnxk/cn9k_cryptodev_ops.c
@@ -221,6 +221,10 @@ cn9k_cpt_enqueue_burst(void *qptr, struct rte_crypto_op **ops, uint16_t nb_ops)
 	uint64_t head;
 	int ret;
 
+	const union cpt_res_s res = {
+		.cn10k.compcode = CPT_COMP_NOT_DONE,
+	};
+
 	pend_q = &qp->pend_q;
 
 	const uint64_t lmt_base = qp->lf.lmt_base;
@@ -274,10 +278,12 @@ cn9k_cpt_enqueue_burst(void *qptr, struct rte_crypto_op **ops, uint16_t nb_ops)
 		infl_req_1->op_flags = 0;
 		infl_req_2->op_flags = 0;
 
-		infl_req_1->res.cn9k.compcode = CPT_COMP_NOT_DONE;
+		__atomic_store_n(&infl_req_1->res.u64[0], res.u64[0],
+				 __ATOMIC_RELAXED);
 		inst[0].res_addr = (uint64_t)&infl_req_1->res;
 
-		infl_req_2->res.cn9k.compcode = CPT_COMP_NOT_DONE;
+		__atomic_store_n(&infl_req_2->res.u64[0], res.u64[0],
+				 __ATOMIC_RELAXED);
 		inst[1].res_addr = (uint64_t)&infl_req_2->res;
 
 		ret = cn9k_cpt_inst_prep(qp, op_1, infl_req_1, &inst[0]);
@@ -410,9 +416,9 @@ cn9k_cpt_sec_post_process(struct rte_crypto_op *cop,
 
 static inline void
 cn9k_cpt_dequeue_post_process(struct cnxk_cpt_qp *qp, struct rte_crypto_op *cop,
-			      struct cpt_inflight_req *infl_req)
+			      struct cpt_inflight_req *infl_req,
+			      struct cpt_cn9k_res_s *res)
 {
-	struct cpt_cn9k_res_s *res = (struct cpt_cn9k_res_s *)&infl_req->res;
 	unsigned int sz;
 
 	if (likely(res->compcode == CPT_COMP_GOOD)) {
@@ -492,12 +498,15 @@ cn9k_cpt_crypto_adapter_dequeue(uintptr_t get_work1)
 	struct cpt_inflight_req *infl_req;
 	struct rte_crypto_op *cop;
 	struct cnxk_cpt_qp *qp;
+	union cpt_res_s res;
 
 	infl_req = (struct cpt_inflight_req *)(get_work1);
 	cop = infl_req->cop;
 	qp = infl_req->qp;
 
-	cn9k_cpt_dequeue_post_process(qp, infl_req->cop, infl_req);
+	res.u64[0] = __atomic_load_n(&infl_req->res.u64[0], __ATOMIC_RELAXED);
+
+	cn9k_cpt_dequeue_post_process(qp, infl_req->cop, infl_req, &res.cn9k);
 
 	if (unlikely(infl_req->op_flags & CPT_OP_FLAGS_METABUF))
 		rte_mempool_put(qp->meta_info.pool, infl_req->mdata);
@@ -512,9 +521,9 @@ cn9k_cpt_dequeue_burst(void *qptr, struct rte_crypto_op **ops, uint16_t nb_ops)
 	struct cpt_inflight_req *infl_req;
 	struct cnxk_cpt_qp *qp = qptr;
 	struct pending_queue *pend_q;
-	struct cpt_cn9k_res_s *res;
 	uint64_t infl_cnt, pq_tail;
 	struct rte_crypto_op *cop;
+	union cpt_res_s res;
 	int i;
 
 	pend_q = &qp->pend_q;
@@ -531,9 +540,10 @@ cn9k_cpt_dequeue_burst(void *qptr, struct rte_crypto_op **ops, uint16_t nb_ops)
 	for (i = 0; i < nb_ops; i++) {
 		infl_req = &pend_q->req_queue[pq_tail];
 
-		res = (struct cpt_cn9k_res_s *)&infl_req->res;
+		res.u64[0] = __atomic_load_n(&infl_req->res.u64[0],
+					     __ATOMIC_RELAXED);
 
-		if (unlikely(res->compcode == CPT_COMP_NOT_DONE)) {
+		if (unlikely(res.cn9k.compcode == CPT_COMP_NOT_DONE)) {
 			if (unlikely(rte_get_timer_cycles() >
 				     pend_q->time_out)) {
 				plt_err("Request timed out");
@@ -550,7 +560,7 @@ cn9k_cpt_dequeue_burst(void *qptr, struct rte_crypto_op **ops, uint16_t nb_ops)
 
 		ops[i] = cop;
 
-		cn9k_cpt_dequeue_post_process(qp, cop, infl_req);
+		cn9k_cpt_dequeue_post_process(qp, cop, infl_req, &res.cn9k);
 
 		if (unlikely(infl_req->op_flags & CPT_OP_FLAGS_METABUF))
 			rte_mempool_put(qp->meta_info.pool, infl_req->mdata);
-- 
2.7.4

[PATCH v3 21/29] crypto/cnxk: add more info on command timeout

[Anoob Joseph]

Print more info when command timeout happens. Print software and
hardware queue information.

Signed-off-by: Anoob Joseph <anoobj@marvell.com>
Signed-off-by: Tejasree Kondoj <ktejasree@marvell.com>
---
 drivers/common/cnxk/hw/cpt.h              | 11 ++++++++
 drivers/crypto/cnxk/cn10k_cryptodev_ops.c |  1 +
 drivers/crypto/cnxk/cn9k_cryptodev_ops.c  |  1 +
 drivers/crypto/cnxk/cnxk_cryptodev_ops.c  | 43 +++++++++++++++++++++++++++++++
 drivers/crypto/cnxk/cnxk_cryptodev_ops.h  |  1 +
 5 files changed, 57 insertions(+)

diff --git a/drivers/common/cnxk/hw/cpt.h b/drivers/common/cnxk/hw/cpt.h
index 412dd76..4d9c896 100644
--- a/drivers/common/cnxk/hw/cpt.h
+++ b/drivers/common/cnxk/hw/cpt.h
@@ -91,6 +91,17 @@ union cpt_lf_inprog {
 	} s;
 };
 
+union cpt_lf_q_inst_ptr {
+	uint64_t u;
+	struct cpt_lf_q_inst_ptr_s {
+		uint64_t dq_ptr : 20;
+		uint64_t reserved_20_31 : 12;
+		uint64_t nq_ptr : 20;
+		uint64_t reserved_52_62 : 11;
+		uint64_t xq_xor : 1;
+	} s;
+};
+
 union cpt_lf_q_base {
 	uint64_t u;
 	struct cpt_lf_q_base_s {
diff --git a/drivers/crypto/cnxk/cn10k_cryptodev_ops.c b/drivers/crypto/cnxk/cn10k_cryptodev_ops.c
index f8240e1..1905ea3 100644
--- a/drivers/crypto/cnxk/cn10k_cryptodev_ops.c
+++ b/drivers/crypto/cnxk/cn10k_cryptodev_ops.c
@@ -548,6 +548,7 @@ cn10k_cpt_dequeue_burst(void *qptr, struct rte_crypto_op **ops, uint16_t nb_ops)
 			if (unlikely(rte_get_timer_cycles() >
 				     pend_q->time_out)) {
 				plt_err("Request timed out");
+				cnxk_cpt_dump_on_err(qp);
 				pend_q->time_out = rte_get_timer_cycles() +
 						   DEFAULT_COMMAND_TIMEOUT *
 							   rte_get_timer_hz();
diff --git a/drivers/crypto/cnxk/cn9k_cryptodev_ops.c b/drivers/crypto/cnxk/cn9k_cryptodev_ops.c
index cf80d47..ac1953b 100644
--- a/drivers/crypto/cnxk/cn9k_cryptodev_ops.c
+++ b/drivers/crypto/cnxk/cn9k_cryptodev_ops.c
@@ -547,6 +547,7 @@ cn9k_cpt_dequeue_burst(void *qptr, struct rte_crypto_op **ops, uint16_t nb_ops)
 			if (unlikely(rte_get_timer_cycles() >
 				     pend_q->time_out)) {
 				plt_err("Request timed out");
+				cnxk_cpt_dump_on_err(qp);
 				pend_q->time_out = rte_get_timer_cycles() +
 						   DEFAULT_COMMAND_TIMEOUT *
 							   rte_get_timer_hz();
diff --git a/drivers/crypto/cnxk/cnxk_cryptodev_ops.c b/drivers/crypto/cnxk/cnxk_cryptodev_ops.c
index 7953a08..0ce54d7 100644
--- a/drivers/crypto/cnxk/cnxk_cryptodev_ops.c
+++ b/drivers/crypto/cnxk/cnxk_cryptodev_ops.c
@@ -703,3 +703,46 @@ cnxk_ae_session_cfg(struct rte_cryptodev *dev,
 
 	return 0;
 }
+
+void
+cnxk_cpt_dump_on_err(struct cnxk_cpt_qp *qp)
+{
+	struct pending_queue *pend_q = &qp->pend_q;
+	uint64_t inflight, enq_ptr, deq_ptr, insts;
+	union cpt_lf_q_inst_ptr inst_ptr;
+	union cpt_lf_inprog lf_inprog;
+
+	plt_print("Lcore ID: %d, LF/QP ID: %d", rte_lcore_id(), qp->lf.lf_id);
+	plt_print("");
+	plt_print("S/w pending queue:");
+	plt_print("\tHead: %ld", pend_q->head);
+	plt_print("\tTail: %ld", pend_q->tail);
+	plt_print("\tMask: 0x%lx", pend_q->pq_mask);
+	plt_print("\tInflight count: %ld",
+		  pending_queue_infl_cnt(pend_q->head, pend_q->tail,
+					 pend_q->pq_mask));
+
+	plt_print("");
+	plt_print("H/w pending queue:");
+
+	lf_inprog.u = plt_read64(qp->lf.rbase + CPT_LF_INPROG);
+	inflight = lf_inprog.s.inflight;
+	plt_print("\tInflight in engines: %ld", inflight);
+
+	inst_ptr.u = plt_read64(qp->lf.rbase + CPT_LF_Q_INST_PTR);
+
+	enq_ptr = inst_ptr.s.nq_ptr;
+	deq_ptr = inst_ptr.s.dq_ptr;
+
+	if (enq_ptr >= deq_ptr)
+		insts = enq_ptr - deq_ptr;
+	else
+		insts = (enq_ptr + pend_q->pq_mask + 1 + 320 + 40) - deq_ptr;
+
+	plt_print("\tNQ ptr: 0x%lx", enq_ptr);
+	plt_print("\tDQ ptr: 0x%lx", deq_ptr);
+	plt_print("Insts waiting in CPT: %ld", insts);
+
+	plt_print("");
+	roc_cpt_afs_print(qp->lf.roc_cpt);
+}
diff --git a/drivers/crypto/cnxk/cnxk_cryptodev_ops.h b/drivers/crypto/cnxk/cnxk_cryptodev_ops.h
index 0336ae1..e521f07 100644
--- a/drivers/crypto/cnxk/cnxk_cryptodev_ops.h
+++ b/drivers/crypto/cnxk/cnxk_cryptodev_ops.h
@@ -122,6 +122,7 @@ int cnxk_ae_session_cfg(struct rte_cryptodev *dev,
 			struct rte_crypto_asym_xform *xform,
 			struct rte_cryptodev_asym_session *sess,
 			struct rte_mempool *pool);
+void cnxk_cpt_dump_on_err(struct cnxk_cpt_qp *qp);
 
 static inline union rte_event_crypto_metadata *
 cnxk_event_crypto_mdata_get(struct rte_crypto_op *op)
-- 
2.7.4

Re: [PATCH v3 21/29] crypto/cnxk: add more info on command timeout

[Thomas Monjalon]

17/12/2021 10:20, Anoob Joseph:
> Print more info when command timeout happens. Print software and
> hardware queue information.
> 
> Signed-off-by: Anoob Joseph <anoobj@marvell.com>
> Signed-off-by: Tejasree Kondoj <ktejasree@marvell.com>
> ---
> +void
> +cnxk_cpt_dump_on_err(struct cnxk_cpt_qp *qp)
> +{
> +	struct pending_queue *pend_q = &qp->pend_q;
> +	uint64_t inflight, enq_ptr, deq_ptr, insts;
> +	union cpt_lf_q_inst_ptr inst_ptr;
> +	union cpt_lf_inprog lf_inprog;
> +
> +	plt_print("Lcore ID: %d, LF/QP ID: %d", rte_lcore_id(), qp->lf.lf_id);
> +	plt_print("");
> +	plt_print("S/w pending queue:");
> +	plt_print("\tHead: %ld", pend_q->head);
> +	plt_print("\tTail: %ld", pend_q->tail);
> +	plt_print("\tMask: 0x%lx", pend_q->pq_mask);
> +	plt_print("\tInflight count: %ld",
> +		  pending_queue_infl_cnt(pend_q->head, pend_q->tail,
> +					 pend_q->pq_mask));
> +
> +	plt_print("");
> +	plt_print("H/w pending queue:");
> +
> +	lf_inprog.u = plt_read64(qp->lf.rbase + CPT_LF_INPROG);
> +	inflight = lf_inprog.s.inflight;
> +	plt_print("\tInflight in engines: %ld", inflight);
> +
> +	inst_ptr.u = plt_read64(qp->lf.rbase + CPT_LF_Q_INST_PTR);
> +
> +	enq_ptr = inst_ptr.s.nq_ptr;
> +	deq_ptr = inst_ptr.s.dq_ptr;
> +
> +	if (enq_ptr >= deq_ptr)
> +		insts = enq_ptr - deq_ptr;
> +	else
> +		insts = (enq_ptr + pend_q->pq_mask + 1 + 320 + 40) - deq_ptr;
> +
> +	plt_print("\tNQ ptr: 0x%lx", enq_ptr);
> +	plt_print("\tDQ ptr: 0x%lx", deq_ptr);
> +	plt_print("Insts waiting in CPT: %ld", insts);
> +
> +	plt_print("");
> +	roc_cpt_afs_print(qp->lf.roc_cpt);
> +}

This functions is wrong. You cannot print 64-bit values with %l.
In 32-bit mode, compilation will fail.
Please use PRIx64.

Note: this mistake is warned by the script devtools/checkpatches.sh
  Warning in drivers/crypto/cnxk/cnxk_cryptodev_ops.c:
  Using %l format, prefer %PRI*64 if type is [u]int64_t

I will wait for the next-crypto tree to be fixed.

RE: [EXT] Re: [PATCH v3 21/29] crypto/cnxk: add more info on command timeout

[Akhil Goyal]

> 17/12/2021 10:20, Anoob Joseph:
> > Print more info when command timeout happens. Print software and
> > hardware queue information.
> >
> > Signed-off-by: Anoob Joseph <anoobj@marvell.com>
> > Signed-off-by: Tejasree Kondoj <ktejasree@marvell.com>
> > ---
> > +void
> > +cnxk_cpt_dump_on_err(struct cnxk_cpt_qp *qp)
> > +{
> > +	struct pending_queue *pend_q = &qp->pend_q;
> > +	uint64_t inflight, enq_ptr, deq_ptr, insts;
> > +	union cpt_lf_q_inst_ptr inst_ptr;
> > +	union cpt_lf_inprog lf_inprog;
> > +
> > +	plt_print("Lcore ID: %d, LF/QP ID: %d", rte_lcore_id(), qp->lf.lf_id);
> > +	plt_print("");
> > +	plt_print("S/w pending queue:");
> > +	plt_print("\tHead: %ld", pend_q->head);
> > +	plt_print("\tTail: %ld", pend_q->tail);
> > +	plt_print("\tMask: 0x%lx", pend_q->pq_mask);
> > +	plt_print("\tInflight count: %ld",
> > +		  pending_queue_infl_cnt(pend_q->head, pend_q->tail,
> > +					 pend_q->pq_mask));
> > +
> > +	plt_print("");
> > +	plt_print("H/w pending queue:");
> > +
> > +	lf_inprog.u = plt_read64(qp->lf.rbase + CPT_LF_INPROG);
> > +	inflight = lf_inprog.s.inflight;
> > +	plt_print("\tInflight in engines: %ld", inflight);
> > +
> > +	inst_ptr.u = plt_read64(qp->lf.rbase + CPT_LF_Q_INST_PTR);
> > +
> > +	enq_ptr = inst_ptr.s.nq_ptr;
> > +	deq_ptr = inst_ptr.s.dq_ptr;
> > +
> > +	if (enq_ptr >= deq_ptr)
> > +		insts = enq_ptr - deq_ptr;
> > +	else
> > +		insts = (enq_ptr + pend_q->pq_mask + 1 + 320 + 40) - deq_ptr;
> > +
> > +	plt_print("\tNQ ptr: 0x%lx", enq_ptr);
> > +	plt_print("\tDQ ptr: 0x%lx", deq_ptr);
> > +	plt_print("Insts waiting in CPT: %ld", insts);
> > +
> > +	plt_print("");
> > +	roc_cpt_afs_print(qp->lf.roc_cpt);
> > +}
> 
> This functions is wrong. You cannot print 64-bit values with %l.
> In 32-bit mode, compilation will fail.
> Please use PRIx64.
> 
> Note: this mistake is warned by the script devtools/checkpatches.sh
>   Warning in drivers/crypto/cnxk/cnxk_cryptodev_ops.c:
>   Using %l format, prefer %PRI*64 if type is [u]int64_t
I believe there is something wrong in the reporting;
it said 1 warning which is for spell check of head
and in the end a line is added for another warning.
I skipped this issue as it was a false positive for spelling. Did not see the last line.

WARNING:TYPO_SPELLING: 'tHead' may be misspelled - perhaps 'thread'?
#157: FILE: drivers/crypto/cnxk/cnxk_cryptodev_ops.c:718:
+	plt_print("	Head: %ld", pend_q->head);

total: 0 errors, 1 warnings, 84 lines checked
	            ^^^^^^^^^^	
Warning in drivers/crypto/cnxk/cnxk_cryptodev_ops.c:
Using %l format, prefer %PRI*64 if type is [u]int64_t

> 
> I will wait for the next-crypto tree to be fixed.
> 
Following changes are mage in this patch on crypto tree.
diff --git a/drivers/crypto/cnxk/cnxk_cryptodev_ops.c b/drivers/crypto/cnxk/cnxk_cryptodev_ops.c
index 0ce54d7bf0..67a2d9b08e 100644
--- a/drivers/crypto/cnxk/cnxk_cryptodev_ops.c
+++ b/drivers/crypto/cnxk/cnxk_cryptodev_ops.c
@@ -715,10 +715,10 @@ cnxk_cpt_dump_on_err(struct cnxk_cpt_qp *qp)
        plt_print("Lcore ID: %d, LF/QP ID: %d", rte_lcore_id(), qp->lf.lf_id);
        plt_print("");
        plt_print("S/w pending queue:");
-       plt_print("\tHead: %ld", pend_q->head);
-       plt_print("\tTail: %ld", pend_q->tail);
-       plt_print("\tMask: 0x%lx", pend_q->pq_mask);
-       plt_print("\tInflight count: %ld",
+       plt_print("\tHead: %"PRIu64"", pend_q->head);
+       plt_print("\tTail: %"PRIu64"", pend_q->tail);
+       plt_print("\tMask: 0x%"PRIx64"", pend_q->pq_mask);
+       plt_print("\tInflight count: %"PRIu64"",
                  pending_queue_infl_cnt(pend_q->head, pend_q->tail,
                                         pend_q->pq_mask));

@@ -727,7 +727,7 @@ cnxk_cpt_dump_on_err(struct cnxk_cpt_qp *qp)

        lf_inprog.u = plt_read64(qp->lf.rbase + CPT_LF_INPROG);
        inflight = lf_inprog.s.inflight;
-       plt_print("\tInflight in engines: %ld", inflight);
+       plt_print("\tInflight in engines: %"PRIu64"", inflight);

        inst_ptr.u = plt_read64(qp->lf.rbase + CPT_LF_Q_INST_PTR);

@@ -739,9 +739,9 @@ cnxk_cpt_dump_on_err(struct cnxk_cpt_qp *qp)
        else
                insts = (enq_ptr + pend_q->pq_mask + 1 + 320 + 40) - deq_ptr;

-       plt_print("\tNQ ptr: 0x%lx", enq_ptr);
-       plt_print("\tDQ ptr: 0x%lx", deq_ptr);
-       plt_print("Insts waiting in CPT: %ld", insts);
+       plt_print("\tNQ ptr: 0x%"PRIx64"", enq_ptr);
+       plt_print("\tDQ ptr: 0x%"PRIx64"", deq_ptr);
+       plt_print("Insts waiting in CPT: %"PRIu64"", insts);

        plt_print("");
        roc_cpt_afs_print(qp->lf.roc_cpt);

Re: [EXT] Re: [PATCH v3 21/29] crypto/cnxk: add more info on command timeout

[Thomas Monjalon]

21/01/2022 10:16, Akhil Goyal:
> > Note: this mistake is warned by the script devtools/checkpatches.sh
> >   Warning in drivers/crypto/cnxk/cnxk_cryptodev_ops.c:
> >   Using %l format, prefer %PRI*64 if type is [u]int64_t
> I believe there is something wrong in the reporting;
> it said 1 warning which is for spell check of head
> and in the end a line is added for another warning.
> I skipped this issue as it was a false positive for spelling. Did not see the last line.
> 
> WARNING:TYPO_SPELLING: 'tHead' may be misspelled - perhaps 'thread'?
> #157: FILE: drivers/crypto/cnxk/cnxk_cryptodev_ops.c:718:
> +	plt_print("	Head: %ld", pend_q->head);
> 
> total: 0 errors, 1 warnings, 84 lines checked
> 	            ^^^^^^^^^^	
> Warning in drivers/crypto/cnxk/cnxk_cryptodev_ops.c:
> Using %l format, prefer %PRI*64 if type is [u]int64_t

That's because they are 2 different checks.
The first total is from checkpatch.pl.

[PATCH v3 22/29] crypto/cnxk: support lookaside IPsec AES-CTR

[Anoob Joseph]

From: Tejasree Kondoj <ktejasree@marvell.com>

Adding AES-CTR support to cnxk CPT in
lookaside IPsec mode.

Signed-off-by: Tejasree Kondoj <ktejasree@marvell.com>
---
 doc/guides/cryptodevs/cnxk.rst                    |  2 ++
 doc/guides/rel_notes/release_22_03.rst            |  1 +
 drivers/common/cnxk/cnxk_security.c               |  6 ++++++
 drivers/crypto/cnxk/cn9k_ipsec.c                  |  3 +++
 drivers/crypto/cnxk/cnxk_cryptodev.h              |  2 +-
 drivers/crypto/cnxk/cnxk_cryptodev_capabilities.c | 20 ++++++++++++++++++++
 drivers/crypto/cnxk/cnxk_ipsec.h                  |  3 ++-
 7 files changed, 35 insertions(+), 2 deletions(-)

diff --git a/doc/guides/cryptodevs/cnxk.rst b/doc/guides/cryptodevs/cnxk.rst
index c49a779..1239155 100644
--- a/doc/guides/cryptodevs/cnxk.rst
+++ b/doc/guides/cryptodevs/cnxk.rst
@@ -261,6 +261,7 @@ Cipher algorithms
 +++++++++++++++++
 
 * AES-128/192/256-CBC
+* AES-128/192/256-CTR
 
 Auth algorithms
 +++++++++++++++
@@ -288,6 +289,7 @@ Cipher algorithms
 +++++++++++++++++
 
 * AES-128/192/256-CBC
+* AES-128/192/256-CTR
 
 Auth algorithms
 +++++++++++++++
diff --git a/doc/guides/rel_notes/release_22_03.rst b/doc/guides/rel_notes/release_22_03.rst
index 8df9092..4b272e4 100644
--- a/doc/guides/rel_notes/release_22_03.rst
+++ b/doc/guides/rel_notes/release_22_03.rst
@@ -60,6 +60,7 @@ New Features
   * Added SHA256-HMAC support in lookaside protocol (IPsec) for CN10K.
   * Added SHA384-HMAC support in lookaside protocol (IPsec) for CN9K & CN10K.
   * Added SHA512-HMAC support in lookaside protocol (IPsec) for CN9K & CN10K.
+  * Added AES-CTR support in lookaside protocol (IPsec) for CN9K & CN10K.
 
 
 Removed Items
diff --git a/drivers/common/cnxk/cnxk_security.c b/drivers/common/cnxk/cnxk_security.c
index 1c86f82..0d4baa9 100644
--- a/drivers/common/cnxk/cnxk_security.c
+++ b/drivers/common/cnxk/cnxk_security.c
@@ -123,6 +123,9 @@ ot_ipsec_sa_common_param_fill(union roc_ot_ipsec_sa_word2 *w2,
 		case RTE_CRYPTO_CIPHER_AES_CBC:
 			w2->s.enc_type = ROC_IE_OT_SA_ENC_AES_CBC;
 			break;
+		case RTE_CRYPTO_CIPHER_AES_CTR:
+			w2->s.enc_type = ROC_IE_OT_SA_ENC_AES_CTR;
+			break;
 		default:
 			return -ENOTSUP;
 		}
@@ -630,6 +633,9 @@ onf_ipsec_sa_common_param_fill(struct roc_ie_onf_sa_ctl *ctl, uint8_t *salt,
 		case RTE_CRYPTO_CIPHER_AES_CBC:
 			ctl->enc_type = ROC_IE_ON_SA_ENC_AES_CBC;
 			break;
+		case RTE_CRYPTO_CIPHER_AES_CTR:
+			ctl->enc_type = ROC_IE_ON_SA_ENC_AES_CTR;
+			break;
 		default:
 			return -ENOTSUP;
 		}
diff --git a/drivers/crypto/cnxk/cn9k_ipsec.c b/drivers/crypto/cnxk/cn9k_ipsec.c
index c27845c..1e2269c 100644
--- a/drivers/crypto/cnxk/cn9k_ipsec.c
+++ b/drivers/crypto/cnxk/cn9k_ipsec.c
@@ -166,6 +166,9 @@ ipsec_sa_ctl_set(struct rte_security_ipsec_xform *ipsec,
 	} else if (cipher_xform->cipher.algo == RTE_CRYPTO_CIPHER_AES_CBC) {
 		ctl->enc_type = ROC_IE_ON_SA_ENC_AES_CBC;
 		aes_key_len = cipher_xform->cipher.key.length;
+	} else if (cipher_xform->cipher.algo == RTE_CRYPTO_CIPHER_AES_CTR) {
+		ctl->enc_type = ROC_IE_ON_SA_ENC_AES_CTR;
+		aes_key_len = cipher_xform->cipher.key.length;
 	} else {
 		return -ENOTSUP;
 	}
diff --git a/drivers/crypto/cnxk/cnxk_cryptodev.h b/drivers/crypto/cnxk/cnxk_cryptodev.h
index f701c26..4a1e377 100644
--- a/drivers/crypto/cnxk/cnxk_cryptodev.h
+++ b/drivers/crypto/cnxk/cnxk_cryptodev.h
@@ -11,7 +11,7 @@
 #include "roc_cpt.h"
 
 #define CNXK_CPT_MAX_CAPS	 34
-#define CNXK_SEC_CRYPTO_MAX_CAPS 8
+#define CNXK_SEC_CRYPTO_MAX_CAPS 9
 #define CNXK_SEC_MAX_CAPS	 5
 #define CNXK_AE_EC_ID_MAX	 8
 /**
diff --git a/drivers/crypto/cnxk/cnxk_cryptodev_capabilities.c b/drivers/crypto/cnxk/cnxk_cryptodev_capabilities.c
index 0fdd91a..fae433e 100644
--- a/drivers/crypto/cnxk/cnxk_cryptodev_capabilities.c
+++ b/drivers/crypto/cnxk/cnxk_cryptodev_capabilities.c
@@ -754,6 +754,26 @@ static const struct rte_cryptodev_capabilities sec_caps_aes[] = {
 			}, }
 		}, }
 	},
+	{	/* AES CTR */
+		.op = RTE_CRYPTO_OP_TYPE_SYMMETRIC,
+		{.sym = {
+			.xform_type = RTE_CRYPTO_SYM_XFORM_CIPHER,
+			{.cipher = {
+				.algo = RTE_CRYPTO_CIPHER_AES_CTR,
+				.block_size = 16,
+				.key_size = {
+					.min = 16,
+					.max = 32,
+					.increment = 8
+				},
+				.iv_size = {
+					.min = 12,
+					.max = 16,
+					.increment = 4
+				}
+			}, }
+		}, }
+	},
 	{	/* AES CBC */
 		.op = RTE_CRYPTO_OP_TYPE_SYMMETRIC,
 		{.sym = {
diff --git a/drivers/crypto/cnxk/cnxk_ipsec.h b/drivers/crypto/cnxk/cnxk_ipsec.h
index 426eaa8..f5a51b5 100644
--- a/drivers/crypto/cnxk/cnxk_ipsec.h
+++ b/drivers/crypto/cnxk/cnxk_ipsec.h
@@ -20,7 +20,8 @@ struct cnxk_cpt_inst_tmpl {
 static inline int
 ipsec_xform_cipher_verify(struct rte_crypto_sym_xform *crypto_xform)
 {
-	if (crypto_xform->cipher.algo == RTE_CRYPTO_CIPHER_AES_CBC) {
+	if (crypto_xform->cipher.algo == RTE_CRYPTO_CIPHER_AES_CBC ||
+	    crypto_xform->cipher.algo == RTE_CRYPTO_CIPHER_AES_CTR) {
 		switch (crypto_xform->cipher.key.length) {
 		case 16:
 		case 24:
-- 
2.7.4

[PATCH v3 23/29] crypto/cnxk: fix extend tail calculation

[Anoob Joseph]

If the packet size to be incremented after IPsec processing is less
than size of hdr (size incremented before submitting), then extend_tail
can become negative. Allow negative values for the variable.

Fixes: 67a87e89561c ("crypto/cnxk: add cn9k lookaside IPsec datapath")
Cc: marchana@marvell.com

Signed-off-by: Anoob Joseph <anoobj@marvell.com>
---
 drivers/crypto/cnxk/cn9k_ipsec_la_ops.h | 6 ++++--
 1 file changed, 4 insertions(+), 2 deletions(-)

diff --git a/drivers/crypto/cnxk/cn9k_ipsec_la_ops.h b/drivers/crypto/cnxk/cn9k_ipsec_la_ops.h
index 2dc8913..2b0261e 100644
--- a/drivers/crypto/cnxk/cn9k_ipsec_la_ops.h
+++ b/drivers/crypto/cnxk/cn9k_ipsec_la_ops.h
@@ -77,9 +77,10 @@ process_outb_sa(struct rte_crypto_op *cop, struct cn9k_ipsec_sa *sa,
 	const unsigned int hdr_len = sizeof(struct roc_ie_on_outb_hdr);
 	struct rte_crypto_sym_op *sym_op = cop->sym;
 	struct rte_mbuf *m_src = sym_op->m_src;
-	uint32_t dlen, rlen, extend_tail;
 	struct roc_ie_on_outb_sa *out_sa;
 	struct roc_ie_on_outb_hdr *hdr;
+	uint32_t dlen, rlen;
+	int32_t extend_tail;
 
 	out_sa = &sa->out_sa;
 
@@ -88,7 +89,8 @@ process_outb_sa(struct rte_crypto_op *cop, struct cn9k_ipsec_sa *sa,
 
 	extend_tail = rlen - dlen;
 	if (unlikely(extend_tail > rte_pktmbuf_tailroom(m_src))) {
-		plt_dp_err("Not enough tail room");
+		plt_dp_err("Not enough tail room (required: %d, available: %d",
+			   extend_tail, rte_pktmbuf_tailroom(m_src));
 		return -ENOMEM;
 	}
 
-- 
2.7.4

[PATCH v3 24/29] crypto/cnxk: add aes xcbc and null cipher

[Anoob Joseph]

Add support for AES XCBC and NULL cipher.

Signed-off-by: Anoob Joseph <anoobj@marvell.com>
---
 doc/guides/cryptodevs/cnxk.rst                    |  4 +
 doc/guides/rel_notes/release_22_03.rst            |  2 +
 drivers/common/cnxk/cnxk_security.c               | 48 ++++++++----
 drivers/common/cnxk/roc_ie_on.h                   | 10 +++
 drivers/crypto/cnxk/cn9k_ipsec.c                  | 93 ++++++++++++++++-------
 drivers/crypto/cnxk/cnxk_cryptodev.h              |  2 +-
 drivers/crypto/cnxk/cnxk_cryptodev_capabilities.c | 45 +++++++++++
 drivers/crypto/cnxk/cnxk_ipsec.h                  |  7 ++
 8 files changed, 169 insertions(+), 42 deletions(-)

diff --git a/doc/guides/cryptodevs/cnxk.rst b/doc/guides/cryptodevs/cnxk.rst
index 1239155..6e844f5 100644
--- a/doc/guides/cryptodevs/cnxk.rst
+++ b/doc/guides/cryptodevs/cnxk.rst
@@ -260,6 +260,7 @@ AEAD algorithms
 Cipher algorithms
 +++++++++++++++++
 
+* NULL
 * AES-128/192/256-CBC
 * AES-128/192/256-CTR
 
@@ -270,6 +271,7 @@ Auth algorithms
 * SHA256-128-HMAC
 * SHA384-192-HMAC
 * SHA512-256-HMAC
+* AES-XCBC-96
 
 CN10XX Features supported
 ~~~~~~~~~~~~~~~~~~~~~~~~~
@@ -288,6 +290,7 @@ AEAD algorithms
 Cipher algorithms
 +++++++++++++++++
 
+* NULL
 * AES-128/192/256-CBC
 * AES-128/192/256-CTR
 
@@ -299,3 +302,4 @@ Auth algorithms
 * SHA256-128-HMAC
 * SHA384-192-HMAC
 * SHA512-256-HMAC
+* AES-XCBC-96
diff --git a/doc/guides/rel_notes/release_22_03.rst b/doc/guides/rel_notes/release_22_03.rst
index 4b272e4..e8fec00 100644
--- a/doc/guides/rel_notes/release_22_03.rst
+++ b/doc/guides/rel_notes/release_22_03.rst
@@ -61,6 +61,8 @@ New Features
   * Added SHA384-HMAC support in lookaside protocol (IPsec) for CN9K & CN10K.
   * Added SHA512-HMAC support in lookaside protocol (IPsec) for CN9K & CN10K.
   * Added AES-CTR support in lookaside protocol (IPsec) for CN9K & CN10K.
+  * Added NULL cipher support in lookaside protocol (IPsec) for CN9K & CN10K.
+  * Added AES-XCBC support in lookaside protocol (IPsec) for CN9K & CN10K.
 
 
 Removed Items
diff --git a/drivers/common/cnxk/cnxk_security.c b/drivers/common/cnxk/cnxk_security.c
index 0d4baa9..6ebf084 100644
--- a/drivers/common/cnxk/cnxk_security.c
+++ b/drivers/common/cnxk/cnxk_security.c
@@ -120,6 +120,9 @@ ot_ipsec_sa_common_param_fill(union roc_ot_ipsec_sa_word2 *w2,
 		}
 	} else {
 		switch (cipher_xfrm->cipher.algo) {
+		case RTE_CRYPTO_CIPHER_NULL:
+			w2->s.enc_type = ROC_IE_OT_SA_ENC_NULL;
+			break;
 		case RTE_CRYPTO_CIPHER_AES_CBC:
 			w2->s.enc_type = ROC_IE_OT_SA_ENC_AES_CBC;
 			break;
@@ -146,11 +149,19 @@ ot_ipsec_sa_common_param_fill(union roc_ot_ipsec_sa_word2 *w2,
 		case RTE_CRYPTO_AUTH_SHA512_HMAC:
 			w2->s.auth_type = ROC_IE_OT_SA_AUTH_SHA2_512;
 			break;
+		case RTE_CRYPTO_AUTH_AES_XCBC_MAC:
+			w2->s.auth_type = ROC_IE_OT_SA_AUTH_AES_XCBC_128;
+			break;
 		default:
 			return -ENOTSUP;
 		}
 
-		ipsec_hmac_opad_ipad_gen(auth_xfrm, hmac_opad_ipad);
+		if (auth_xfrm->auth.algo == RTE_CRYPTO_AUTH_AES_XCBC_MAC) {
+			const uint8_t *auth_key = auth_xfrm->auth.key.data;
+			roc_aes_xcbc_key_derive(auth_key, hmac_opad_ipad);
+		} else {
+			ipsec_hmac_opad_ipad_gen(auth_xfrm, hmac_opad_ipad);
+		}
 
 		tmp_key = (uint64_t *)hmac_opad_ipad;
 		for (i = 0;
@@ -174,18 +185,26 @@ ot_ipsec_sa_common_param_fill(union roc_ot_ipsec_sa_word2 *w2,
 	for (i = 0; i < (int)(ROC_CTX_MAX_CKEY_LEN / sizeof(uint64_t)); i++)
 		tmp_key[i] = rte_be_to_cpu_64(tmp_key[i]);
 
-	switch (length) {
-	case ROC_CPT_AES128_KEY_LEN:
-		w2->s.aes_key_len = ROC_IE_SA_AES_KEY_LEN_128;
-		break;
-	case ROC_CPT_AES192_KEY_LEN:
-		w2->s.aes_key_len = ROC_IE_SA_AES_KEY_LEN_192;
-		break;
-	case ROC_CPT_AES256_KEY_LEN:
-		w2->s.aes_key_len = ROC_IE_SA_AES_KEY_LEN_256;
-		break;
-	default:
-		return -EINVAL;
+	/* Set AES key length */
+	if (w2->s.enc_type == ROC_IE_OT_SA_ENC_AES_CBC ||
+	    w2->s.enc_type == ROC_IE_OT_SA_ENC_AES_CCM ||
+	    w2->s.enc_type == ROC_IE_OT_SA_ENC_AES_CTR ||
+	    w2->s.enc_type == ROC_IE_OT_SA_ENC_AES_GCM ||
+	    w2->s.auth_type == ROC_IE_OT_SA_AUTH_AES_GMAC) {
+		switch (length) {
+		case ROC_CPT_AES128_KEY_LEN:
+			w2->s.aes_key_len = ROC_IE_SA_AES_KEY_LEN_128;
+			break;
+		case ROC_CPT_AES192_KEY_LEN:
+			w2->s.aes_key_len = ROC_IE_SA_AES_KEY_LEN_192;
+			break;
+		case ROC_CPT_AES256_KEY_LEN:
+			w2->s.aes_key_len = ROC_IE_SA_AES_KEY_LEN_256;
+			break;
+		default:
+			plt_err("Invalid AES key length");
+			return -EINVAL;
+		}
 	}
 
 	if (ipsec_xfrm->life.packets_soft_limit != 0 ||
@@ -815,6 +834,9 @@ cnxk_ipsec_icvlen_get(enum rte_crypto_cipher_algorithm c_algo,
 	case RTE_CRYPTO_AUTH_SHA512_HMAC:
 		icv = 32;
 		break;
+	case RTE_CRYPTO_AUTH_AES_XCBC_MAC:
+		icv = 12;
+		break;
 	default:
 		break;
 	}
diff --git a/drivers/common/cnxk/roc_ie_on.h b/drivers/common/cnxk/roc_ie_on.h
index 817ef33..cb56a70 100644
--- a/drivers/common/cnxk/roc_ie_on.h
+++ b/drivers/common/cnxk/roc_ie_on.h
@@ -181,6 +181,11 @@ struct roc_ie_on_outb_sa {
 			struct roc_ie_on_ip_template template;
 		} sha1;
 		struct {
+			uint8_t key[16];
+			uint8_t unused[32];
+			struct roc_ie_on_ip_template template;
+		} aes_xcbc;
+		struct {
 			uint8_t hmac_key[64];
 			uint8_t hmac_iv[64];
 			struct roc_ie_on_ip_template template;
@@ -202,6 +207,11 @@ struct roc_ie_on_inb_sa {
 			struct roc_ie_on_traffic_selector selector;
 		} sha1_or_gcm;
 		struct {
+			uint8_t key[16];
+			uint8_t unused[32];
+			struct roc_ie_on_traffic_selector selector;
+		} aes_xcbc;
+		struct {
 			uint8_t hmac_key[64];
 			uint8_t hmac_iv[64];
 			struct roc_ie_on_traffic_selector selector;
diff --git a/drivers/crypto/cnxk/cn9k_ipsec.c b/drivers/crypto/cnxk/cn9k_ipsec.c
index 1e2269c..c9f5825 100644
--- a/drivers/crypto/cnxk/cn9k_ipsec.c
+++ b/drivers/crypto/cnxk/cn9k_ipsec.c
@@ -118,7 +118,7 @@ ipsec_sa_ctl_set(struct rte_security_ipsec_xform *ipsec,
 		 struct roc_ie_on_sa_ctl *ctl)
 {
 	struct rte_crypto_sym_xform *cipher_xform, *auth_xform;
-	int aes_key_len;
+	int aes_key_len = 0;
 
 	if (ipsec->direction == RTE_SECURITY_IPSEC_SA_DIR_EGRESS) {
 		ctl->direction = ROC_IE_SA_DIR_OUTBOUND;
@@ -157,37 +157,33 @@ ipsec_sa_ctl_set(struct rte_security_ipsec_xform *ipsec,
 		return -EINVAL;
 
 	if (crypto_xform->type == RTE_CRYPTO_SYM_XFORM_AEAD) {
-		if (crypto_xform->aead.algo == RTE_CRYPTO_AEAD_AES_GCM) {
+		switch (crypto_xform->aead.algo) {
+		case RTE_CRYPTO_AEAD_AES_GCM:
 			ctl->enc_type = ROC_IE_ON_SA_ENC_AES_GCM;
 			aes_key_len = crypto_xform->aead.key.length;
-		} else {
+			break;
+		default:
+			plt_err("Unsupported AEAD algorithm");
 			return -ENOTSUP;
 		}
-	} else if (cipher_xform->cipher.algo == RTE_CRYPTO_CIPHER_AES_CBC) {
-		ctl->enc_type = ROC_IE_ON_SA_ENC_AES_CBC;
-		aes_key_len = cipher_xform->cipher.key.length;
-	} else if (cipher_xform->cipher.algo == RTE_CRYPTO_CIPHER_AES_CTR) {
-		ctl->enc_type = ROC_IE_ON_SA_ENC_AES_CTR;
-		aes_key_len = cipher_xform->cipher.key.length;
 	} else {
-		return -ENOTSUP;
-	}
-
-	switch (aes_key_len) {
-	case 16:
-		ctl->aes_key_len = ROC_IE_SA_AES_KEY_LEN_128;
-		break;
-	case 24:
-		ctl->aes_key_len = ROC_IE_SA_AES_KEY_LEN_192;
-		break;
-	case 32:
-		ctl->aes_key_len = ROC_IE_SA_AES_KEY_LEN_256;
-		break;
-	default:
-		return -EINVAL;
-	}
+		switch (cipher_xform->cipher.algo) {
+		case RTE_CRYPTO_CIPHER_NULL:
+			ctl->enc_type = ROC_IE_ON_SA_ENC_NULL;
+			break;
+		case RTE_CRYPTO_CIPHER_AES_CBC:
+			ctl->enc_type = ROC_IE_ON_SA_ENC_AES_CBC;
+			aes_key_len = cipher_xform->cipher.key.length;
+			break;
+		case RTE_CRYPTO_CIPHER_AES_CTR:
+			ctl->enc_type = ROC_IE_ON_SA_ENC_AES_CTR;
+			aes_key_len = cipher_xform->cipher.key.length;
+			break;
+		default:
+			plt_err("Unsupported cipher algorithm");
+			return -ENOTSUP;
+		}
 
-	if (crypto_xform->type != RTE_CRYPTO_SYM_XFORM_AEAD) {
 		switch (auth_xform->auth.algo) {
 		case RTE_CRYPTO_AUTH_NULL:
 			ctl->auth_type = ROC_IE_ON_SA_AUTH_NULL;
@@ -217,10 +213,33 @@ ipsec_sa_ctl_set(struct rte_security_ipsec_xform *ipsec,
 			ctl->auth_type = ROC_IE_ON_SA_AUTH_AES_XCBC_128;
 			break;
 		default:
+			plt_err("Unsupported auth algorithm");
 			return -ENOTSUP;
 		}
 	}
 
+	/* Set AES key length */
+	if (ctl->enc_type == ROC_IE_ON_SA_ENC_AES_CBC ||
+	    ctl->enc_type == ROC_IE_ON_SA_ENC_AES_CCM ||
+	    ctl->enc_type == ROC_IE_ON_SA_ENC_AES_CTR ||
+	    ctl->enc_type == ROC_IE_ON_SA_ENC_AES_GCM ||
+	    ctl->auth_type == ROC_IE_ON_SA_AUTH_AES_GMAC) {
+		switch (aes_key_len) {
+		case 16:
+			ctl->aes_key_len = ROC_IE_SA_AES_KEY_LEN_128;
+			break;
+		case 24:
+			ctl->aes_key_len = ROC_IE_SA_AES_KEY_LEN_192;
+			break;
+		case 32:
+			ctl->aes_key_len = ROC_IE_SA_AES_KEY_LEN_256;
+			break;
+		default:
+			plt_err("Invalid AES key length");
+			return -EINVAL;
+		}
+	}
+
 	if (ipsec->options.esn)
 		ctl->esn_en = 1;
 
@@ -267,8 +286,6 @@ fill_ipsec_common_sa(struct rte_security_ipsec_xform *ipsec,
 
 	if (cipher_key_len != 0)
 		memcpy(common_sa->cipher_key, cipher_key, cipher_key_len);
-	else
-		return -EINVAL;
 
 	return 0;
 }
@@ -337,7 +354,13 @@ cn9k_ipsec_outb_sa_create(struct cnxk_cpt_qp *qp,
 			ctx_len = offsetof(struct roc_ie_on_outb_sa,
 					   sha2.template);
 			break;
+		case ROC_IE_ON_SA_AUTH_AES_XCBC_128:
+			template = &out_sa->aes_xcbc.template;
+			ctx_len = offsetof(struct roc_ie_on_outb_sa,
+					   aes_xcbc.template);
+			break;
 		default:
+			plt_err("Unsupported auth algorithm");
 			return -EINVAL;
 		}
 	}
@@ -419,6 +442,9 @@ cn9k_ipsec_outb_sa_create(struct cnxk_cpt_qp *qp,
 		case RTE_CRYPTO_AUTH_SHA512_HMAC:
 			memcpy(out_sa->sha2.hmac_key, auth_key, auth_key_len);
 			break;
+		case RTE_CRYPTO_AUTH_AES_XCBC_MAC:
+			memcpy(out_sa->aes_xcbc.key, auth_key, auth_key_len);
+			break;
 		default:
 			plt_err("Unsupported auth algorithm %u",
 				auth_xform->auth.algo);
@@ -505,6 +531,11 @@ cn9k_ipsec_inb_sa_create(struct cnxk_cpt_qp *qp,
 			ctx_len = offsetof(struct roc_ie_on_inb_sa,
 					   sha2.selector);
 			break;
+		case RTE_CRYPTO_AUTH_AES_XCBC_MAC:
+			memcpy(in_sa->aes_xcbc.key, auth_key, auth_key_len);
+			ctx_len = offsetof(struct roc_ie_on_inb_sa,
+					   aes_xcbc.selector);
+			break;
 		default:
 			plt_err("Unsupported auth algorithm %u",
 				auth_xform->auth.algo);
@@ -597,6 +628,12 @@ cn9k_ipsec_xform_verify(struct rte_security_ipsec_xform *ipsec,
 				plt_err("Transport mode AES-CBC SHA2 HMAC 512 is not supported");
 				return -ENOTSUP;
 			}
+
+			if ((cipher->algo == RTE_CRYPTO_CIPHER_AES_CBC) &&
+			    (auth->algo == RTE_CRYPTO_AUTH_AES_XCBC_MAC)) {
+				plt_err("Transport mode AES-CBC AES-XCBC is not supported");
+				return -ENOTSUP;
+			}
 		}
 	}
 
diff --git a/drivers/crypto/cnxk/cnxk_cryptodev.h b/drivers/crypto/cnxk/cnxk_cryptodev.h
index 4a1e377..16e7572 100644
--- a/drivers/crypto/cnxk/cnxk_cryptodev.h
+++ b/drivers/crypto/cnxk/cnxk_cryptodev.h
@@ -11,7 +11,7 @@
 #include "roc_cpt.h"
 
 #define CNXK_CPT_MAX_CAPS	 34
-#define CNXK_SEC_CRYPTO_MAX_CAPS 9
+#define CNXK_SEC_CRYPTO_MAX_CAPS 11
 #define CNXK_SEC_MAX_CAPS	 5
 #define CNXK_AE_EC_ID_MAX	 8
 /**
diff --git a/drivers/crypto/cnxk/cnxk_cryptodev_capabilities.c b/drivers/crypto/cnxk/cnxk_cryptodev_capabilities.c
index fae433e..a0b2a1f 100644
--- a/drivers/crypto/cnxk/cnxk_cryptodev_capabilities.c
+++ b/drivers/crypto/cnxk/cnxk_cryptodev_capabilities.c
@@ -794,6 +794,26 @@ static const struct rte_cryptodev_capabilities sec_caps_aes[] = {
 			}, }
 		}, }
 	},
+	{	/* AES-XCBC */
+		.op = RTE_CRYPTO_OP_TYPE_SYMMETRIC,
+		{ .sym = {
+			.xform_type = RTE_CRYPTO_SYM_XFORM_AUTH,
+			{.auth = {
+				.algo = RTE_CRYPTO_AUTH_AES_XCBC_MAC,
+				.block_size = 16,
+				.key_size = {
+					.min = 16,
+					.max = 16,
+					.increment = 0
+				},
+				.digest_size = {
+					.min = 12,
+					.max = 12,
+					.increment = 0,
+				},
+			}, }
+		}, }
+	},
 };
 
 static const struct rte_cryptodev_capabilities sec_caps_sha1_sha2[] = {
@@ -879,6 +899,29 @@ static const struct rte_cryptodev_capabilities sec_caps_sha1_sha2[] = {
 	},
 };
 
+static const struct rte_cryptodev_capabilities sec_caps_null[] = {
+	{	/* NULL (CIPHER) */
+		.op = RTE_CRYPTO_OP_TYPE_SYMMETRIC,
+		{.sym = {
+			.xform_type = RTE_CRYPTO_SYM_XFORM_CIPHER,
+			{.cipher = {
+				.algo = RTE_CRYPTO_CIPHER_NULL,
+				.block_size = 1,
+				.key_size = {
+					.min = 0,
+					.max = 0,
+					.increment = 0
+				},
+				.iv_size = {
+					.min = 0,
+					.max = 0,
+					.increment = 0
+				}
+			}, },
+		}, }
+	},
+};
+
 static const struct rte_security_capability sec_caps_templ[] = {
 	{	/* IPsec Lookaside Protocol ESP Tunnel Ingress */
 		.action = RTE_SECURITY_ACTION_TYPE_LOOKASIDE_PROTOCOL,
@@ -1069,6 +1112,8 @@ sec_crypto_caps_populate(struct rte_cryptodev_capabilities cnxk_caps[],
 	else
 		cn9k_sec_crypto_caps_update(cnxk_caps);
 
+	sec_caps_add(cnxk_caps, &cur_pos, sec_caps_null,
+		     RTE_DIM(sec_caps_null));
 	sec_caps_add(cnxk_caps, &cur_pos, caps_end, RTE_DIM(caps_end));
 }
 
diff --git a/drivers/crypto/cnxk/cnxk_ipsec.h b/drivers/crypto/cnxk/cnxk_ipsec.h
index f5a51b5..f50d9fa 100644
--- a/drivers/crypto/cnxk/cnxk_ipsec.h
+++ b/drivers/crypto/cnxk/cnxk_ipsec.h
@@ -20,6 +20,9 @@ struct cnxk_cpt_inst_tmpl {
 static inline int
 ipsec_xform_cipher_verify(struct rte_crypto_sym_xform *crypto_xform)
 {
+	if (crypto_xform->cipher.algo == RTE_CRYPTO_CIPHER_NULL)
+		return 0;
+
 	if (crypto_xform->cipher.algo == RTE_CRYPTO_CIPHER_AES_CBC ||
 	    crypto_xform->cipher.algo == RTE_CRYPTO_CIPHER_AES_CTR) {
 		switch (crypto_xform->cipher.key.length) {
@@ -58,6 +61,10 @@ ipsec_xform_auth_verify(struct rte_crypto_sym_xform *crypto_xform)
 			return 0;
 	}
 
+	if (crypto_xform->auth.algo == RTE_CRYPTO_AUTH_AES_XCBC_MAC &&
+	    keylen == ROC_CPT_AES_XCBC_KEY_LENGTH)
+		return 0;
+
 	return -ENOTSUP;
 }
 
-- 
2.7.4

[PATCH v3 25/29] crypto/cnxk: add copy and set DF

[Anoob Joseph]

Add support for copy and set DF bit.

Signed-off-by: Anoob Joseph <anoobj@marvell.com>
---
 drivers/crypto/cnxk/cn9k_ipsec.c                  | 7 ++++++-
 drivers/crypto/cnxk/cnxk_cryptodev_capabilities.c | 1 +
 2 files changed, 7 insertions(+), 1 deletion(-)

diff --git a/drivers/crypto/cnxk/cn9k_ipsec.c b/drivers/crypto/cnxk/cn9k_ipsec.c
index c9f5825..62b9c26 100644
--- a/drivers/crypto/cnxk/cn9k_ipsec.c
+++ b/drivers/crypto/cnxk/cn9k_ipsec.c
@@ -246,6 +246,8 @@ ipsec_sa_ctl_set(struct rte_security_ipsec_xform *ipsec,
 	if (ipsec->options.udp_encap == 1)
 		ctl->encap_type = ROC_IE_ON_SA_ENCAP_UDP;
 
+	ctl->copy_df = ipsec->options.copy_df;
+
 	ctl->spi = rte_cpu_to_be_32(ipsec->spi);
 
 	rte_io_wmb();
@@ -376,13 +378,16 @@ cn9k_ipsec_outb_sa_create(struct cnxk_cpt_qp *qp,
 
 	if (ipsec->mode == RTE_SECURITY_IPSEC_SA_MODE_TUNNEL) {
 		if (ipsec->tunnel.type == RTE_SECURITY_IPSEC_TUNNEL_IPV4) {
+			uint16_t frag_off = 0;
 			ctx_len += sizeof(template->ip4);
 
 			ip4->version_ihl = RTE_IPV4_VHL_DEF;
 			ip4->time_to_live = ipsec->tunnel.ipv4.ttl;
 			ip4->type_of_service |= (ipsec->tunnel.ipv4.dscp << 2);
 			if (ipsec->tunnel.ipv4.df)
-				ip4->fragment_offset = BIT(14);
+				frag_off |= RTE_IPV4_HDR_DF_FLAG;
+			ip4->fragment_offset = rte_cpu_to_be_16(frag_off);
+
 			memcpy(&ip4->src_addr, &ipsec->tunnel.ipv4.src_ip,
 			       sizeof(struct in_addr));
 			memcpy(&ip4->dst_addr, &ipsec->tunnel.ipv4.dst_ip,
diff --git a/drivers/crypto/cnxk/cnxk_cryptodev_capabilities.c b/drivers/crypto/cnxk/cnxk_cryptodev_capabilities.c
index a0b2a1f..69ee0d9 100644
--- a/drivers/crypto/cnxk/cnxk_cryptodev_capabilities.c
+++ b/drivers/crypto/cnxk/cnxk_cryptodev_capabilities.c
@@ -1121,6 +1121,7 @@ static void
 cnxk_sec_caps_update(struct rte_security_capability *sec_cap)
 {
 	sec_cap->ipsec.options.udp_encap = 1;
+	sec_cap->ipsec.options.copy_df = 1;
 }
 
 static void
-- 
2.7.4

[PATCH v3 26/29] crypto/cnxk: add aes cmac

[Anoob Joseph]

Add support for AES CMAC auth algorithm.

Signed-off-by: Anoob Joseph <anoobj@marvell.com>
---
 doc/guides/cryptodevs/cnxk.rst                    |  1 +
 doc/guides/cryptodevs/features/cn10k.ini          | 37 +++++++-------
 doc/guides/cryptodevs/features/cn9k.ini           | 37 +++++++-------
 doc/guides/rel_notes/release_22_03.rst            |  1 +
 drivers/common/cnxk/roc_se.h                      |  8 +--
 drivers/crypto/cnxk/cnxk_cryptodev_capabilities.c | 20 ++++++++
 drivers/crypto/cnxk/cnxk_se.h                     | 60 ++++++++++++++---------
 7 files changed, 103 insertions(+), 61 deletions(-)

diff --git a/doc/guides/cryptodevs/cnxk.rst b/doc/guides/cryptodevs/cnxk.rst
index 6e844f5..3c58517 100644
--- a/doc/guides/cryptodevs/cnxk.rst
+++ b/doc/guides/cryptodevs/cnxk.rst
@@ -61,6 +61,7 @@ Hash algorithms:
 * ``RTE_CRYPTO_AUTH_SHA512_HMAC``
 * ``RTE_CRYPTO_AUTH_SNOW3G_UIA2``
 * ``RTE_CRYPTO_AUTH_ZUC_EIA3``
+* ``RTE_CRYPTO_AUTH_AES_CMAC``
 
 AEAD algorithms:
 
diff --git a/doc/guides/cryptodevs/features/cn10k.ini b/doc/guides/cryptodevs/features/cn10k.ini
index ab21d9d..c8193c2 100644
--- a/doc/guides/cryptodevs/features/cn10k.ini
+++ b/doc/guides/cryptodevs/features/cn10k.ini
@@ -41,23 +41,26 @@ ZUC EEA3       = Y
 ; Supported authentication algorithms of 'cn10k' crypto driver.
 ;
 [Auth]
-NULL         = Y
-AES GMAC     = Y
-KASUMI F9    = Y
-MD5          = Y
-MD5 HMAC     = Y
-SHA1         = Y
-SHA1 HMAC    = Y
-SHA224       = Y
-SHA224 HMAC  = Y
-SHA256       = Y
-SHA256 HMAC  = Y
-SHA384       = Y
-SHA384 HMAC  = Y
-SHA512       = Y
-SHA512 HMAC  = Y
-SNOW3G UIA2  = Y
-ZUC EIA3     = Y
+NULL            = Y
+AES GMAC        = Y
+KASUMI F9       = Y
+MD5             = Y
+MD5 HMAC        = Y
+SHA1            = Y
+SHA1 HMAC       = Y
+SHA224          = Y
+SHA224 HMAC     = Y
+SHA256          = Y
+SHA256 HMAC     = Y
+SHA384          = Y
+SHA384 HMAC     = Y
+SHA512          = Y
+SHA512 HMAC     = Y
+SNOW3G UIA2     = Y
+ZUC EIA3        = Y
+AES CMAC (128)  = Y
+AES CMAC (192)  = Y
+AES CMAC (256)  = Y
 
 ;
 ; Supported AEAD algorithms of 'cn10k' crypto driver.
diff --git a/doc/guides/cryptodevs/features/cn9k.ini b/doc/guides/cryptodevs/features/cn9k.ini
index d834659..f215ee0 100644
--- a/doc/guides/cryptodevs/features/cn9k.ini
+++ b/doc/guides/cryptodevs/features/cn9k.ini
@@ -40,23 +40,26 @@ ZUC EEA3       = Y
 ; Supported authentication algorithms of 'cn9k' crypto driver.
 ;
 [Auth]
-NULL         = Y
-AES GMAC     = Y
-KASUMI F9    = Y
-MD5          = Y
-MD5 HMAC     = Y
-SHA1         = Y
-SHA1 HMAC    = Y
-SHA224       = Y
-SHA224 HMAC  = Y
-SHA256       = Y
-SHA256 HMAC  = Y
-SHA384       = Y
-SHA384 HMAC  = Y
-SHA512       = Y
-SHA512 HMAC  = Y
-SNOW3G UIA2  = Y
-ZUC EIA3     = Y
+NULL            = Y
+AES GMAC        = Y
+KASUMI F9       = Y
+MD5             = Y
+MD5 HMAC        = Y
+SHA1            = Y
+SHA1 HMAC       = Y
+SHA224          = Y
+SHA224 HMAC     = Y
+SHA256          = Y
+SHA256 HMAC     = Y
+SHA384          = Y
+SHA384 HMAC     = Y
+SHA512          = Y
+SHA512 HMAC     = Y
+SNOW3G UIA2     = Y
+ZUC EIA3        = Y
+AES CMAC (128)  = Y
+AES CMAC (192)  = Y
+AES CMAC (256)  = Y
 
 ;
 ; Supported AEAD algorithms of 'cn9k' crypto driver.
diff --git a/doc/guides/rel_notes/release_22_03.rst b/doc/guides/rel_notes/release_22_03.rst
index e8fec00..72e758e 100644
--- a/doc/guides/rel_notes/release_22_03.rst
+++ b/doc/guides/rel_notes/release_22_03.rst
@@ -63,6 +63,7 @@ New Features
   * Added AES-CTR support in lookaside protocol (IPsec) for CN9K & CN10K.
   * Added NULL cipher support in lookaside protocol (IPsec) for CN9K & CN10K.
   * Added AES-XCBC support in lookaside protocol (IPsec) for CN9K & CN10K.
+  * Added AES-CMAC support in CN9K & CN10K.
 
 
 Removed Items
diff --git a/drivers/common/cnxk/roc_se.h b/drivers/common/cnxk/roc_se.h
index 253575a..145a182 100644
--- a/drivers/common/cnxk/roc_se.h
+++ b/drivers/common/cnxk/roc_se.h
@@ -11,10 +11,10 @@
 #define ROC_SE_FC_MINOR_OP_DECRYPT    0x1
 #define ROC_SE_FC_MINOR_OP_HMAC_FIRST 0x10
 
-#define ROC_SE_MAJOR_OP_HASH	   0x34
-#define ROC_SE_MAJOR_OP_HMAC	   0x35
-#define ROC_SE_MAJOR_OP_ZUC_SNOW3G 0x37
-#define ROC_SE_MAJOR_OP_KASUMI	   0x38
+#define ROC_SE_MAJOR_OP_HASH   0x34
+#define ROC_SE_MAJOR_OP_HMAC   0x35
+#define ROC_SE_MAJOR_OP_PDCP   0x37
+#define ROC_SE_MAJOR_OP_KASUMI 0x38
 
 #define ROC_SE_MAJOR_OP_MISC		 0x01
 #define ROC_SE_MISC_MINOR_OP_PASSTHROUGH 0x03
diff --git a/drivers/crypto/cnxk/cnxk_cryptodev_capabilities.c b/drivers/crypto/cnxk/cnxk_cryptodev_capabilities.c
index 69ee0d9..457e166 100644
--- a/drivers/crypto/cnxk/cnxk_cryptodev_capabilities.c
+++ b/drivers/crypto/cnxk/cnxk_cryptodev_capabilities.c
@@ -568,6 +568,26 @@ static const struct rte_cryptodev_capabilities caps_aes[] = {
 			}, }
 		}, }
 	},
+	{	/* AES CMAC */
+		.op = RTE_CRYPTO_OP_TYPE_SYMMETRIC,
+		{.sym = {
+			.xform_type = RTE_CRYPTO_SYM_XFORM_AUTH,
+			{.auth = {
+				.algo = RTE_CRYPTO_AUTH_AES_CMAC,
+				.block_size = 16,
+				.key_size = {
+					.min = 16,
+					.max = 32,
+					.increment = 8
+				},
+				.digest_size = {
+					.min = 4,
+					.max = 4,
+					.increment = 0
+				},
+			}, }
+		}, }
+	},
 };
 
 static const struct rte_cryptodev_capabilities caps_kasumi[] = {
diff --git a/drivers/crypto/cnxk/cnxk_se.h b/drivers/crypto/cnxk/cnxk_se.h
index a8cd2c5..e988d57 100644
--- a/drivers/crypto/cnxk/cnxk_se.h
+++ b/drivers/crypto/cnxk/cnxk_se.h
@@ -73,11 +73,15 @@ pdcp_iv_copy(uint8_t *iv_d, uint8_t *iv_s, const uint8_t pdcp_alg_type,
 		for (j = 0; j < 4; j++)
 			iv_temp[j] = iv_s_temp[3 - j];
 		memcpy(iv_d, iv_temp, 16);
-	} else {
+	} else if (pdcp_alg_type == ROC_SE_PDCP_ALG_TYPE_ZUC) {
 		/* ZUC doesn't need a swap */
 		memcpy(iv_d, iv_s, 16);
 		if (pack_iv)
 			cpt_pack_iv(iv_s, iv_d);
+	} else {
+		/* AES-CMAC EIA2, microcode expects 16B zeroized IV */
+		for (j = 0; j < 4; j++)
+			iv_d[j] = 0;
 	}
 }
 
@@ -992,8 +996,8 @@ cpt_dec_hmac_prep(uint32_t flags, uint64_t d_offs, uint64_t d_lens,
 }
 
 static __rte_always_inline int
-cpt_zuc_snow3g_prep(uint32_t req_flags, uint64_t d_offs, uint64_t d_lens,
-		    struct roc_se_fc_params *params, struct cpt_inst_s *inst)
+cpt_pdcp_alg_prep(uint32_t req_flags, uint64_t d_offs, uint64_t d_lens,
+		  struct roc_se_fc_params *params, struct cpt_inst_s *inst)
 {
 	uint32_t size;
 	int32_t inputlen, outputlen;
@@ -1014,33 +1018,43 @@ cpt_zuc_snow3g_prep(uint32_t req_flags, uint64_t d_offs, uint64_t d_lens,
 	mac_len = se_ctx->mac_len;
 	pdcp_alg_type = se_ctx->pdcp_alg_type;
 
-	cpt_inst_w4.s.opcode_major = ROC_SE_MAJOR_OP_ZUC_SNOW3G;
-
+	cpt_inst_w4.s.opcode_major = ROC_SE_MAJOR_OP_PDCP;
 	cpt_inst_w4.s.opcode_minor = se_ctx->template_w4.s.opcode_minor;
 
 	if (flags == 0x1) {
 		iv_s = params->auth_iv_buf;
-		iv_len = params->auth_iv_len;
-
-		if (iv_len == 25) {
-			iv_len -= 2;
-			pack_iv = 1;
-		}
 
 		/*
 		 * Microcode expects offsets in bytes
 		 * TODO: Rounding off
 		 */
 		auth_data_len = ROC_SE_AUTH_DLEN(d_lens);
-
-		/* EIA3 or UIA2 */
 		auth_offset = ROC_SE_AUTH_OFFSET(d_offs);
-		auth_offset = auth_offset / 8;
 
-		/* consider iv len */
-		auth_offset += iv_len;
+		if (se_ctx->pdcp_alg_type != ROC_SE_PDCP_ALG_TYPE_AES_CTR) {
+			iv_len = params->auth_iv_len;
+
+			if (iv_len == 25) {
+				iv_len -= 2;
+				pack_iv = 1;
+			}
+
+			auth_offset = auth_offset / 8;
+
+			/* consider iv len */
+			auth_offset += iv_len;
+
+			inputlen =
+				auth_offset + (RTE_ALIGN(auth_data_len, 8) / 8);
+		} else {
+			iv_len = 16;
+
+			/* consider iv len */
+			auth_offset += iv_len;
+
+			inputlen = auth_offset + auth_data_len;
+		}
 
-		inputlen = auth_offset + (RTE_ALIGN(auth_data_len, 8) / 8);
 		outputlen = mac_len;
 
 		offset_ctrl = rte_cpu_to_be_64((uint64_t)auth_offset);
@@ -1056,7 +1070,6 @@ cpt_zuc_snow3g_prep(uint32_t req_flags, uint64_t d_offs, uint64_t d_lens,
 			pack_iv = 1;
 		}
 
-		/* EEA3 or UEA2 */
 		/*
 		 * Microcode expects offsets in bytes
 		 * TODO: Rounding off
@@ -1589,8 +1602,7 @@ cpt_fc_dec_hmac_prep(uint32_t flags, uint64_t d_offs, uint64_t d_lens,
 	if (likely(fc_type == ROC_SE_FC_GEN)) {
 		ret = cpt_dec_hmac_prep(flags, d_offs, d_lens, fc_params, inst);
 	} else if (fc_type == ROC_SE_PDCP) {
-		ret = cpt_zuc_snow3g_prep(flags, d_offs, d_lens, fc_params,
-					  inst);
+		ret = cpt_pdcp_alg_prep(flags, d_offs, d_lens, fc_params, inst);
 	} else if (fc_type == ROC_SE_KASUMI) {
 		ret = cpt_kasumi_dec_prep(d_offs, d_lens, fc_params, inst);
 	}
@@ -1618,8 +1630,7 @@ cpt_fc_enc_hmac_prep(uint32_t flags, uint64_t d_offs, uint64_t d_lens,
 	if (likely(fc_type == ROC_SE_FC_GEN)) {
 		ret = cpt_enc_hmac_prep(flags, d_offs, d_lens, fc_params, inst);
 	} else if (fc_type == ROC_SE_PDCP) {
-		ret = cpt_zuc_snow3g_prep(flags, d_offs, d_lens, fc_params,
-					  inst);
+		ret = cpt_pdcp_alg_prep(flags, d_offs, d_lens, fc_params, inst);
 	} else if (fc_type == ROC_SE_KASUMI) {
 		ret = cpt_kasumi_enc_prep(flags, d_offs, d_lens, fc_params,
 					  inst);
@@ -1883,8 +1894,11 @@ fill_sess_auth(struct rte_crypto_sym_xform *xform, struct cnxk_se_sess *sess)
 		auth_type = 0;
 		is_null = 1;
 		break;
-	case RTE_CRYPTO_AUTH_AES_XCBC_MAC:
 	case RTE_CRYPTO_AUTH_AES_CMAC:
+		auth_type = ROC_SE_AES_CMAC_EIA2;
+		zsk_flag = ROC_SE_ZS_IA;
+		break;
+	case RTE_CRYPTO_AUTH_AES_XCBC_MAC:
 	case RTE_CRYPTO_AUTH_AES_CBC_MAC:
 		plt_dp_err("Crypto: Unsupported hash algo %u", a_form->algo);
 		return -1;
-- 
2.7.4

[PATCH v3 27/29] crypto/cnxk: add per pkt IV in lookaside IPsec debug mode

[Anoob Joseph]

From: Archana Muniganti <marchana@marvell.com>

For cn9k, use HW GEN IV as default and add per pkt IV
in lookaside IPsec debug mode. Debug mode helps to verify
lookaside PMD using known outbound vectors in lookaside
autotest.

Signed-off-by: Archana Muniganti <marchana@marvell.com>
---
 drivers/common/cnxk/roc_ie_on.h                   |  7 +++++
 drivers/crypto/cnxk/cn9k_ipsec.c                  | 34 +++++++++++++++++------
 drivers/crypto/cnxk/cn9k_ipsec.h                  |  2 ++
 drivers/crypto/cnxk/cn9k_ipsec_la_ops.h           | 14 +++++++---
 drivers/crypto/cnxk/cnxk_cryptodev_capabilities.c |  2 ++
 5 files changed, 47 insertions(+), 12 deletions(-)

diff --git a/drivers/common/cnxk/roc_ie_on.h b/drivers/common/cnxk/roc_ie_on.h
index cb56a70..aaad872 100644
--- a/drivers/common/cnxk/roc_ie_on.h
+++ b/drivers/common/cnxk/roc_ie_on.h
@@ -22,6 +22,8 @@ enum roc_ie_on_ucc_ipsec {
 
 /* Helper macros */
 #define ROC_IE_ON_INB_RPTR_HDR 0x8
+#define ROC_IE_ON_MAX_IV_LEN   16
+#define ROC_IE_ON_PER_PKT_IV   BIT(43)
 
 enum {
 	ROC_IE_ON_SA_ENC_NULL = 0,
@@ -55,6 +57,11 @@ enum {
 	ROC_IE_ON_SA_ENCAP_UDP = 1,
 };
 
+enum {
+	ROC_IE_ON_IV_SRC_HW_GEN_DEFAULT = 0,
+	ROC_IE_ON_IV_SRC_FROM_DPTR = 1,
+};
+
 struct roc_ie_on_outb_hdr {
 	uint32_t ip_id;
 	uint32_t seq;
diff --git a/drivers/crypto/cnxk/cn9k_ipsec.c b/drivers/crypto/cnxk/cn9k_ipsec.c
index 62b9c26..9f876f7 100644
--- a/drivers/crypto/cnxk/cn9k_ipsec.c
+++ b/drivers/crypto/cnxk/cn9k_ipsec.c
@@ -426,13 +426,7 @@ cn9k_ipsec_outb_sa_create(struct cnxk_cpt_qp *qp,
 
 	ctx_len += RTE_ALIGN_CEIL(ctx_len, 8);
 
-	if (crypto_xform->type == RTE_CRYPTO_SYM_XFORM_AEAD) {
-		sa->cipher_iv_off = crypto_xform->aead.iv.offset;
-		sa->cipher_iv_len = crypto_xform->aead.iv.length;
-	} else {
-		sa->cipher_iv_off = crypto_xform->cipher.iv.offset;
-		sa->cipher_iv_len = crypto_xform->cipher.iv.length;
-
+	if (crypto_xform->type != RTE_CRYPTO_SYM_XFORM_AEAD) {
 		auth_key = auth_xform->auth.key.data;
 		auth_key_len = auth_xform->auth.key.length;
 
@@ -465,7 +459,31 @@ cn9k_ipsec_outb_sa_create(struct cnxk_cpt_qp *qp,
 
 	param1.u16 = 0;
 	param1.s.ikev2 = 1;
-	param1.s.per_pkt_iv = 1;
+
+	sa->custom_hdr_len = sizeof(struct roc_ie_on_outb_hdr) -
+			     ROC_IE_ON_MAX_IV_LEN;
+
+#ifdef LA_IPSEC_DEBUG
+	/* Use IV from application in debug mode */
+	if (ipsec->options.iv_gen_disable == 1) {
+		param1.s.per_pkt_iv = ROC_IE_ON_IV_SRC_FROM_DPTR;
+		sa->custom_hdr_len = sizeof(struct roc_ie_on_outb_hdr);
+
+		if (crypto_xform->type == RTE_CRYPTO_SYM_XFORM_AEAD) {
+			sa->cipher_iv_off = crypto_xform->aead.iv.offset;
+			sa->cipher_iv_len = crypto_xform->aead.iv.length;
+		} else {
+			sa->cipher_iv_off = crypto_xform->cipher.iv.offset;
+			sa->cipher_iv_len = crypto_xform->cipher.iv.length;
+		}
+	}
+#else
+	if (ipsec->options.iv_gen_disable != 0) {
+		plt_err("Application provided IV is not supported");
+		return -ENOTSUP;
+	}
+#endif
+
 	w4.s.param1 = param1.u16;
 
 	inst_tmpl->w4 = w4.u64;
diff --git a/drivers/crypto/cnxk/cn9k_ipsec.h b/drivers/crypto/cnxk/cn9k_ipsec.h
index fc440d5..f3acad5 100644
--- a/drivers/crypto/cnxk/cn9k_ipsec.h
+++ b/drivers/crypto/cnxk/cn9k_ipsec.h
@@ -24,6 +24,8 @@ struct cn9k_ipsec_sa {
 	uint16_t cipher_iv_off;
 	/** Cipher IV length in bytes */
 	uint8_t cipher_iv_len;
+	/** Outbound custom header length */
+	uint8_t custom_hdr_len;
 	/** Response length calculation data */
 	struct cnxk_ipsec_outb_rlens rlens;
 	/** Outbound IP-ID */
diff --git a/drivers/crypto/cnxk/cn9k_ipsec_la_ops.h b/drivers/crypto/cnxk/cn9k_ipsec_la_ops.h
index 2b0261e..9a1e217 100644
--- a/drivers/crypto/cnxk/cn9k_ipsec_la_ops.h
+++ b/drivers/crypto/cnxk/cn9k_ipsec_la_ops.h
@@ -74,7 +74,7 @@ static __rte_always_inline int
 process_outb_sa(struct rte_crypto_op *cop, struct cn9k_ipsec_sa *sa,
 		struct cpt_inst_s *inst)
 {
-	const unsigned int hdr_len = sizeof(struct roc_ie_on_outb_hdr);
+	const unsigned int hdr_len = sa->custom_hdr_len;
 	struct rte_crypto_sym_op *sym_op = cop->sym;
 	struct rte_mbuf *m_src = sym_op->m_src;
 	struct roc_ie_on_outb_sa *out_sa;
@@ -103,9 +103,15 @@ process_outb_sa(struct rte_crypto_op *cop, struct cn9k_ipsec_sa *sa,
 		return -ENOMEM;
 	}
 
-	memcpy(&hdr->iv[0],
-	       rte_crypto_op_ctod_offset(cop, uint8_t *, sa->cipher_iv_off),
-	       sa->cipher_iv_len);
+#ifdef LA_IPSEC_DEBUG
+	if (sa->inst.w4 & ROC_IE_ON_PER_PKT_IV) {
+		memcpy(&hdr->iv[0],
+		       rte_crypto_op_ctod_offset(cop, uint8_t *,
+						 sa->cipher_iv_off),
+		       sa->cipher_iv_len);
+	}
+#endif
+
 	hdr->seq = rte_cpu_to_be_32(sa->seq_lo);
 	hdr->ip_id = rte_cpu_to_be_32(sa->ip_id);
 
diff --git a/drivers/crypto/cnxk/cnxk_cryptodev_capabilities.c b/drivers/crypto/cnxk/cnxk_cryptodev_capabilities.c
index 457e166..f79e4d7 100644
--- a/drivers/crypto/cnxk/cnxk_cryptodev_capabilities.c
+++ b/drivers/crypto/cnxk/cnxk_cryptodev_capabilities.c
@@ -1166,7 +1166,9 @@ static void
 cn9k_sec_caps_update(struct rte_security_capability *sec_cap)
 {
 	if (sec_cap->ipsec.direction == RTE_SECURITY_IPSEC_SA_DIR_EGRESS) {
+#ifdef LA_IPSEC_DEBUG
 		sec_cap->ipsec.options.iv_gen_disable = 1;
+#endif
 	}
 }
 
-- 
2.7.4

[PATCH v3 28/29] crypto/cnxk: enable copy dscp

[Anoob Joseph]

Copy DSCP is supported. Enable it in capabilities.

Signed-off-by: Anoob Joseph <anoobj@marvell.com>
---
 drivers/crypto/cnxk/cnxk_cryptodev_capabilities.c | 1 +
 1 file changed, 1 insertion(+)

diff --git a/drivers/crypto/cnxk/cnxk_cryptodev_capabilities.c b/drivers/crypto/cnxk/cnxk_cryptodev_capabilities.c
index f79e4d7..f8c007e 100644
--- a/drivers/crypto/cnxk/cnxk_cryptodev_capabilities.c
+++ b/drivers/crypto/cnxk/cnxk_cryptodev_capabilities.c
@@ -1142,6 +1142,7 @@ cnxk_sec_caps_update(struct rte_security_capability *sec_cap)
 {
 	sec_cap->ipsec.options.udp_encap = 1;
 	sec_cap->ipsec.options.copy_df = 1;
+	sec_cap->ipsec.options.copy_dscp = 1;
 }
 
 static void
-- 
2.7.4

[PATCH v3 29/29] crypto/cnxk: update microcode completion handling

[Anoob Joseph]

Update microcode completion code handling to update the required mbuf &
crypto op flags. IP checksum good case is now reported by specific
microcode completion code.

Signed-off-by: Anoob Joseph <anoobj@marvell.com>
---
 drivers/crypto/cnxk/cn10k_cryptodev_ops.c | 59 ++++++++++---------------------
 drivers/crypto/cnxk/cn10k_ipsec.c         |  1 -
 drivers/crypto/cnxk/cn10k_ipsec.h         |  1 -
 3 files changed, 18 insertions(+), 43 deletions(-)

diff --git a/drivers/crypto/cnxk/cn10k_cryptodev_ops.c b/drivers/crypto/cnxk/cn10k_cryptodev_ops.c
index 1905ea3..d217bbf 100644
--- a/drivers/crypto/cnxk/cn10k_cryptodev_ops.c
+++ b/drivers/crypto/cnxk/cn10k_cryptodev_ops.c
@@ -50,8 +50,7 @@ cn10k_cpt_sym_temp_sess_create(struct cnxk_cpt_qp *qp, struct rte_crypto_op *op)
 
 static __rte_always_inline int __rte_hot
 cpt_sec_inst_fill(struct cnxk_cpt_qp *qp, struct rte_crypto_op *op,
-		  struct cn10k_sec_session *sess,
-		  struct cpt_inflight_req *infl_req, struct cpt_inst_s *inst)
+		  struct cn10k_sec_session *sess, struct cpt_inst_s *inst)
 {
 	struct rte_crypto_sym_op *sym_op = op->sym;
 	struct cn10k_ipsec_sa *sa;
@@ -71,10 +70,8 @@ cpt_sec_inst_fill(struct cnxk_cpt_qp *qp, struct rte_crypto_op *op,
 
 	if (sa->is_outbound)
 		ret = process_outb_sa(&qp->lf, op, sa, inst);
-	else {
-		infl_req->op_flags |= CPT_OP_FLAGS_IPSEC_DIR_INBOUND;
+	else
 		ret = process_inb_sa(op, sa, inst);
-	}
 
 	return ret;
 }
@@ -127,8 +124,7 @@ cn10k_cpt_fill_inst(struct cnxk_cpt_qp *qp, struct rte_crypto_op *ops[],
 		if (op->sess_type == RTE_CRYPTO_OP_SECURITY_SESSION) {
 			sec_sess = get_sec_session_private_data(
 				sym_op->sec_session);
-			ret = cpt_sec_inst_fill(qp, op, sec_sess, infl_req,
-						&inst[0]);
+			ret = cpt_sec_inst_fill(qp, op, sec_sess, &inst[0]);
 			if (unlikely(ret))
 				return 0;
 			w7 = sec_sess->sa.inst.w7;
@@ -346,52 +342,34 @@ static inline void
 cn10k_cpt_sec_post_process(struct rte_crypto_op *cop,
 			   struct cpt_cn10k_res_s *res)
 {
-	struct rte_mbuf *m = cop->sym->m_src;
+	struct rte_mbuf *mbuf = cop->sym->m_src;
 	const uint16_t m_len = res->rlen;
 
-	m->data_len = m_len;
-	m->pkt_len = m_len;
-}
-
-static inline void
-cn10k_cpt_sec_ucc_process(struct rte_crypto_op *cop,
-			  struct cpt_inflight_req *infl_req,
-			  const uint8_t uc_compcode)
-{
-	struct cn10k_sec_session *sess;
-	struct cn10k_ipsec_sa *sa;
-	struct rte_mbuf *mbuf;
-
-	if (uc_compcode == ROC_IE_OT_UCC_SUCCESS_SA_SOFTEXP_FIRST)
-		cop->aux_flags = RTE_CRYPTO_OP_AUX_FLAGS_IPSEC_SOFT_EXPIRY;
-
-	if (!(infl_req->op_flags & CPT_OP_FLAGS_IPSEC_DIR_INBOUND))
-		return;
-
-	sess = get_sec_session_private_data(cop->sym->sec_session);
-	sa = &sess->sa;
+	mbuf->data_len = m_len;
+	mbuf->pkt_len = m_len;
 
-	mbuf = cop->sym->m_src;
-
-	switch (uc_compcode) {
+	switch (res->uc_compcode) {
 	case ROC_IE_OT_UCC_SUCCESS:
-		if (sa->ip_csum_enable)
-			mbuf->ol_flags |= RTE_MBUF_F_RX_IP_CKSUM_GOOD;
 		break;
 	case ROC_IE_OT_UCC_SUCCESS_PKT_IP_BADCSUM:
 		mbuf->ol_flags |= RTE_MBUF_F_RX_IP_CKSUM_BAD;
 		break;
 	case ROC_IE_OT_UCC_SUCCESS_PKT_L4_GOODCSUM:
-		mbuf->ol_flags |= RTE_MBUF_F_RX_L4_CKSUM_GOOD;
-		if (sa->ip_csum_enable)
-			mbuf->ol_flags |= RTE_MBUF_F_RX_IP_CKSUM_GOOD;
+		mbuf->ol_flags |= RTE_MBUF_F_RX_L4_CKSUM_GOOD |
+				  RTE_MBUF_F_RX_IP_CKSUM_GOOD;
 		break;
 	case ROC_IE_OT_UCC_SUCCESS_PKT_L4_BADCSUM:
-		mbuf->ol_flags |= RTE_MBUF_F_RX_L4_CKSUM_BAD;
-		if (sa->ip_csum_enable)
-			mbuf->ol_flags |= RTE_MBUF_F_RX_IP_CKSUM_GOOD;
+		mbuf->ol_flags |= RTE_MBUF_F_RX_L4_CKSUM_BAD |
+				  RTE_MBUF_F_RX_IP_CKSUM_GOOD;
+		break;
+	case ROC_IE_OT_UCC_SUCCESS_PKT_IP_GOODCSUM:
+		mbuf->ol_flags |= RTE_MBUF_F_RX_IP_CKSUM_GOOD;
+		break;
+	case ROC_IE_OT_UCC_SUCCESS_SA_SOFTEXP_FIRST:
+		cop->aux_flags = RTE_CRYPTO_OP_AUX_FLAGS_IPSEC_SOFT_EXPIRY;
 		break;
 	default:
+		plt_dp_err("Success with unknown microcode completion code");
 		break;
 	}
 }
@@ -412,7 +390,6 @@ cn10k_cpt_dequeue_post_process(struct cnxk_cpt_qp *qp,
 	    cop->sess_type == RTE_CRYPTO_OP_SECURITY_SESSION) {
 		if (likely(compcode == CPT_COMP_WARN)) {
 			/* Success with additional info */
-			cn10k_cpt_sec_ucc_process(cop, infl_req, uc_compcode);
 			cn10k_cpt_sec_post_process(cop, res);
 		} else {
 			cop->status = RTE_CRYPTO_OP_STATUS_ERROR;
diff --git a/drivers/crypto/cnxk/cn10k_ipsec.c b/drivers/crypto/cnxk/cn10k_ipsec.c
index a93c211..7f4ccaf 100644
--- a/drivers/crypto/cnxk/cn10k_ipsec.c
+++ b/drivers/crypto/cnxk/cn10k_ipsec.c
@@ -201,7 +201,6 @@ cn10k_ipsec_inb_sa_create(struct roc_cpt *roc_cpt, struct roc_cpt_lf *lf,
 	if (ipsec_xfrm->options.ip_csum_enable) {
 		param1.s.ip_csum_disable =
 			ROC_IE_OT_SA_INNER_PKT_IP_CSUM_ENABLE;
-		sa->ip_csum_enable = true;
 	}
 
 	/* Disable L4 checksum verification by default */
diff --git a/drivers/crypto/cnxk/cn10k_ipsec.h b/drivers/crypto/cnxk/cn10k_ipsec.h
index cc7ca19..647a71c 100644
--- a/drivers/crypto/cnxk/cn10k_ipsec.h
+++ b/drivers/crypto/cnxk/cn10k_ipsec.h
@@ -19,7 +19,6 @@ struct cn10k_ipsec_sa {
 	uint16_t max_extended_len;
 	uint16_t iv_offset;
 	uint8_t iv_length;
-	bool ip_csum_enable;
 	bool is_outbound;
 
 	/**
-- 
2.7.4

RE: [PATCH v3 00/29] New features and improvements in cnxk crypto PMD

[Akhil Goyal]

> Subject: [PATCH v3 00/29] New features and improvements in cnxk crypto PMD
> 
> New features and fixes to cnxk crypto PMDs
> - Support for more algorithms in lookaside crypto & protocol
> - Support for copy & set DF bit
> - Support for CPT CTX update
> - Support for security session stats in cn10k
> 
> Changes in v3
> - Fixed build error from CI
Series Acked-by: Akhil Goyal <gakhil@marvell.com>

Applied to dpdk-next-crypto
Changed patch titles in some of the patches. Added/fixed missing/wrong fixes tag in the patches while merging.