DPDK patches and discussions
 help / color / mirror / Atom feed
* [dpdk-dev] [PATCH 00/17] bnxt PMD fixes
@ 2022-04-27 14:58 Kalesh A P
  2022-04-27 14:58 ` [PATCH 01/17] net/bnxt: update HWRM structures Kalesh A P
                   ` (17 more replies)
  0 siblings, 18 replies; 19+ messages in thread
From: Kalesh A P @ 2022-04-27 14:58 UTC (permalink / raw)
  To: dev; +Cc: ajit.khaparde

From: Kalesh AP <kalesh-anakkur.purayil@broadcom.com>

This patch set contains bug fixes in bnxt PMD. Please apply.

Kalesh AP (12):
  net/bnxt: update HWRM structures
  net/bnxt: fix device capability reporting
  net/bnxt: fix to remove an unused macro
  net/bnxt: fix Rxq configure
  net/bnxt: fix support for tunnel stateless offloads
  net/bnxt: fix RSS action support
  net/bnxt: add check for dupliate queue ids
  net/bnxt: avoid unnecessary endianness conversion
  net/bnxt: fix setting autoneg speed
  net/bnxt: force PHY update on certain configurations
  net/bnxt: fix reporting link status when port is stopped
  net/bnxt: recheck FW readiness if FW is in reset process

Somnath Kotur (5):
  net/bnxt: remove support for COUNT action
  net/bnxt: fix to reconfigure the VNIC's default receive ring
  net/bnxt: fix to handle queue stop during RSS flow create
  net/bnxt: fix freeing of VNIC filters
  net/bnxt: don't wait for link up completion in dev start

 drivers/net/bnxt/bnxt.h                |   29 +-
 drivers/net/bnxt/bnxt_ethdev.c         |   58 +-
 drivers/net/bnxt/bnxt_filter.c         |    2 +
 drivers/net/bnxt/bnxt_flow.c           |   92 +-
 drivers/net/bnxt/bnxt_hwrm.c           |   15 +-
 drivers/net/bnxt/bnxt_hwrm.h           |   20 +
 drivers/net/bnxt/bnxt_reps.c           |    6 +-
 drivers/net/bnxt/bnxt_rxq.c            |   75 +-
 drivers/net/bnxt/bnxt_rxq.h            |    1 +
 drivers/net/bnxt/bnxt_txq.c            |   29 +
 drivers/net/bnxt/bnxt_txq.h            |    1 +
 drivers/net/bnxt/hsi_struct_def_dpdk.h | 4025 ++++++++++++++++++++++++++++----
 12 files changed, 3809 insertions(+), 544 deletions(-)

-- 
2.10.1


^ permalink raw reply	[flat|nested] 19+ messages in thread

* [PATCH 01/17] net/bnxt: update HWRM structures
  2022-04-27 14:58 [dpdk-dev] [PATCH 00/17] bnxt PMD fixes Kalesh A P
@ 2022-04-27 14:58 ` Kalesh A P
  2022-04-27 14:58 ` [PATCH 02/17] net/bnxt: fix device capability reporting Kalesh A P
                   ` (16 subsequent siblings)
  17 siblings, 0 replies; 19+ messages in thread
From: Kalesh A P @ 2022-04-27 14:58 UTC (permalink / raw)
  To: dev; +Cc: ajit.khaparde

[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #1: Type: text/plain; charset=y, Size: 190026 bytes --]

From: Kalesh AP <kalesh-anakkur.purayil@broadcom.com>

Brought in the latest hsi_struct_def_dpdk.h.
HWRM API is now updated to version "1.10.2.83".

Signed-off-by: Kalesh AP <kalesh-anakkur.purayil@broadcom.com>
Reviewed-by: Ajit Khaparde <ajit.khaparde@broadcom.com>
Reviewed-by: Somnath Kotur <somnath.kotur@broadcom.com>
---
 drivers/net/bnxt/hsi_struct_def_dpdk.h | 4025 ++++++++++++++++++++++++++++----
 1 file changed, 3606 insertions(+), 419 deletions(-)

diff --git a/drivers/net/bnxt/hsi_struct_def_dpdk.h b/drivers/net/bnxt/hsi_struct_def_dpdk.h
index 88624f8..380dec4 100644
--- a/drivers/net/bnxt/hsi_struct_def_dpdk.h
+++ b/drivers/net/bnxt/hsi_struct_def_dpdk.h
@@ -1,5 +1,5 @@
 /* SPDX-License-Identifier: BSD-3-Clause
- * Copyright (c) 2014-2021 Broadcom Inc.
+ * Copyright (c) 2014-2022 Broadcom Inc.
  * All rights reserved.
  *
  * DO NOT MODIFY!!! This file is automatically generated.
@@ -543,6 +543,10 @@ struct cmd_nums {
 	#define HWRM_CFA_LAG_GROUP_MEMBER_RGTR            UINT32_C(0x126)
 	/* Experimental */
 	#define HWRM_CFA_LAG_GROUP_MEMBER_UNRGTR          UINT32_C(0x127)
+	/* Experimental */
+	#define HWRM_CFA_TLS_FILTER_ALLOC                 UINT32_C(0x128)
+	/* Experimental */
+	#define HWRM_CFA_TLS_FILTER_FREE                  UINT32_C(0x129)
 	/* Engine CKV - Get the current allocation status of keys provisioned in the key vault. */
 	#define HWRM_ENGINE_CKV_STATUS                    UINT32_C(0x12e)
 	/* Engine CKV - Add a new CKEK used to encrypt keys. */
@@ -657,12 +661,23 @@ struct cmd_nums {
 	#define HWRM_FUNC_PTP_EXT_CFG                     UINT32_C(0x1a0)
 	/* PTP - Query extended PTP configuration. */
 	#define HWRM_FUNC_PTP_EXT_QCFG                    UINT32_C(0x1a1)
-	/* The command is used to allocate KTLS crypto key contexts. */
+	/* The command is used to allocate KTLS or QUIC key contexts. */
 	#define HWRM_FUNC_KEY_CTX_ALLOC                   UINT32_C(0x1a2)
 	/* The is the new API to configure backing stores. */
 	#define HWRM_FUNC_BACKING_STORE_CFG_V2            UINT32_C(0x1a3)
 	/* The is the new API to query backing store configurations. */
 	#define HWRM_FUNC_BACKING_STORE_QCFG_V2           UINT32_C(0x1a4)
+	/* To support doorbell pacing configuration. */
+	#define HWRM_FUNC_DBR_PACING_CFG                  UINT32_C(0x1a5)
+	/* To query doorbell pacing configuration. */
+	#define HWRM_FUNC_DBR_PACING_QCFG                 UINT32_C(0x1a6)
+	/*
+	 * To broadcast the doorbell event to the drivers to
+	 * initiate pacing of doorbells.
+	 */
+	#define HWRM_FUNC_DBR_PACING_BROADCAST_EVENT      UINT32_C(0x1a7)
+	/* The is the new API to query backing store capabilities. */
+	#define HWRM_FUNC_BACKING_STORE_QCAPS_V2          UINT32_C(0x1a8)
 	/* Experimental */
 	#define HWRM_SELFTEST_QLIST                       UINT32_C(0x200)
 	/* Experimental */
@@ -1074,8 +1089,8 @@ struct hwrm_err_output {
 #define HWRM_VERSION_MINOR 10
 #define HWRM_VERSION_UPDATE 2
 /* non-zero means beta version */
-#define HWRM_VERSION_RSVD 68
-#define HWRM_VERSION_STR "1.10.2.68"
+#define HWRM_VERSION_RSVD 83
+#define HWRM_VERSION_STR "1.10.2.83"
 
 /****************
  * hwrm_ver_get *
@@ -2021,9 +2036,8 @@ struct ce_bds_add_data_msg {
 	/*
 	 * Version number of TLS connection. HW will provide registers that
 	 * converts the 4b encoded version number to 16b of actual version
-	 * number in the TLS Header. * Initialized --> By mid-path command *
-	 * Updated --> Never though another mid-path command will result in an
-	 * update.
+	 * number in the TLS Header. This field is initialized/updated by
+	 * this "KTLS crypto add" mid-path command.
 	 */
 	#define CE_BDS_ADD_DATA_MSG_VERSION_MASK \
 		UINT32_C(0xf0000000)
@@ -2036,19 +2050,28 @@ struct ce_bds_add_data_msg {
 		(UINT32_C(0x1) << 28)
 	#define CE_BDS_ADD_DATA_MSG__LAST \
 		CE_BDS_ADD_DATA_MSG__TLS1_3
+	uint8_t	cmd_type_ctx_kind;
 	/*
 	 * Command Type in the TLS header. HW will provide registers that
-	 * converts the 3b encoded command type to 8b of actual command type in
-	 * the TLS Header. * Initialized --> By mid-path command * Updated -->
-	 * Never though another mid-path command will result in an update
+	 * converts the 3b encoded command type to 8b of actual command
+	 * type in the TLS Header. This field is initialized/updated by
+	 * this "KTLS crypto add" mid-path command.
 	 */
-	uint8_t	cmd_type;
 	#define CE_BDS_ADD_DATA_MSG_CMD_TYPE_MASK UINT32_C(0x7)
-	#define CE_BDS_ADD_DATA_MSG_CMD_TYPE_SFT 0
+	#define CE_BDS_ADD_DATA_MSG_CMD_TYPE_SFT  0
 	/* Application */
-	#define CE_BDS_ADD_DATA_MSG_CMD_TYPE_APP   UINT32_C(0x0)
+	#define CE_BDS_ADD_DATA_MSG_CMD_TYPE_APP    UINT32_C(0x0)
 	#define CE_BDS_ADD_DATA_MSG_CMD_TYPE_LAST \
 		CE_BDS_ADD_DATA_MSG_CMD_TYPE_APP
+	/* This field selects the context kind for the request. */
+	#define CE_BDS_ADD_DATA_MSG_CTX_KIND_MASK UINT32_C(0xf8)
+	#define CE_BDS_ADD_DATA_MSG_CTX_KIND_SFT  3
+	/* Crypto key transmit context */
+	#define CE_BDS_ADD_DATA_MSG_CTX_KIND_CK_TX  (UINT32_C(0x11) << 3)
+	/* Crypto key receive context */
+	#define CE_BDS_ADD_DATA_MSG_CTX_KIND_CK_RX  (UINT32_C(0x12) << 3)
+	#define CE_BDS_ADD_DATA_MSG_CTX_KIND_LAST \
+		CE_BDS_ADD_DATA_MSG_CTX_KIND_CK_RX
 	uint8_t	unused0[3];
 	/*
 	 * Salt is part of the nonce that is used as the Initial Vector (IV) in
@@ -2099,13 +2122,13 @@ struct ce_bds_add_data_msg {
 
 /* ce_bds_delete_data_msg (size:64b/8B) */
 struct ce_bds_delete_data_msg {
-	uint32_t	kid_opcode;
+	uint32_t	kid_opcode_ctx_kind;
 	/*
 	 * This value selects the operation for the mid-path command for the
 	 * crypto blocks.
 	 */
-	#define CE_BDS_DELETE_DATA_MSG_OPCODE_MASK  UINT32_C(0xf)
-	#define CE_BDS_DELETE_DATA_MSG_OPCODE_SFT   0
+	#define CE_BDS_DELETE_DATA_MSG_OPCODE_MASK     UINT32_C(0xf)
+	#define CE_BDS_DELETE_DATA_MSG_OPCODE_SFT      0
 	/*
 	 * This is the delete command. Using this opcode, the host Driver
 	 * can remove a key context from the CFCK. If context is deleted
@@ -2115,15 +2138,28 @@ struct ce_bds_delete_data_msg {
 	 * receive packets, no crypto operation will be performed,
 	 * payload will be unmodified.
 	 */
-	#define CE_BDS_DELETE_DATA_MSG_OPCODE_DELETE  UINT32_C(0x2)
+	#define CE_BDS_DELETE_DATA_MSG_OPCODE_DELETE     UINT32_C(0x2)
 	#define CE_BDS_DELETE_DATA_MSG_OPCODE_LAST \
 		CE_BDS_DELETE_DATA_MSG_OPCODE_DELETE
 	/*
 	 * This field is the Crypto Context ID. The KID is used to store
 	 * information used by the associated kTLS offloaded connection.
 	 */
-	#define CE_BDS_DELETE_DATA_MSG_KID_MASK     UINT32_C(0xfffff0)
-	#define CE_BDS_DELETE_DATA_MSG_KID_SFT      4
+	#define CE_BDS_DELETE_DATA_MSG_KID_MASK        UINT32_C(0xfffff0)
+	#define CE_BDS_DELETE_DATA_MSG_KID_SFT         4
+	/* This field selects the context kind for the request. */
+	#define CE_BDS_DELETE_DATA_MSG_CTX_KIND_MASK   UINT32_C(0x1f000000)
+	#define CE_BDS_DELETE_DATA_MSG_CTX_KIND_SFT    24
+	/* Crypto Key Transmit Context. */
+	#define CE_BDS_DELETE_DATA_MSG_CTX_KIND_CK_TX    (UINT32_C(0x11) << 24)
+	/* Crypto Key Receive Context. */
+	#define CE_BDS_DELETE_DATA_MSG_CTX_KIND_CK_RX    (UINT32_C(0x12) << 24)
+	/* QUIC Key Transmit Context. */
+	#define CE_BDS_DELETE_DATA_MSG_CTX_KIND_QUIC_TX  (UINT32_C(0x14) << 24)
+	/* QUIC Key Receive Context. */
+	#define CE_BDS_DELETE_DATA_MSG_CTX_KIND_QUIC_RX  (UINT32_C(0x15) << 24)
+	#define CE_BDS_DELETE_DATA_MSG_CTX_KIND_LAST \
+		CE_BDS_DELETE_DATA_MSG_CTX_KIND_QUIC_RX
 	uint32_t	unused0;
 } __rte_packed;
 
@@ -2352,6 +2388,12 @@ struct bd_base {
 	 */
 	#define BD_BASE_TYPE_TX_BD_PRESYNC_CMD  UINT32_C(0x9)
 	/*
+	 * Indicates a timed transmit BD. This is a 16b BD that is inserted
+	 * into a packet BD chain immediately after the first BD. It is used
+	 * to control the flow in a timed transmit operation.
+	 */
+	#define BD_BASE_TYPE_TX_BD_TIMEDTX      UINT32_C(0xa)
+	/*
 	 * Indicates that this BD is 32B long and is used for
 	 * normal L2 packet transmission.
 	 */
@@ -2632,10 +2674,22 @@ struct tx_bd_long_hi {
 	 */
 	#define TX_BD_LONG_LFLAGS_NOCRC              UINT32_C(0x4)
 	/*
-	 * If set to 1, the device will record the time at which the packet
-	 * was actually transmitted at the TX MAC for 2-step time sync.
+	 * This bit, in conjunction with the stamp_1step bit, controls whether
+	 * a TX packet timestamp is collected and the type of timestamp that
+	 * is collected.
 	 *
 	 * This bit must be valid on the first BD of a packet.
+	 *
+	 * Enumerations of the concatenation { stamp, stamp_1step } are
+	 * as follows:
+	 *
+	 * - 2'b00: ts_none - no timestamp
+	 * - 2'b01: ts_ptp_1step - 1-step PTP
+	 * - 2'b10: ts_2cmpl - 2-step PTP timestamp or PA timestamp
+	 * - 2'b11: ts_rsvd - reserved, same behavior as ts_none
+	 * For the ts_2cmpl enumeration, an additional completion is returned.
+	 * This additional completion may carry a 2-step PTP timestamp or a PA
+	 * timestamp, depending on parsing of the transmitted packet.
 	 */
 	#define TX_BD_LONG_LFLAGS_STAMP              UINT32_C(0x8)
 	/*
@@ -2644,15 +2698,15 @@ struct tx_bd_long_hi {
 	 * of the packet associated with this descriptor.
 	 *
 	 * For outer UDP checksum, global outer UDP checksum TE_NIC register
-	 * needs to be enabled. If the global outer UDP checksum TE_NIC register
-	 * bit is set, outer UDP checksum will be calculated for the following
-	 * cases:
-	 * 1. Packets with tcp_udp_chksum flag set to offload checksum for inner
-	 * packet AND the inner packet is TCP/UDP. If the inner packet is ICMP for
-	 * example (non-TCP/UDP), even if the tcp_udp_chksum is set, the outer UDP
-	 * checksum will not be calculated.
-	 * 2. Packets with lso flag set which implies inner TCP checksum calculation
-	 * as part of LSO operation.
+	 * needs to be enabled. If the global outer UDP checksum TE_NIC
+	 * register bit is set, outer UDP checksum will be calculated for
+	 * the following cases:
+	 * 1. Packets with tcp_udp_chksum flag set to offload checksum for
+	 * inner packet AND the inner packet is TCP/UDP. If the inner packet
+	 * is ICMP for example (non-TCP/UDP), even if the tcp_udp_chksum is
+	 * set, the outer UDP checksum will not be calculated.
+	 * 2. Packets with lso flag set which implies inner TCP checksum
+	 * calculation as part of LSO operation.
 	 */
 	#define TX_BD_LONG_LFLAGS_T_IP_CHKSUM        UINT32_C(0x10)
 	/*
@@ -2710,26 +2764,32 @@ struct tx_bd_long_hi {
 	 */
 	#define TX_BD_LONG_LFLAGS_DEBUG_TRACE        UINT32_C(0x800)
 	/*
-	 * If set to '1', the device will record the time at which the packet
-	 * was actually transmitted at the TX MAC for 1-step time sync. This
-	 * bit must be valid on the first BD of a packet.
+	 * This bit, in conjunction with the stamp bit, controls whether a
+	 * TX packet timestamp is collected and the type of timestamp that
+	 * is collected.
+	 *
+	 * See the stamp field for a description of the valid combinations of
+	 * stamp and stamp_1step.
+	 *
+	 * This bit must be valid on the first BD of a packet.
 	 */
 	#define TX_BD_LONG_LFLAGS_STAMP_1STEP        UINT32_C(0x1000)
 	/*
 	 * If set to '1', the controller replaces the Outer-tunnel IP checksum
 	 * field with hardware calculated IP checksum for the IP header of the
 	 * packet associated with this descriptor. For outer UDP checksum, it
-	 * will be the following behavior for all cases independent of settings
-	 * of inner LSO and checksum offload BD flags. If outer UDP checksum
-	 * is 0, then do not update it. If outer UDP checksum is non zero, then
-	 * the hardware should compute and update it.
+	 * will be the following behavior for all cases independent of
+	 * settings of inner LSO and checksum offload BD flags.
+	 * If outer UDP checksum is 0, then do not update it.
+	 * If outer UDP checksum is non zero, then  the hardware should
+	 * compute and update it.
 	 */
 	#define TX_BD_LONG_LFLAGS_OT_IP_CHKSUM       UINT32_C(0x2000)
 	/*
-	 * If set to zero when LSO is '1', then the IPID of the Outer-tunnel IP
-	 * header will not be modified during LSO operations. If set to one
-	 * when LSO is '1', then the IPID of the Outer-tunnel IP header will be
-	 * incremented for each subsequent segment of an LSO operation. The
+	 * If set to zero when LSO is '1', then the IPID of the Outer-tunnel
+	 * IP header will not be modified during LSO operations. If set to one
+	 * when LSO is '1', then the IPID of the Outer-tunnel IP header will
+	 * be incremented for each subsequent segment of an LSO operation. The
 	 * flag is ignored if the LSO packet is a normal (non-tunneled) TCP
 	 * packet.
 	 */
@@ -2753,8 +2813,8 @@ struct tx_bd_long_hi {
 	#define TX_BD_LONG_HDR_SIZE_SFT      0
 	/*
 	 * If lflags.bd_ts_en is 1, this is the lower 7 bits of the 24-bit
-	 * timestamp. If lflags.crypto_en is 1, this is the lower 7 bits of the
-	 * 20-bit KID.
+	 * timestamp. If lflags.crypto_en is 1, this is the lower 7 bits of
+	 * the 20-bit KID.
 	 */
 	#define TX_BD_LONG_KID_OR_TS_LOW_MASK UINT32_C(0xfe00)
 	#define TX_BD_LONG_KID_OR_TS_LOW_SFT 9
@@ -2861,11 +2921,11 @@ struct tx_bd_long_hi {
 } __rte_packed;
 
 /*
- * This structure is used to inform the NIC of packet data that needs to be
- * transmitted with additional processing that requires extra data such as
- * VLAN insertion plus attached inline data. This BD type may be used to
- * improve latency for small packets needing the additional extended features
- * supported by long BDs.
+ * This structure is used to inform the NIC of packet data that needs to
+ * be transmitted with additional processing that requires extra data
+ * such as VLAN insertion plus attached inline data.
+ * This BD type may be used to improve latency for small packets needing
+ * the additional extended features supported by long BDs.
  */
 /* tx_bd_long_inline (size:256b/32B) */
 struct tx_bd_long_inline {
@@ -2933,13 +2993,13 @@ struct tx_bd_long_inline {
 	uint16_t	len;
 	/*
 	 * The opaque data field is passed through to the completion and can be
-	 * used for any data that the driver wants to associate with the transmit
-	 * BD. This field must be valid on the first BD of a packet. If
-	 * completion coalescing is enabled on the TX ring, it is suggested that
-	 * the driver populate the opaque field to indicate the specific TX ring
-	 * with which the completion is associated, then utilize the opaque and
-	 * sq_cons_idx fields in the coalesced completion record to determine
-	 * the specific packets that are to be completed on that ring.
+	 * used for any data that the driver wants to associate with the
+	 * transmit BD. This field must be valid on the first BD of a packet.
+	 * If completion coalescing is enabled on the TX ring, it is suggested
+	 * that the driver populate the opaque field to indicate the specific
+	 * TX ring with which the completion is associated, then utilize the
+	 * opaque and sq_cons_idx fields in the coalesced completion record to
+	 * determine the specific packets that are to be completed on that ring.
 	 *
 	 * This field must be valid on the first BD of a packet.
 	 */
@@ -3644,8 +3704,20 @@ struct ce_cmpls_cmp_data_msg {
 	/* FID check error. */
 	#define CE_CMPLS_CMP_DATA_MSG_STATUS_FID_CHK_ERR \
 		(UINT32_C(0x2) << 10)
+	/* Context kind / MP version mismatch error. */
+	#define CE_CMPLS_CMP_DATA_MSG_STATUS_CTX_VER_ERR \
+		(UINT32_C(0x3) << 10)
+	/* Unsupported Destination Connection ID Length. */
+	#define CE_CMPLS_CMP_DATA_MSG_STATUS_DST_ID_ERR \
+		(UINT32_C(0x4) << 10)
+	/*
+	 * Invalid MP Command [anything other than ADD or DELETE
+	 * triggers this for QUIC].
+	 */
+	#define CE_CMPLS_CMP_DATA_MSG_STATUS_MP_CMD_ERR \
+		(UINT32_C(0x5) << 10)
 	#define CE_CMPLS_CMP_DATA_MSG_STATUS_LAST \
-		CE_CMPLS_CMP_DATA_MSG_STATUS_FID_CHK_ERR
+		CE_CMPLS_CMP_DATA_MSG_STATUS_MP_CMD_ERR
 	uint8_t	unused0;
 	uint8_t	mp_clients;
 	#define CE_CMPLS_CMP_DATA_MSG_UNUSED1_MASK   UINT32_C(0xf)
@@ -3717,6 +3789,11 @@ struct cmpl_base {
 	 */
 	#define CMPL_BASE_TYPE_TX_L2_PTP         UINT32_C(0x3)
 	/*
+	 * TX L2 Packet Timestamp completion:
+	 * Completion of an L2 Packet Timestamp Packet. Length = 16B
+	 */
+	#define CMPL_BASE_TYPE_TX_L2_PTP_TS      UINT32_C(0x4)
+	/*
 	 * RX L2 TPA Start V2 Completion:
 	 * Completion of and L2 RX packet. Length = 32B
 	 * This is the new version of the RX_TPA_START completion used
@@ -3732,11 +3809,17 @@ struct cmpl_base {
 	#define CMPL_BASE_TYPE_RX_L2_V2          UINT32_C(0xf)
 	/*
 	 * RX L2 completion:
+	 * This is the compressed version of Rx Completion for performance
+	 * applications. Length = 16B
+	 */
+	#define CMPL_BASE_TYPE_RX_L2_COMPRESS    UINT32_C(0x10)
+	/*
+	 * RX L2 completion:
 	 * Completion of and L2 RX packet. Length = 32B
 	 */
 	#define CMPL_BASE_TYPE_RX_L2             UINT32_C(0x11)
 	/*
-	 * RX Aggregation Buffer completion :
+	 * RX Aggregation Buffer completion:
 	 * Completion of an L2 aggregation buffer in support of
 	 * TPA, HDS, or Jumbo packet completion. Length = 16B
 	 */
@@ -3754,6 +3837,24 @@ struct cmpl_base {
 	 */
 	#define CMPL_BASE_TYPE_RX_TPA_END        UINT32_C(0x15)
 	/*
+	 * RX TPA Aggregation Buffer Completion:
+	 * Completion of an L2 aggregation buffer in support of TPA packet
+	 * completion.
+	 * Length = 16B
+	 */
+	#define CMPL_BASE_TYPE_RX_TPA_AGG        UINT32_C(0x16)
+	/*
+	 * RX L2 completion: Completion of and L2 RX packet.
+	 * Length = 32B
+	 */
+	#define CMPL_BASE_TYPE_RX_L2_V3          UINT32_C(0x17)
+	/*
+	 * RX L2 TPA Start completion: Completion at the beginning of a TPA
+	 * operation.
+	 * Length = 32B
+	 */
+	#define CMPL_BASE_TYPE_RX_TPA_START_V3   UINT32_C(0x19)
+	/*
 	 * Statistics Ejection Completion:
 	 * Completion of statistics data ejection buffer.
 	 * Length = 16B
@@ -3920,7 +4021,7 @@ struct tx_cmpl {
 	 */
 	#define TX_CMPL_ERRORS_INTERNAL_ERROR               UINT32_C(0x200)
 	/*
-	 * When this bit is '1', it was not possible to collect a a timestamp
+	 * When this bit is '1', it was not possible to collect a timestamp
 	 * for a PTP completion, in which case the timestamp_hi and
 	 * timestamp_lo fields are invalid. When this bit is '0' for a PTP
 	 * completion, the timestamp_hi and timestamp_lo fields are valid.
@@ -4205,8 +4306,9 @@ struct tx_cmpl_ptp_hi {
 	uint64_t	v2;
 	/*
 	 * This value is written by the NIC such that it will be different for
-	 * each pass through the completion queue.The even passes will write 1.
-	 * The odd passes will write 0
+	 * each pass through the completion queue.
+	 * The even passes will write 1.
+	 * The odd passes will write 0.
 	 */
 	#define TX_CMPL_PTP_HI_V2     UINT32_C(0x1)
 } __rte_packed;
@@ -4221,53 +4323,60 @@ struct rx_pkt_cmpl {
 	 * records. Odd values indicate 32B
 	 * records.
 	 */
-	#define RX_PKT_CMPL_TYPE_MASK                   UINT32_C(0x3f)
-	#define RX_PKT_CMPL_TYPE_SFT                    0
+	#define RX_PKT_CMPL_TYPE_MASK                      UINT32_C(0x3f)
+	#define RX_PKT_CMPL_TYPE_SFT                       0
 	/*
 	 * RX L2 completion:
 	 * Completion of and L2 RX packet. Length = 32B
 	 */
-	#define RX_PKT_CMPL_TYPE_RX_L2                    UINT32_C(0x11)
-	#define RX_PKT_CMPL_TYPE_LAST                    RX_PKT_CMPL_TYPE_RX_L2
-	#define RX_PKT_CMPL_FLAGS_MASK                  UINT32_C(0xffc0)
-	#define RX_PKT_CMPL_FLAGS_SFT                   6
+	#define RX_PKT_CMPL_TYPE_RX_L2                       UINT32_C(0x11)
+	#define RX_PKT_CMPL_TYPE_LAST \
+		RX_PKT_CMPL_TYPE_RX_L2
+	#define RX_PKT_CMPL_FLAGS_MASK                     UINT32_C(0xffc0)
+	#define RX_PKT_CMPL_FLAGS_SFT                      6
 	/*
 	 * When this bit is '1', it indicates a packet that has an
 	 * error of some type. Type of error is indicated in
 	 * error_flags.
 	 */
-	#define RX_PKT_CMPL_FLAGS_ERROR                  UINT32_C(0x40)
+	#define RX_PKT_CMPL_FLAGS_ERROR                     UINT32_C(0x40)
 	/* This field indicates how the packet was placed in the buffer. */
-	#define RX_PKT_CMPL_FLAGS_PLACEMENT_MASK         UINT32_C(0x380)
-	#define RX_PKT_CMPL_FLAGS_PLACEMENT_SFT          7
+	#define RX_PKT_CMPL_FLAGS_PLACEMENT_MASK            UINT32_C(0x380)
+	#define RX_PKT_CMPL_FLAGS_PLACEMENT_SFT             7
 	/*
 	 * Normal:
 	 * Packet was placed using normal algorithm.
 	 */
-	#define RX_PKT_CMPL_FLAGS_PLACEMENT_NORMAL         (UINT32_C(0x0) << 7)
+	#define RX_PKT_CMPL_FLAGS_PLACEMENT_NORMAL \
+		(UINT32_C(0x0) << 7)
 	/*
 	 * Jumbo:
 	 * Packet was placed using jumbo algorithm.
 	 */
-	#define RX_PKT_CMPL_FLAGS_PLACEMENT_JUMBO          (UINT32_C(0x1) << 7)
+	#define RX_PKT_CMPL_FLAGS_PLACEMENT_JUMBO \
+		(UINT32_C(0x1) << 7)
 	/*
 	 * Header/Data Separation:
 	 * Packet was placed using Header/Data separation algorithm.
 	 * The separation location is indicated by the itype field.
 	 */
-	#define RX_PKT_CMPL_FLAGS_PLACEMENT_HDS            (UINT32_C(0x2) << 7)
+	#define RX_PKT_CMPL_FLAGS_PLACEMENT_HDS \
+		(UINT32_C(0x2) << 7)
 	#define RX_PKT_CMPL_FLAGS_PLACEMENT_LAST \
 		RX_PKT_CMPL_FLAGS_PLACEMENT_HDS
 	/* This bit is '1' if the RSS field in this completion is valid. */
-	#define RX_PKT_CMPL_FLAGS_RSS_VALID              UINT32_C(0x400)
-	/* unused is 1 b */
-	#define RX_PKT_CMPL_FLAGS_UNUSED                 UINT32_C(0x800)
+	#define RX_PKT_CMPL_FLAGS_RSS_VALID                 UINT32_C(0x400)
+	/*
+	 * This bit is '1' if metadata has been added to the end of the
+	 * packet in host memory.
+	 */
+	#define RX_PKT_CMPL_FLAGS_PKT_METADATA_PRESENT      UINT32_C(0x800)
 	/*
 	 * This value indicates what the inner packet determined for the
 	 * packet was.
 	 */
-	#define RX_PKT_CMPL_FLAGS_ITYPE_MASK             UINT32_C(0xf000)
-	#define RX_PKT_CMPL_FLAGS_ITYPE_SFT              12
+	#define RX_PKT_CMPL_FLAGS_ITYPE_MASK                UINT32_C(0xf000)
+	#define RX_PKT_CMPL_FLAGS_ITYPE_SFT                 12
 	/*
 	 * Not Known:
 	 * Indicates that the packet type was not known.
@@ -4317,15 +4426,15 @@ struct rx_pkt_cmpl {
 	#define RX_PKT_CMPL_FLAGS_ITYPE_ICMP \
 		(UINT32_C(0x7) << 12)
 	/*
-	 * PtP packet wo/timestamp:
-	 * Indicates that the packet was recognized as a PtP
+	 * PTP packet wo/timestamp:
+	 * Indicates that the packet was recognized as a PTP
 	 * packet.
 	 */
 	#define RX_PKT_CMPL_FLAGS_ITYPE_PTP_WO_TIMESTAMP \
 		(UINT32_C(0x8) << 12)
 	/*
-	 * PtP packet w/timestamp:
-	 * Indicates that the packet was recognized as a PtP
+	 * PTP packet w/timestamp:
+	 * Indicates that the packet was recognized as a PTP
 	 * packet and that a timestamp was taken for the packet.
 	 */
 	#define RX_PKT_CMPL_FLAGS_ITYPE_PTP_W_TIMESTAMP \
@@ -4365,37 +4474,49 @@ struct rx_pkt_cmpl {
 	/*
 	 * This is the RSS hash type for the packet. The value is packed
 	 * {tuple_extrac_op[1:0],rss_profile_id[4:0],tuple_extrac_op[2]}.
-	 *
 	 * The value of tuple_extrac_op provides the information about
 	 * what fields the hash was computed on.
-	 * * 0: The RSS hash was computed over source IP address,
+	 * Note that 4-tuples values listed below are applicable
+	 * for layer 4 protocols supported and enabled for RSS in the hardware,
+	 * HWRM firmware, and drivers. For example, if RSS hash is supported and
+	 * enabled for TCP traffic only, then the values of tuple_extract_op
+	 * corresponding to 4-tuples are only valid for TCP traffic.
+	 */
+	uint8_t	rss_hash_type;
+	/*
+	 * The RSS hash was computed over source IP address,
 	 * destination IP address, source port, and destination port of inner
 	 * IP and TCP or UDP headers. Note: For non-tunneled packets,
 	 * the packet headers are considered inner packet headers for the RSS
 	 * hash computation purpose.
-	 * * 1: The RSS hash was computed over source IP address and destination
+	 */
+	#define RX_PKT_CMPL_RSS_HASH_TYPE_ENUM_0 UINT32_C(0x0)
+	/*
+	 * The RSS hash was computed over source IP address and destination
 	 * IP address of inner IP header. Note: For non-tunneled packets,
 	 * the packet headers are considered inner packet headers for the RSS
 	 * hash computation purpose.
-	 * * 2: The RSS hash was computed over source IP address,
+	 */
+	#define RX_PKT_CMPL_RSS_HASH_TYPE_ENUM_1 UINT32_C(0x1)
+	/*
+	 * The RSS hash was computed over source IP address,
 	 * destination IP address, source port, and destination port of
 	 * IP and TCP or UDP headers of outer tunnel headers.
 	 * Note: For non-tunneled packets, this value is not applicable.
-	 * * 3: The RSS hash was computed over source IP address and
+	 */
+	#define RX_PKT_CMPL_RSS_HASH_TYPE_ENUM_2 UINT32_C(0x2)
+	/*
+	 * The RSS hash was computed over source IP address and
 	 * destination IP address of IP header of outer tunnel headers.
 	 * Note: For non-tunneled packets, this value is not applicable.
-	 *
-	 * Note that 4-tuples values listed above are applicable
-	 * for layer 4 protocols supported and enabled for RSS in the hardware,
-	 * HWRM firmware, and drivers. For example, if RSS hash is supported and
-	 * enabled for TCP traffic only, then the values of tuple_extract_op
-	 * corresponding to 4-tuples are only valid for TCP traffic.
 	 */
-	uint8_t	rss_hash_type;
+	#define RX_PKT_CMPL_RSS_HASH_TYPE_ENUM_3 UINT32_C(0x3)
+	#define RX_PKT_CMPL_RSS_HASH_TYPE_LAST \
+		RX_PKT_CMPL_RSS_HASH_TYPE_ENUM_3
 	/*
-	 * This value indicates the offset in bytes from the beginning of the packet
-	 * where the inner payload starts. This value is valid for TCP, UDP,
-	 * FCoE, and RoCE packets.
+	 * This value indicates the offset in bytes from the beginning of the
+	 * packet where the inner payload starts. This value is valid for TCP,
+	 * UDP, FCoE, and RoCE packets.
 	 *
 	 * A value of zero indicates that header is 256B into the packet.
 	 */
@@ -4882,15 +5003,15 @@ struct rx_pkt_v2_cmpl {
 	#define RX_PKT_V2_CMPL_FLAGS_ITYPE_ICMP \
 		(UINT32_C(0x7) << 12)
 	/*
-	 * PtP packet wo/timestamp:
-	 * Indicates that the packet was recognized as a PtP
+	 * PTP packet wo/timestamp:
+	 * Indicates that the packet was recognized as a PTP
 	 * packet.
 	 */
 	#define RX_PKT_V2_CMPL_FLAGS_ITYPE_PTP_WO_TIMESTAMP \
 		(UINT32_C(0x8) << 12)
 	/*
-	 * PtP packet w/timestamp:
-	 * Indicates that the packet was recognized as a PtP
+	 * PTP packet w/timestamp:
+	 * Indicates that the packet was recognized as a PTP
 	 * packet and that a timestamp was taken for the packet.
 	 */
 	#define RX_PKT_V2_CMPL_FLAGS_ITYPE_PTP_W_TIMESTAMP \
@@ -4930,33 +5051,45 @@ struct rx_pkt_v2_cmpl {
 	/*
 	 * This is the RSS hash type for the packet. The value is packed
 	 * {tuple_extrac_op[1:0],rss_profile_id[4:0],tuple_extrac_op[2]}.
-	 *
 	 * The value of tuple_extrac_op provides the information about
 	 * what fields the hash was computed on.
-	 * * 0: The RSS hash was computed over source IP address,
+	 * Note that 4-tuples values listed below are applicable
+	 * for layer 4 protocols supported and enabled for RSS in the hardware,
+	 * HWRM firmware, and drivers. For example, if RSS hash is supported and
+	 * enabled for TCP traffic only, then the values of tuple_extract_op
+	 * corresponding to 4-tuples are only valid for TCP traffic.
+	 */
+	uint8_t	rss_hash_type;
+	/*
+	 * The RSS hash was computed over source IP address,
 	 * destination IP address, source port, and destination port of inner
 	 * IP and TCP or UDP headers. Note: For non-tunneled packets,
 	 * the packet headers are considered inner packet headers for the RSS
 	 * hash computation purpose.
-	 * * 1: The RSS hash was computed over source IP address and destination
+	 */
+	#define RX_PKT_V2_CMPL_RSS_HASH_TYPE_ENUM_0 UINT32_C(0x0)
+	/*
+	 * The RSS hash was computed over source IP address and destination
 	 * IP address of inner IP header. Note: For non-tunneled packets,
 	 * the packet headers are considered inner packet headers for the RSS
 	 * hash computation purpose.
-	 * * 2: The RSS hash was computed over source IP address,
+	 */
+	#define RX_PKT_V2_CMPL_RSS_HASH_TYPE_ENUM_1 UINT32_C(0x1)
+	/*
+	 * The RSS hash was computed over source IP address,
 	 * destination IP address, source port, and destination port of
 	 * IP and TCP or UDP headers of outer tunnel headers.
 	 * Note: For non-tunneled packets, this value is not applicable.
-	 * * 3: The RSS hash was computed over source IP address and
+	 */
+	#define RX_PKT_V2_CMPL_RSS_HASH_TYPE_ENUM_2 UINT32_C(0x2)
+	/*
+	 * The RSS hash was computed over source IP address and
 	 * destination IP address of IP header of outer tunnel headers.
 	 * Note: For non-tunneled packets, this value is not applicable.
-	 *
-	 * Note that 4-tuples values listed above are applicable
-	 * for layer 4 protocols supported and enabled for RSS in the hardware,
-	 * HWRM firmware, and drivers. For example, if RSS hash is supported and
-	 * enabled for TCP traffic only, then the values of tuple_extract_op
-	 * corresponding to 4-tuples are only valid for TCP traffic.
 	 */
-	uint8_t	rss_hash_type;
+	#define RX_PKT_V2_CMPL_RSS_HASH_TYPE_ENUM_3 UINT32_C(0x3)
+	#define RX_PKT_V2_CMPL_RSS_HASH_TYPE_LAST \
+		RX_PKT_V2_CMPL_RSS_HASH_TYPE_ENUM_3
 	uint16_t	metadata1_payload_offset;
 	/*
 	 * This is data from the CFA as indicated by the meta_format field.
@@ -5048,7 +5181,7 @@ struct rx_pkt_v2_cmpl_hi {
 	 * information:
 	 * - vtag[19:0] = {valid, tpid_sel[2:0], pri[2:0], de, vid[11:0]}
 	 * The metadata2 field contains the Tunnel ID
-	 * value, justified to LSB. i
+	 * value, justified to LSB.
 	 * - VXLAN = VNI[23:0] -> VXLAN Network ID
 	 * - Geneve (NGE) = VNI[23:0] a-> Virtual Network Identifier
 	 * - NVGRE = TNI[23:0] -> Tenant Network ID
@@ -5408,6 +5541,1105 @@ struct rx_pkt_v2_cmpl_hi {
 	uint32_t	timestamp;
 } __rte_packed;
 
+/* rx_pkt_v3_cmpl (size:128b/16B) */
+struct rx_pkt_v3_cmpl {
+	uint16_t	flags_type;
+	/*
+	 * This field indicates the exact type of the completion.
+	 * By convention, the LSB identifies the length of the
+	 * record in 16B units. Even values indicate 16B
+	 * records. Odd values indicate 32B
+	 * records.
+	 */
+	#define RX_PKT_V3_CMPL_TYPE_MASK                      UINT32_C(0x3f)
+	#define RX_PKT_V3_CMPL_TYPE_SFT                       0
+	/*
+	 * RX L2 V3 completion:
+	 * Completion of and L2 RX packet. Length = 32B
+	 * This is the new version of the RX_L2 completion used in Thor2
+	 * and later chips.
+	 */
+	#define RX_PKT_V3_CMPL_TYPE_RX_L2_V3                    UINT32_C(0x17)
+	#define RX_PKT_V3_CMPL_TYPE_LAST \
+		RX_PKT_V3_CMPL_TYPE_RX_L2_V3
+	#define RX_PKT_V3_CMPL_FLAGS_MASK                     UINT32_C(0xffc0)
+	#define RX_PKT_V3_CMPL_FLAGS_SFT                      6
+	/*
+	 * When this bit is '1', it indicates a packet that has an
+	 * error of some type. Type of error is indicated in
+	 * error_flags.
+	 */
+	#define RX_PKT_V3_CMPL_FLAGS_ERROR                     UINT32_C(0x40)
+	/* This field indicates how the packet was placed in the buffer. */
+	#define RX_PKT_V3_CMPL_FLAGS_PLACEMENT_MASK            UINT32_C(0x380)
+	#define RX_PKT_V3_CMPL_FLAGS_PLACEMENT_SFT             7
+	/*
+	 * Normal:
+	 * Packet was placed using normal algorithm.
+	 */
+	#define RX_PKT_V3_CMPL_FLAGS_PLACEMENT_NORMAL \
+		(UINT32_C(0x0) << 7)
+	/*
+	 * Jumbo:
+	 * Packet was placed using jumbo algorithm.
+	 */
+	#define RX_PKT_V3_CMPL_FLAGS_PLACEMENT_JUMBO \
+		(UINT32_C(0x1) << 7)
+	/*
+	 * Header/Data Separation:
+	 * Packet was placed using Header/Data separation algorithm.
+	 * The separation location is indicated by the itype field.
+	 */
+	#define RX_PKT_V3_CMPL_FLAGS_PLACEMENT_HDS \
+		(UINT32_C(0x2) << 7)
+	/*
+	 * Truncation:
+	 * Packet was placed using truncation algorithm. The
+	 * placed (truncated) length is indicated in the payload_offset
+	 * field. The original length is indicated in the len field.
+	 */
+	#define RX_PKT_V3_CMPL_FLAGS_PLACEMENT_TRUNCATION \
+		(UINT32_C(0x3) << 7)
+	#define RX_PKT_V3_CMPL_FLAGS_PLACEMENT_LAST \
+		RX_PKT_V3_CMPL_FLAGS_PLACEMENT_TRUNCATION
+	/* This bit is '1' if the RSS field in this completion is valid. */
+	#define RX_PKT_V3_CMPL_FLAGS_RSS_VALID                 UINT32_C(0x400)
+	/*
+	 * This bit is '1' if metadata has been added to the end of the
+	 * packet in host memory. Metadata starts at the first 32B boundary
+	 * after the end of the packet for regular and jumbo placement.
+	 * It starts at the first 32B boundary after the end of the header
+	 * for HDS placement. The length of the metadata is indicated in the
+	 * metadata itself.
+	 */
+	#define RX_PKT_V3_CMPL_FLAGS_PKT_METADATA_PRESENT      UINT32_C(0x800)
+	/*
+	 * This value indicates what the inner packet determined for the
+	 * packet was.
+	 */
+	#define RX_PKT_V3_CMPL_FLAGS_ITYPE_MASK                UINT32_C(0xf000)
+	#define RX_PKT_V3_CMPL_FLAGS_ITYPE_SFT                 12
+	/*
+	 * Not Known:
+	 * Indicates that the packet type was not known.
+	 */
+	#define RX_PKT_V3_CMPL_FLAGS_ITYPE_NOT_KNOWN \
+		(UINT32_C(0x0) << 12)
+	/*
+	 * IP Packet:
+	 * Indicates that the packet was an IP packet, but further
+	 * classification was not possible.
+	 */
+	#define RX_PKT_V3_CMPL_FLAGS_ITYPE_IP \
+		(UINT32_C(0x1) << 12)
+	/*
+	 * TCP Packet:
+	 * Indicates that the packet was IP and TCP.
+	 * This indicates that the payload_offset field is valid.
+	 */
+	#define RX_PKT_V3_CMPL_FLAGS_ITYPE_TCP \
+		(UINT32_C(0x2) << 12)
+	/*
+	 * UDP Packet:
+	 * Indicates that the packet was IP and UDP.
+	 * This indicates that the payload_offset field is valid.
+	 */
+	#define RX_PKT_V3_CMPL_FLAGS_ITYPE_UDP \
+		(UINT32_C(0x3) << 12)
+	/*
+	 * FCoE Packet:
+	 * Indicates that the packet was recognized as a FCoE.
+	 * This also indicates that the payload_offset field is valid.
+	 */
+	#define RX_PKT_V3_CMPL_FLAGS_ITYPE_FCOE \
+		(UINT32_C(0x4) << 12)
+	/*
+	 * RoCE Packet:
+	 * Indicates that the packet was recognized as a RoCE.
+	 * This also indicates that the payload_offset field is valid.
+	 */
+	#define RX_PKT_V3_CMPL_FLAGS_ITYPE_ROCE \
+		(UINT32_C(0x5) << 12)
+	/*
+	 * ICMP Packet:
+	 * Indicates that the packet was recognized as ICMP.
+	 * This indicates that the payload_offset field is valid.
+	 */
+	#define RX_PKT_V3_CMPL_FLAGS_ITYPE_ICMP \
+		(UINT32_C(0x7) << 12)
+	/*
+	 * PTP packet wo/timestamp:
+	 * Indicates that the packet was recognized as a PTP
+	 * packet.
+	 */
+	#define RX_PKT_V3_CMPL_FLAGS_ITYPE_PTP_WO_TIMESTAMP \
+		(UINT32_C(0x8) << 12)
+	/*
+	 * PTP packet w/timestamp:
+	 * Indicates that the packet was recognized as a PTP
+	 * packet and that a timestamp was taken for the packet.
+	 * The 4b sub-nanosecond portion of the timestamp is in
+	 * the payload_offset field.
+	 */
+	#define RX_PKT_V3_CMPL_FLAGS_ITYPE_PTP_W_TIMESTAMP \
+		(UINT32_C(0x9) << 12)
+	#define RX_PKT_V3_CMPL_FLAGS_ITYPE_LAST \
+		RX_PKT_V3_CMPL_FLAGS_ITYPE_PTP_W_TIMESTAMP
+	/*
+	 * This is the length of the data for the packet stored in the
+	 * buffer(s) identified by the opaque value. This includes
+	 * the packet BD and any associated buffer BDs. This does not include
+	 * the length of any data places in aggregation BDs.
+	 */
+	uint16_t	len;
+	/*
+	 * This is a copy of the opaque field from the RX BD this completion
+	 * corresponds to.
+	 */
+	uint32_t	opaque;
+	uint16_t	rss_hash_type_agg_bufs_v1;
+	/*
+	 * This value is written by the NIC such that it will be different
+	 * for each pass through the completion queue. The even passes
+	 * will write 1. The odd passes will write 0.
+	 */
+	#define RX_PKT_V3_CMPL_V1                   UINT32_C(0x1)
+	/*
+	 * This value is the number of aggregation buffers that follow this
+	 * entry in the completion ring that are a part of this packet.
+	 * If the value is zero, then the packet is completely contained
+	 * in the buffer space provided for the packet in the RX ring.
+	 */
+	#define RX_PKT_V3_CMPL_AGG_BUFS_MASK        UINT32_C(0x3e)
+	#define RX_PKT_V3_CMPL_AGG_BUFS_SFT         1
+	/* unused1 is 1 b */
+	#define RX_PKT_V3_CMPL_UNUSED1              UINT32_C(0x40)
+	/*
+	 * This is the RSS hash type for the packet. The value is packed
+	 * {tuple_extrac_op[1:0],rss_profile_id[4:0],tuple_extrac_op[2]}.
+	 * The value of tuple_extrac_op provides the information about
+	 * what fields the hash was computed on.
+	 * Note that 4-tuples values listed below are applicable
+	 * for layer 4 protocols supported and enabled for RSS in the
+	 * hardware, HWRM firmware, and drivers. For example, if RSS hash
+	 * is supported and enabled for TCP traffic only, then the values of
+	 * tuple_extract_op corresponding to 4-tuples are only valid for
+	 * TCP traffic.
+	 */
+	#define RX_PKT_V3_CMPL_RSS_HASH_TYPE_MASK   UINT32_C(0xff80)
+	#define RX_PKT_V3_CMPL_RSS_HASH_TYPE_SFT    7
+	/*
+	 * The RSS hash was computed over source IP address,
+	 * destination IP address, source port, and destination port of
+	 * inner IP and TCP or UDP headers.
+	 */
+	#define RX_PKT_V3_CMPL_RSS_HASH_TYPE_ENUM_0   (UINT32_C(0x0) << 7)
+	/*
+	 * The RSS hash was computed over source IP address and
+	 * destination IP address of inner IP header.
+	 */
+	#define RX_PKT_V3_CMPL_RSS_HASH_TYPE_ENUM_1   (UINT32_C(0x1) << 7)
+	/*
+	 * The RSS hash was computed over source IP address,
+	 * destination IP address, source port, and destination port of
+	 * IP and TCP or UDP headers of outer tunnel headers.
+	 * Note: For non-tunneled packets, this value is not applicable.
+	 */
+	#define RX_PKT_V3_CMPL_RSS_HASH_TYPE_ENUM_2   (UINT32_C(0x2) << 7)
+	/*
+	 * The RSS hash was computed over source IP address and
+	 * destination IP address of IP header of outer tunnel headers.
+	 * Note: For non-tunneled packets, this value is not applicable.
+	 */
+	#define RX_PKT_V3_CMPL_RSS_HASH_TYPE_ENUM_3   (UINT32_C(0x3) << 7)
+	/*
+	 * The RSS hash was computed over source IP address of the inner
+	 * IP header.
+	 */
+	#define RX_PKT_V3_CMPL_RSS_HASH_TYPE_ENUM_4   (UINT32_C(0x4) << 7)
+	/*
+	 * The RSS hash was computed over destination IP address of the
+	 * inner IP header.
+	 */
+	#define RX_PKT_V3_CMPL_RSS_HASH_TYPE_ENUM_5   (UINT32_C(0x5) << 7)
+	/*
+	 * The RSS hash was computed over source IP address of the outer
+	 * IP header.
+	 * Note: For non-tunneled packets, this value is not applicable.
+	 */
+	#define RX_PKT_V3_CMPL_RSS_HASH_TYPE_ENUM_6   (UINT32_C(0x6) << 7)
+	/*
+	 * The RSS hash was computed over destination IP address of the
+	 * outer IP header.
+	 * Note: For non-tunneled packets, this value is not applicable.
+	 */
+	#define RX_PKT_V3_CMPL_RSS_HASH_TYPE_ENUM_7   (UINT32_C(0x7) << 7)
+	/*
+	 * The RSS hash was computed over source IP address, destination
+	 * IP address, and flow label of the inner IP header.
+	 * Note: For packets without an inner IPv6 header, this value is not
+	 * this value is not applicable.
+	 */
+	#define RX_PKT_V3_CMPL_RSS_HASH_TYPE_ENUM_8   (UINT32_C(0x8) << 7)
+	/*
+	 * The RSS hash was computed over the flow label of the inner
+	 * IP header.
+	 * Note: For packets without an inner IPv6 header, this value
+	 * is not applicable.
+	 */
+	#define RX_PKT_V3_CMPL_RSS_HASH_TYPE_ENUM_9   (UINT32_C(0x9) << 7)
+	/*
+	 * The RSS hash was computed over source IP address, destination
+	 * IP address, and flow label of the outer IP header.
+	 * Note: For packets without an outer IPv6 header, this value is not
+	 * applicable.
+	 */
+	#define RX_PKT_V3_CMPL_RSS_HASH_TYPE_ENUM_10  (UINT32_C(0xa) << 7)
+	/*
+	 * The RSS hash was computed over the flow label of the outer
+	 * IP header.
+	 * Note: For packets without an outer IPv6 header, this value
+	 * is not applicable.
+	 */
+	#define RX_PKT_V3_CMPL_RSS_HASH_TYPE_ENUM_11  (UINT32_C(0xb) << 7)
+	#define RX_PKT_V3_CMPL_RSS_HASH_TYPE_LAST \
+		RX_PKT_V3_CMPL_RSS_HASH_TYPE_ENUM_11
+	uint16_t	metadata1_payload_offset;
+	/*
+	 * If truncation placement is not used, this value indicates the offset
+	 * in bytes from the beginning of the packet where the inner payload
+	 * starts. This value is valid for TCP, UDP, FCoE, and RoCE packets.
+	 * For PTP packets with timestamp (as indicated by the flags_itype
+	 * field), this field contains the 4b sub-nanosecond portion of the
+	 * timestamp.
+	 *
+	 * If truncation placement is used, this value represents the placed
+	 * (truncated) length of the packet.
+	 */
+	#define RX_PKT_V3_CMPL_PAYLOAD_OFFSET_MASK        UINT32_C(0x1ff)
+	#define RX_PKT_V3_CMPL_PAYLOAD_OFFSET_SFT         0
+	/* This is data from the CFA as indicated by the meta_format field. */
+	#define RX_PKT_V3_CMPL_METADATA1_MASK             UINT32_C(0xf000)
+	#define RX_PKT_V3_CMPL_METADATA1_SFT              12
+	/* When meta_format != 0, this value is the VLAN TPID_SEL. */
+	#define RX_PKT_V3_CMPL_METADATA1_TPID_SEL_MASK     UINT32_C(0x7000)
+	#define RX_PKT_V3_CMPL_METADATA1_TPID_SEL_SFT      12
+	/* 0x88a8 */
+	#define RX_PKT_V3_CMPL_METADATA1_TPID_SEL_TPID88A8 \
+		(UINT32_C(0x0) << 12)
+	/* 0x8100 */
+	#define RX_PKT_V3_CMPL_METADATA1_TPID_SEL_TPID8100 \
+		(UINT32_C(0x1) << 12)
+	/* 0x9100 */
+	#define RX_PKT_V3_CMPL_METADATA1_TPID_SEL_TPID9100 \
+		(UINT32_C(0x2) << 12)
+	/* 0x9200 */
+	#define RX_PKT_V3_CMPL_METADATA1_TPID_SEL_TPID9200 \
+		(UINT32_C(0x3) << 12)
+	/* 0x9300 */
+	#define RX_PKT_V3_CMPL_METADATA1_TPID_SEL_TPID9300 \
+		(UINT32_C(0x4) << 12)
+	/* Value programmed in CFA VLANTPID register. */
+	#define RX_PKT_V3_CMPL_METADATA1_TPID_SEL_TPIDCFG \
+		(UINT32_C(0x5) << 12)
+	#define RX_PKT_V3_CMPL_METADATA1_TPID_SEL_LAST \
+		RX_PKT_V3_CMPL_METADATA1_TPID_SEL_TPIDCFG
+	/* When meta_format != 0, this value is the VLAN valid. */
+	#define RX_PKT_V3_CMPL_METADATA1_VALID             UINT32_C(0x8000)
+	/*
+	 * This value is the RSS hash value calculated for the packet
+	 * based on the mode bits and key value in the VNIC. When hairpin_en
+	 * is set in VNIC context, this is the lower 32b of the host address
+	 * from the first BD used to place the packet.
+	 */
+	uint32_t	rss_hash;
+} __rte_packed;
+
+/* Last 16 bytes of RX Packet V3 Completion Record */
+/* rx_pkt_v3_cmpl_hi (size:128b/16B) */
+struct rx_pkt_v3_cmpl_hi {
+	uint32_t	flags2;
+	/*
+	 * This indicates that the ip checksum was calculated for the inner
+	 * packet and that the ip_cs_error field indicates if there was an
+	 * error.
+	 */
+	#define RX_PKT_V3_CMPL_HI_FLAGS2_IP_CS_CALC \
+		UINT32_C(0x1)
+	/*
+	 * This indicates that the TCP, UDP or ICMP checksum was calculated
+	 * for the inner packet and that the l4_cs_error field indicates if
+	 * there was an error.
+	 */
+	#define RX_PKT_V3_CMPL_HI_FLAGS2_L4_CS_CALC \
+		UINT32_C(0x2)
+	/*
+	 * This indicates that the ip checksum was calculated for the tunnel
+	 * header and that the t_ip_cs_error field indicates if there was an
+	 * error.
+	 */
+	#define RX_PKT_V3_CMPL_HI_FLAGS2_T_IP_CS_CALC \
+		UINT32_C(0x4)
+	/*
+	 * This indicates that the UDP checksum was calculated for the tunnel
+	 * packet and that the t_l4_cs_error field indicates if there was an
+	 * error.
+	 */
+	#define RX_PKT_V3_CMPL_HI_FLAGS2_T_L4_CS_CALC \
+		UINT32_C(0x8)
+	/* This value indicates what format the metadata field is. */
+	#define RX_PKT_V3_CMPL_HI_FLAGS2_META_FORMAT_MASK \
+		UINT32_C(0xf0)
+	#define RX_PKT_V3_CMPL_HI_FLAGS2_META_FORMAT_SFT            4
+	/* There is no metadata information. Values are zero. */
+	#define RX_PKT_V3_CMPL_HI_FLAGS2_META_FORMAT_NONE \
+		(UINT32_C(0x0) << 4)
+	/*
+	 * The {metadata1, metadata0} fields contain the vtag
+	 * information: - vtag[19:0] = {valid, tpid_sel[2:0], pri[2:0],
+	 * de, vid[11:0]} The metadata2 field contains the table scope
+	 * and action record pointer. - metadata2[25:0] contains the
+	 * action record pointer. - metadata2[31:26] contains the table
+	 * scope.
+	 */
+	#define RX_PKT_V3_CMPL_HI_FLAGS2_META_FORMAT_ACT_REC_PTR \
+		(UINT32_C(0x1) << 4)
+	/*
+	 * The {metadata1, metadata0} fields contain the vtag
+	 * information:
+	 * - vtag[19:0] = {valid, tpid_sel[2:0], pri[2:0], de, vid[11:0]}
+	 * The metadata2 field contains the Tunnel ID
+	 * value, justified to LSB.
+	 * - VXLAN = VNI[23:0] -> VXLAN Network ID
+	 * - Geneve (NGE) = VNI[23:0] a-> Virtual Network Identifier
+	 * - NVGRE = TNI[23:0] -> Tenant Network ID
+	 * - GRE = KEY[31:0] -> key field with bit mask. zero if K=0
+	 * - IPv4 = 0 (not populated)
+	 * - IPv6 = Flow Label[19:0]
+	 * - PPPoE = sessionID[15:0]
+	 * - MPLs = Outer label[19:0]
+	 * - UPAR = Selected[31:0] with bit mask
+	 */
+	#define RX_PKT_V3_CMPL_HI_FLAGS2_META_FORMAT_TUNNEL_ID \
+		(UINT32_C(0x2) << 4)
+	/*
+	 * The {metadata1, metadata0} fields contain the vtag
+	 * information:
+	 * - vtag[19:0] = {valid, tpid_sel[2:0], pri[2:0],de, vid[11:0]}
+	 * The metadata2 field contains the 32b metadata from the prepended
+	 * header (chdr_data).
+	 */
+	#define RX_PKT_V3_CMPL_HI_FLAGS2_META_FORMAT_CHDR_DATA \
+		(UINT32_C(0x3) << 4)
+	/*
+	 * The {metadata1, metadata0} fields contain the vtag
+	 * information:
+	 * - vtag[19:0] = {valid, tpid_sel[2:0], pri[2:0], de, vid[11:0]}
+	 * The metadata2 field contains the outer_l3_offset,
+	 * inner_l2_offset, inner_l3_offset, and inner_l4_size.
+	 * - metadata2[8:0] contains the outer_l3_offset.
+	 * - metadata2[17:9] contains the inner_l2_offset.
+	 * - metadata2[26:18] contains the inner_l3_offset.
+	 * - metadata2[31:27] contains the inner_l4_size.
+	 */
+	#define RX_PKT_V3_CMPL_HI_FLAGS2_META_FORMAT_HDR_OFFSET \
+		(UINT32_C(0x4) << 4)
+	#define RX_PKT_V3_CMPL_HI_FLAGS2_META_FORMAT_LAST \
+		RX_PKT_V3_CMPL_HI_FLAGS2_META_FORMAT_HDR_OFFSET
+	/*
+	 * This field indicates the IP type for the inner-most IP header.
+	 * A value of '0' indicates IPv4. A value of '1' indicates IPv6.
+	 * This value is only valid if itype indicates a packet
+	 * with an IP header.
+	 */
+	#define RX_PKT_V3_CMPL_HI_FLAGS2_IP_TYPE \
+		UINT32_C(0x100)
+	/*
+	 * This indicates that the complete 1's complement checksum was
+	 * calculated for the packet.
+	 */
+	#define RX_PKT_V3_CMPL_HI_FLAGS2_COMPLETE_CHECKSUM_CALC \
+		UINT32_C(0x200)
+	/*
+	 * This field indicates the status of IP and L4 CS calculations done
+	 * by the chip. The format of this field is indicated by the
+	 * cs_all_ok_mode bit.
+	 */
+	#define RX_PKT_V3_CMPL_HI_FLAGS2_T_IP_TYPE \
+		UINT32_C(0x400)
+	/* Indicates that the Tunnel IP type was IPv4 */
+	#define RX_PKT_V3_CMPL_HI_FLAGS2_T_IP_TYPE_IPV4 \
+		(UINT32_C(0x0) << 10)
+	/* Indicates that the Tunnel IP type was IPv6 */
+	#define RX_PKT_V3_CMPL_HI_FLAGS2_T_IP_TYPE_IPV6 \
+		(UINT32_C(0x1) << 10)
+	#define RX_PKT_V3_CMPL_HI_FLAGS2_T_IP_TYPE_LAST \
+		RX_PKT_V3_CMPL_HI_FLAGS2_T_IP_TYPE_IPV6
+	/*
+	 * This value is the complete 1's complement checksum calculated from
+	 * the start of the outer L3 header to the end of the packet (not
+	 * including the ethernet crc). It is valid when the
+	 * 'complete_checksum_calc' flag is set.
+	 */
+	#define RX_PKT_V3_CMPL_HI_FLAGS2_COMPLETE_CHECKSUM_MASK \
+		UINT32_C(0xffff0000)
+	#define RX_PKT_V3_CMPL_HI_FLAGS2_COMPLETE_CHECKSUM_SFT      16
+	/*
+	 * This is data from the CFA block as indicated by the meta_format
+	 * field.
+	 * - meta_format 0 - none - metadata2 = 0 - not valid/not stripped
+	 * - meta_format 1 - act_rec_ptr - metadata2 = {table_scope[5:0],
+	 *   act_rec_ptr[25:0]}
+	 * - meta_format 2 - tunnel_id - metadata2 = tunnel_id[31:0]
+	 * - meta_format 3 - chdr_data - metadata2 = updated_chdr_data[31:0]
+	 * - meta_format 4 - hdr_offsets - metadata2 = hdr_offsets[31:0]
+	 */
+	uint32_t	metadata2;
+	uint16_t	errors_v2;
+	/*
+	 * This value is written by the NIC such that it will be different
+	 * for each pass through the completion queue. The even passes
+	 * will write 1. The odd passes will write 0.
+	 */
+	#define RX_PKT_V3_CMPL_HI_V2 \
+		UINT32_C(0x1)
+	#define RX_PKT_V3_CMPL_HI_ERRORS_MASK \
+		UINT32_C(0xfffe)
+	#define RX_PKT_V3_CMPL_HI_ERRORS_SFT                               1
+	/*
+	 * This error indicates that there was some sort of problem with
+	 * the BDs for the packet that was found after part of the
+	 * packet was already placed. The packet should be treated as
+	 * invalid.
+	 */
+	#define RX_PKT_V3_CMPL_HI_ERRORS_BUFFER_ERROR_MASK \
+		UINT32_C(0xe)
+	#define RX_PKT_V3_CMPL_HI_ERRORS_BUFFER_ERROR_SFT                   1
+	/* No buffer error */
+	#define RX_PKT_V3_CMPL_HI_ERRORS_BUFFER_ERROR_NO_BUFFER \
+		(UINT32_C(0x0) << 1)
+	/*
+	 * Did Not Fit: Packet did not fit into packet buffer provided.
+	 * For regular placement, this means the packet did not fit in
+	 * the buffer provided. For HDS and jumbo placement, this means
+	 * that the packet could not be placed into 8 physical buffers.
+	 */
+	#define RX_PKT_V3_CMPL_HI_ERRORS_BUFFER_ERROR_DID_NOT_FIT \
+		(UINT32_C(0x1) << 1)
+	/*
+	 * Not On Chip: All BDs needed for the packet were not on-chip
+	 * when the packet arrived. For regular placement, this error is
+	 * not valid. For HDS and jumbo placement, this means that not
+	 * enough agg BDs were posted to place the packet.
+	 */
+	#define RX_PKT_V3_CMPL_HI_ERRORS_BUFFER_ERROR_NOT_ON_CHIP \
+		(UINT32_C(0x2) << 1)
+	/*
+	 * Bad Format:
+	 * BDs were not formatted correctly.
+	 */
+	#define RX_PKT_V3_CMPL_HI_ERRORS_BUFFER_ERROR_BAD_FORMAT \
+		(UINT32_C(0x3) << 1)
+	/*
+	 * Flush:
+	 * There was a bad_format error on the previous operation
+	 */
+	#define RX_PKT_V3_CMPL_HI_ERRORS_BUFFER_ERROR_FLUSH \
+		(UINT32_C(0x5) << 1)
+	#define RX_PKT_V3_CMPL_HI_ERRORS_BUFFER_ERROR_LAST \
+		RX_PKT_V3_CMPL_HI_ERRORS_BUFFER_ERROR_FLUSH
+	/* This indicates that there was an error in the IP header checksum. */
+	#define RX_PKT_V3_CMPL_HI_ERRORS_IP_CS_ERROR \
+		UINT32_C(0x10)
+	/*
+	 * This indicates that there was an error in the TCP, UDP or ICMP
+	 * checksum.
+	 */
+	#define RX_PKT_V3_CMPL_HI_ERRORS_L4_CS_ERROR \
+		UINT32_C(0x20)
+	/*
+	 * This indicates that there was an error in the tunnel IP header
+	 * checksum.
+	 */
+	#define RX_PKT_V3_CMPL_HI_ERRORS_T_IP_CS_ERROR \
+		UINT32_C(0x40)
+	/* This indicates that there was an error in the tunnel UDP checksum. */
+	#define RX_PKT_V3_CMPL_HI_ERRORS_T_L4_CS_ERROR \
+		UINT32_C(0x80)
+	/*
+	 * This indicates that there was a CRC error on either an FCoE
+	 * or RoCE packet. The itype indicates the packet type.
+	 */
+	#define RX_PKT_V3_CMPL_HI_ERRORS_CRC_ERROR \
+		UINT32_C(0x100)
+	/*
+	 * This indicates that there was an error in the tunnel portion
+	 * of the packet when this field is non-zero.
+	 */
+	#define RX_PKT_V3_CMPL_HI_ERRORS_T_PKT_ERROR_MASK \
+		UINT32_C(0xe00)
+	#define RX_PKT_V3_CMPL_HI_ERRORS_T_PKT_ERROR_SFT                    9
+	/*
+	 * No additional error occurred on the tunnel portion
+	 * of the packet or the packet does not have a tunnel.
+	 */
+	#define RX_PKT_V3_CMPL_HI_ERRORS_T_PKT_ERROR_NO_ERROR \
+		(UINT32_C(0x0) << 9)
+	/*
+	 * Indicates that IP header version does not match expectation
+	 * from L2 Ethertype for IPv4 and IPv6 in the tunnel header.
+	 */
+	#define RX_PKT_V3_CMPL_HI_ERRORS_T_PKT_ERROR_T_L3_BAD_VERSION \
+		(UINT32_C(0x1) << 9)
+	/*
+	 * Indicates that header length is out of range in the tunnel
+	 * header. Valid for IPv4.
+	 */
+	#define RX_PKT_V3_CMPL_HI_ERRORS_T_PKT_ERROR_T_L3_BAD_HDR_LEN \
+		(UINT32_C(0x2) << 9)
+	/*
+	 * Indicates that physical packet is shorter than that claimed
+	 * by the tunnel l3 header length. Valid for IPv4, or IPv6 tunnel
+	 * packet packets.
+	 */
+	#define RX_PKT_V3_CMPL_HI_ERRORS_T_PKT_ERROR_T_IP_TOTAL_ERROR \
+		(UINT32_C(0x3) << 9)
+	/*
+	 * Indicates that the physical packet is shorter than that claimed
+	 * by the tunnel UDP header length for a tunnel UDP packet that is
+	 * not fragmented.
+	 */
+	#define RX_PKT_V3_CMPL_HI_ERRORS_T_PKT_ERROR_T_UDP_TOTAL_ERROR \
+		(UINT32_C(0x4) << 9)
+	/*
+	 * Indicates that the IPv4 TTL or IPv6 hop limit check have failed
+	 * (e.g. TTL = 0) in the tunnel header. Valid for IPv4, and IPv6.
+	 */
+	#define RX_PKT_V3_CMPL_HI_ERRORS_T_PKT_ERROR_T_L3_BAD_TTL \
+		(UINT32_C(0x5) << 9)
+	/*
+	 * Indicates that the IP checksum failed its check in the tunnel
+	 * header.
+	 */
+	#define RX_PKT_V3_CMPL_HI_ERRORS_T_PKT_ERROR_T_IP_CS_ERROR \
+		(UINT32_C(0x6) << 9)
+	/*
+	 * Indicates that the L4 checksum failed its check in the tunnel
+	 * header.
+	 */
+	#define RX_PKT_V3_CMPL_HI_ERRORS_T_PKT_ERROR_T_L4_CS_ERROR \
+		(UINT32_C(0x7) << 9)
+	#define RX_PKT_V3_CMPL_HI_ERRORS_T_PKT_ERROR_LAST \
+		RX_PKT_V3_CMPL_HI_ERRORS_T_PKT_ERROR_T_L4_CS_ERROR
+	/*
+	 * This indicates that there was an error in the inner
+	 * portion of the packet when this
+	 * field is non-zero.
+	 */
+	#define RX_PKT_V3_CMPL_HI_ERRORS_PKT_ERROR_MASK \
+		UINT32_C(0xf000)
+	#define RX_PKT_V3_CMPL_HI_ERRORS_PKT_ERROR_SFT                      12
+	/*
+	 * No additional error occurred on the tunnel portion
+	 * or the packet of the packet does not have a tunnel.
+	 */
+	#define RX_PKT_V3_CMPL_HI_ERRORS_PKT_ERROR_NO_ERROR \
+		(UINT32_C(0x0) << 12)
+	/*
+	 * Indicates that IP header version does not match
+	 * expectation from L2 Ethertype for IPv4 and IPv6 or that
+	 * option other than VFT was parsed on
+	 * FCoE packet.
+	 */
+	#define RX_PKT_V3_CMPL_HI_ERRORS_PKT_ERROR_L3_BAD_VERSION \
+		(UINT32_C(0x1) << 12)
+	/*
+	 * indicates that header length is out of range. Valid for
+	 * IPv4 and RoCE
+	 */
+	#define RX_PKT_V3_CMPL_HI_ERRORS_PKT_ERROR_L3_BAD_HDR_LEN \
+		(UINT32_C(0x2) << 12)
+	/*
+	 * indicates that the IPv4 TTL or IPv6 hop limit check
+	 * have failed (e.g. TTL = 0). Valid for IPv4, and IPv6
+	 */
+	#define RX_PKT_V3_CMPL_HI_ERRORS_PKT_ERROR_L3_BAD_TTL \
+		(UINT32_C(0x3) << 12)
+	/*
+	 * Indicates that physical packet is shorter than that
+	 * claimed by the l3 header length. Valid for IPv4,
+	 * IPv6 packet or RoCE packets.
+	 */
+	#define RX_PKT_V3_CMPL_HI_ERRORS_PKT_ERROR_IP_TOTAL_ERROR \
+		(UINT32_C(0x4) << 12)
+	/*
+	 * Indicates that the physical packet is shorter than that
+	 * claimed by the UDP header length for a UDP packet that is
+	 * not fragmented.
+	 */
+	#define RX_PKT_V3_CMPL_HI_ERRORS_PKT_ERROR_UDP_TOTAL_ERROR \
+		(UINT32_C(0x5) << 12)
+	/*
+	 * Indicates that TCP header length > IP payload. Valid for
+	 * TCP packets only.
+	 */
+	#define RX_PKT_V3_CMPL_HI_ERRORS_PKT_ERROR_L4_BAD_HDR_LEN \
+		(UINT32_C(0x6) << 12)
+	/* Indicates that TCP header length < 5. Valid for TCP. */
+	#define RX_PKT_V3_CMPL_HI_ERRORS_PKT_ERROR_L4_BAD_HDR_LEN_TOO_SMALL \
+		(UINT32_C(0x7) << 12)
+	/*
+	 * Indicates that TCP option headers result in a TCP header
+	 * size that does not match data offset in TCP header. Valid
+	 * for TCP.
+	 */
+	#define RX_PKT_V3_CMPL_HI_ERRORS_PKT_ERROR_L4_BAD_OPT_LEN \
+		(UINT32_C(0x8) << 12)
+	/*
+	 * Indicates that the IP checksum failed its check in the
+	 * inner header.
+	 */
+	#define RX_PKT_V3_CMPL_HI_ERRORS_PKT_ERROR_IP_CS_ERROR \
+		(UINT32_C(0x9) << 12)
+	/*
+	 * Indicates that the L4 checksum failed its check in the
+	 * inner header.
+	 */
+	#define RX_PKT_V3_CMPL_HI_ERRORS_PKT_ERROR_L4_CS_ERROR \
+		(UINT32_C(0xa) << 12)
+	#define RX_PKT_V3_CMPL_HI_ERRORS_PKT_ERROR_LAST \
+		RX_PKT_V3_CMPL_HI_ERRORS_PKT_ERROR_L4_CS_ERROR
+	/*
+	 * This is data from the CFA block as indicated by the meta_format
+	 * field.
+	 */
+	uint16_t	metadata0;
+	/* When meta_format=1, this value is the VLAN VID. */
+	#define RX_PKT_V3_CMPL_HI_METADATA0_VID_MASK UINT32_C(0xfff)
+	#define RX_PKT_V3_CMPL_HI_METADATA0_VID_SFT 0
+	/* When meta_format=1, this value is the VLAN DE. */
+	#define RX_PKT_V3_CMPL_HI_METADATA0_DE      UINT32_C(0x1000)
+	/* When meta_format=1, this value is the VLAN PRI. */
+	#define RX_PKT_V3_CMPL_HI_METADATA0_PRI_MASK UINT32_C(0xe000)
+	#define RX_PKT_V3_CMPL_HI_METADATA0_PRI_SFT 13
+	/*
+	 * The timestamp field contains the 32b timestamp for the packet from
+	 * the MAC.
+	 *
+	 * When hairpin_en is set in VNIC context, this is the upper 32b of the
+	 * host address from the first BD used to place the packet.
+	 */
+	uint32_t	timestamp;
+} __rte_packed;
+
+/* rx_pkt_compress_cmpl (size:128b/16B) */
+struct rx_pkt_compress_cmpl {
+	uint16_t	flags_type;
+	/*
+	 * This field indicates the exact type of the completion.
+	 * By convention, the LSB identifies the length of the
+	 * record in 16B units. Even values indicate 16B
+	 * records. Odd values indicate 32B
+	 * records.
+	 */
+	#define RX_PKT_COMPRESS_CMPL_TYPE_MASK                   UINT32_C(0x3f)
+	#define RX_PKT_COMPRESS_CMPL_TYPE_SFT                    0
+	/*
+	 * RX L2 completion:
+	 * This is the compressed version of Rx Completion for performance
+	 * applications. Length = 16B
+	 * This version of the completion record is used in Thor2 and later
+	 * chips.
+	 */
+	#define RX_PKT_COMPRESS_CMPL_TYPE_RX_L2_COMPRESS \
+		UINT32_C(0x10)
+	#define RX_PKT_COMPRESS_CMPL_TYPE_LAST \
+		RX_PKT_COMPRESS_CMPL_TYPE_RX_L2_COMPRESS
+	#define RX_PKT_COMPRESS_CMPL_FLAGS_MASK \
+		UINT32_C(0xffc0)
+	#define RX_PKT_COMPRESS_CMPL_FLAGS_SFT                   6
+	/*
+	 * When this bit is '1', it indicates a packet that has an
+	 * error of some type. Type of error is indicated in
+	 * error_flags.
+	 */
+	#define RX_PKT_COMPRESS_CMPL_FLAGS_ERROR \
+		UINT32_C(0x40)
+	/*
+	 * This field indicates the status of IP and L4 CS calculations done
+	 * by the chip. The format of this field is indicated by the
+	 * cs_all_ok_mode bit.
+	 */
+	#define RX_PKT_COMPRESS_CMPL_FLAGS_T_IP_TYPE \
+		UINT32_C(0x100)
+	/* Indicates that the Tunnel IP type was IPv4 */
+	#define RX_PKT_COMPRESS_CMPL_FLAGS_T_IP_TYPE_IPV4 \
+		(UINT32_C(0x0) << 8)
+	/* Indicates that the Tunnel IP type was IPv6 */
+	#define RX_PKT_COMPRESS_CMPL_FLAGS_T_IP_TYPE_IPV6 \
+		(UINT32_C(0x1) << 8)
+	#define RX_PKT_COMPRESS_CMPL_FLAGS_T_IP_TYPE_LAST \
+		RX_PKT_COMPRESS_CMPL_FLAGS_T_IP_TYPE_IPV6
+	/*
+	 * This field indicates the IP type for the inner-most IP header.
+	 * A value of '0' indicates IPv4. A value of '1' indicates IPv6.
+	 * This value is only valid if itype indicates a packet
+	 * with an IP header.
+	 */
+	#define RX_PKT_COMPRESS_CMPL_FLAGS_IP_TYPE \
+		UINT32_C(0x200)
+	/* This bit is '1' if the RSS field in this completion is valid. */
+	#define RX_PKT_COMPRESS_CMPL_FLAGS_RSS_VALID \
+		UINT32_C(0x400)
+	/*
+	 * This value indicates what the inner packet determined for the
+	 * packet was.
+	 */
+	#define RX_PKT_COMPRESS_CMPL_FLAGS_ITYPE_MASK \
+		UINT32_C(0xf000)
+	#define RX_PKT_COMPRESS_CMPL_FLAGS_ITYPE_SFT              12
+	/*
+	 * Not Known:
+	 * Indicates that the packet type was not known.
+	 */
+	#define RX_PKT_COMPRESS_CMPL_FLAGS_ITYPE_NOT_KNOWN \
+		(UINT32_C(0x0) << 12)
+	/*
+	 * IP Packet:
+	 * Indicates that the packet was an IP packet, but further
+	 * classification was not possible.
+	 */
+	#define RX_PKT_COMPRESS_CMPL_FLAGS_ITYPE_IP \
+		(UINT32_C(0x1) << 12)
+	/*
+	 * TCP Packet:
+	 * Indicates that the packet was IP and TCP.
+	 * This indicates that the payload_offset field is valid.
+	 */
+	#define RX_PKT_COMPRESS_CMPL_FLAGS_ITYPE_TCP \
+		(UINT32_C(0x2) << 12)
+	/*
+	 * UDP Packet:
+	 * Indicates that the packet was IP and UDP.
+	 * This indicates that the payload_offset field is valid.
+	 */
+	#define RX_PKT_COMPRESS_CMPL_FLAGS_ITYPE_UDP \
+		(UINT32_C(0x3) << 12)
+	/*
+	 * FCoE Packet:
+	 * Indicates that the packet was recognized as a FCoE.
+	 * This also indicates that the payload_offset field is valid.
+	 */
+	#define RX_PKT_COMPRESS_CMPL_FLAGS_ITYPE_FCOE \
+		(UINT32_C(0x4) << 12)
+	/*
+	 * RoCE Packet:
+	 * Indicates that the packet was recognized as a RoCE.
+	 * This also indicates that the payload_offset field is valid.
+	 */
+	#define RX_PKT_COMPRESS_CMPL_FLAGS_ITYPE_ROCE \
+		(UINT32_C(0x5) << 12)
+	/*
+	 * ICMP Packet:
+	 * Indicates that the packet was recognized as ICMP.
+	 * This indicates that the payload_offset field is valid.
+	 */
+	#define RX_PKT_COMPRESS_CMPL_FLAGS_ITYPE_ICMP \
+		(UINT32_C(0x7) << 12)
+	/*
+	 * PTP packet wo/timestamp:
+	 * Indicates that the packet was recognized as a PTP
+	 * packet.
+	 */
+	#define RX_PKT_COMPRESS_CMPL_FLAGS_ITYPE_PTP_WO_TIMESTAMP \
+		(UINT32_C(0x8) << 12)
+	/*
+	 * PTP packet w/timestamp:
+	 * Indicates that the packet was recognized as a PTP
+	 * packet and that a timestamp was taken for the packet.
+	 * The 4b sub-nanosecond portion of the timestamp is in
+	 * the payload_offset field.
+	 */
+	#define RX_PKT_COMPRESS_CMPL_FLAGS_ITYPE_PTP_W_TIMESTAMP \
+		(UINT32_C(0x9) << 12)
+	#define RX_PKT_COMPRESS_CMPL_FLAGS_ITYPE_LAST \
+		RX_PKT_COMPRESS_CMPL_FLAGS_ITYPE_PTP_W_TIMESTAMP
+	/*
+	 * This is the length of the data for the packet stored in the
+	 * buffer(s) identified by the opaque value. This includes
+	 * the packet BD and any associated buffer BDs. This does not include
+	 * the length of any data places in aggregation BDs.
+	 */
+	uint16_t	len;
+	/*
+	 * This value is the RSS hash value calculated for the packet
+	 * based on the mode bits and key value in the VNIC. When hairpin_en
+	 * is set in VNIC context, this is the lower 32b of the host address
+	 * from the first BD used to place the packet.
+	 */
+	uint32_t	rss_hash;
+	uint16_t	metadata1_cs_error_calc_v1;
+	/*
+	 * This value is written by the NIC such that it will be different
+	 * for each pass through the completion queue. The even passes
+	 * will write 1. The odd passes will write 0.
+	 */
+	#define RX_PKT_COMPRESS_CMPL_V1 \
+		UINT32_C(0x1)
+	/* unused is 3 b */
+	#define RX_PKT_COMPRESS_CMPL_UNUSED_MASK \
+		UINT32_C(0xe)
+	#define RX_PKT_COMPRESS_CMPL_UNUSED_SFT                      1
+	#define RX_PKT_COMPRESS_CMPL_CS_ERROR_CALC_MASK \
+		UINT32_C(0xff0)
+	#define RX_PKT_COMPRESS_CMPL_CS_ERROR_CALC_SFT               4
+	/* This indicates that there was an error in the IP header checksum. */
+	#define RX_PKT_COMPRESS_CMPL_CS_ERROR_CALC_IP_CS_ERROR \
+		UINT32_C(0x10)
+	/*
+	 * This indicates that there was an error in the TCP, UDP or ICMP
+	 * checksum.
+	 */
+	#define RX_PKT_COMPRESS_CMPL_CS_ERROR_CALC_L4_CS_ERROR \
+		UINT32_C(0x20)
+	/*
+	 * This indicates that there was an error in the tunnel IP header
+	 * checksum.
+	 */
+	#define RX_PKT_COMPRESS_CMPL_CS_ERROR_CALC_T_IP_CS_ERROR \
+		UINT32_C(0x40)
+	/* This indicates that there was an error in the tunnel UDP checksum. */
+	#define RX_PKT_COMPRESS_CMPL_CS_ERROR_CALC_T_L4_CS_ERROR \
+		UINT32_C(0x80)
+	/*
+	 * This indicates that the ip checksum was calculated for the inner
+	 * packet and that the ip_cs_error field indicates if there was an
+	 * error.
+	 */
+	#define RX_PKT_COMPRESS_CMPL_CS_ERROR_CALC_IP_CS_CALC \
+		UINT32_C(0x100)
+	/*
+	 * This indicates that the TCP, UDP or ICMP checksum was calculated
+	 * for the inner packet and that the l4_cs_error field indicates if
+	 * there was an error.
+	 */
+	#define RX_PKT_COMPRESS_CMPL_CS_ERROR_CALC_L4_CS_CALC \
+		UINT32_C(0x200)
+	/*
+	 * This indicates that the ip checksum was calculated for the tunnel
+	 * header and that the t_ip_cs_error field indicates if there was an
+	 * error.
+	 */
+	#define RX_PKT_COMPRESS_CMPL_CS_ERROR_CALC_T_IP_CS_CALC \
+		UINT32_C(0x400)
+	/*
+	 * This indicates that the UDP checksum was calculated for the tunnel
+	 * packet and that the t_l4_cs_error field indicates if there was an
+	 * error.
+	 */
+	#define RX_PKT_COMPRESS_CMPL_CS_ERROR_CALC_T_L4_CS_CALC \
+		UINT32_C(0x800)
+	/* This is data from the CFA as indicated by the meta_format field. */
+	#define RX_PKT_COMPRESS_CMPL_METADATA1_MASK \
+		UINT32_C(0xf000)
+	#define RX_PKT_COMPRESS_CMPL_METADATA1_SFT                   12
+	/* When meta_format != 0, this value is the VLAN TPID_SEL. */
+	#define RX_PKT_COMPRESS_CMPL_METADATA1_TPID_SEL_MASK \
+		UINT32_C(0x7000)
+	#define RX_PKT_COMPRESS_CMPL_METADATA1_TPID_SEL_SFT           12
+	/* 0x88a8 */
+	#define RX_PKT_COMPRESS_CMPL_METADATA1_TPID_SEL_TPID88A8 \
+		(UINT32_C(0x0) << 12)
+	/* 0x8100 */
+	#define RX_PKT_COMPRESS_CMPL_METADATA1_TPID_SEL_TPID8100 \
+		(UINT32_C(0x1) << 12)
+	/* 0x9100 */
+	#define RX_PKT_COMPRESS_CMPL_METADATA1_TPID_SEL_TPID9100 \
+		(UINT32_C(0x2) << 12)
+	/* 0x9200 */
+	#define RX_PKT_COMPRESS_CMPL_METADATA1_TPID_SEL_TPID9200 \
+		(UINT32_C(0x3) << 12)
+	/* 0x9300 */
+	#define RX_PKT_COMPRESS_CMPL_METADATA1_TPID_SEL_TPID9300 \
+		(UINT32_C(0x4) << 12)
+	/* Value programmed in CFA VLANTPID register. */
+	#define RX_PKT_COMPRESS_CMPL_METADATA1_TPID_SEL_TPIDCFG \
+		(UINT32_C(0x5) << 12)
+	#define RX_PKT_COMPRESS_CMPL_METADATA1_TPID_SEL_LAST \
+		RX_PKT_COMPRESS_CMPL_METADATA1_TPID_SEL_TPIDCFG
+	/* When meta_format != 0, this value is the VLAN valid. */
+	#define RX_PKT_COMPRESS_CMPL_METADATA1_VALID \
+		UINT32_C(0x8000)
+	/* This is data from the CFA as indicated by the meta_format field. */
+	uint16_t	vlanc_tcid;
+	/* When meta_format!=0, this value is the VLAN VID. */
+	#define RX_PKT_COMPRESS_CMPL_VLANC_TCID_VID_MASK UINT32_C(0xfff)
+	#define RX_PKT_COMPRESS_CMPL_VLANC_TCID_VID_SFT 0
+	/* When meta_format!=0, this value is the VLAN DE. */
+	#define RX_PKT_COMPRESS_CMPL_VLANC_TCID_DE      UINT32_C(0x1000)
+	/* When meta_format!=0, this value is the VLAN PRI. */
+	#define RX_PKT_COMPRESS_CMPL_VLANC_TCID_PRI_MASK UINT32_C(0xe000)
+	#define RX_PKT_COMPRESS_CMPL_VLANC_TCID_PRI_SFT 13
+	uint32_t	errors_agg_bufs_opaque;
+	/* Lower 16bits of the Opaque field provided in the Rx BD. */
+	#define RX_PKT_COMPRESS_CMPL_ERRORS_AGG_BUFS_OPAQUE_OPAQUE_MASK \
+		UINT32_C(0xffff)
+	#define RX_PKT_COMPRESS_CMPL_ERRORS_AGG_BUFS_OPAQUE_OPAQUE_SFT \
+		0
+	/*
+	 * This value is the number of aggregation buffers that follow this
+	 * entry in the completion ring that are a part of this packet.
+	 * If the value is zero, then the packet is completely contained
+	 * in the buffer space provided for the packet in the RX ring.
+	 */
+	#define RX_PKT_COMPRESS_CMPL_ERRORS_AGG_BUFS_OPAQUE_AGG_BUFS_MASK \
+		UINT32_C(0x1f0000)
+	#define RX_PKT_COMPRESS_CMPL_ERRORS_AGG_BUFS_OPAQUE_AGG_BUFS_SFT \
+		16
+	#define RX_PKT_COMPRESS_CMPL_ERRORS_AGG_BUFS_OPAQUE_ERRORS_MASK \
+		UINT32_C(0x1fe00000)
+	#define RX_PKT_COMPRESS_CMPL_ERRORS_AGG_BUFS_OPAQUE_ERRORS_SFT \
+		21
+	/*
+	 * This indicates that there was an error in the inner
+	 * portion of the packet when this
+	 * field is non-zero.
+	 */
+	#define RX_PKT_COMPRESS_CMPL_ERRORS_AGG_BUFS_OPAQUE_ERRORS_PKT_ERROR_MASK \
+		UINT32_C(0x1e00000)
+	#define RX_PKT_COMPRESS_CMPL_ERRORS_AGG_BUFS_OPAQUE_ERRORS_PKT_ERROR_SFT \
+		21
+	/*
+	 * No additional error occurred on the tunnel portion
+	 * or the packet of the packet does not have a tunnel.
+	 */
+	#define RX_PKT_COMPRESS_CMPL_ERRORS_AGG_BUFS_OPAQUE_ERRORS_PKT_ERROR_NO_ERROR \
+		(UINT32_C(0x0) << 21)
+	/*
+	 * Indicates that IP header version does not match
+	 * expectation from L2 Ethertype for IPv4 and IPv6 or that
+	 * option other than VFT was parsed on
+	 * FCoE packet.
+	 */
+	#define RX_PKT_COMPRESS_CMPL_ERRORS_AGG_BUFS_OPAQUE_ERRORS_PKT_ERROR_L3_BAD_VERSION \
+		(UINT32_C(0x1) << 21)
+	/*
+	 * indicates that header length is out of range. Valid for
+	 * IPv4 and RoCE
+	 */
+	#define RX_PKT_COMPRESS_CMPL_ERRORS_AGG_BUFS_OPAQUE_ERRORS_PKT_ERROR_L3_BAD_HDR_LEN \
+		(UINT32_C(0x2) << 21)
+	/*
+	 * indicates that the IPv4 TTL or IPv6 hop limit check
+	 * have failed (e.g. TTL = 0). Valid for IPv4, and IPv6
+	 */
+	#define RX_PKT_COMPRESS_CMPL_ERRORS_AGG_BUFS_OPAQUE_ERRORS_PKT_ERROR_L3_BAD_TTL \
+		(UINT32_C(0x3) << 21)
+	/*
+	 * Indicates that physical packet is shorter than that
+	 * claimed by the l3 header length. Valid for IPv4,
+	 * IPv6 packet or RoCE packets.
+	 */
+	#define RX_PKT_COMPRESS_CMPL_ERRORS_AGG_BUFS_OPAQUE_ERRORS_PKT_ERROR_IP_TOTAL_ERROR \
+		(UINT32_C(0x4) << 21)
+	/*
+	 * Indicates that the physical packet is shorter than that
+	 * claimed by the UDP header length for a UDP packet that is
+	 * not fragmented.
+	 */
+	#define RX_PKT_COMPRESS_CMPL_ERRORS_AGG_BUFS_OPAQUE_ERRORS_PKT_ERROR_UDP_TOTAL_ERROR \
+		(UINT32_C(0x5) << 21)
+	/*
+	 * Indicates that TCP header length > IP payload. Valid for
+	 * TCP packets only.
+	 */
+	#define RX_PKT_COMPRESS_CMPL_ERRORS_AGG_BUFS_OPAQUE_ERRORS_PKT_ERROR_L4_BAD_HDR_LEN \
+		(UINT32_C(0x6) << 21)
+	/* Indicates that TCP header length < 5. Valid for TCP. */
+	#define RX_PKT_COMPRESS_CMPL_ERRORS_AGG_BUFS_OPAQUE_ERRORS_PKT_ERROR_L4_BAD_HDR_LEN_TOO_SMALL \
+		(UINT32_C(0x7) << 21)
+	/*
+	 * Indicates that TCP option headers result in a TCP header
+	 * size that does not match data offset in TCP header. Valid
+	 * for TCP.
+	 */
+	#define RX_PKT_COMPRESS_CMPL_ERRORS_AGG_BUFS_OPAQUE_ERRORS_PKT_ERROR_L4_BAD_OPT_LEN \
+		(UINT32_C(0x8) << 21)
+	#define RX_PKT_COMPRESS_CMPL_ERRORS_AGG_BUFS_OPAQUE_ERRORS_PKT_ERROR_LAST \
+		RX_PKT_COMPRESS_CMPL_ERRORS_AGG_BUFS_OPAQUE_ERRORS_PKT_ERROR_L4_BAD_OPT_LEN
+	/*
+	 * This indicates that there was an error in the tunnel portion
+	 * of the packet when this field is non-zero.
+	 */
+	#define RX_PKT_COMPRESS_CMPL_ERRORS_AGG_BUFS_OPAQUE_ERRORS_T_PKT_ERROR_MASK \
+		UINT32_C(0xe000000)
+	#define RX_PKT_COMPRESS_CMPL_ERRORS_AGG_BUFS_OPAQUE_ERRORS_T_PKT_ERROR_SFT \
+		25
+	/*
+	 * No additional error occurred on the tunnel portion
+	 * of the packet or the packet does not have a tunnel.
+	 */
+	#define RX_PKT_COMPRESS_CMPL_ERRORS_AGG_BUFS_OPAQUE_ERRORS_T_PKT_ERROR_NO_ERROR \
+		(UINT32_C(0x0) << 25)
+	/*
+	 * Indicates that IP header version does not match expectation
+	 * from L2 Ethertype for IPv4 and IPv6 in the tunnel header.
+	 */
+	#define RX_PKT_COMPRESS_CMPL_ERRORS_AGG_BUFS_OPAQUE_ERRORS_T_PKT_ERROR_T_L3_BAD_VERSION \
+		(UINT32_C(0x1) << 25)
+	/*
+	 * Indicates that header length is out of range in the tunnel
+	 * header. Valid for IPv4.
+	 */
+	#define RX_PKT_COMPRESS_CMPL_ERRORS_AGG_BUFS_OPAQUE_ERRORS_T_PKT_ERROR_T_L3_BAD_HDR_LEN \
+		(UINT32_C(0x2) << 25)
+	/*
+	 * Indicates that physical packet is shorter than that claimed
+	 * by the tunnel l3 header length. Valid for IPv4, or IPv6 tunnel
+	 * packet packets.
+	 */
+	#define RX_PKT_COMPRESS_CMPL_ERRORS_AGG_BUFS_OPAQUE_ERRORS_T_PKT_ERROR_T_IP_TOTAL_ERROR \
+		(UINT32_C(0x3) << 25)
+	/*
+	 * Indicates that the physical packet is shorter than that claimed
+	 * by the tunnel UDP header length for a tunnel UDP packet that is
+	 * not fragmented.
+	 */
+	#define RX_PKT_COMPRESS_CMPL_ERRORS_AGG_BUFS_OPAQUE_ERRORS_T_PKT_ERROR_T_UDP_TOTAL_ERROR \
+		(UINT32_C(0x4) << 25)
+	/*
+	 * Indicates that the IPv4 TTL or IPv6 hop limit check have failed
+	 * (e.g. TTL = 0) in the tunnel header. Valid for IPv4, and IPv6.
+	 */
+	#define RX_PKT_COMPRESS_CMPL_ERRORS_AGG_BUFS_OPAQUE_ERRORS_T_PKT_ERROR_T_L3_BAD_TTL \
+		(UINT32_C(0x5) << 25)
+	/*
+	 * Indicates that the IP checksum failed its check in the tunnel
+	 * header.
+	 */
+	#define RX_PKT_COMPRESS_CMPL_ERRORS_AGG_BUFS_OPAQUE_ERRORS_T_PKT_ERROR_T_IP_CS_ERROR \
+		(UINT32_C(0x6) << 25)
+	/*
+	 * Indicates that the L4 checksum failed its check in the tunnel
+	 * header.
+	 */
+	#define RX_PKT_COMPRESS_CMPL_ERRORS_AGG_BUFS_OPAQUE_ERRORS_T_PKT_ERROR_T_L4_CS_ERROR \
+		(UINT32_C(0x7) << 25)
+	#define RX_PKT_COMPRESS_CMPL_ERRORS_AGG_BUFS_OPAQUE_ERRORS_T_PKT_ERROR_LAST \
+		RX_PKT_COMPRESS_CMPL_ERRORS_AGG_BUFS_OPAQUE_ERRORS_T_PKT_ERROR_T_L4_CS_ERROR
+	/*
+	 * This indicates that there was a CRC error on either an FCoE
+	 * or RoCE packet. The itype indicates the packet type.
+	 */
+	#define RX_PKT_COMPRESS_CMPL_ERRORS_AGG_BUFS_OPAQUE_ERRORS_CRC_ERROR \
+		UINT32_C(0x10000000)
+	/* unused1 is 3 b */
+	#define RX_PKT_COMPRESS_CMPL_ERRORS_AGG_BUFS_OPAQUE_UNUSED1_MASK \
+		UINT32_C(0xe0000000)
+	#define RX_PKT_COMPRESS_CMPL_ERRORS_AGG_BUFS_OPAQUE_UNUSED1_SFT \
+		29
+} __rte_packed;
+
 /*
  * This TPA completion structure is used on devices where the
  * `hwrm_vnic_qcaps.max_aggs_supported` value is 0.
@@ -5977,7 +7209,7 @@ struct rx_tpa_start_v2_cmpl_hi {
 	 * information:
 	 * - vtag[19:0] = {valid, tpid_sel[2:0], pri[2:0], de, vid[11:0]}
 	 * The metadata2 field contains the Tunnel ID
-	 * value, justified to LSB. i
+	 * value, justified to LSB.
 	 * - VXLAN = VNI[23:0] -> VXLAN Network ID
 	 * - Geneve (NGE) = VNI[23:0] a-> Virtual Network Identifier
 	 * - NVGRE = TNI[23:0] -> Tenant Network ID
@@ -6140,6 +7372,487 @@ struct rx_tpa_start_v2_cmpl_hi {
 /*
  * This TPA completion structure is used on devices where the
  * `hwrm_vnic_qcaps.max_aggs_supported` value is 0.
+ * RX L2 TPA Start V3 Completion Record (32 bytes split to 2 16-byte
+ * struct)
+ */
+/* rx_tpa_start_v3_cmpl (size:128b/16B) */
+struct rx_tpa_start_v3_cmpl {
+	uint16_t	flags_type;
+	/*
+	 * This field indicates the exact type of the completion.
+	 * By convention, the LSB identifies the length of the
+	 * record in 16B units. Even values indicate 16B
+	 * records. Odd values indicate 32B
+	 * records.
+	 */
+	#define RX_TPA_START_V3_CMPL_TYPE_MASK \
+		UINT32_C(0x3f)
+	#define RX_TPA_START_V3_CMPL_TYPE_SFT                       0
+	/*
+	 * RX L2 TPA Start V3 completion:
+	 * Completion at the beginning of a TPA operation.
+	 * Length = 32B
+	 * This is the new version of the RX_TPA_START completion used
+	 * in Thor2 and later chips.
+	 */
+	#define RX_TPA_START_V3_CMPL_TYPE_RX_TPA_START_V3 \
+		UINT32_C(0x19)
+	#define RX_TPA_START_V3_CMPL_TYPE_LAST \
+		RX_TPA_START_V3_CMPL_TYPE_RX_TPA_START_V3
+	#define RX_TPA_START_V3_CMPL_FLAGS_MASK \
+		UINT32_C(0xffc0)
+	#define RX_TPA_START_V3_CMPL_FLAGS_SFT                      6
+	/*
+	 * When this bit is '1', it indicates a packet that has an error
+	 * of some type. Type of error is indicated in error_flags.
+	 */
+	#define RX_TPA_START_V3_CMPL_FLAGS_ERROR \
+		UINT32_C(0x40)
+	/* This field indicates how the packet was placed in the buffer. */
+	#define RX_TPA_START_V3_CMPL_FLAGS_PLACEMENT_MASK \
+		UINT32_C(0x380)
+	#define RX_TPA_START_V3_CMPL_FLAGS_PLACEMENT_SFT             7
+	/*
+	 * Jumbo:
+	 * TPA Packet was placed using jumbo algorithm. This means
+	 * that the first buffer will be filled with data before
+	 * moving to aggregation buffers. Each aggregation buffer
+	 * will be filled before moving to the next aggregation
+	 * buffer.
+	 */
+	#define RX_TPA_START_V3_CMPL_FLAGS_PLACEMENT_JUMBO \
+		(UINT32_C(0x1) << 7)
+	/*
+	 * Header/Data Separation:
+	 * Packet was placed using Header/Data separation algorithm.
+	 * The separation location is indicated by the itype field.
+	 */
+	#define RX_TPA_START_V3_CMPL_FLAGS_PLACEMENT_HDS \
+		(UINT32_C(0x2) << 7)
+	/*
+	 * IOC/Jumbo:
+	 * Packet will be placed using In-Order Completion/Jumbo where
+	 * the first packet of the aggregation is placed using Jumbo
+	 * Placement. Subsequent packets will be placed such that each
+	 * packet starts at the beginning of an aggregation buffer.
+	 */
+	#define RX_TPA_START_V3_CMPL_FLAGS_PLACEMENT_IOC_JUMBO \
+		(UINT32_C(0x4) << 7)
+	/*
+	 * GRO/Jumbo:
+	 * Packet will be placed using GRO/Jumbo where the first
+	 * packet is filled with data. Subsequent packets will be
+	 * placed such that any one packet does not span two
+	 * aggregation buffers unless it starts at the beginning of
+	 * an aggregation buffer.
+	 */
+	#define RX_TPA_START_V3_CMPL_FLAGS_PLACEMENT_GRO_JUMBO \
+		(UINT32_C(0x5) << 7)
+	/*
+	 * GRO/Header-Data Separation:
+	 * Packet will be placed using GRO/HDS where the header
+	 * is in the first packet.
+	 * Payload of each packet will be
+	 * placed such that any one packet does not span two
+	 * aggregation buffers unless it starts at the beginning of
+	 * an aggregation buffer.
+	 */
+	#define RX_TPA_START_V3_CMPL_FLAGS_PLACEMENT_GRO_HDS \
+		(UINT32_C(0x6) << 7)
+	/*
+	 * IOC/Header-Data Separation:
+	 * Packet will be placed using In-Order Completion/HDS where
+	 * the header is in the first packet buffer. Payload of each
+	 * packet will be placed such that each packet starts at the
+	 * beginning of an aggregation buffer.
+	 */
+	#define RX_TPA_START_V3_CMPL_FLAGS_PLACEMENT_IOC_HDS \
+		(UINT32_C(0x7) << 7)
+	#define RX_TPA_START_V3_CMPL_FLAGS_PLACEMENT_LAST \
+		RX_TPA_START_V3_CMPL_FLAGS_PLACEMENT_IOC_HDS
+	/* This bit is '1' if the RSS field in this completion is valid. */
+	#define RX_TPA_START_V3_CMPL_FLAGS_RSS_VALID \
+		UINT32_C(0x400)
+	/*
+	 * This bit is '1' if metadata has been added to the end of the
+	 * packet in host memory. Metadata starts at the first 32B boundary
+	 * after the end of the packet for regular and jumbo placement. It
+	 * starts at the first 32B boundary after the end of the header for
+	 * HDS placement. The length of the metadata is indicated in the
+	 * metadata itself.
+	 */
+	#define RX_TPA_START_V3_CMPL_FLAGS_PKT_METADATA_PRESENT \
+		UINT32_C(0x800)
+	/*
+	 * This value indicates what the inner packet determined for the
+	 * packet was.
+	 */
+	#define RX_TPA_START_V3_CMPL_FLAGS_ITYPE_MASK \
+		UINT32_C(0xf000)
+	#define RX_TPA_START_V3_CMPL_FLAGS_ITYPE_SFT                 12
+	/*
+	 * TCP Packet:
+	 * Indicates that the packet was IP and TCP.
+	 */
+	#define RX_TPA_START_V3_CMPL_FLAGS_ITYPE_TCP \
+		(UINT32_C(0x2) << 12)
+	#define RX_TPA_START_V3_CMPL_FLAGS_ITYPE_LAST \
+		RX_TPA_START_V3_CMPL_FLAGS_ITYPE_TCP
+	/*
+	 * This value indicates the amount of packet data written to the
+	 * buffer the opaque field in this completion corresponds to.
+	 */
+	uint16_t	len;
+	/*
+	 * This is a copy of the opaque field from the RX BD this completion
+	 * corresponds to. If the VNIC is configured to not use an Rx BD for
+	 * the TPA Start completion, then this is a copy of the opaque field
+	 * from the first BD used to place the TPA Start packet.
+	 */
+	uint32_t	opaque;
+	uint16_t	rss_hash_type_v1;
+	/*
+	 * This value is written by the NIC such that it will be different
+	 * for each pass through the completion queue. The even passes
+	 * will write 1. The odd passes will write 0.
+	 */
+	#define RX_TPA_START_V3_CMPL_V1                UINT32_C(0x1)
+	/* unused1 is 6 b. */
+	#define RX_TPA_START_V3_CMPL_UNUSED1_MASK      UINT32_C(0x7e)
+	#define RX_TPA_START_V3_CMPL_UNUSED1_SFT       1
+	/*
+	 * This is the RSS hash type for the packet. The value is packed
+	 * {tuple_extrac_op[1:0],rss_profile_id[4:0],tuple_extrac_op[2]}.
+	 *
+	 * The value of tuple_extrac_op provides the information about
+	 * what fields the hash was computed on.
+	 * * 0: The RSS hash was computed over source IP address,
+	 * destination IP address, source port, and destination port of inner
+	 * IP and TCP or UDP headers.
+	 * * 1: The RSS hash was computed over source IP address and
+	 * destination IP address of inner IP header.
+	 * * 2: The RSS hash was computed over source IP address,
+	 * destination IP address, source port, and destination port of
+	 * IP and TCP or UDP headers of outer tunnel headers.
+	 * Note: For non-tunneled packets, this value is not applicable.
+	 * * 3: The RSS hash was computed over source IP address and
+	 * destination IP address of IP header of outer tunnel headers.
+	 * Note: For non-tunneled packets, this value is not applicable.
+	 * * 4: The RSS hash was computed over source IP address of the inner
+	 * IP header.
+	 * * 5: The RSS hash was computed over destination IP address of the
+	 * inner IP header.
+	 * * 6: The RSS hash was computed over source IP address of the outer
+	 * IP header. Note: For non-tunneled packets, this value is not
+	 * applicable
+	 * * 7: The RSS hash was computed over destination IP address of the
+	 * outer IP header.
+	 * Note: For non-tunneled packets, this value is not applicable.
+	 * * 8: The RSS hash was computed over source IP address, destination
+	 * IP address, and flow label of the inner IP header.
+	 * Note: For packets without an inner IPv6 header, this value is not
+	 * applicable.
+	 * * 9: The RSS hash was computed over the flow label of the inner
+	 * IP header.
+	 * Note: For packets without an inner IPv6 header, this value
+	 * is not applicable.
+	 * * 10: The RSS hash was computed over source IP address, destination
+	 * IP address, and flow label of the outer IP header.
+	 * Note: For packets without an outer IPv6 header, this value is not
+	 * applicable.
+	 * * 11: The RSS hash was computed over the flow label of the outer
+	 * IP header. Note: For packets without an outer IPv6 header, this
+	 * value is not applicable.
+	 *
+	 * Note that 4-tuples values listed above are applicable
+	 * for layer 4 protocols supported and enabled for RSS in the hardware,
+	 * HWRM firmware, and drivers. For example, if RSS hash is supported
+	 * and enabled for TCP traffic only, then the values of
+	 * tuple_extract_op corresponding to 4-tuples are only valid for TCP
+	 * traffic
+	 */
+	#define RX_TPA_START_V3_CMPL_RSS_HASH_TYPE_MASK UINT32_C(0xff80)
+	#define RX_TPA_START_V3_CMPL_RSS_HASH_TYPE_SFT 7
+	/*
+	 * This is the aggregation ID that the completion is associated
+	 * with. Use this number to correlate the TPA start completion
+	 * with the TPA end completion.
+	 */
+	uint16_t	agg_id;
+	/*
+	 * This is the aggregation ID that the completion is associated
+	 * with. Use this number to correlate the TPA start completion
+	 * with the TPA end completion.
+	 */
+	#define RX_TPA_START_V3_CMPL_AGG_ID_MASK                UINT32_C(0xfff)
+	#define RX_TPA_START_V3_CMPL_AGG_ID_SFT                 0
+	#define RX_TPA_START_V3_CMPL_METADATA1_MASK \
+		UINT32_C(0xf000)
+	#define RX_TPA_START_V3_CMPL_METADATA1_SFT              12
+	/* When meta_format != 0, this value is the VLAN TPID_SEL. */
+	#define RX_TPA_START_V3_CMPL_METADATA1_TPID_SEL_MASK \
+		UINT32_C(0x7000)
+	#define RX_TPA_START_V3_CMPL_METADATA1_TPID_SEL_SFT      12
+	/* 0x88a8 */
+	#define RX_TPA_START_V3_CMPL_METADATA1_TPID_SEL_TPID88A8 \
+		(UINT32_C(0x0) << 12)
+	/* 0x8100 */
+	#define RX_TPA_START_V3_CMPL_METADATA1_TPID_SEL_TPID8100 \
+		(UINT32_C(0x1) << 12)
+	/* 0x9100 */
+	#define RX_TPA_START_V3_CMPL_METADATA1_TPID_SEL_TPID9100 \
+		(UINT32_C(0x2) << 12)
+	/* 0x9200 */
+	#define RX_TPA_START_V3_CMPL_METADATA1_TPID_SEL_TPID9200 \
+		(UINT32_C(0x3) << 12)
+	/* 0x9300 */
+	#define RX_TPA_START_V3_CMPL_METADATA1_TPID_SEL_TPID9300 \
+		(UINT32_C(0x4) << 12)
+	/* Value programmed in CFA VLANTPID register. */
+	#define RX_TPA_START_V3_CMPL_METADATA1_TPID_SEL_TPIDCFG \
+		(UINT32_C(0x5) << 12)
+	#define RX_TPA_START_V3_CMPL_METADATA1_TPID_SEL_LAST \
+		RX_TPA_START_V3_CMPL_METADATA1_TPID_SEL_TPIDCFG
+	/* When meta_format != 0, this value is the VLAN valid. */
+	#define RX_TPA_START_V3_CMPL_METADATA1_VALID \
+		UINT32_C(0x8000)
+	/*
+	 * This value is the RSS hash value calculated for the packet
+	 * based on the mode bits and key value in the VNIC.
+	 * When vee_cmpl_mode is set in VNIC context, this is the lower
+	 * 32b of the host address from the first BD used to place the packet.
+	 */
+	uint32_t	rss_hash;
+} __rte_packed;
+
+/*
+ * Last 16 bytes of RX L2 TPA Start V3 Completion Record
+ *
+ * This TPA completion structure is used on devices where the
+ * `hwrm_vnic_qcaps.max_aggs_supported` value is 0.
+ */
+/* rx_tpa_start_v3_cmpl_hi (size:128b/16B) */
+struct rx_tpa_start_v3_cmpl_hi {
+	uint32_t	flags2;
+	/*
+	 * This indicates that the ip checksum was calculated for the inner
+	 * packet and that the ip_cs_error field indicates if there was an
+	 * error.
+	 */
+	#define RX_TPA_START_V3_CMPL_FLAGS2_IP_CS_CALC \
+		UINT32_C(0x1)
+	/*
+	 * This indicates that the TCP, UDP or ICMP checksum was calculated
+	 * for the inner packet and that the l4_cs_error field indicates if
+	 * there was an error.
+	 */
+	#define RX_TPA_START_V3_CMPL_FLAGS2_L4_CS_CALC \
+		UINT32_C(0x2)
+	/*
+	 * This indicates that the ip checksum was calculated for the tunnel
+	 * header and that the t_ip_cs_error field indicates if there was an
+	 * error.
+	 */
+	#define RX_TPA_START_V3_CMPL_FLAGS2_T_IP_CS_CALC \
+		UINT32_C(0x4)
+	/*
+	 * This indicates that the UDP checksum was calculated for the tunnel
+	 * packet and that the t_l4_cs_error field indicates if there was an
+	 * error.
+	 */
+	#define RX_TPA_START_V3_CMPL_FLAGS2_T_L4_CS_CALC \
+		UINT32_C(0x8)
+	/* This value indicates what format the metadata field is. */
+	#define RX_TPA_START_V3_CMPL_FLAGS2_META_FORMAT_MASK \
+		UINT32_C(0xf0)
+	#define RX_TPA_START_V3_CMPL_FLAGS2_META_FORMAT_SFT            4
+	/* There is no metadata information. Values are zero. */
+	#define RX_TPA_START_V3_CMPL_FLAGS2_META_FORMAT_NONE \
+		(UINT32_C(0x0) << 4)
+	/*
+	 * The {metadata1, metadata0} fields contain the vtag
+	 * information: - vtag[19:0] = {valid, tpid_sel[2:0], pri[2:0],
+	 * de, vid[11:0]} The metadata2 field contains the table scope
+	 * and action record pointer. - metadata2[25:0] contains the
+	 * action record pointer. - metadata2[31:26] contains the table
+	 * scope.
+	 */
+	#define RX_TPA_START_V3_CMPL_FLAGS2_META_FORMAT_ACT_REC_PTR \
+		(UINT32_C(0x1) << 4)
+	/*
+	 * The {metadata1, metadata0} fields contain the vtag
+	 * information:
+	 * - vtag[19:0] = {valid, tpid_sel[2:0], pri[2:0], de, vid[11:0]}
+	 * The metadata2 field contains the Tunnel ID
+	 * value, justified to LSB.
+	 * - VXLAN = VNI[23:0] -> VXLAN Network ID
+	 * - Geneve (NGE) = VNI[23:0] a-> Virtual Network Identifier
+	 * - NVGRE = TNI[23:0] -> Tenant Network ID
+	 * - GRE = KEY[31:0] -> key field with bit mask. zero if K=0
+	 * - IPv4 = 0 (not populated)
+	 * - IPv6 = Flow Label[19:0]
+	 * - PPPoE = sessionID[15:0]
+	 * - MPLs = Outer label[19:0]
+	 * - UPAR = Selected[31:0] with bit mask
+	 */
+	#define RX_TPA_START_V3_CMPL_FLAGS2_META_FORMAT_TUNNEL_ID \
+		(UINT32_C(0x2) << 4)
+	/*
+	 * The {metadata1, metadata0} fields contain the vtag
+	 * information:
+	 * - vtag[19:0] = {valid, tpid_sel[2:0], pri[2:0],de, vid[11:0]}
+	 * The metadata2 field contains the 32b metadata from the prepended
+	 * header (chdr_data).
+	 */
+	#define RX_TPA_START_V3_CMPL_FLAGS2_META_FORMAT_CHDR_DATA \
+		(UINT32_C(0x3) << 4)
+	/*
+	 * The {metadata1, metadata0} fields contain the vtag
+	 * information:
+	 * - vtag[19:0] = {valid, tpid_sel[2:0], pri[2:0], de, vid[11:0]}
+	 * The metadata2 field contains the outer_l3_offset,
+	 * inner_l2_offset, inner_l3_offset, and inner_l4_size.
+	 * - metadata2[8:0] contains the outer_l3_offset.
+	 * - metadata2[17:9] contains the inner_l2_offset.
+	 * - metadata2[26:18] contains the inner_l3_offset.
+	 * - metadata2[31:27] contains the inner_l4_size.
+	 */
+	#define RX_TPA_START_V3_CMPL_FLAGS2_META_FORMAT_HDR_OFFSET \
+		(UINT32_C(0x4) << 4)
+	#define RX_TPA_START_V3_CMPL_FLAGS2_META_FORMAT_LAST \
+		RX_TPA_START_V3_CMPL_FLAGS2_META_FORMAT_HDR_OFFSET
+	/*
+	 * This field indicates the IP type for the inner-most IP header.
+	 * A value of '0' indicates IPv4. A value of '1' indicates IPv6.
+	 * This value is only valid if itype indicates a packet
+	 * with an IP header.
+	 */
+	#define RX_TPA_START_V3_CMPL_FLAGS2_IP_TYPE \
+		UINT32_C(0x100)
+	/*
+	 * This indicates that the complete 1's complement checksum was
+	 * calculated for the packet.
+	 */
+	#define RX_TPA_START_V3_CMPL_FLAGS2_COMPLETE_CHECKSUM_CALC \
+		UINT32_C(0x200)
+	/*
+	 * This field indicates the status of IP and L4 CS calculations done
+	 * by the chip. The format of this field is indicated by the
+	 * cs_all_ok_mode bit.
+	 */
+	#define RX_TPA_START_V3_CMPL_FLAGS2_T_IP_TYPE \
+		UINT32_C(0x400)
+	/* Indicates that the Tunnel IP type was IPv4 */
+	#define RX_TPA_START_V3_CMPL_FLAGS2_T_IP_TYPE_IPV4 \
+		(UINT32_C(0x0) << 10)
+	/* Indicates that the Tunnel IP type was IPv6 */
+	#define RX_TPA_START_V3_CMPL_FLAGS2_T_IP_TYPE_IPV6 \
+		(UINT32_C(0x1) << 10)
+	#define RX_TPA_START_V3_CMPL_FLAGS2_T_IP_TYPE_LAST \
+		RX_TPA_START_V3_CMPL_FLAGS2_T_IP_TYPE_IPV6
+	/* This indicates that the aggregation was done using GRO rules. */
+	#define RX_TPA_START_V3_CMPL_FLAGS2_AGG_GRO \
+		UINT32_C(0x800)
+	/*
+	 * This value is the complete 1's complement checksum calculated from
+	 * the start of the outer L3 header to the end of the packet (not
+	 * including the ethernet crc). It is valid when the
+	 * 'complete_checksum_calc' flag is set. For TPA Start completions,
+	 * the complete checksum is calculated for the first packet in the
+	 * aggregation only.
+	 */
+	#define RX_TPA_START_V3_CMPL_FLAGS2_COMPLETE_CHECKSUM_MASK \
+		UINT32_C(0xffff0000)
+	#define RX_TPA_START_V3_CMPL_FLAGS2_COMPLETE_CHECKSUM_SFT      16
+	/*
+	 * This is data from the CFA block as indicated by the meta_format
+	 * field.
+	 * - meta_format 0 - none - metadata2 = 0 - not valid/not stripped
+	 * - meta_format 1 - act_rec_ptr - metadata2 = {table_scope[5:0],
+	 *   act_rec_ptr[25:0]}
+	 * - meta_format 2 - tunnel_id - metadata2 = tunnel_id[31:0]
+	 * - meta_format 3 - chdr_data - metadata2 = updated_chdr_data[31:0]
+	 * - meta_format 4 - hdr_offsets - metadata2 = hdr_offsets[31:0]
+	 * When vee_cmpl_mode is set in VNIC context, this is the upper 32b
+	 * of the host address from the first BD used to place the packet.
+	 */
+	uint32_t	metadata2;
+	uint16_t	errors_v2;
+	/*
+	 * This value is written by the NIC such that it will be different
+	 * for each pass through the completion queue. The even passes
+	 * will write 1. The odd passes will write 0.
+	 */
+	#define RX_TPA_START_V3_CMPL_V2 \
+		UINT32_C(0x1)
+	#define RX_TPA_START_V3_CMPL_ERRORS_MASK \
+		UINT32_C(0xfffe)
+	#define RX_TPA_START_V3_CMPL_ERRORS_SFT                     1
+	/*
+	 * This error indicates that there was some sort of problem with
+	 * the BDs for the packetThe packet should be treated as
+	 * invalid.
+	 */
+	#define RX_TPA_START_V3_CMPL_ERRORS_BUFFER_ERROR_MASK \
+		UINT32_C(0xe)
+	#define RX_TPA_START_V3_CMPL_ERRORS_BUFFER_ERROR_SFT         1
+	/* No buffer error */
+	#define RX_TPA_START_V3_CMPL_ERRORS_BUFFER_ERROR_NO_BUFFER \
+		(UINT32_C(0x0) << 1)
+	/*
+	 * Did Not Fit:
+	 * Packet did not fit into packet buffer provided. This means
+	 * that the TPA Start packet was too big to be placed into the
+	 * per-packet maximum number of physical buffers configured for
+	 * the VNIC, or that it was too big to be placed into the
+	 * per-aggregation maximum number of physical buffers configured
+	 * for the VNIC. This error only occurs when the VNIC is
+	 * configured for variable size receive buffers.
+	 */
+	#define RX_TPA_START_V3_CMPL_ERRORS_BUFFER_ERROR_DID_NOT_FIT \
+		(UINT32_C(0x1) << 1)
+	/*
+	 * Bad Format:
+	 * BDs were not formatted correctly.
+	 */
+	#define RX_TPA_START_V3_CMPL_ERRORS_BUFFER_ERROR_BAD_FORMAT \
+		(UINT32_C(0x3) << 1)
+	/*
+	 * Flush:
+	 * There was a bad_format error on the previous operation
+	 */
+	#define RX_TPA_START_V3_CMPL_ERRORS_BUFFER_ERROR_FLUSH \
+		(UINT32_C(0x5) << 1)
+	#define RX_TPA_START_V3_CMPL_ERRORS_BUFFER_ERROR_LAST \
+		RX_TPA_START_V3_CMPL_ERRORS_BUFFER_ERROR_FLUSH
+	/*
+	 * This is data from the CFA block as indicated by the meta_format
+	 * field.
+	 */
+	uint16_t	metadata0;
+	/* When meta_format != 0, this value is the VLAN VID. */
+	#define RX_TPA_START_V3_CMPL_METADATA0_VID_MASK UINT32_C(0xfff)
+	#define RX_TPA_START_V3_CMPL_METADATA0_VID_SFT 0
+	/* When meta_format != 0, this value is the VLAN DE. */
+	#define RX_TPA_START_V3_CMPL_METADATA0_DE      UINT32_C(0x1000)
+	/* When meta_format != 0, this value is the VLAN PRI. */
+	#define RX_TPA_START_V3_CMPL_METADATA0_PRI_MASK UINT32_C(0xe000)
+	#define RX_TPA_START_V3_CMPL_METADATA0_PRI_SFT 13
+	/*
+	 * This field contains the outer_l3_offset, inner_l2_offset,
+	 * inner_l3_offset, and inner_l4_size.
+	 *
+	 * hdr_offsets[8:0] contains the outer_l3_offset.
+	 * hdr_offsets[17:9] contains the inner_l2_offset.
+	 * hdr_offsets[26:18] contains the inner_l3_offset.
+	 * hdr_offsets[31:27] contains the inner_l4_size.
+	 */
+	uint32_t	hdr_offsets;
+} __rte_packed;
+
+/*
+ * This TPA completion structure is used on devices where the
+ * `hwrm_vnic_qcaps.max_aggs_supported` value is 0.
  */
 /* rx_tpa_end_cmpl (size:128b/16B) */
 struct rx_tpa_end_cmpl {
@@ -6230,8 +7943,8 @@ struct rx_tpa_end_cmpl {
 		(UINT32_C(0x7) << 7)
 	#define RX_TPA_END_CMPL_FLAGS_PLACEMENT_LAST \
 		RX_TPA_END_CMPL_FLAGS_PLACEMENT_IOC_HDS
-	/* unused is 1 b */
-	#define RX_TPA_END_CMPL_FLAGS_UNUSED                    UINT32_C(0x400)
+	/* When set, this bit indicates that the timestamp field is valid. */
+	#define RX_TPA_END_CMPL_FLAGS_TIMESTAMP_VALID           UINT32_C(0x400)
 	/*
 	 * This bit is '1' if metadata has been added to the end of the
 	 * packet in host memory. Metadata starts at the first 32B boundary
@@ -6288,9 +8001,9 @@ struct rx_tpa_end_cmpl {
 	/* This value is the number of segments in the TPA operation. */
 	uint8_t	tpa_segs;
 	/*
-	 * This value indicates the offset in bytes from the beginning of the packet
-	 * where the inner payload starts. This value is valid for TCP, UDP,
-	 * FCoE, and RoCE packets.
+	 * This value indicates the offset in bytes from the beginning of the
+	 * packet where the inner payload starts. This value is valid for TCP,
+	 * UDP, FCoE, and RoCE packets.
 	 *
 	 * A value of zero indicates an offset of 256 bytes.
 	 */
@@ -6347,8 +8060,11 @@ struct rx_tpa_end_cmpl_hi {
 	 * to indicate MSS size to the stack.
 	 */
 	uint16_t	tpa_seg_len;
-	/* unused4 is 16 b */
-	uint16_t	unused3;
+	/*
+	 * The lower 16b of the timestamp of the last packet added to the
+	 * aggregation. Only valid when flags.timestamp_valid is set.
+	 */
+	uint16_t	timestamp_lower;
 	uint16_t	errors_v2;
 	/*
 	 * This value is written by the NIC such that it will be different
@@ -6388,8 +8104,11 @@ struct rx_tpa_end_cmpl_hi {
 		(UINT32_C(0x4) << 1)
 	#define RX_TPA_END_CMPL_ERRORS_BUFFER_ERROR_LAST \
 		RX_TPA_END_CMPL_ERRORS_BUFFER_ERROR_RSV_ERROR
-	/* unused5 is 16 b */
-	uint16_t	unused_4;
+	/*
+	 * The upper 16b of the timestamp of the last packet added to the
+	 * aggregation. Only valid when flags.timestamp_valid is set.
+	 */
+	uint16_t	timestamp_upper;
 	/*
 	 * This is the opaque value that was completed for the TPA start
 	 * completion that corresponds to this TPA end completion.
@@ -7066,7 +8785,7 @@ struct rx_tpa_v2_abuf_cmpl {
 	#define RX_TPA_V2_ABUF_CMPL_TYPE_MASK      UINT32_C(0x3f)
 	#define RX_TPA_V2_ABUF_CMPL_TYPE_SFT       0
 	/*
-	 * RX TPA Aggregation Buffer completion :
+	 * RX TPA Aggregation Buffer completion:
 	 * Completion of an L2 aggregation buffer in support of
 	 * TPA packet completion. Length = 16B
 	 */
@@ -7116,7 +8835,7 @@ struct rx_abuf_cmpl {
 	#define RX_ABUF_CMPL_TYPE_MASK  UINT32_C(0x3f)
 	#define RX_ABUF_CMPL_TYPE_SFT   0
 	/*
-	 * RX Aggregation Buffer completion :
+	 * RX Aggregation Buffer completion:
 	 * Completion of an L2 aggregation buffer in support of
 	 * TPA, HDS, or Jumbo packet completion. Length = 16B
 	 */
@@ -7570,9 +9289,9 @@ struct hwrm_async_event_cmpl {
 	/*
 	 * An event from firmware indicating who has been selected as the
 	 * PHC Master or secondary. Also indicates the last time a failover
-	 * happens.
+	 * happens. Event will also be sent when PHC rolls over.
 	 */
-	#define HWRM_ASYNC_EVENT_CMPL_EVENT_ID_PHC_MASTER \
+	#define HWRM_ASYNC_EVENT_CMPL_EVENT_ID_PHC_UPDATE \
 		UINT32_C(0x43)
 	/*
 	 * An event from firmware showing the last PPS timestamp that has been
@@ -7587,9 +9306,17 @@ struct hwrm_async_event_cmpl {
 	 */
 	#define HWRM_ASYNC_EVENT_CMPL_EVENT_ID_ERROR_REPORT \
 		UINT32_C(0x45)
+	/*
+	 * An event from firmware indicating that the programmed pacing
+	 * threshold for the doorbell global FIFO has been crossed. The driver
+	 * needs to take appropriate action to pace the doorbells when this
+	 * event is received from the firmware.
+	 */
+	#define HWRM_ASYNC_EVENT_CMPL_EVENT_ID_DOORBELL_PACING_THRESHOLD \
+		UINT32_C(0x46)
 	/* Maximum Registrable event id. */
 	#define HWRM_ASYNC_EVENT_CMPL_EVENT_ID_MAX_RGTR_EVENT_ID \
-		UINT32_C(0x46)
+		UINT32_C(0x47)
 	/*
 	 * A trace log message. This contains firmware trace logs string
 	 * embedded in the asynchronous message. This is an experimental
@@ -8255,15 +9982,15 @@ struct hwrm_async_event_cmpl_reset_notify {
 	#define HWRM_ASYNC_EVENT_CMPL_RESET_NOTIFY_OPAQUE_SFT 1
 	/*
 	 * 8-lsb timestamp (100-msec resolution)
-	 * The Minimum time required for the Firmware readiness after sending this
-	 * notification to the driver instances.
+	 * The Minimum time required for the Firmware readiness after sending
+	 * this notification to the driver instances.
 	 */
 	uint8_t	timestamp_lo;
 	/*
 	 * 16-lsb timestamp (100-msec resolution)
 	 * The Maximum Firmware Reset bail out value in the order of 100
-	 * milli seconds. The driver instances will use this value to re-initiate the
-	 * registration process again if the core firmware didn’t set the ready
+	 * milliseconds. The driver instances will use this value to reinitiate
+	 * the registration process again if the core firmware didn’t set the
 	 * state bit.
 	 */
 	uint16_t	timestamp_hi;
@@ -8316,7 +10043,8 @@ struct hwrm_async_event_cmpl_reset_notify {
 	#define HWRM_ASYNC_EVENT_CMPL_RESET_NOTIFY_EVENT_DATA1_REASON_CODE_LAST \
 		HWRM_ASYNC_EVENT_CMPL_RESET_NOTIFY_EVENT_DATA1_REASON_CODE_FW_ACTIVATION
 	/*
-	 * Minimum time before driver should attempt access - units 100ms ticks.
+	 * Minimum time before driver should attempt access - units 100ms
+	 * ticks.
 	 * Range 0-65535
 	 */
 	#define HWRM_ASYNC_EVENT_CMPL_RESET_NOTIFY_EVENT_DATA1_DELAY_IN_100MS_TICKS_MASK \
@@ -8453,7 +10181,7 @@ struct hwrm_async_event_cmpl_ring_monitor_msg {
 	/* 16-lsb timestamp from POR (100-msec resolution) */
 	uint16_t	timestamp_hi;
 	/*
-	 * Event specific data. If ring_type_disabled indicates a tx,rx or cmpl
+	 * Event specific data. If ring_type_disabled indicates a tx, rx or cmpl
 	 * then this field will indicate the ring id.
 	 */
 	uint32_t	event_data1;
@@ -9808,8 +11536,8 @@ struct hwrm_async_event_cmpl_echo_request {
 	uint32_t	event_data1;
 } __rte_packed;
 
-/* hwrm_async_event_cmpl_phc_master (size:128b/16B) */
-struct hwrm_async_event_cmpl_phc_master {
+/* hwrm_async_event_cmpl_phc_update (size:128b/16B) */
+struct hwrm_async_event_cmpl_phc_update {
 	uint16_t	type;
 	/*
 	 * This field indicates the exact type of the completion.
@@ -9818,14 +11546,14 @@ struct hwrm_async_event_cmpl_phc_master {
 	 * records. Odd values indicate 32B
 	 * records.
 	 */
-	#define HWRM_ASYNC_EVENT_CMPL_PHC_MASTER_TYPE_MASK \
+	#define HWRM_ASYNC_EVENT_CMPL_PHC_UPDATE_TYPE_MASK \
 		UINT32_C(0x3f)
-	#define HWRM_ASYNC_EVENT_CMPL_PHC_MASTER_TYPE_SFT             0
+	#define HWRM_ASYNC_EVENT_CMPL_PHC_UPDATE_TYPE_SFT             0
 	/* HWRM Asynchronous Event Information */
-	#define HWRM_ASYNC_EVENT_CMPL_PHC_MASTER_TYPE_HWRM_ASYNC_EVENT \
+	#define HWRM_ASYNC_EVENT_CMPL_PHC_UPDATE_TYPE_HWRM_ASYNC_EVENT \
 		UINT32_C(0x2e)
-	#define HWRM_ASYNC_EVENT_CMPL_PHC_MASTER_TYPE_LAST \
-		HWRM_ASYNC_EVENT_CMPL_PHC_MASTER_TYPE_HWRM_ASYNC_EVENT
+	#define HWRM_ASYNC_EVENT_CMPL_PHC_UPDATE_TYPE_LAST \
+		HWRM_ASYNC_EVENT_CMPL_PHC_UPDATE_TYPE_HWRM_ASYNC_EVENT
 	/* Identifiers of events. */
 	uint16_t	event_id;
 	/*
@@ -9833,21 +11561,21 @@ struct hwrm_async_event_cmpl_phc_master {
 	 * in PHC master. Only one master function can configure
 	 * PHC.
 	 */
-	#define HWRM_ASYNC_EVENT_CMPL_PHC_MASTER_EVENT_ID_PHC_MASTER \
+	#define HWRM_ASYNC_EVENT_CMPL_PHC_UPDATE_EVENT_ID_PHC_UPDATE \
 		UINT32_C(0x43)
-	#define HWRM_ASYNC_EVENT_CMPL_PHC_MASTER_EVENT_ID_LAST \
-		HWRM_ASYNC_EVENT_CMPL_PHC_MASTER_EVENT_ID_PHC_MASTER
+	#define HWRM_ASYNC_EVENT_CMPL_PHC_UPDATE_EVENT_ID_LAST \
+		HWRM_ASYNC_EVENT_CMPL_PHC_UPDATE_EVENT_ID_PHC_UPDATE
 	/* Event specific data */
 	uint32_t	event_data2;
 	/* This field provides the current master function. */
-	#define HWRM_ASYNC_EVENT_CMPL_PHC_MASTER_EVENT_DATA2_PHC_MASTER_FID_MASK \
+	#define HWRM_ASYNC_EVENT_CMPL_PHC_UPDATE_EVENT_DATA2_PHC_MASTER_FID_MASK \
 		UINT32_C(0xffff)
-	#define HWRM_ASYNC_EVENT_CMPL_PHC_MASTER_EVENT_DATA2_PHC_MASTER_FID_SFT \
+	#define HWRM_ASYNC_EVENT_CMPL_PHC_UPDATE_EVENT_DATA2_PHC_MASTER_FID_SFT \
 		0
 	/* This field provides the current secondary function. */
-	#define HWRM_ASYNC_EVENT_CMPL_PHC_MASTER_EVENT_DATA2_PHC_SEC_FID_MASK \
+	#define HWRM_ASYNC_EVENT_CMPL_PHC_UPDATE_EVENT_DATA2_PHC_SEC_FID_MASK \
 		UINT32_C(0xffff0000)
-	#define HWRM_ASYNC_EVENT_CMPL_PHC_MASTER_EVENT_DATA2_PHC_SEC_FID_SFT \
+	#define HWRM_ASYNC_EVENT_CMPL_PHC_UPDATE_EVENT_DATA2_PHC_SEC_FID_SFT \
 		16
 	uint8_t	opaque_v;
 	/*
@@ -9855,10 +11583,10 @@ struct hwrm_async_event_cmpl_phc_master {
 	 * for each pass through the completion queue. The even passes
 	 * will write 1. The odd passes will write 0.
 	 */
-	#define HWRM_ASYNC_EVENT_CMPL_PHC_MASTER_V          UINT32_C(0x1)
+	#define HWRM_ASYNC_EVENT_CMPL_PHC_UPDATE_V          UINT32_C(0x1)
 	/* opaque is 7 b */
-	#define HWRM_ASYNC_EVENT_CMPL_PHC_MASTER_OPAQUE_MASK UINT32_C(0xfe)
-	#define HWRM_ASYNC_EVENT_CMPL_PHC_MASTER_OPAQUE_SFT 1
+	#define HWRM_ASYNC_EVENT_CMPL_PHC_UPDATE_OPAQUE_MASK UINT32_C(0xfe)
+	#define HWRM_ASYNC_EVENT_CMPL_PHC_UPDATE_OPAQUE_SFT 1
 	/* 8-lsb timestamp (100-msec resolution) */
 	uint8_t	timestamp_lo;
 	/* 16-lsb timestamp (100-msec resolution) */
@@ -9866,30 +11594,45 @@ struct hwrm_async_event_cmpl_phc_master {
 	/* Event specific data */
 	uint32_t	event_data1;
 	/* Indicates to the driver the type of PHC event. */
-	#define HWRM_ASYNC_EVENT_CMPL_PHC_MASTER_EVENT_DATA1_FLAGS_MASK \
+	#define HWRM_ASYNC_EVENT_CMPL_PHC_UPDATE_EVENT_DATA1_FLAGS_MASK \
 		UINT32_C(0xf)
-	#define HWRM_ASYNC_EVENT_CMPL_PHC_MASTER_EVENT_DATA1_FLAGS_SFT \
+	#define HWRM_ASYNC_EVENT_CMPL_PHC_UPDATE_EVENT_DATA1_FLAGS_SFT \
 		0
 	/*
 	 * Indicates PHC Master selection event. The master fid is
 	 * specified in event_data2.phc_master_fid.
 	 */
-	#define HWRM_ASYNC_EVENT_CMPL_PHC_MASTER_EVENT_DATA1_FLAGS_PHC_MASTER \
+	#define HWRM_ASYNC_EVENT_CMPL_PHC_UPDATE_EVENT_DATA1_FLAGS_PHC_MASTER \
 		UINT32_C(0x1)
 	/*
 	 * Indicates PHC Secondary selection event. The secondary fid is
 	 * specified in event_data2.phc_sec_fid.
 	 */
-	#define HWRM_ASYNC_EVENT_CMPL_PHC_MASTER_EVENT_DATA1_FLAGS_PHC_SECONDARY \
+	#define HWRM_ASYNC_EVENT_CMPL_PHC_UPDATE_EVENT_DATA1_FLAGS_PHC_SECONDARY \
 		UINT32_C(0x2)
 	/*
 	 * Indicates PHC failover event. Failover happens from
 	 * event_data2.phc_master_fid to event_data2.phc_sec_fid.
 	 */
-	#define HWRM_ASYNC_EVENT_CMPL_PHC_MASTER_EVENT_DATA1_FLAGS_PHC_FAILOVER \
+	#define HWRM_ASYNC_EVENT_CMPL_PHC_UPDATE_EVENT_DATA1_FLAGS_PHC_FAILOVER \
 		UINT32_C(0x3)
-	#define HWRM_ASYNC_EVENT_CMPL_PHC_MASTER_EVENT_DATA1_FLAGS_LAST \
-		HWRM_ASYNC_EVENT_CMPL_PHC_MASTER_EVENT_DATA1_FLAGS_PHC_FAILOVER
+	/*
+	 * Indicates that the 64bit Real time clock upper 16bits
+	 * have been updated due to PHC rollover. The updated
+	 * upper 16bits is in event_data1.phc_time_msb
+	 */
+	#define HWRM_ASYNC_EVENT_CMPL_PHC_UPDATE_EVENT_DATA1_FLAGS_PHC_RTC_UPDATE \
+		UINT32_C(0x4)
+	#define HWRM_ASYNC_EVENT_CMPL_PHC_UPDATE_EVENT_DATA1_FLAGS_LAST \
+		HWRM_ASYNC_EVENT_CMPL_PHC_UPDATE_EVENT_DATA1_FLAGS_PHC_RTC_UPDATE
+	/*
+	 * This field provides the upper 16bits of the 64bit real
+	 * time clock.
+	 */
+	#define HWRM_ASYNC_EVENT_CMPL_PHC_UPDATE_EVENT_DATA1_PHC_TIME_MSB_MASK \
+		UINT32_C(0xffff0)
+	#define HWRM_ASYNC_EVENT_CMPL_PHC_UPDATE_EVENT_DATA1_PHC_TIME_MSB_SFT \
+		4
 } __rte_packed;
 
 /* hwrm_async_event_cmpl_pps_timestamp (size:128b/16B) */
@@ -10032,6 +11775,59 @@ struct hwrm_async_event_cmpl_error_report {
 	#define HWRM_ASYNC_EVENT_CMPL_ERROR_REPORT_EVENT_DATA1_ERROR_TYPE_SFT 0
 } __rte_packed;
 
+/* hwrm_async_event_cmpl_doorbell_pacing_threshold (size:128b/16B) */
+struct hwrm_async_event_cmpl_doorbell_pacing_threshold {
+	uint16_t	type;
+	/*
+	 * This field indicates the exact type of the completion.
+	 * By convention, the LSB identifies the length of the
+	 * record in 16B units. Even values indicate 16B
+	 * records. Odd values indicate 32B
+	 * records.
+	 */
+	#define HWRM_ASYNC_EVENT_CMPL_DOORBELL_PACING_THRESHOLD_TYPE_MASK \
+		UINT32_C(0x3f)
+	#define HWRM_ASYNC_EVENT_CMPL_DOORBELL_PACING_THRESHOLD_TYPE_SFT \
+		0
+	/* HWRM Asynchronous Event Information */
+	#define HWRM_ASYNC_EVENT_CMPL_DOORBELL_PACING_THRESHOLD_TYPE_HWRM_ASYNC_EVENT \
+		UINT32_C(0x2e)
+	#define HWRM_ASYNC_EVENT_CMPL_DOORBELL_PACING_THRESHOLD_TYPE_LAST \
+		HWRM_ASYNC_EVENT_CMPL_DOORBELL_PACING_THRESHOLD_TYPE_HWRM_ASYNC_EVENT
+	/* Identifiers of events. */
+	uint16_t	event_id;
+	/*
+	 * This async notification message is used to inform the driver
+	 * that the programmable pacing threshold for the doorbell FIFO is
+	 * reached. The driver will take appropriate action to pace the
+	 * doorbells when this async event is received from the firmware.
+	 */
+	#define HWRM_ASYNC_EVENT_CMPL_DOORBELL_PACING_THRESHOLD_EVENT_ID_DOORBELL_PACING_THRESHOLD \
+		UINT32_C(0x46)
+	#define HWRM_ASYNC_EVENT_CMPL_DOORBELL_PACING_THRESHOLD_EVENT_ID_LAST \
+		HWRM_ASYNC_EVENT_CMPL_DOORBELL_PACING_THRESHOLD_EVENT_ID_DOORBELL_PACING_THRESHOLD
+	/* Event specific data. */
+	uint32_t	event_data2;
+	uint8_t	opaque_v;
+	/*
+	 * This value is written by the NIC such that it will be different
+	 * for each pass through the completion queue. The even passes
+	 * will write 1. The odd passes will write 0.
+	 */
+	#define HWRM_ASYNC_EVENT_CMPL_DOORBELL_PACING_THRESHOLD_V \
+		UINT32_C(0x1)
+	/* opaque is 7 b */
+	#define HWRM_ASYNC_EVENT_CMPL_DOORBELL_PACING_THRESHOLD_OPAQUE_MASK \
+		UINT32_C(0xfe)
+	#define HWRM_ASYNC_EVENT_CMPL_DOORBELL_PACING_THRESHOLD_OPAQUE_SFT 1
+	/* 8-lsb timestamp (100-msec resolution) */
+	uint8_t	timestamp_lo;
+	/* 16-lsb timestamp (100-msec resolution) */
+	uint16_t	timestamp_hi;
+	/* Event specific data */
+	uint32_t	event_data1;
+} __rte_packed;
+
 /* hwrm_async_event_cmpl_fw_trace_msg (size:128b/16B) */
 struct hwrm_async_event_cmpl_fw_trace_msg {
 	uint16_t	type;
@@ -10287,8 +12083,14 @@ struct hwrm_async_event_cmpl_error_report_base {
 	 */
 	#define HWRM_ASYNC_EVENT_CMPL_ERROR_REPORT_BASE_EVENT_DATA1_ERROR_TYPE_DOORBELL_DROP_THRESHOLD \
 		UINT32_C(0x4)
+	/*
+	 * Indicates the NIC's temperature has crossed one of the thermal
+	 * thresholds.
+	 */
+	#define HWRM_ASYNC_EVENT_CMPL_ERROR_REPORT_BASE_EVENT_DATA1_ERROR_TYPE_THERMAL_THRESHOLD \
+		UINT32_C(0x5)
 	#define HWRM_ASYNC_EVENT_CMPL_ERROR_REPORT_BASE_EVENT_DATA1_ERROR_TYPE_LAST \
-		HWRM_ASYNC_EVENT_CMPL_ERROR_REPORT_BASE_EVENT_DATA1_ERROR_TYPE_DOORBELL_DROP_THRESHOLD
+		HWRM_ASYNC_EVENT_CMPL_ERROR_REPORT_BASE_EVENT_DATA1_ERROR_TYPE_THERMAL_THRESHOLD
 } __rte_packed;
 
 /* hwrm_async_event_cmpl_error_report_pause_storm (size:128b/16B) */
@@ -10585,6 +12387,125 @@ struct hwrm_async_event_cmpl_error_report_doorbell_drop_threshold {
 		HWRM_ASYNC_EVENT_CMPL_ERROR_REPORT_DOORBELL_DROP_THRESHOLD_EVENT_DATA1_ERROR_TYPE_DOORBELL_DROP_THRESHOLD
 } __rte_packed;
 
+/* hwrm_async_event_cmpl_error_report_thermal (size:128b/16B) */
+struct hwrm_async_event_cmpl_error_report_thermal {
+	uint16_t	type;
+	/*
+	 * This field indicates the exact type of the completion.
+	 * By convention, the LSB identifies the length of the
+	 * record in 16B units. Even values indicate 16B
+	 * records. Odd values indicate 32B
+	 * records.
+	 */
+	#define HWRM_ASYNC_EVENT_CMPL_ERROR_REPORT_THERMAL_TYPE_MASK \
+		UINT32_C(0x3f)
+	#define HWRM_ASYNC_EVENT_CMPL_ERROR_REPORT_THERMAL_TYPE_SFT \
+		0
+	/* HWRM Asynchronous Event Information */
+	#define HWRM_ASYNC_EVENT_CMPL_ERROR_REPORT_THERMAL_TYPE_HWRM_ASYNC_EVENT \
+		UINT32_C(0x2e)
+	#define HWRM_ASYNC_EVENT_CMPL_ERROR_REPORT_THERMAL_TYPE_LAST \
+		HWRM_ASYNC_EVENT_CMPL_ERROR_REPORT_THERMAL_TYPE_HWRM_ASYNC_EVENT
+	/* Identifiers of events. */
+	uint16_t	event_id;
+	/*
+	 * This async notification message is used to inform
+	 * the driver that an error has occurred which may need
+	 * the attention of the administrator.
+	 */
+	#define HWRM_ASYNC_EVENT_CMPL_ERROR_REPORT_THERMAL_EVENT_ID_ERROR_REPORT \
+		UINT32_C(0x45)
+	#define HWRM_ASYNC_EVENT_CMPL_ERROR_REPORT_THERMAL_EVENT_ID_LAST \
+		HWRM_ASYNC_EVENT_CMPL_ERROR_REPORT_THERMAL_EVENT_ID_ERROR_REPORT
+	/* Event specific data. */
+	uint32_t	event_data2;
+	/* Current temperature.  In Celsius */
+	#define HWRM_ASYNC_EVENT_CMPL_ERROR_REPORT_THERMAL_EVENT_DATA2_CURRENT_TEMP_MASK \
+		UINT32_C(0xff)
+	#define HWRM_ASYNC_EVENT_CMPL_ERROR_REPORT_THERMAL_EVENT_DATA2_CURRENT_TEMP_SFT \
+		0
+	/*
+	 * The temperature setting of the threshold that was just crossed.
+	 * In Celsius
+	 */
+	#define HWRM_ASYNC_EVENT_CMPL_ERROR_REPORT_THERMAL_EVENT_DATA2_THRESHOLD_TEMP_MASK \
+		UINT32_C(0xff00)
+	#define HWRM_ASYNC_EVENT_CMPL_ERROR_REPORT_THERMAL_EVENT_DATA2_THRESHOLD_TEMP_SFT \
+		8
+	uint8_t	opaque_v;
+	/*
+	 * This value is written by the NIC such that it will be different
+	 * for each pass through the completion queue. The even passes
+	 * will write 1. The odd passes will write 0.
+	 */
+	#define HWRM_ASYNC_EVENT_CMPL_ERROR_REPORT_THERMAL_V \
+		UINT32_C(0x1)
+	/* opaque is 7 b */
+	#define HWRM_ASYNC_EVENT_CMPL_ERROR_REPORT_THERMAL_OPAQUE_MASK \
+		UINT32_C(0xfe)
+	#define HWRM_ASYNC_EVENT_CMPL_ERROR_REPORT_THERMAL_OPAQUE_SFT 1
+	/* 8-lsb timestamp (100-msec resolution) */
+	uint8_t	timestamp_lo;
+	/* 16-lsb timestamp (100-msec resolution) */
+	uint16_t	timestamp_hi;
+	/* Event specific data */
+	uint32_t	event_data1;
+	/* Indicates the type of error being reported. */
+	#define HWRM_ASYNC_EVENT_CMPL_ERROR_REPORT_THERMAL_EVENT_DATA1_ERROR_TYPE_MASK \
+		UINT32_C(0xff)
+	#define HWRM_ASYNC_EVENT_CMPL_ERROR_REPORT_THERMAL_EVENT_DATA1_ERROR_TYPE_SFT \
+		0
+	/*
+	 * There was thermal event. The type will be specified in the
+	 * field threshold_type. event_data2 will contain the current
+	 * temperature and the configured value for the threshold that
+	 * was just crossed. The threshold values are lower thresholds,
+	 * so the event will trigger with an active flag when the
+	 * temperature is on an increasing trajectory.
+	 */
+	#define HWRM_ASYNC_EVENT_CMPL_ERROR_REPORT_THERMAL_EVENT_DATA1_ERROR_TYPE_THERMAL_EVENT \
+		UINT32_C(0x5)
+	#define HWRM_ASYNC_EVENT_CMPL_ERROR_REPORT_THERMAL_EVENT_DATA1_ERROR_TYPE_LAST \
+		HWRM_ASYNC_EVENT_CMPL_ERROR_REPORT_THERMAL_EVENT_DATA1_ERROR_TYPE_THERMAL_EVENT
+	/* The specific type of thermal threshold error */
+	#define HWRM_ASYNC_EVENT_CMPL_ERROR_REPORT_THERMAL_EVENT_DATA1_THRESHOLD_TYPE_MASK \
+		UINT32_C(0x700)
+	#define HWRM_ASYNC_EVENT_CMPL_ERROR_REPORT_THERMAL_EVENT_DATA1_THRESHOLD_TYPE_SFT \
+		8
+	/* Warning thermal threshold was crossed */
+	#define HWRM_ASYNC_EVENT_CMPL_ERROR_REPORT_THERMAL_EVENT_DATA1_THRESHOLD_TYPE_WARN \
+		(UINT32_C(0x0) << 8)
+	/* Critical thermal threshold was crossed */
+	#define HWRM_ASYNC_EVENT_CMPL_ERROR_REPORT_THERMAL_EVENT_DATA1_THRESHOLD_TYPE_CRITICAL \
+		(UINT32_C(0x1) << 8)
+	/* Fatal thermal threshold was crossed */
+	#define HWRM_ASYNC_EVENT_CMPL_ERROR_REPORT_THERMAL_EVENT_DATA1_THRESHOLD_TYPE_FATAL \
+		(UINT32_C(0x2) << 8)
+	/*
+	 * Thermal shutdown threshold was crossed and a shutdown is
+	 * imminent. This event will not occur if self shutdown
+	 * is disabled.
+	 */
+	#define HWRM_ASYNC_EVENT_CMPL_ERROR_REPORT_THERMAL_EVENT_DATA1_THRESHOLD_TYPE_SHUTDOWN \
+		(UINT32_C(0x3) << 8)
+	#define HWRM_ASYNC_EVENT_CMPL_ERROR_REPORT_THERMAL_EVENT_DATA1_THRESHOLD_TYPE_LAST \
+		HWRM_ASYNC_EVENT_CMPL_ERROR_REPORT_THERMAL_EVENT_DATA1_THRESHOLD_TYPE_SHUTDOWN
+	/*
+	 * Indicates if the thermal crossing occurs while the temperature is
+	 * increasing or decreasing.
+	 */
+	#define HWRM_ASYNC_EVENT_CMPL_ERROR_REPORT_THERMAL_EVENT_DATA1_TRANSITION_DIR \
+		UINT32_C(0x800)
+	/* Threshold is crossed while the temperature is falling. */
+	#define HWRM_ASYNC_EVENT_CMPL_ERROR_REPORT_THERMAL_EVENT_DATA1_TRANSITION_DIR_DECREASING \
+		(UINT32_C(0x0) << 11)
+	/* Threshold is crossed while the temperature is rising. */
+	#define HWRM_ASYNC_EVENT_CMPL_ERROR_REPORT_THERMAL_EVENT_DATA1_TRANSITION_DIR_INCREASING \
+		(UINT32_C(0x1) << 11)
+	#define HWRM_ASYNC_EVENT_CMPL_ERROR_REPORT_THERMAL_EVENT_DATA1_TRANSITION_DIR_LAST \
+		HWRM_ASYNC_EVENT_CMPL_ERROR_REPORT_THERMAL_EVENT_DATA1_TRANSITION_DIR_INCREASING
+} __rte_packed;
+
 /* metadata_base_msg (size:64b/8B) */
 struct metadata_base_msg {
 	uint16_t	md_type_link;
@@ -12183,6 +14104,30 @@ struct hwrm_func_qcaps_output {
 	 */
 	#define HWRM_FUNC_QCAPS_OUTPUT_FLAGS_EXT_BS_V2_REQUIRED \
 		UINT32_C(0x8000000)
+	/*
+	 * When this bit is '1', it indicates that FW will support a single
+	 * 64bit real time clock for PTP.
+	 */
+	#define HWRM_FUNC_QCAPS_OUTPUT_FLAGS_EXT_PTP_64BIT_RTC_SUPPORTED \
+		UINT32_C(0x10000000)
+	/*
+	 * When this bit is '1', it indicates the FW is capable of
+	 * supporting Doorbell Pacing.
+	 */
+	#define HWRM_FUNC_QCAPS_OUTPUT_FLAGS_EXT_DBR_PACING_SUPPORTED \
+		UINT32_C(0x20000000)
+	/*
+	 * When this bit is '1', it indicates the FW is capable of
+	 * supporting HW based doorbell drop recovery.
+	 */
+	#define HWRM_FUNC_QCAPS_OUTPUT_FLAGS_EXT_HW_DBR_DROP_RECOV_SUPPORTED \
+		UINT32_C(0x40000000)
+	/*
+	 * When this bit is '1', it indicates the driver can disable the CQ
+	 * overflow detection and can also skip the index updates for CQ.
+	 */
+	#define HWRM_FUNC_QCAPS_OUTPUT_FLAGS_EXT_DISABLE_CQ_OVERFLOW_DETECTION_SUPPORTED \
+		UINT32_C(0x80000000)
 	/* The maximum number of SCHQs supported by this device. */
 	uint8_t	max_schqs;
 	uint8_t	mpc_chnls_cap;
@@ -12221,7 +14166,66 @@ struct hwrm_func_qcaps_output {
 	 * function call for allocating Key Contexts.
 	 */
 	uint16_t	max_key_ctxs_alloc;
-	uint8_t	unused_1[7];
+	uint32_t	flags_ext2;
+	/*
+	 * When this bit is '1', it indicates that FW will support
+	 * timestamping on all RX packets, not just PTP type packets.
+	 */
+	#define HWRM_FUNC_QCAPS_OUTPUT_FLAGS_EXT2_RX_ALL_PKTS_TIMESTAMPS_SUPPORTED \
+		UINT32_C(0x1)
+	/* When this bit is '1', it indicates that HW and FW support QUIC. */
+	#define HWRM_FUNC_QCAPS_OUTPUT_FLAGS_EXT2_QUIC_SUPPORTED \
+		UINT32_C(0x2)
+	uint16_t	tunnel_disable_flag;
+	/*
+	 * When this bit is '1', it indicates that the VXLAN parsing
+	 * is disabled in hardware
+	 */
+	#define HWRM_FUNC_QCAPS_OUTPUT_TUNNEL_DISABLE_FLAG_DISABLE_VXLAN \
+		UINT32_C(0x1)
+	/*
+	 * When this bit is '1', it indicates that the NGE parsing
+	 * is disabled in hardware
+	 */
+	#define HWRM_FUNC_QCAPS_OUTPUT_TUNNEL_DISABLE_FLAG_DISABLE_NGE \
+		UINT32_C(0x2)
+	/*
+	 * When this bit is '1', it indicates that the NVGRE parsing
+	 * is disabled in hardware
+	 */
+	#define HWRM_FUNC_QCAPS_OUTPUT_TUNNEL_DISABLE_FLAG_DISABLE_NVGRE \
+		UINT32_C(0x4)
+	/*
+	 * When this bit is '1', it indicates that the L2GRE parsing
+	 * is disabled in hardware
+	 */
+	#define HWRM_FUNC_QCAPS_OUTPUT_TUNNEL_DISABLE_FLAG_DISABLE_L2GRE \
+		UINT32_C(0x8)
+	/*
+	 * When this bit is '1', it indicates that the GRE parsing
+	 * is disabled in hardware
+	 */
+	#define HWRM_FUNC_QCAPS_OUTPUT_TUNNEL_DISABLE_FLAG_DISABLE_GRE \
+		UINT32_C(0x10)
+	/*
+	 * When this bit is '1', it indicates that the IPINIP parsing
+	 * is disabled in hardware
+	 */
+	#define HWRM_FUNC_QCAPS_OUTPUT_TUNNEL_DISABLE_FLAG_DISABLE_IPINIP \
+		UINT32_C(0x20)
+	/*
+	 * When this bit is '1', it indicates that the MPLS parsing
+	 * is disabled in hardware
+	 */
+	#define HWRM_FUNC_QCAPS_OUTPUT_TUNNEL_DISABLE_FLAG_DISABLE_MPLS \
+		UINT32_C(0x40)
+	/*
+	 * When this bit is '1', it indicates that the PPPOE parsing
+	 * is disabled in hardware
+	 */
+	#define HWRM_FUNC_QCAPS_OUTPUT_TUNNEL_DISABLE_FLAG_DISABLE_PPPOE \
+		UINT32_C(0x80)
+	uint8_t	unused_1;
 	/*
 	 * This field is used in Output records to indicate that the output
 	 * is completely written to RAM.  This field should be read as '1'
@@ -15475,13 +17479,8 @@ struct hwrm_func_backing_store_qcaps_output {
 	 * function.
 	 */
 	uint32_t	rkc_max_entries;
-	/*
-	 * Number of mid-path TQM rings to be used for allocating
-	 * backing stores.
-	 */
-	uint8_t	mp_tqm_rings_count;
 	/* Reserved for future. */
-	uint8_t	rsvd1[6];
+	uint8_t	rsvd1[7];
 	/*
 	 * This field is used in Output records to indicate that the output
 	 * is completely written to RAM.  This field should be read as '1'
@@ -18876,7 +20875,7 @@ struct hwrm_func_ptp_pin_cfg_output {
  *********************/
 
 
-/* hwrm_func_ptp_cfg_input (size:320b/40B) */
+/* hwrm_func_ptp_cfg_input (size:384b/48B) */
 struct hwrm_func_ptp_cfg_input {
 	/* The HWRM command request type. */
 	uint16_t	req_type;
@@ -18943,6 +20942,9 @@ struct hwrm_func_ptp_cfg_input {
 	 */
 	#define HWRM_FUNC_PTP_CFG_INPUT_ENABLES_PTP_FREQ_ADJ_EXT_PHASE \
 		UINT32_C(0x20)
+	/* This bit must be '1' for ptp_set_time field to be configured. */
+	#define HWRM_FUNC_PTP_CFG_INPUT_ENABLES_PTP_SET_TIME \
+		UINT32_C(0x40)
 	/* This field is used to enable interrupt for a specific PPS event. */
 	uint8_t	ptp_pps_event;
 	/*
@@ -19043,6 +21045,13 @@ struct hwrm_func_ptp_cfg_input {
 	 * to provide a 48 bit value input for Phase.
 	 */
 	uint32_t	ptp_freq_adj_ext_phase_upper;
+	/*
+	 * Allows driver to set the full 64bit time in FW. The upper 16 bits
+	 * will be stored in FW and the lower 48bits will be programmed in
+	 * PHC. Firmware will send a broadcast async event to all functions
+	 * to indicate the programmed upper 16 bits.
+	 */
+	uint64_t	ptp_set_time;
 } __rte_packed;
 
 /* hwrm_func_ptp_cfg_output (size:128b/16B) */
@@ -19423,12 +21432,20 @@ struct hwrm_func_key_ctx_alloc_input {
 	uint32_t	dma_bufr_size_bytes;
 	/* Key Context type. */
 	uint8_t	key_ctx_type;
-	/* Tx Key Context. */
-	#define HWRM_FUNC_KEY_CTX_ALLOC_INPUT_KEY_CTX_TYPE_TX UINT32_C(0x0)
-	/* Rx KTLS Context. */
-	#define HWRM_FUNC_KEY_CTX_ALLOC_INPUT_KEY_CTX_TYPE_RX UINT32_C(0x1)
+	/* KTLS Tx Key Context type. */
+	#define HWRM_FUNC_KEY_CTX_ALLOC_INPUT_KEY_CTX_TYPE_TX \
+		UINT32_C(0x0)
+	/* KTLS Rx Key Context type. */
+	#define HWRM_FUNC_KEY_CTX_ALLOC_INPUT_KEY_CTX_TYPE_RX \
+		UINT32_C(0x1)
+	/* QUIC Tx Key Context type. */
+	#define HWRM_FUNC_KEY_CTX_ALLOC_INPUT_KEY_CTX_TYPE_QUIC_TX \
+		UINT32_C(0x2)
+	/* QUIC Rx Key Context type. */
+	#define HWRM_FUNC_KEY_CTX_ALLOC_INPUT_KEY_CTX_TYPE_QUIC_RX \
+		UINT32_C(0x3)
 	#define HWRM_FUNC_KEY_CTX_ALLOC_INPUT_KEY_CTX_TYPE_LAST \
-		HWRM_FUNC_KEY_CTX_ALLOC_INPUT_KEY_CTX_TYPE_RX
+		HWRM_FUNC_KEY_CTX_ALLOC_INPUT_KEY_CTX_TYPE_QUIC_RX
 	uint8_t	unused_0[7];
 	/* Host DMA address to send back KTLS context IDs. */
 	uint64_t	host_dma_addr;
@@ -19462,7 +21479,7 @@ struct hwrm_func_key_ctx_alloc_output {
  **********************************/
 
 
-/* hwrm_func_backing_store_cfg_v2_input (size:320b/40B) */
+/* hwrm_func_backing_store_cfg_v2_input (size:448b/56B) */
 struct hwrm_func_backing_store_cfg_v2_input {
 	/* The HWRM command request type. */
 	uint16_t	req_type;
@@ -19515,7 +21532,7 @@ struct hwrm_func_backing_store_cfg_v2_input {
 	/* Fast-path TQM ring. */
 	#define HWRM_FUNC_BACKING_STORE_CFG_V2_INPUT_TYPE_FP_TQM_RING \
 		UINT32_C(0x6)
-	/* MRAV. */
+	/* Memory Region and Memory Address Vector Context. */
 	#define HWRM_FUNC_BACKING_STORE_CFG_V2_INPUT_TYPE_MRAV \
 		UINT32_C(0xe)
 	/* TIM. */
@@ -19530,8 +21547,29 @@ struct hwrm_func_backing_store_cfg_v2_input {
 	/* Mid-path TQM ring. */
 	#define HWRM_FUNC_BACKING_STORE_CFG_V2_INPUT_TYPE_MP_TQM_RING \
 		UINT32_C(0x15)
+	/* SQ Doorbell shadow region. */
+	#define HWRM_FUNC_BACKING_STORE_CFG_V2_INPUT_TYPE_SQ_DB_SHADOW \
+		UINT32_C(0x16)
+	/* RQ Doorbell shadow region. */
+	#define HWRM_FUNC_BACKING_STORE_CFG_V2_INPUT_TYPE_RQ_DB_SHADOW \
+		UINT32_C(0x17)
+	/* SRQ Doorbell shadow region. */
+	#define HWRM_FUNC_BACKING_STORE_CFG_V2_INPUT_TYPE_SRQ_DB_SHADOW \
+		UINT32_C(0x18)
+	/* CQ Doorbell shadow region. */
+	#define HWRM_FUNC_BACKING_STORE_CFG_V2_INPUT_TYPE_CQ_DB_SHADOW \
+		UINT32_C(0x19)
+	/* QUIC Tx key context. */
+	#define HWRM_FUNC_BACKING_STORE_CFG_V2_INPUT_TYPE_QUIC_TKC \
+		UINT32_C(0x1a)
+	/* QUIC Rx key context. */
+	#define HWRM_FUNC_BACKING_STORE_CFG_V2_INPUT_TYPE_QUIC_RKC \
+		UINT32_C(0x1b)
+	/* Invalid type. */
+	#define HWRM_FUNC_BACKING_STORE_CFG_V2_INPUT_TYPE_INVALID \
+		UINT32_C(0xffff)
 	#define HWRM_FUNC_BACKING_STORE_CFG_V2_INPUT_TYPE_LAST \
-		HWRM_FUNC_BACKING_STORE_CFG_V2_INPUT_TYPE_MP_TQM_RING
+		HWRM_FUNC_BACKING_STORE_CFG_V2_INPUT_TYPE_INVALID
 	/*
 	 * Instance of the backing store type. It is zero-based,
 	 * which means "0" indicates the first instance. For backing
@@ -19598,7 +21636,38 @@ struct hwrm_func_backing_store_cfg_v2_input {
 		(UINT32_C(0x5) << 4)
 	#define HWRM_FUNC_BACKING_STORE_CFG_V2_INPUT_PAGE_SIZE_LAST \
 		HWRM_FUNC_BACKING_STORE_CFG_V2_INPUT_PAGE_SIZE_PG_1G
-	uint8_t	rsvd;
+	/*
+	 * This field counts how many split entries contain valid data.
+	 * Below is the table that maps the count value:
+	 * | Count |                     Indication                     |
+	 * | ----- | -------------------------------------------------- |
+	 * |   0   | None of the split entries has valid data.          |
+	 * |   1   | Only "split_entry_0" contains valid data.          |
+	 * |   2   | Only "split_entry_0" and "1" have valid data.      |
+	 * |   3   | Only "split_entry_0", "1" and "2" have valid data. |
+	 * |   4   | All four split entries have valid data.            |
+	 */
+	uint8_t	subtype_valid_cnt;
+	/*
+	 * Split entry #0. Note that the four split entries (as a group)
+	 * must be cast to a type-specific data structure first before
+	 * accessing it! Below is the table that maps a backing store
+	 * type to the associated split entry casting data structure.
+	 * | Type |       Split Entry Casting Data Structure           |
+	 * | ---- | -------------------------------------------------- |
+	 * | QPC  |             qpc_split_entries                      |
+	 * | SRQ  |             srq_split_entries                      |
+	 * | CQ   |             cq_split_entries                       |
+	 * | VINC |            vnic_split_entries                      |
+	 * | MRAV |            marv_split_entries                      |
+	 */
+	uint32_t	split_entry_0;
+	/* Split entry #1. */
+	uint32_t	split_entry_1;
+	/* Split entry #2. */
+	uint32_t	split_entry_2;
+	/* Split entry #3. */
+	uint32_t	split_entry_3;
 } __rte_packed;
 
 /* hwrm_func_backing_store_cfg_v2_output (size:128b/16B) */
@@ -19658,57 +21727,642 @@ struct hwrm_func_backing_store_qcfg_v2_input {
 	 * point to a physically contiguous block of memory.
 	 */
 	uint64_t	resp_addr;
-	/* Type of backing store to be configured. */
-	uint16_t	type;
-	/* Queue pair. */
-	#define HWRM_FUNC_BACKING_STORE_QCFG_V2_INPUT_TYPE_QP \
-		UINT32_C(0x0)
-	/* Shared receive queue. */
-	#define HWRM_FUNC_BACKING_STORE_QCFG_V2_INPUT_TYPE_SRQ \
-		UINT32_C(0x1)
-	/* Completion queue. */
-	#define HWRM_FUNC_BACKING_STORE_QCFG_V2_INPUT_TYPE_CQ \
-		UINT32_C(0x2)
-	/* Virtual NIC. */
-	#define HWRM_FUNC_BACKING_STORE_QCFG_V2_INPUT_TYPE_VNIC \
-		UINT32_C(0x3)
-	/* Statistic context. */
-	#define HWRM_FUNC_BACKING_STORE_QCFG_V2_INPUT_TYPE_STAT \
-		UINT32_C(0x4)
-	/* Slow-path TQM ring. */
-	#define HWRM_FUNC_BACKING_STORE_QCFG_V2_INPUT_TYPE_SP_TQM_RING \
-		UINT32_C(0x5)
-	/* Fast-path TQM ring. */
-	#define HWRM_FUNC_BACKING_STORE_QCFG_V2_INPUT_TYPE_FP_TQM_RING \
-		UINT32_C(0x6)
-	/* MRAV. */
-	#define HWRM_FUNC_BACKING_STORE_QCFG_V2_INPUT_TYPE_MRAV \
-		UINT32_C(0xe)
-	/* TIM. */
-	#define HWRM_FUNC_BACKING_STORE_QCFG_V2_INPUT_TYPE_TIM \
-		UINT32_C(0xf)
-	/* Tx key context. */
-	#define HWRM_FUNC_BACKING_STORE_QCFG_V2_INPUT_TYPE_TKC \
-		UINT32_C(0x13)
-	/* Rx key context. */
-	#define HWRM_FUNC_BACKING_STORE_QCFG_V2_INPUT_TYPE_RKC \
-		UINT32_C(0x14)
-	/* Mid-path TQM ring. */
-	#define HWRM_FUNC_BACKING_STORE_QCFG_V2_INPUT_TYPE_MP_TQM_RING \
-		UINT32_C(0x15)
-	#define HWRM_FUNC_BACKING_STORE_QCFG_V2_INPUT_TYPE_LAST \
-		HWRM_FUNC_BACKING_STORE_QCFG_V2_INPUT_TYPE_MP_TQM_RING
-	/*
-	 * Instance of the backing store type. It is zero-based,
-	 * which means "0" indicates the first instance. For backing
-	 * stores with single instance only, leave this field to 0.
-	 */
-	uint16_t	instance;
-	uint8_t	rsvd[4];
+	/* Type of backing store to be configured. */
+	uint16_t	type;
+	/* Queue pair. */
+	#define HWRM_FUNC_BACKING_STORE_QCFG_V2_INPUT_TYPE_QP \
+		UINT32_C(0x0)
+	/* Shared receive queue. */
+	#define HWRM_FUNC_BACKING_STORE_QCFG_V2_INPUT_TYPE_SRQ \
+		UINT32_C(0x1)
+	/* Completion queue. */
+	#define HWRM_FUNC_BACKING_STORE_QCFG_V2_INPUT_TYPE_CQ \
+		UINT32_C(0x2)
+	/* Virtual NIC. */
+	#define HWRM_FUNC_BACKING_STORE_QCFG_V2_INPUT_TYPE_VNIC \
+		UINT32_C(0x3)
+	/* Statistic context. */
+	#define HWRM_FUNC_BACKING_STORE_QCFG_V2_INPUT_TYPE_STAT \
+		UINT32_C(0x4)
+	/* Slow-path TQM ring. */
+	#define HWRM_FUNC_BACKING_STORE_QCFG_V2_INPUT_TYPE_SP_TQM_RING \
+		UINT32_C(0x5)
+	/* Fast-path TQM ring. */
+	#define HWRM_FUNC_BACKING_STORE_QCFG_V2_INPUT_TYPE_FP_TQM_RING \
+		UINT32_C(0x6)
+	/* Memory Region and Memory Address Vector Context. */
+	#define HWRM_FUNC_BACKING_STORE_QCFG_V2_INPUT_TYPE_MRAV \
+		UINT32_C(0xe)
+	/* TIM. */
+	#define HWRM_FUNC_BACKING_STORE_QCFG_V2_INPUT_TYPE_TIM \
+		UINT32_C(0xf)
+	/* Tx key context. */
+	#define HWRM_FUNC_BACKING_STORE_QCFG_V2_INPUT_TYPE_TKC \
+		UINT32_C(0x13)
+	/* Rx key context. */
+	#define HWRM_FUNC_BACKING_STORE_QCFG_V2_INPUT_TYPE_RKC \
+		UINT32_C(0x14)
+	/* Mid-path TQM ring. */
+	#define HWRM_FUNC_BACKING_STORE_QCFG_V2_INPUT_TYPE_MP_TQM_RING \
+		UINT32_C(0x15)
+	/* SQ Doorbell shadow region. */
+	#define HWRM_FUNC_BACKING_STORE_QCFG_V2_INPUT_TYPE_SQ_DB_SHADOW \
+		UINT32_C(0x16)
+	/* RQ Doorbell shadow region. */
+	#define HWRM_FUNC_BACKING_STORE_QCFG_V2_INPUT_TYPE_RQ_DB_SHADOW \
+		UINT32_C(0x17)
+	/* SRQ Doorbell shadow region. */
+	#define HWRM_FUNC_BACKING_STORE_QCFG_V2_INPUT_TYPE_SRQ_DB_SHADOW \
+		UINT32_C(0x18)
+	/* CQ Doorbell shadow region. */
+	#define HWRM_FUNC_BACKING_STORE_QCFG_V2_INPUT_TYPE_CQ_DB_SHADOW \
+		UINT32_C(0x19)
+	/* QUIC Tx key context. */
+	#define HWRM_FUNC_BACKING_STORE_QCFG_V2_INPUT_TYPE_QUIC_TKC \
+		UINT32_C(0x1a)
+	/* QUIC Rx key context. */
+	#define HWRM_FUNC_BACKING_STORE_QCFG_V2_INPUT_TYPE_QUIC_RKC \
+		UINT32_C(0x1b)
+	/* Invalid type. */
+	#define HWRM_FUNC_BACKING_STORE_QCFG_V2_INPUT_TYPE_INVALID \
+		UINT32_C(0xffff)
+	#define HWRM_FUNC_BACKING_STORE_QCFG_V2_INPUT_TYPE_LAST \
+		HWRM_FUNC_BACKING_STORE_QCFG_V2_INPUT_TYPE_INVALID
+	/*
+	 * Instance of the backing store type. It is zero-based,
+	 * which means "0" indicates the first instance. For backing
+	 * stores with single instance only, leave this field to 0.
+	 */
+	uint16_t	instance;
+	uint8_t	rsvd[4];
+} __rte_packed;
+
+/* hwrm_func_backing_store_qcfg_v2_output (size:448b/56B) */
+struct hwrm_func_backing_store_qcfg_v2_output {
+	/* The specific error status for the command. */
+	uint16_t	error_code;
+	/* The HWRM command request type. */
+	uint16_t	req_type;
+	/* The sequence ID from the original command. */
+	uint16_t	seq_id;
+	/* The length of the response data in number of bytes. */
+	uint16_t	resp_len;
+	/* Type of backing store to be configured. */
+	uint16_t	type;
+	/* Queue pair. */
+	#define HWRM_FUNC_BACKING_STORE_QCFG_V2_OUTPUT_TYPE_QP \
+		UINT32_C(0x0)
+	/* Shared receive queue. */
+	#define HWRM_FUNC_BACKING_STORE_QCFG_V2_OUTPUT_TYPE_SRQ \
+		UINT32_C(0x1)
+	/* Completion queue. */
+	#define HWRM_FUNC_BACKING_STORE_QCFG_V2_OUTPUT_TYPE_CQ \
+		UINT32_C(0x2)
+	/* Virtual NIC. */
+	#define HWRM_FUNC_BACKING_STORE_QCFG_V2_OUTPUT_TYPE_VNIC \
+		UINT32_C(0x3)
+	/* Statistic context. */
+	#define HWRM_FUNC_BACKING_STORE_QCFG_V2_OUTPUT_TYPE_STAT \
+		UINT32_C(0x4)
+	/* Slow-path TQM ring. */
+	#define HWRM_FUNC_BACKING_STORE_QCFG_V2_OUTPUT_TYPE_SP_TQM_RING \
+		UINT32_C(0x5)
+	/* Fast-path TQM ring. */
+	#define HWRM_FUNC_BACKING_STORE_QCFG_V2_OUTPUT_TYPE_FP_TQM_RING \
+		UINT32_C(0x6)
+	/* Memory Region and Memory Address Vector Context. */
+	#define HWRM_FUNC_BACKING_STORE_QCFG_V2_OUTPUT_TYPE_MRAV \
+		UINT32_C(0xe)
+	/* TIM. */
+	#define HWRM_FUNC_BACKING_STORE_QCFG_V2_OUTPUT_TYPE_TIM \
+		UINT32_C(0xf)
+	/* Tx key context. */
+	#define HWRM_FUNC_BACKING_STORE_QCFG_V2_OUTPUT_TYPE_TKC \
+		UINT32_C(0x13)
+	/* Rx key context. */
+	#define HWRM_FUNC_BACKING_STORE_QCFG_V2_OUTPUT_TYPE_RKC \
+		UINT32_C(0x14)
+	/* Mid-path TQM ring. */
+	#define HWRM_FUNC_BACKING_STORE_QCFG_V2_OUTPUT_TYPE_MP_TQM_RING \
+		UINT32_C(0x15)
+	/* QUIC Tx key context. */
+	#define HWRM_FUNC_BACKING_STORE_QCFG_V2_OUTPUT_TYPE_QUIC_TKC \
+		UINT32_C(0x1a)
+	/* QUIC Rx key context. */
+	#define HWRM_FUNC_BACKING_STORE_QCFG_V2_OUTPUT_TYPE_QUIC_RKC \
+		UINT32_C(0x1b)
+	/* Invalid type. */
+	#define HWRM_FUNC_BACKING_STORE_QCFG_V2_OUTPUT_TYPE_INVALID \
+		UINT32_C(0xffff)
+	#define HWRM_FUNC_BACKING_STORE_QCFG_V2_OUTPUT_TYPE_LAST \
+		HWRM_FUNC_BACKING_STORE_QCFG_V2_OUTPUT_TYPE_INVALID
+	/*
+	 * Instance of the backing store type. It is zero-based,
+	 * which means "0" indicates the first instance. For backing
+	 * stores with single instance only, leave this field to 0.
+	 */
+	uint16_t	instance;
+	/* Control flags. */
+	uint32_t	flags;
+	/* Page directory. */
+	uint64_t	page_dir;
+	/* Number of entries */
+	uint32_t	num_entries;
+	/* Page size and pbl level. */
+	uint8_t	page_size_pbl_level;
+	/* PBL indirect levels. */
+	#define HWRM_FUNC_BACKING_STORE_QCFG_V2_OUTPUT_PBL_LEVEL_MASK \
+		UINT32_C(0xf)
+	#define HWRM_FUNC_BACKING_STORE_QCFG_V2_OUTPUT_PBL_LEVEL_SFT   0
+	/* PBL pointer is physical start address. */
+	#define HWRM_FUNC_BACKING_STORE_QCFG_V2_OUTPUT_PBL_LEVEL_LVL_0 \
+		UINT32_C(0x0)
+	/* PBL pointer points to PTE table. */
+	#define HWRM_FUNC_BACKING_STORE_QCFG_V2_OUTPUT_PBL_LEVEL_LVL_1 \
+		UINT32_C(0x1)
+	/*
+	 * PBL pointer points to PDE table with each entry pointing to
+	 * PTE tables.
+	 */
+	#define HWRM_FUNC_BACKING_STORE_QCFG_V2_OUTPUT_PBL_LEVEL_LVL_2 \
+		UINT32_C(0x2)
+	#define HWRM_FUNC_BACKING_STORE_QCFG_V2_OUTPUT_PBL_LEVEL_LAST \
+		HWRM_FUNC_BACKING_STORE_QCFG_V2_OUTPUT_PBL_LEVEL_LVL_2
+	/* Page size. */
+	#define HWRM_FUNC_BACKING_STORE_QCFG_V2_OUTPUT_PAGE_SIZE_MASK \
+		UINT32_C(0xf0)
+	#define HWRM_FUNC_BACKING_STORE_QCFG_V2_OUTPUT_PAGE_SIZE_SFT   4
+	/* 4KB. */
+	#define HWRM_FUNC_BACKING_STORE_QCFG_V2_OUTPUT_PAGE_SIZE_PG_4K \
+		(UINT32_C(0x0) << 4)
+	/* 8KB. */
+	#define HWRM_FUNC_BACKING_STORE_QCFG_V2_OUTPUT_PAGE_SIZE_PG_8K \
+		(UINT32_C(0x1) << 4)
+	/* 64KB. */
+	#define HWRM_FUNC_BACKING_STORE_QCFG_V2_OUTPUT_PAGE_SIZE_PG_64K \
+		(UINT32_C(0x2) << 4)
+	/* 2MB. */
+	#define HWRM_FUNC_BACKING_STORE_QCFG_V2_OUTPUT_PAGE_SIZE_PG_2M \
+		(UINT32_C(0x3) << 4)
+	/* 8MB. */
+	#define HWRM_FUNC_BACKING_STORE_QCFG_V2_OUTPUT_PAGE_SIZE_PG_8M \
+		(UINT32_C(0x4) << 4)
+	/* 1GB. */
+	#define HWRM_FUNC_BACKING_STORE_QCFG_V2_OUTPUT_PAGE_SIZE_PG_1G \
+		(UINT32_C(0x5) << 4)
+	#define HWRM_FUNC_BACKING_STORE_QCFG_V2_OUTPUT_PAGE_SIZE_LAST \
+		HWRM_FUNC_BACKING_STORE_QCFG_V2_OUTPUT_PAGE_SIZE_PG_1G
+	/*
+	 * This field counts how many split entries contain valid data.
+	 * Below is the table that maps the count value:
+	 * | count |                     Indication                     |
+	 * | ----- | -------------------------------------------------- |
+	 * |   0   | None of the split entries has valid data.          |
+	 * |   1   | Only "split_entry_0" contains valid data.          |
+	 * |   2   | Only "split_entry_0" and "1" have valid data.      |
+	 * |   3   | Only "split_entry_0", "1" and "2" have valid data. |
+	 * |   4   | All four split entries have valid data.            |
+	 */
+	uint8_t	subtype_valid_cnt;
+	uint8_t	rsvd[2];
+	/*
+	 * Split entry #0. Note that the four split entries (as a group)
+	 * must be cast to a type-specific data structure first before
+	 * accessing it! Below is the table that maps a backing store
+	 * type to the associated split entry casting data structure.
+	 * | Type |       Split Entry Casting Data Structure           |
+	 * | ---- | -------------------------------------------------- |
+	 * | QPC  |             qpc_split_entries                      |
+	 * | SRQ  |             srq_split_entries                      |
+	 * | CQ   |             cq_split_entries                       |
+	 * | VINC |            vnic_split_entries                      |
+	 * | MRAV |            marv_split_entries                      |
+	 */
+	uint32_t	split_entry_0;
+	/* Split entry #1. */
+	uint32_t	split_entry_1;
+	/* Split entry #2. */
+	uint32_t	split_entry_2;
+	/* Split entry #3. */
+	uint32_t	split_entry_3;
+	uint8_t	rsvd2[7];
+	/*
+	 * This field is used in Output records to indicate that the
+	 * output is completely written to RAM.  This field should be
+	 * read as '1' to indicate that the output has been completely
+	 * written. When writing a command completion or response to
+	 * an internal processor, the order of writes has to be such
+	 * that this field is written last.
+	 */
+	uint8_t	valid;
+} __rte_packed;
+
+/************************************
+ * hwrm_func_backing_store_qcaps_v2 *
+ ************************************/
+
+
+/* hwrm_func_backing_store_qcaps_v2_input (size:192b/24B) */
+struct hwrm_func_backing_store_qcaps_v2_input {
+	/* The HWRM command request type. */
+	uint16_t	req_type;
+	/*
+	 * The completion ring to send the completion event on. This should
+	 * be the NQ ID returned from the `nq_alloc` HWRM command.
+	 */
+	uint16_t	cmpl_ring;
+	/*
+	 * The sequence ID is used by the driver for tracking multiple
+	 * commands. This ID is treated as opaque data by the firmware and
+	 * the value is returned in the `hwrm_resp_hdr` upon completion.
+	 */
+	uint16_t	seq_id;
+	/*
+	 * The target ID of the command:
+	 * * 0x0-0xFFF8 - The function ID
+	 * * 0xFFF8-0xFFFC, 0xFFFE - Reserved for internal processors
+	 * * 0xFFFD - Reserved for user-space HWRM interface
+	 * * 0xFFFF - HWRM
+	 */
+	uint16_t	target_id;
+	/*
+	 * A physical address pointer pointing to a host buffer that the
+	 * command's response data will be written. This can be either a host
+	 * physical address (HPA) or a guest physical address (GPA) and must
+	 * point to a physically contiguous block of memory.
+	 */
+	uint64_t	resp_addr;
+	/* Type of backing store to be queried. */
+	uint16_t	type;
+	/* Queue pair. */
+	#define HWRM_FUNC_BACKING_STORE_QCAPS_V2_INPUT_TYPE_QP \
+		UINT32_C(0x0)
+	/* Shared receive queue. */
+	#define HWRM_FUNC_BACKING_STORE_QCAPS_V2_INPUT_TYPE_SRQ \
+		UINT32_C(0x1)
+	/* Completion queue. */
+	#define HWRM_FUNC_BACKING_STORE_QCAPS_V2_INPUT_TYPE_CQ \
+		UINT32_C(0x2)
+	/* Virtual NIC. */
+	#define HWRM_FUNC_BACKING_STORE_QCAPS_V2_INPUT_TYPE_VNIC \
+		UINT32_C(0x3)
+	/* Statistic context. */
+	#define HWRM_FUNC_BACKING_STORE_QCAPS_V2_INPUT_TYPE_STAT \
+		UINT32_C(0x4)
+	/* Slow-path TQM ring. */
+	#define HWRM_FUNC_BACKING_STORE_QCAPS_V2_INPUT_TYPE_SP_TQM_RING \
+		UINT32_C(0x5)
+	/* Fast-path TQM ring. */
+	#define HWRM_FUNC_BACKING_STORE_QCAPS_V2_INPUT_TYPE_FP_TQM_RING \
+		UINT32_C(0x6)
+	/* Memory Region and Memory Address Vector Context. */
+	#define HWRM_FUNC_BACKING_STORE_QCAPS_V2_INPUT_TYPE_MRAV \
+		UINT32_C(0xe)
+	/* TIM. */
+	#define HWRM_FUNC_BACKING_STORE_QCAPS_V2_INPUT_TYPE_TIM \
+		UINT32_C(0xf)
+	/* Tx key context. */
+	#define HWRM_FUNC_BACKING_STORE_QCAPS_V2_INPUT_TYPE_TKC \
+		UINT32_C(0x13)
+	/* Rx key context. */
+	#define HWRM_FUNC_BACKING_STORE_QCAPS_V2_INPUT_TYPE_RKC \
+		UINT32_C(0x14)
+	/* Mid-path TQM ring. */
+	#define HWRM_FUNC_BACKING_STORE_QCAPS_V2_INPUT_TYPE_MP_TQM_RING \
+		UINT32_C(0x15)
+	/* SQ Doorbell shadow region. */
+	#define HWRM_FUNC_BACKING_STORE_QCAPS_V2_INPUT_TYPE_SQ_DB_SHADOW \
+		UINT32_C(0x16)
+	/* RQ Doorbell shadow region. */
+	#define HWRM_FUNC_BACKING_STORE_QCAPS_V2_INPUT_TYPE_RQ_DB_SHADOW \
+		UINT32_C(0x17)
+	/* SRQ Doorbell shadow region. */
+	#define HWRM_FUNC_BACKING_STORE_QCAPS_V2_INPUT_TYPE_SRQ_DB_SHADOW \
+		UINT32_C(0x18)
+	/* CQ Doorbell shadow region. */
+	#define HWRM_FUNC_BACKING_STORE_QCAPS_V2_INPUT_TYPE_CQ_DB_SHADOW \
+		UINT32_C(0x19)
+	/* QUIC Tx key context. */
+	#define HWRM_FUNC_BACKING_STORE_QCAPS_V2_INPUT_TYPE_QUIC_TKC \
+		UINT32_C(0x1a)
+	/* QUIC Rx key context. */
+	#define HWRM_FUNC_BACKING_STORE_QCAPS_V2_INPUT_TYPE_QUIC_RKC \
+		UINT32_C(0x1b)
+	/* Invalid type. */
+	#define HWRM_FUNC_BACKING_STORE_QCAPS_V2_INPUT_TYPE_INVALID \
+		UINT32_C(0xffff)
+	#define HWRM_FUNC_BACKING_STORE_QCAPS_V2_INPUT_TYPE_LAST \
+		HWRM_FUNC_BACKING_STORE_QCAPS_V2_INPUT_TYPE_INVALID
+	uint8_t	rsvd[6];
+} __rte_packed;
+
+/* hwrm_func_backing_store_qcaps_v2_output (size:448b/56B) */
+struct hwrm_func_backing_store_qcaps_v2_output {
+	/* The specific error status for the command. */
+	uint16_t	error_code;
+	/* The HWRM command request type. */
+	uint16_t	req_type;
+	/* The sequence ID from the original command. */
+	uint16_t	seq_id;
+	/* The length of the response data in number of bytes. */
+	uint16_t	resp_len;
+	/* Type of backing store to be queried. */
+	uint16_t	type;
+	/* Queue pair. */
+	#define HWRM_FUNC_BACKING_STORE_QCAPS_V2_OUTPUT_TYPE_QP \
+		UINT32_C(0x0)
+	/* Shared receive queue. */
+	#define HWRM_FUNC_BACKING_STORE_QCAPS_V2_OUTPUT_TYPE_SRQ \
+		UINT32_C(0x1)
+	/* Completion queue. */
+	#define HWRM_FUNC_BACKING_STORE_QCAPS_V2_OUTPUT_TYPE_CQ \
+		UINT32_C(0x2)
+	/* Virtual NIC. */
+	#define HWRM_FUNC_BACKING_STORE_QCAPS_V2_OUTPUT_TYPE_VNIC \
+		UINT32_C(0x3)
+	/* Statistic context. */
+	#define HWRM_FUNC_BACKING_STORE_QCAPS_V2_OUTPUT_TYPE_STAT \
+		UINT32_C(0x4)
+	/* Slow-path TQM ring. */
+	#define HWRM_FUNC_BACKING_STORE_QCAPS_V2_OUTPUT_TYPE_SP_TQM_RING \
+		UINT32_C(0x5)
+	/* Fast-path TQM ring. */
+	#define HWRM_FUNC_BACKING_STORE_QCAPS_V2_OUTPUT_TYPE_FP_TQM_RING \
+		UINT32_C(0x6)
+	/* Memory Region and Memory Address Vector Context. */
+	#define HWRM_FUNC_BACKING_STORE_QCAPS_V2_OUTPUT_TYPE_MRAV \
+		UINT32_C(0xe)
+	/* TIM. */
+	#define HWRM_FUNC_BACKING_STORE_QCAPS_V2_OUTPUT_TYPE_TIM \
+		UINT32_C(0xf)
+	/* KTLS Tx key context. */
+	#define HWRM_FUNC_BACKING_STORE_QCAPS_V2_OUTPUT_TYPE_TKC \
+		UINT32_C(0x13)
+	/* KTLS Rx key context. */
+	#define HWRM_FUNC_BACKING_STORE_QCAPS_V2_OUTPUT_TYPE_RKC \
+		UINT32_C(0x14)
+	/* Mid-path TQM ring. */
+	#define HWRM_FUNC_BACKING_STORE_QCAPS_V2_OUTPUT_TYPE_MP_TQM_RING \
+		UINT32_C(0x15)
+	/* SQ Doorbell shadow region. */
+	#define HWRM_FUNC_BACKING_STORE_QCAPS_V2_OUTPUT_TYPE_SQ_DB_SHADOW \
+		UINT32_C(0x16)
+	/* RQ Doorbell shadow region. */
+	#define HWRM_FUNC_BACKING_STORE_QCAPS_V2_OUTPUT_TYPE_RQ_DB_SHADOW \
+		UINT32_C(0x17)
+	/* SRQ Doorbell shadow region. */
+	#define HWRM_FUNC_BACKING_STORE_QCAPS_V2_OUTPUT_TYPE_SRQ_DB_SHADOW \
+		UINT32_C(0x18)
+	/* CQ Doorbell shadow region. */
+	#define HWRM_FUNC_BACKING_STORE_QCAPS_V2_OUTPUT_TYPE_CQ_DB_SHADOW \
+		UINT32_C(0x19)
+	/* QUIC Tx key context. */
+	#define HWRM_FUNC_BACKING_STORE_QCAPS_V2_OUTPUT_TYPE_QUIC_TKC \
+		UINT32_C(0x1a)
+	/* QUIC Rx key context. */
+	#define HWRM_FUNC_BACKING_STORE_QCAPS_V2_OUTPUT_TYPE_QUIC_RKC \
+		UINT32_C(0x1b)
+	/* Invalid type. */
+	#define HWRM_FUNC_BACKING_STORE_QCAPS_V2_OUTPUT_TYPE_INVALID \
+		UINT32_C(0xffff)
+	#define HWRM_FUNC_BACKING_STORE_QCAPS_V2_OUTPUT_TYPE_LAST \
+		HWRM_FUNC_BACKING_STORE_QCAPS_V2_OUTPUT_TYPE_INVALID
+	/* Number of bytes per backing store entry. */
+	uint16_t	entry_size;
+	/* Control flags. */
+	uint32_t	flags;
+	/*
+	 * When set, it indicates the context type should be initialized
+	 * with the “ctx_init_value” at the specified offset.
+	 */
+	#define HWRM_FUNC_BACKING_STORE_QCAPS_V2_OUTPUT_FLAGS_ENABLE_CTX_KIND_INIT \
+		UINT32_C(0x1)
+	/* When set, it indicates the context type is valid. */
+	#define HWRM_FUNC_BACKING_STORE_QCAPS_V2_OUTPUT_FLAGS_TYPE_VALID \
+		UINT32_C(0x2)
+	/*
+	 * When set, it indicates the region for this type is not a regular
+	 * context memory but a driver managed memory that is created,
+	 * initialized and managed by the driver.
+	 */
+	#define HWRM_FUNC_BACKING_STORE_QCAPS_V2_OUTPUT_FLAGS_DRIVER_MANAGED_MEMORY \
+		UINT32_C(0x4)
+	/*
+	 * Bit map of the valid instances associated with the
+	 * backing store type.
+	 */
+	uint32_t	instance_bit_map;
+	/*
+	 * Initializer to be used by drivers to initialize context memory
+	 * to ensure context subsystem flags an error for an attack before
+	 * the first time context load.
+	 */
+	uint8_t	ctx_init_value;
+	/*
+	 * Specifies the doubleword offset of ctx_init_value for this
+	 * context type.
+	 */
+	uint8_t	ctx_init_offset;
+	/*
+	 * Some backing store types, e.g., TQM rings, require the number
+	 * of entries to be a multiple of this value to prevent any
+	 * resource allocation limitations. If not applicable, leave
+	 * this field with "0".
+	 */
+	uint8_t	entry_multiple;
+	uint8_t	rsvd;
+	/* Maximum number of backing store entries supported for this type. */
+	uint32_t	max_num_entries;
+	/*
+	 * Minimum number of backing store entries required for this type.
+	 * This field is only valid for some backing store types, e.g.,
+	 * TQM rings. If not applicable, leave this field with "0".
+	 */
+	uint32_t	min_num_entries;
+	/*
+	 * Next valid backing store type. If current type queried is already
+	 * the last valid type, firmware must set this field to invalid type.
+	 */
+	uint16_t	next_valid_type;
+	/*
+	 * This field counts how many split entries contain valid data.
+	 * Below is the table that maps the count value:
+	 * | count |                     Indication                     |
+	 * | ----- | -------------------------------------------------- |
+	 * |   0   | None of the split entries has valid data.          |
+	 * |   1   | Only "split_entry_0" contains valid data.          |
+	 * |   2   | Only "split_entry_0" and "1" have valid data.      |
+	 * |   3   | Only "split_entry_0", "1" and "2" have valid data. |
+	 * |   4   | All four split entries have valid data.            |
+	 */
+	uint8_t	subtype_valid_cnt;
+	uint8_t	rsvd2;
+	/*
+	 * Split entry #0. Note that the four split entries (as a group)
+	 * must be cast to a type-specific data structure first before
+	 * accessing it! Below is the table that maps a backing store
+	 * type to the associated split entry casting data structure.
+	 * | Type |       Split Entry Casting Data Structure           |
+	 * | ---- | -------------------------------------------------- |
+	 * | QPC  |             qpc_split_entries                      |
+	 * | SRQ  |             srq_split_entries                      |
+	 * | CQ   |             cq_split_entries                       |
+	 * | VINC |            vnic_split_entries                      |
+	 * | MRAV |            marv_split_entries                      |
+	 */
+	uint32_t	split_entry_0;
+	/* Split entry #1. */
+	uint32_t	split_entry_1;
+	/* Split entry #2. */
+	uint32_t	split_entry_2;
+	/* Split entry #3. */
+	uint32_t	split_entry_3;
+	uint8_t	rsvd3[3];
+	/*
+	 * This field is used in Output records to indicate that the
+	 * output is completely written to RAM.  This field should be
+	 * read as '1' to indicate that the output has been completely
+	 * written. When writing a command completion or response to
+	 * an internal processor, the order of writes has to be such
+	 * that this field is written last.
+	 */
+	uint8_t	valid;
+} __rte_packed;
+
+/****************************
+ * hwrm_func_dbr_pacing_cfg *
+ ****************************/
+
+
+/* hwrm_func_dbr_pacing_cfg_input (size:320b/40B) */
+struct hwrm_func_dbr_pacing_cfg_input {
+	/* The HWRM command request type. */
+	uint16_t	req_type;
+	/*
+	 * The completion ring to send the completion event on. This should
+	 * be the NQ ID returned from the `nq_alloc` HWRM command.
+	 */
+	uint16_t	cmpl_ring;
+	/*
+	 * The sequence ID is used by the driver for tracking multiple
+	 * commands. This ID is treated as opaque data by the firmware and
+	 * the value is returned in the `hwrm_resp_hdr` upon completion.
+	 */
+	uint16_t	seq_id;
+	/*
+	 * The target ID of the command:
+	 * * 0x0-0xFFF8 - The function ID
+	 * * 0xFFF8-0xFFFC, 0xFFFE - Reserved for internal processors
+	 * * 0xFFFD - Reserved for user-space HWRM interface
+	 * * 0xFFFF - HWRM
+	 */
+	uint16_t	target_id;
+	/*
+	 * A physical address pointer pointing to a host buffer that the
+	 * command's response data will be written. This can be either a host
+	 * physical address (HPA) or a guest physical address (GPA) and must
+	 * point to a physically contiguous block of memory.
+	 */
+	uint64_t	resp_addr;
+	uint8_t	flags;
+	/*
+	 * This bit must be '1' to enable DBR NQ events. The NQ ID to
+	 * receive the events must be specified in the primary_nq_id
+	 * field.
+	 */
+	#define HWRM_FUNC_DBR_PACING_CFG_INPUT_FLAGS_DBR_NQ_EVENT_ENABLE \
+		UINT32_C(0x1)
+	/* This bit must be '1' to disable DBR NQ events. */
+	#define HWRM_FUNC_DBR_PACING_CFG_INPUT_FLAGS_DBR_NQ_EVENT_DISABLE \
+		UINT32_C(0x2)
+	uint8_t	unused_0[7];
+	uint32_t	enables;
+	/*
+	 * This bit must be '1' for the primary_nq_id field to be
+	 * configured.
+	 */
+	#define HWRM_FUNC_DBR_PACING_CFG_INPUT_ENABLES_PRIMARY_NQ_ID_VALID \
+		UINT32_C(0x1)
+	/*
+	 * This bit must be '1' for the pacing_threshold field to be
+	 * configured.
+	 */
+	#define HWRM_FUNC_DBR_PACING_CFG_INPUT_ENABLES_PACING_THRESHOLD_VALID \
+		UINT32_C(0x2)
+	/*
+	 * Specify primary function’s NQ ID to receive the doorbell pacing
+	 * threshold crossing events.
+	 */
+	uint32_t	primary_nq_id;
+	/*
+	 * Specify pacing threshold value, as a percentage of the max
+	 * doorbell FIFO depth. The range is 1 to 36.
+	 */
+	uint32_t	pacing_threshold;
+	uint8_t	unused_1[4];
+} __rte_packed;
+
+/* hwrm_func_dbr_pacing_cfg_output (size:128b/16B) */
+struct hwrm_func_dbr_pacing_cfg_output {
+	/* The specific error status for the command. */
+	uint16_t	error_code;
+	/* The HWRM command request type. */
+	uint16_t	req_type;
+	/* The sequence ID from the original command. */
+	uint16_t	seq_id;
+	/* The length of the response data in number of bytes. */
+	uint16_t	resp_len;
+	uint8_t	unused_0[7];
+	/*
+	 * This field is used in Output records to indicate that the output
+	 * is completely written to RAM.  This field should be read as '1'
+	 * to indicate that the output has been completely written.
+	 * When writing a command completion or response to an internal
+	 * processor, the order of writes has to be such that this field is
+	 * written last.
+	 */
+	uint8_t	valid;
+} __rte_packed;
+
+/*****************************
+ * hwrm_func_dbr_pacing_qcfg *
+ *****************************/
+
+
+/* hwrm_func_dbr_pacing_qcfg_input (size:128b/16B) */
+struct hwrm_func_dbr_pacing_qcfg_input {
+	/* The HWRM command request type. */
+	uint16_t	req_type;
+	/*
+	 * The completion ring to send the completion event on. This should
+	 * be the NQ ID returned from the `nq_alloc` HWRM command.
+	 */
+	uint16_t	cmpl_ring;
+	/*
+	 * The sequence ID is used by the driver for tracking multiple
+	 * commands. This ID is treated as opaque data by the firmware and
+	 * the value is returned in the `hwrm_resp_hdr` upon completion.
+	 */
+	uint16_t	seq_id;
+	/*
+	 * The target ID of the command:
+	 * * 0x0-0xFFF8 - The function ID
+	 * * 0xFFF8-0xFFFC, 0xFFFE - Reserved for internal processors
+	 * * 0xFFFD - Reserved for user-space HWRM interface
+	 * * 0xFFFF - HWRM
+	 */
+	uint16_t	target_id;
+	/*
+	 * A physical address pointer pointing to a host buffer that the
+	 * command's response data will be written. This can be either a host
+	 * physical address (HPA) or a guest physical address (GPA) and must
+	 * point to a physically contiguous block of memory.
+	 */
+	uint64_t	resp_addr;
 } __rte_packed;
 
-/* hwrm_func_backing_store_qcfg_v2_output (size:256b/32B) */
-struct hwrm_func_backing_store_qcfg_v2_output {
+/* hwrm_func_dbr_pacing_qcfg_output (size:512b/64B) */
+struct hwrm_func_dbr_pacing_qcfg_output {
 	/* The specific error status for the command. */
 	uint16_t	error_code;
 	/* The HWRM command request type. */
@@ -19717,110 +22371,216 @@ struct hwrm_func_backing_store_qcfg_v2_output {
 	uint16_t	seq_id;
 	/* The length of the response data in number of bytes. */
 	uint16_t	resp_len;
-	/* Type of backing store to be configured. */
-	uint16_t	type;
-	/* Queue pair. */
-	#define HWRM_FUNC_BACKING_STORE_QCFG_V2_OUTPUT_TYPE_QP \
+	uint8_t	flags;
+	/* When this bit is '1', it indicates DBR NQ events are enabled. */
+	#define HWRM_FUNC_DBR_PACING_QCFG_OUTPUT_FLAGS_DBR_NQ_EVENT_ENABLED \
+		UINT32_C(0x1)
+	uint8_t	unused_0[7];
+	/*
+	 * The Doorbell global FIFO occupancy register. This field should be
+	 * used by the driver and user library in the doorbell pacing
+	 * algorithm. Lower 2 bits indicates address space location and upper
+	 * 30 bits indicates upper 30bits of the register address. A value of
+	 * 0xFFFF-FFFF indicates this register does not exist.
+	 */
+	uint32_t	dbr_stat_db_fifo_reg;
+	/* Lower 2 bits indicates address space location. */
+	#define HWRM_FUNC_DBR_PACING_QCFG_OUTPUT_DBR_STAT_DB_FIFO_REG_ADDR_SPACE_MASK \
+		UINT32_C(0x3)
+	#define HWRM_FUNC_DBR_PACING_QCFG_OUTPUT_DBR_STAT_DB_FIFO_REG_ADDR_SPACE_SFT \
+		0
+	/*
+	 * If value is 0, this register is located in PCIe config space.
+	 * Drivers have to map appropriate window to access this
+	 * register.
+	 */
+	#define HWRM_FUNC_DBR_PACING_QCFG_OUTPUT_DBR_STAT_DB_FIFO_REG_ADDR_SPACE_PCIE_CFG \
 		UINT32_C(0x0)
-	/* Shared receive queue. */
-	#define HWRM_FUNC_BACKING_STORE_QCFG_V2_OUTPUT_TYPE_SRQ \
+	/*
+	 * If value is 1, this register is located in GRC address space.
+	 * Drivers have to map appropriate window to access this
+	 * register.
+	 */
+	#define HWRM_FUNC_DBR_PACING_QCFG_OUTPUT_DBR_STAT_DB_FIFO_REG_ADDR_SPACE_GRC \
 		UINT32_C(0x1)
-	/* Completion queue. */
-	#define HWRM_FUNC_BACKING_STORE_QCFG_V2_OUTPUT_TYPE_CQ \
+	/*
+	 * If value is 2, this register is located in first BAR address
+	 * space. Drivers have to map appropriate window to access this
+	 * register.
+	 */
+	#define HWRM_FUNC_DBR_PACING_QCFG_OUTPUT_DBR_STAT_DB_FIFO_REG_ADDR_SPACE_BAR0 \
 		UINT32_C(0x2)
-	/* Virtual NIC. */
-	#define HWRM_FUNC_BACKING_STORE_QCFG_V2_OUTPUT_TYPE_VNIC \
+	/*
+	 * If value is 3, this register is located in second BAR address
+	 * space. Drivers have to map appropriate window to access this
+	 * Drivers have to map appropriate window to access this
+	 * register.
+	 */
+	#define HWRM_FUNC_DBR_PACING_QCFG_OUTPUT_DBR_STAT_DB_FIFO_REG_ADDR_SPACE_BAR1 \
 		UINT32_C(0x3)
-	/* Statistic context. */
-	#define HWRM_FUNC_BACKING_STORE_QCFG_V2_OUTPUT_TYPE_STAT \
-		UINT32_C(0x4)
-	/* Slow-path TQM ring. */
-	#define HWRM_FUNC_BACKING_STORE_QCFG_V2_OUTPUT_TYPE_SP_TQM_RING \
-		UINT32_C(0x5)
-	/* Fast-path TQM ring. */
-	#define HWRM_FUNC_BACKING_STORE_QCFG_V2_OUTPUT_TYPE_FP_TQM_RING \
-		UINT32_C(0x6)
-	/* MRAV. */
-	#define HWRM_FUNC_BACKING_STORE_QCFG_V2_OUTPUT_TYPE_MRAV \
-		UINT32_C(0xe)
-	/* TIM. */
-	#define HWRM_FUNC_BACKING_STORE_QCFG_V2_OUTPUT_TYPE_TIM \
-		UINT32_C(0xf)
-	/* Tx key context. */
-	#define HWRM_FUNC_BACKING_STORE_QCFG_V2_OUTPUT_TYPE_TKC \
-		UINT32_C(0x13)
-	/* Rx key context. */
-	#define HWRM_FUNC_BACKING_STORE_QCFG_V2_OUTPUT_TYPE_RKC \
-		UINT32_C(0x14)
-	/* Mid-path TQM ring. */
-	#define HWRM_FUNC_BACKING_STORE_QCFG_V2_OUTPUT_TYPE_MP_TQM_RING \
-		UINT32_C(0x15)
-	#define HWRM_FUNC_BACKING_STORE_QCFG_V2_OUTPUT_TYPE_LAST \
-		HWRM_FUNC_BACKING_STORE_QCFG_V2_OUTPUT_TYPE_MP_TQM_RING
+	#define HWRM_FUNC_DBR_PACING_QCFG_OUTPUT_DBR_STAT_DB_FIFO_REG_ADDR_SPACE_LAST \
+		HWRM_FUNC_DBR_PACING_QCFG_OUTPUT_DBR_STAT_DB_FIFO_REG_ADDR_SPACE_BAR1
+	/* Upper 30bits of the register address. */
+	#define HWRM_FUNC_DBR_PACING_QCFG_OUTPUT_DBR_STAT_DB_FIFO_REG_ADDR_MASK \
+		UINT32_C(0xfffffffc)
+	#define HWRM_FUNC_DBR_PACING_QCFG_OUTPUT_DBR_STAT_DB_FIFO_REG_ADDR_SFT \
+		2
 	/*
-	 * Instance of the backing store type. It is zero-based,
-	 * which means "0" indicates the first instance. For backing
-	 * stores with single instance only, leave this field to 0.
+	 * This field indicates the mask value for dbr_stat_db_fifo_reg
+	 * to get the high watermark for doorbell FIFO.
 	 */
-	uint16_t	instance;
-	/* Control flags. */
-	uint32_t	flags;
-	/* Page directory. */
-	uint64_t	page_dir;
-	/* Number of entries */
-	uint32_t	num_entries;
-	/* Page size and pbl level. */
-	uint8_t	page_size_pbl_level;
-	/* PBL indirect levels. */
-	#define HWRM_FUNC_BACKING_STORE_QCFG_V2_OUTPUT_PBL_LEVEL_MASK \
-		UINT32_C(0xf)
-	#define HWRM_FUNC_BACKING_STORE_QCFG_V2_OUTPUT_PBL_LEVEL_SFT   0
-	/* PBL pointer is physical start address. */
-	#define HWRM_FUNC_BACKING_STORE_QCFG_V2_OUTPUT_PBL_LEVEL_LVL_0 \
+	uint32_t	dbr_stat_db_fifo_reg_watermark_mask;
+	/*
+	 * This field indicates the shift value for dbr_stat_db_fifo_reg
+	 * to get the high watermark for doorbell FIFO.
+	 */
+	uint8_t	dbr_stat_db_fifo_reg_watermark_shift;
+	uint8_t	unused_1[3];
+	/*
+	 * This field indicates the mask value for dbr_stat_db_fifo_reg
+	 * to get the amount of room left for doorbell FIFO.
+	 */
+	uint32_t	dbr_stat_db_fifo_reg_fifo_room_mask;
+	/*
+	 * This field indicates the shift value for dbr_stat_db_fifo_reg
+	 * to get the amount of room left for doorbell FIFO.
+	 */
+	uint8_t	dbr_stat_db_fifo_reg_fifo_room_shift;
+	uint8_t	unused_2[3];
+	/*
+	 * DBR_REG_AEQ_ARM register. This field should be used by the driver
+	 * to rearm the interrupt for regeneration of a notification to the
+	 * host from the hardware when the global doorbell occupancy threshold
+	 * is above the threshold value. Lower 2 bits indicates address space
+	 * location and upper 30 bits indicates upper 30bits of the register
+	 * address. A value of 0xFFFF-FFFF indicates this register does not
+	 * exist.
+	 */
+	uint32_t	dbr_throttling_aeq_arm_reg;
+	/* Lower 2 bits indicates address space location. */
+	#define HWRM_FUNC_DBR_PACING_QCFG_OUTPUT_DBR_THROTTLING_AEQ_ARM_REG_ADDR_SPACE_MASK \
+		UINT32_C(0x3)
+	#define HWRM_FUNC_DBR_PACING_QCFG_OUTPUT_DBR_THROTTLING_AEQ_ARM_REG_ADDR_SPACE_SFT \
+		0
+	/*
+	 * If value is 0, this register is located in PCIe config space.
+	 * Drivers have to map appropriate window to access this
+	 * register.
+	 */
+	#define HWRM_FUNC_DBR_PACING_QCFG_OUTPUT_DBR_THROTTLING_AEQ_ARM_REG_ADDR_SPACE_PCIE_CFG \
 		UINT32_C(0x0)
-	/* PBL pointer points to PTE table. */
-	#define HWRM_FUNC_BACKING_STORE_QCFG_V2_OUTPUT_PBL_LEVEL_LVL_1 \
+	/*
+	 * If value is 1, this register is located in GRC address space.
+	 * Drivers have to map appropriate window to access this
+	 * register.
+	 */
+	#define HWRM_FUNC_DBR_PACING_QCFG_OUTPUT_DBR_THROTTLING_AEQ_ARM_REG_ADDR_SPACE_GRC \
 		UINT32_C(0x1)
 	/*
-	 * PBL pointer points to PDE table with each entry pointing to
-	 * PTE tables.
+	 * If value is 2, this register is located in first BAR address
+	 * space. Drivers have to map appropriate window to access this
+	 * register.
 	 */
-	#define HWRM_FUNC_BACKING_STORE_QCFG_V2_OUTPUT_PBL_LEVEL_LVL_2 \
+	#define HWRM_FUNC_DBR_PACING_QCFG_OUTPUT_DBR_THROTTLING_AEQ_ARM_REG_ADDR_SPACE_BAR0 \
 		UINT32_C(0x2)
-	#define HWRM_FUNC_BACKING_STORE_QCFG_V2_OUTPUT_PBL_LEVEL_LAST \
-		HWRM_FUNC_BACKING_STORE_QCFG_V2_OUTPUT_PBL_LEVEL_LVL_2
-	/* Page size. */
-	#define HWRM_FUNC_BACKING_STORE_QCFG_V2_OUTPUT_PAGE_SIZE_MASK \
-		UINT32_C(0xf0)
-	#define HWRM_FUNC_BACKING_STORE_QCFG_V2_OUTPUT_PAGE_SIZE_SFT   4
-	/* 4KB. */
-	#define HWRM_FUNC_BACKING_STORE_QCFG_V2_OUTPUT_PAGE_SIZE_PG_4K \
-		(UINT32_C(0x0) << 4)
-	/* 8KB. */
-	#define HWRM_FUNC_BACKING_STORE_QCFG_V2_OUTPUT_PAGE_SIZE_PG_8K \
-		(UINT32_C(0x1) << 4)
-	/* 64KB. */
-	#define HWRM_FUNC_BACKING_STORE_QCFG_V2_OUTPUT_PAGE_SIZE_PG_64K \
-		(UINT32_C(0x2) << 4)
-	/* 2MB. */
-	#define HWRM_FUNC_BACKING_STORE_QCFG_V2_OUTPUT_PAGE_SIZE_PG_2M \
-		(UINT32_C(0x3) << 4)
-	/* 8MB. */
-	#define HWRM_FUNC_BACKING_STORE_QCFG_V2_OUTPUT_PAGE_SIZE_PG_8M \
-		(UINT32_C(0x4) << 4)
-	/* 1GB. */
-	#define HWRM_FUNC_BACKING_STORE_QCFG_V2_OUTPUT_PAGE_SIZE_PG_1G \
-		(UINT32_C(0x5) << 4)
-	#define HWRM_FUNC_BACKING_STORE_QCFG_V2_OUTPUT_PAGE_SIZE_LAST \
-		HWRM_FUNC_BACKING_STORE_QCFG_V2_OUTPUT_PAGE_SIZE_PG_1G
-	uint8_t	rsvd[2];
 	/*
-	 * This field is used in Output records to indicate that the
-	 * output is completely written to RAM.  This field should be
-	 * read as '1' to indicate that the output has been completely
-	 * written. When writing a command completion or response to
-	 * an internal processor, the order of writes has to be such
-	 * that this field is written last.
+	 * If value is 3, this register is located in second BAR address
+	 * space. Drivers have to map appropriate window to access this
+	 * Drivers have to map appropriate window to access this
+	 * register.
+	 */
+	#define HWRM_FUNC_DBR_PACING_QCFG_OUTPUT_DBR_THROTTLING_AEQ_ARM_REG_ADDR_SPACE_BAR1 \
+		UINT32_C(0x3)
+	#define HWRM_FUNC_DBR_PACING_QCFG_OUTPUT_DBR_THROTTLING_AEQ_ARM_REG_ADDR_SPACE_LAST \
+		HWRM_FUNC_DBR_PACING_QCFG_OUTPUT_DBR_THROTTLING_AEQ_ARM_REG_ADDR_SPACE_BAR1
+	/* Upper 30bits of the register address. */
+	#define HWRM_FUNC_DBR_PACING_QCFG_OUTPUT_DBR_THROTTLING_AEQ_ARM_REG_ADDR_MASK \
+		UINT32_C(0xfffffffc)
+	#define HWRM_FUNC_DBR_PACING_QCFG_OUTPUT_DBR_THROTTLING_AEQ_ARM_REG_ADDR_SFT \
+		2
+	/*
+	 * This field indicates the value to be written for
+	 * dbr_throttling_aeq_arm_reg register.
+	 */
+	uint8_t	dbr_throttling_aeq_arm_reg_val;
+	uint8_t	unused_3[7];
+	/*
+	 * Specifies primary function’s NQ ID.
+	 * A value of 0xFFFF indicates NQ ID is invalid.
+	 */
+	uint32_t	primary_nq_id;
+	/*
+	 * Specifies the pacing threshold value, as a percentage of the
+	 * max doorbell FIFO depth. The range is 1 to 100.
+	 */
+	uint32_t	pacing_threshold;
+	uint8_t	unused_4[7];
+	/*
+	 * This field is used in Output records to indicate that the output
+	 * is completely written to RAM.  This field should be read as '1'
+	 * to indicate that the output has been completely written.
+	 * When writing a command completion or response to an internal
+	 * processor, the order of writes has to be such that this field is
+	 * written last.
+	 */
+	uint8_t	valid;
+} __rte_packed;
+
+/****************************************
+ * hwrm_func_dbr_pacing_broadcast_event *
+ ****************************************/
+
+
+/* hwrm_func_dbr_pacing_broadcast_event_input (size:128b/16B) */
+struct hwrm_func_dbr_pacing_broadcast_event_input {
+	/* The HWRM command request type. */
+	uint16_t	req_type;
+	/*
+	 * The completion ring to send the completion event on. This should
+	 * be the NQ ID returned from the `nq_alloc` HWRM command.
+	 */
+	uint16_t	cmpl_ring;
+	/*
+	 * The sequence ID is used by the driver for tracking multiple
+	 * commands. This ID is treated as opaque data by the firmware and
+	 * the value is returned in the `hwrm_resp_hdr` upon completion.
+	 */
+	uint16_t	seq_id;
+	/*
+	 * The target ID of the command:
+	 * * 0x0-0xFFF8 - The function ID
+	 * * 0xFFF8-0xFFFC, 0xFFFE - Reserved for internal processors
+	 * * 0xFFFD - Reserved for user-space HWRM interface
+	 * * 0xFFFF - HWRM
+	 */
+	uint16_t	target_id;
+	/*
+	 * A physical address pointer pointing to a host buffer that the
+	 * command's response data will be written. This can be either a host
+	 * physical address (HPA) or a guest physical address (GPA) and must
+	 * point to a physically contiguous block of memory.
+	 */
+	uint64_t	resp_addr;
+} __rte_packed;
+
+/* hwrm_func_dbr_pacing_broadcast_event_output (size:128b/16B) */
+struct hwrm_func_dbr_pacing_broadcast_event_output {
+	/* The specific error status for the command. */
+	uint16_t	error_code;
+	/* The HWRM command request type. */
+	uint16_t	req_type;
+	/* The sequence ID from the original command. */
+	uint16_t	seq_id;
+	/* The length of the response data in number of bytes. */
+	uint16_t	resp_len;
+	uint8_t	unused_0[7];
+	/*
+	 * This field is used in Output records to indicate that the output
+	 * is completely written to RAM.  This field should be read as '1'
+	 * to indicate that the output has been completely written.
+	 * When writing a command completion or response to an internal
+	 * processor, the order of writes has to be such that this field is
+	 * written last.
 	 */
 	uint8_t	valid;
 } __rte_packed;
@@ -22856,7 +25616,7 @@ struct hwrm_port_phy_qcfg_output {
  *********************/
 
 
-/* hwrm_port_mac_cfg_input (size:384b/48B) */
+/* hwrm_port_mac_cfg_input (size:448b/56B) */
 struct hwrm_port_mac_cfg_input {
 	/* The HWRM command request type. */
 	uint16_t	req_type;
@@ -23002,6 +25762,20 @@ struct hwrm_port_mac_cfg_input {
 	 */
 	#define HWRM_PORT_MAC_CFG_INPUT_FLAGS_PTP_ONE_STEP_TX_TS \
 		UINT32_C(0x2000)
+	/*
+	 * When this bit is '1', the controller is requested to enable
+	 * timestamp capture capability on all packets (not just PTP)
+	 * of the receive side of this port.
+	 */
+	#define HWRM_PORT_MAC_CFG_INPUT_FLAGS_ALL_RX_TS_CAPTURE_ENABLE \
+		UINT32_C(0x4000)
+	/*
+	 * When this bit is '1', the controller is requested to disable
+	 * timestamp capture capability on all packets (not just PTP)
+	 * of the receive side of this port.
+	 */
+	#define HWRM_PORT_MAC_CFG_INPUT_FLAGS_ALL_RX_TS_CAPTURE_DISABLE \
+		UINT32_C(0x8000)
 	uint32_t	enables;
 	/*
 	 * This bit must be '1' for the ipg field to be
@@ -23255,12 +26029,13 @@ struct hwrm_port_mac_cfg_input {
 	 * of sync timer updates (measured in parts per billion).
 	 */
 	int32_t	ptp_freq_adj_ppb;
+	uint8_t	unused_1[4];
 	/*
 	 * This unsigned field specifies the phase offset to be applied
 	 * to the PHC (PTP Hardware Clock). This field is specified in
 	 * nanoseconds.
 	 */
-	uint32_t	ptp_adj_phase;
+	int64_t	ptp_adj_phase;
 } __rte_packed;
 
 /* hwrm_port_mac_cfg_output (size:128b/16B) */
@@ -23701,6 +26476,12 @@ struct hwrm_port_mac_ptp_qcfg_output {
 	 */
 	#define HWRM_PORT_MAC_PTP_QCFG_OUTPUT_FLAGS_PARTIAL_DIRECT_ACCESS_REF_CLOCK \
 		UINT32_C(0x10)
+	/*
+	 * When this bit is set to '1', it indicates that driver has
+	 * configured 64bit RTC.
+	 */
+	#define HWRM_PORT_MAC_PTP_QCFG_OUTPUT_FLAGS_RTC_CONFIGURED \
+		UINT32_C(0x20)
 	uint8_t	unused_0[3];
 	/*
 	 * Offset of the PTP register for the lower 32 bits of timestamp
@@ -31726,7 +34507,11 @@ struct hwrm_queue_dscp2pri_cfg_input {
 	uint64_t	resp_addr;
 	/*
 	 * This is the host address where the 24-bits DSCP-MASK-PRI tuple
-	 * will be copied from.
+	 * will be copied from. A non-zero mask "adds" a tuple, while
+	 * a mask equal to 0 triggers the firmware to remove a tuple.
+	 * Only tuples with unique DSCP values are stored. On chips
+	 * prior to Thor a mask can be 0 - 0x3f, while on Thor it can
+	 * be 0 or 0x3f.
 	 */
 	uint64_t	src_data_addr;
 	uint32_t	flags;
@@ -32891,12 +35676,6 @@ struct hwrm_vnic_update_input {
 	#define HWRM_VNIC_UPDATE_INPUT_ENABLES_METADATA_FORMAT_TYPE_VALID \
 		UINT32_C(0x4)
 	/*
-	 * This bit must be '1' for the rss_hash_function field to be
-	 * configured.
-	 */
-	#define HWRM_VNIC_UPDATE_INPUT_ENABLES_RSS_HASH_FUNCTION_VALID \
-		UINT32_C(0x8)
-	/*
 	 * This will update the context variable with the same name if
 	 * the corresponding enable is set.
 	 */
@@ -32941,32 +35720,7 @@ struct hwrm_vnic_update_input {
 	 * the mru of the port the function is associated with.
 	 */
 	uint16_t	mru;
-	/*
-	 * Used to choose the RSS hash algorithm based on which HW
-	 * can select the destination ring
-	 */
-	uint8_t	rss_hash_function;
-	/*
-	 * NIC calculates the RSS hash using the Toeplitz algorithm on L3/L4
-	 * headers and uses the hash to select the ring.
-	 */
-	#define HWRM_VNIC_UPDATE_INPUT_RSS_HASH_FUNCTION_TOEPLITZ UINT32_C(0x0)
-	/*
-	 * NIC calculates the RSS hash using the XOR algorithm on L3/L4
-	 * headers and uses the hash to select the ring.
-	 */
-	#define HWRM_VNIC_UPDATE_INPUT_RSS_HASH_FUNCTION_XOR      UINT32_C(0x1)
-	/*
-	 * In this mode, RSS hash is calculated with Toeplitz and reported
-	 * in the RX completion. However, the ring selection algorithm is
-	 * based on the checksum. In this mode, only the innermost L3/L4
-	 * packet checksums are used. So this hash function mode will not
-	 * be exposed/valid when the outer RSS mode is enabled.
-	 */
-	#define HWRM_VNIC_UPDATE_INPUT_RSS_HASH_FUNCTION_CHECKSUM UINT32_C(0x2)
-	#define HWRM_VNIC_UPDATE_INPUT_RSS_HASH_FUNCTION_LAST \
-		HWRM_VNIC_UPDATE_INPUT_RSS_HASH_FUNCTION_CHECKSUM
-	uint8_t	unused_1[3];
+	uint8_t	unused_1[4];
 } __rte_packed;
 
 /* hwrm_vnic_update_output (size:128b/16B) */
@@ -33206,6 +35960,9 @@ struct hwrm_vnic_cfg_input {
 	/* This bit must be '1' for the rx_csum_v2_mode field to be configured. */
 	#define HWRM_VNIC_CFG_INPUT_ENABLES_RX_CSUM_V2_MODE \
 		UINT32_C(0x100)
+	/* This bit must be '1' for the l2_cqe_mode field to be configured. */
+	#define HWRM_VNIC_CFG_INPUT_ENABLES_L2_CQE_MODE \
+		UINT32_C(0x200)
 	/* Logical vnic ID */
 	uint16_t	vnic_id;
 	/*
@@ -33297,7 +36054,37 @@ struct hwrm_vnic_cfg_input {
 	#define HWRM_VNIC_CFG_INPUT_RX_CSUM_V2_MODE_MAX     UINT32_C(0x2)
 	#define HWRM_VNIC_CFG_INPUT_RX_CSUM_V2_MODE_LAST \
 		HWRM_VNIC_CFG_INPUT_RX_CSUM_V2_MODE_MAX
-	uint8_t	unused0[5];
+	/*
+	 * If the device supports different L2 RX CQE modes, as indicated by
+	 * the HWRM_VNIC_QCAPS command, this field is used to configure the
+	 * CQE mode.
+	 */
+	uint8_t	l2_cqe_mode;
+	/*
+	 * When configured with this cqe mode, A normal (32B) CQE
+	 * will be generated. This is the default mode.
+	 */
+	#define HWRM_VNIC_CFG_INPUT_L2_CQE_MODE_DEFAULT    UINT32_C(0x0)
+	/*
+	 * When configured with this cqe mode, A compressed (16B) CQE
+	 * will be generated. In this mode TPA and HDS are not supported.
+	 * Host drivers should not configure the TPA and HDS along with
+	 * compressed mode, per VNIC. FW returns error, if host drivers
+	 * try to configure the VNIC with compressed mode and (TPA or HDS).
+	 * The compressed completion does not include PTP data. Host
+	 * drivers should not use this mode to receive the PTP data.
+	 */
+	#define HWRM_VNIC_CFG_INPUT_L2_CQE_MODE_COMPRESSED UINT32_C(0x1)
+	/*
+	 * When configured with this cqe mode, HW generates either a 32B
+	 * completion or a 16B completion depending on use case within a
+	 * VNIC. For ex. a simple L2 packet could use the compressed form
+	 * while a PTP packet on the same VNIC would use the 32B form.
+	 */
+	#define HWRM_VNIC_CFG_INPUT_L2_CQE_MODE_MIXED      UINT32_C(0x2)
+	#define HWRM_VNIC_CFG_INPUT_L2_CQE_MODE_LAST \
+		HWRM_VNIC_CFG_INPUT_L2_CQE_MODE_MIXED
+	uint8_t	unused0[4];
 } __rte_packed;
 
 /* hwrm_vnic_cfg_output (size:128b/16B) */
@@ -33504,7 +36291,31 @@ struct hwrm_vnic_qcfg_output {
 	#define HWRM_VNIC_QCFG_OUTPUT_RX_CSUM_V2_MODE_MAX     UINT32_C(0x2)
 	#define HWRM_VNIC_QCFG_OUTPUT_RX_CSUM_V2_MODE_LAST \
 		HWRM_VNIC_QCFG_OUTPUT_RX_CSUM_V2_MODE_MAX
-	uint8_t	unused_1[4];
+	/*
+	 * If the device supports different L2 RX CQE modes, as indicated by
+	 * the HWRM_VNIC_QCAPS command, this field is used to convey the
+	 * configured CQE mode.
+	 */
+	uint8_t	l2_cqe_mode;
+	/*
+	 * This value indicates that the VNIC is configured with normal
+	 * (32B) CQE mode.
+	 */
+	#define HWRM_VNIC_QCFG_OUTPUT_L2_CQE_MODE_DEFAULT    UINT32_C(0x0)
+	/*
+	 * This value indicates that the VNIC is configured with compressed
+	 * (16B) CQE mode.
+	 */
+	#define HWRM_VNIC_QCFG_OUTPUT_L2_CQE_MODE_COMPRESSED UINT32_C(0x1)
+	/*
+	 * This value indicates that the VNIC is configured with mixed
+	 * CQE mode. HW generates either a 32B completion or a 16B
+	 * completion depending on use case within a VNIC.
+	 */
+	#define HWRM_VNIC_QCFG_OUTPUT_L2_CQE_MODE_MIXED      UINT32_C(0x2)
+	#define HWRM_VNIC_QCFG_OUTPUT_L2_CQE_MODE_LAST \
+		HWRM_VNIC_QCFG_OUTPUT_L2_CQE_MODE_MIXED
+	uint8_t	unused_1[3];
 	/*
 	 * This field is used in Output records to indicate that the output
 	 * is completely written to RAM.  This field should be read as '1'
@@ -33692,27 +36503,37 @@ struct hwrm_vnic_qcaps_output {
 	#define HWRM_VNIC_QCAPS_OUTPUT_FLAGS_RSS_HASH_TYPE_DELTA_CAP \
 		UINT32_C(0x4000)
 	/*
-	 * When this bit is '1', it indicates that HW is capable
-	 * of calculating the RSS hash using Toeplitz algorithm.
+	 * When this bit is '1', it indicates that HW is capable of using
+	 * Toeplitz algorithm. This mode uses Toeplitz algorithm and
+	 * provided Toeplitz hash key to hash the packets according to the
+	 * configured hash type and hash mode. The Toeplitz hash results and
+	 * the provided Toeplitz RSS indirection table are used to determine
+	 * the RSS rings.
 	 */
-	#define HWRM_VNIC_QCAPS_OUTPUT_FLAGS_RSS_HASH_FUNCTION_TOEPLITZ_CAP \
+	#define HWRM_VNIC_QCAPS_OUTPUT_FLAGS_RING_SELECT_MODE_TOEPLITZ_CAP \
 		UINT32_C(0x8000)
 	/*
-	 * When this bit is '1', it indicates that HW is capable
-	 * of calculating the RSS hash using XOR algorithm.
+	 * When this bit is '1', it indicates that HW is capable of using
+	 * XOR algorithm. This mode uses XOR algorithm to hash the packets
+	 * according to the configured hash type and hash mode. The XOR
+	 * hash results and the provided XOR RSS indirection table are
+	 * used to determine the RSS rings. Host drivers provided hash key
+	 * is not honored in this mode.
 	 */
-	#define HWRM_VNIC_QCAPS_OUTPUT_FLAGS_RSS_HASH_FUNCTION_XOR_CAP \
+	#define HWRM_VNIC_QCAPS_OUTPUT_FLAGS_RING_SELECT_MODE_XOR_CAP \
 		UINT32_C(0x10000)
 	/*
-	 * When this bit is '1', it indicates that HW is capable
-	 * of using checksum algorithm.
-	 * In this mode, RSS hash is calculated with Toeplitz and reported
-	 * in the RX completion. However, the ring selection algorithm is
-	 * based on the checksum. In this mode, only the innermost L3/L4
-	 * packet checksums are used. So this hash function mode will not
-	 * be exposed/valid when the outer RSS mode is enabled.
+	 * When this bit is '1', it indicates that HW is capable of using
+	 * checksum algorithm. In this mode, HW uses inner packets checksum
+	 * algorithm to distribute the packets across the rings and Toeplitz
+	 * algorithm to calculate the hash to convey it in the RX
+	 * completions. Host drivers should provide Toeplitz hash key.
+	 * As HW uses innermost packets checksum to distribute the packets
+	 * across the rings, host drivers can't convey hash  mode to choose
+	 * outer headers to calculate Toeplitz hash. FW will fail such
+	 * configuration.
 	 */
-	#define HWRM_VNIC_QCAPS_OUTPUT_FLAGS_RSS_HASH_FUNCTION_CHKSM_CAP \
+	#define HWRM_VNIC_QCAPS_OUTPUT_FLAGS_RING_SELECT_MODE_TOEPLITZ_CHKSM_CAP \
 		UINT32_C(0x20000)
 	/*
 	 * When this bit is '1' HW supports hash calculation
@@ -33721,9 +36542,25 @@ struct hwrm_vnic_qcaps_output {
 	#define HWRM_VNIC_QCAPS_OUTPUT_FLAGS_RSS_IPV6_FLOW_LABEL_CAP \
 		UINT32_C(0x40000)
 	/*
+	 * When this bit is '1', it indicates that HW and firmware supports
+	 * the use of RX V3 and RX TPA start V3 completion records for all
+	 * the RX rings of a VNIC. Once set, this feature is mandatory to
+	 * be used for the RX rings of the VNIC. If set to '0', the
+	 * HW and the firmware does not support this feature.
+	 */
+	#define HWRM_VNIC_QCAPS_OUTPUT_FLAGS_RX_CMPL_V3_CAP \
+		UINT32_C(0x80000)
+	/*
+	 * When this bit is '1' HW supports different RX CQE record types.
+	 * Host drivers can choose the mode based on their application
+	 * requirements like performance, TPA, HDS and PTP.
+	 */
+	#define HWRM_VNIC_QCAPS_OUTPUT_FLAGS_L2_CQE_MODE_CAP \
+		UINT32_C(0x100000)
+	/*
 	 * This field advertises the maximum concurrent TPA aggregations
-	 * supported by the VNIC on new devices that support TPA v2.
-	 * '0' means that TPA v2 is not supported.
+	 * supported by the VNIC on new devices that support TPA v2 or v3.
+	 * '0' means that both the TPA v2 and v3 are not supported.
 	 */
 	uint16_t	max_aggs_supported;
 	uint8_t	unused_1[5];
@@ -33984,37 +36821,54 @@ struct hwrm_vnic_rss_cfg_input {
 	 * over source and destination IPv4 addresses of IPv4
 	 * packets.
 	 */
-	#define HWRM_VNIC_RSS_CFG_INPUT_HASH_TYPE_IPV4         UINT32_C(0x1)
+	#define HWRM_VNIC_RSS_CFG_INPUT_HASH_TYPE_IPV4 \
+		UINT32_C(0x1)
 	/*
 	 * When this bit is '1', the RSS hash shall be computed
 	 * over source/destination IPv4 addresses and
 	 * source/destination ports of TCP/IPv4 packets.
 	 */
-	#define HWRM_VNIC_RSS_CFG_INPUT_HASH_TYPE_TCP_IPV4     UINT32_C(0x2)
+	#define HWRM_VNIC_RSS_CFG_INPUT_HASH_TYPE_TCP_IPV4 \
+		UINT32_C(0x2)
 	/*
 	 * When this bit is '1', the RSS hash shall be computed
 	 * over source/destination IPv4 addresses and
 	 * source/destination ports of UDP/IPv4 packets.
 	 */
-	#define HWRM_VNIC_RSS_CFG_INPUT_HASH_TYPE_UDP_IPV4     UINT32_C(0x4)
+	#define HWRM_VNIC_RSS_CFG_INPUT_HASH_TYPE_UDP_IPV4 \
+		UINT32_C(0x4)
 	/*
 	 * When this bit is '1', the RSS hash shall be computed
 	 * over source and destination IPv6 addresses of IPv6
 	 * packets.
 	 */
-	#define HWRM_VNIC_RSS_CFG_INPUT_HASH_TYPE_IPV6         UINT32_C(0x8)
+	#define HWRM_VNIC_RSS_CFG_INPUT_HASH_TYPE_IPV6 \
+		UINT32_C(0x8)
 	/*
 	 * When this bit is '1', the RSS hash shall be computed
 	 * over source/destination IPv6 addresses and
 	 * source/destination ports of TCP/IPv6 packets.
 	 */
-	#define HWRM_VNIC_RSS_CFG_INPUT_HASH_TYPE_TCP_IPV6     UINT32_C(0x10)
+	#define HWRM_VNIC_RSS_CFG_INPUT_HASH_TYPE_TCP_IPV6 \
+		UINT32_C(0x10)
 	/*
 	 * When this bit is '1', the RSS hash shall be computed
 	 * over source/destination IPv6 addresses and
 	 * source/destination ports of UDP/IPv6 packets.
 	 */
-	#define HWRM_VNIC_RSS_CFG_INPUT_HASH_TYPE_UDP_IPV6     UINT32_C(0x20)
+	#define HWRM_VNIC_RSS_CFG_INPUT_HASH_TYPE_UDP_IPV6 \
+		UINT32_C(0x20)
+	/*
+	 * When this bit is '1', the RSS hash shall be computed
+	 * over source, destination IPv6 addresses and flow label of IPv6
+	 * packets. Hash type ipv6 and ipv6_flow_label are mutually
+	 * exclusive. HW does not include the flow_label in hash
+	 * calculation for the packets that are matching tcp_ipv6 and
+	 * udp_ipv6 hash types. Host drivers should set this bit based on
+	 * rss_ipv6_flow_label_cap.
+	 */
+	#define HWRM_VNIC_RSS_CFG_INPUT_HASH_TYPE_IPV6_FLOW_LABEL \
+		UINT32_C(0x40)
 	/* VNIC ID of VNIC associated with RSS table being configured. */
 	uint16_t	vnic_id;
 	/*
@@ -34097,34 +36951,38 @@ struct hwrm_vnic_rss_cfg_input {
 	 */
 	#define HWRM_VNIC_RSS_CFG_INPUT_FLAGS_HASH_TYPE_EXCLUDE \
 		UINT32_C(0x2)
+	uint8_t	ring_select_mode;
 	/*
-	 * Used to choose the RSS hash algorithm based on which HW
-	 * can select the destination ring
+	 * In this mode, HW uses Toeplitz algorithm and provided Toeplitz
+	 * hash key to hash the packets according to the configured hash
+	 * type and hash mode. The Toeplitz hash results and the provided
+	 * Toeplitz RSS indirection table are used to determine the RSS
+	 * rings.
 	 */
-	uint8_t	rss_hash_function;
-	/*
-	 * NIC calculates the RSS hash using the Toeplitz algorithm on L3/L4
-	 * headers and uses the hash to select the ring.
-	 */
-	#define HWRM_VNIC_RSS_CFG_INPUT_RSS_HASH_FUNCTION_TOEPLITZ \
+	#define HWRM_VNIC_RSS_CFG_INPUT_RING_SELECT_MODE_TOEPLITZ \
 		UINT32_C(0x0)
 	/*
-	 * NIC calculates the RSS hash using the XOR algorithm on L3/L4
-	 * headers and uses the hash to select the ring.
+	 * In this mode, HW uses XOR algorithm to hash the packets according
+	 * to the configured hash type and hash mode. The XOR hash results
+	 * and the provided XOR RSS indirection table are used to determine
+	 * the RSS rings. Host drivers provided hash key is not honored in
+	 * this mode.
 	 */
-	#define HWRM_VNIC_RSS_CFG_INPUT_RSS_HASH_FUNCTION_XOR \
+	#define HWRM_VNIC_RSS_CFG_INPUT_RING_SELECT_MODE_XOR \
 		UINT32_C(0x1)
 	/*
-	 * In this mode, RSS hash is calculated with Toeplitz and reported
-	 * in the RX completion. However, the ring selection algorithm is
-	 * based on the checksum. In this mode, only the innermost L3/L4
-	 * packet checksums are used. So this hash function mode will not
-	 * be exposed/valid when the outer RSS mode is enabled.
+	 * In this mode, HW uses inner packets checksum algorithm to
+	 * distribute the packets across the rings and Toeplitz algorithm
+	 * to calculate the hash to convey it in the RX completions. Host
+	 * drivers should provide Toeplitz hash key. As HW uses innermost
+	 * packets checksum to distribute the packets across the rings,
+	 * host drivers can't convey hash mode to choose outer headers to
+	 * calculate Toeplitz hash. FW will fail such configuration.
 	 */
-	#define HWRM_VNIC_RSS_CFG_INPUT_RSS_HASH_FUNCTION_CHECKSUM \
+	#define HWRM_VNIC_RSS_CFG_INPUT_RING_SELECT_MODE_TOEPLITZ_CHECKSUM \
 		UINT32_C(0x2)
-	#define HWRM_VNIC_RSS_CFG_INPUT_RSS_HASH_FUNCTION_LAST \
-		HWRM_VNIC_RSS_CFG_INPUT_RSS_HASH_FUNCTION_CHECKSUM
+	#define HWRM_VNIC_RSS_CFG_INPUT_RING_SELECT_MODE_LAST \
+		HWRM_VNIC_RSS_CFG_INPUT_RING_SELECT_MODE_TOEPLITZ_CHECKSUM
 	uint8_t	unused_1[4];
 } __rte_packed;
 
@@ -34302,34 +37160,38 @@ struct hwrm_vnic_rss_qcfg_output {
 	 */
 	#define HWRM_VNIC_RSS_QCFG_OUTPUT_HASH_MODE_FLAGS_OUTERMOST_2 \
 		UINT32_C(0x10)
+	uint8_t	ring_select_mode;
 	/*
-	 * Used to choose the RSS hash algorithm based on which HW
-	 * can select the destination ring
-	 */
-	uint8_t	rss_hash_function;
-	/*
-	 * NIC calculates the RSS hash using the Toeplitz algorithm on L3/L4
-	 * headers and uses the hash to select the ring.
+	 * In this mode, HW uses Toeplitz algorithm and provided Toeplitz
+	 * hash key to hash the packets according to the configured hash
+	 * type and hash mode. The Toeplitz hash results and the provided
+	 * Toeplitz RSS indirection table are used to determine the RSS
+	 * rings.
 	 */
-	#define HWRM_VNIC_RSS_QCFG_OUTPUT_RSS_HASH_FUNCTION_TOEPLITZ \
+	#define HWRM_VNIC_RSS_QCFG_OUTPUT_RING_SELECT_MODE_TOEPLITZ \
 		UINT32_C(0x0)
 	/*
-	 * NIC calculates the RSS hash using the XOR algorithm on L3/L4
-	 * headers and uses the hash to select the ring.
+	 * In this mode, HW uses XOR algorithm to hash the packets according
+	 * to the configured hash type and hash mode. The XOR hash results
+	 * and the provided XOR RSS indirection table are used to determine
+	 * the RSS rings. Host drivers provided hash key is not honored in
+	 * this mode.
 	 */
-	#define HWRM_VNIC_RSS_QCFG_OUTPUT_RSS_HASH_FUNCTION_XOR \
+	#define HWRM_VNIC_RSS_QCFG_OUTPUT_RING_SELECT_MODE_XOR \
 		UINT32_C(0x1)
 	/*
-	 * In this mode, RSS hash is calculated with Toeplitz and reported
-	 * in the RX completion. However, the ring selection algorithm is
-	 * based on the checksum. In this mode, only the innermost L3/L4
-	 * packet checksums are used. So this hash function mode will not
-	 * be exposed/valid when the outer RSS mode is enabled.
+	 * In this mode, HW uses inner packets checksum algorithm to
+	 * distribute the packets across the rings and Toeplitz algorithm
+	 * to calculate the hash to convey it in the RX completions. Host
+	 * drivers should provide Toeplitz hash key. As HW uses innermost
+	 * packets checksum to distribute the packets across the rings,
+	 * host drivers can't convey hash mode to choose outer headers to
+	 * calculate Toeplitz hash. FW will fail such configuration.
 	 */
-	#define HWRM_VNIC_RSS_QCFG_OUTPUT_RSS_HASH_FUNCTION_CHECKSUM \
+	#define HWRM_VNIC_RSS_QCFG_OUTPUT_RING_SELECT_MODE_TOEPLITZ_CHECKSUM \
 		UINT32_C(0x2)
-	#define HWRM_VNIC_RSS_QCFG_OUTPUT_RSS_HASH_FUNCTION_LAST \
-		HWRM_VNIC_RSS_QCFG_OUTPUT_RSS_HASH_FUNCTION_CHECKSUM
+	#define HWRM_VNIC_RSS_QCFG_OUTPUT_RING_SELECT_MODE_LAST \
+		HWRM_VNIC_RSS_QCFG_OUTPUT_RING_SELECT_MODE_TOEPLITZ_CHECKSUM
 	uint8_t	unused_1[5];
 	/*
 	 * This field is used in Output records to indicate that the output
@@ -34956,7 +37818,20 @@ struct hwrm_ring_alloc_input {
 	 * Rx rings and is ignored for all other rings included Rx
 	 * Aggregation rings.
 	 */
-	#define HWRM_RING_ALLOC_INPUT_FLAGS_RX_SOP_PAD     UINT32_C(0x1)
+	#define HWRM_RING_ALLOC_INPUT_FLAGS_RX_SOP_PAD \
+		UINT32_C(0x1)
+	/*
+	 * When the HW Doorbell Drop Recovery feature is enabled,
+	 * HW can flag false CQ overflow when CQ consumer index
+	 * doorbells are dropped when there really wasn't any overflow.
+	 * The CQE values could have already been processed by the driver,
+	 * but HW doesn't know about this because of the doorbell drop.
+	 * To avoid false detection of CQ overflow events,
+	 * it is recommended that CQ overflow detection is disabled
+	 * by the driver when HW based doorbell recovery is enabled.
+	 */
+	#define HWRM_RING_ALLOC_INPUT_FLAGS_DISABLE_CQ_OVERFLOW_DETECTION \
+		UINT32_C(0x2)
 	/*
 	 * This value is a pointer to the page table for the
 	 * Ring.
@@ -45758,6 +48633,318 @@ struct hwrm_cfa_lag_group_member_unrgtr_output {
 	uint8_t	valid;
 } __rte_packed;
 
+/*****************************
+ * hwrm_cfa_tls_filter_alloc *
+ *****************************/
+
+
+/* hwrm_cfa_tls_filter_alloc_input (size:704b/88B) */
+struct hwrm_cfa_tls_filter_alloc_input {
+	/* The HWRM command request type. */
+	uint16_t	req_type;
+	/*
+	 * The completion ring to send the completion event on. This should
+	 * be the NQ ID returned from the `nq_alloc` HWRM command.
+	 */
+	uint16_t	cmpl_ring;
+	/*
+	 * The sequence ID is used by the driver for tracking multiple
+	 * commands. This ID is treated as opaque data by the firmware and
+	 * the value is returned in the `hwrm_resp_hdr` upon completion.
+	 */
+	uint16_t	seq_id;
+	/*
+	 * The target ID of the command:
+	 * * 0x0-0xFFF8 - The function ID
+	 * * 0xFFF8-0xFFFC, 0xFFFE - Reserved for internal processors
+	 * * 0xFFFD - Reserved for user-space HWRM interface
+	 * * 0xFFFF - HWRM
+	 */
+	uint16_t	target_id;
+	/*
+	 * A physical address pointer pointing to a host buffer that the
+	 * command's response data will be written. This can be either a host
+	 * physical address (HPA) or a guest physical address (GPA) and must
+	 * point to a physically contiguous block of memory.
+	 */
+	uint64_t	resp_addr;
+	uint32_t	unused_0;
+	uint32_t	enables;
+	/*
+	 * This bit must be '1' for the l2_filter_id field to be
+	 * configured.
+	 */
+	#define HWRM_CFA_TLS_FILTER_ALLOC_INPUT_ENABLES_L2_FILTER_ID \
+		UINT32_C(0x1)
+	/*
+	 * This bit must be '1' for the ethertype field to be
+	 * configured.
+	 */
+	#define HWRM_CFA_TLS_FILTER_ALLOC_INPUT_ENABLES_ETHERTYPE \
+		UINT32_C(0x2)
+	/*
+	 * This bit must be '1' for the ipaddr_type field to be
+	 * configured.
+	 */
+	#define HWRM_CFA_TLS_FILTER_ALLOC_INPUT_ENABLES_IPADDR_TYPE \
+		UINT32_C(0x4)
+	/*
+	 * This bit must be '1' for the src_ipaddr field to be
+	 * configured.
+	 */
+	#define HWRM_CFA_TLS_FILTER_ALLOC_INPUT_ENABLES_SRC_IPADDR \
+		UINT32_C(0x8)
+	/*
+	 * This bit must be '1' for the dst_ipaddr field to be
+	 * configured.
+	 */
+	#define HWRM_CFA_TLS_FILTER_ALLOC_INPUT_ENABLES_DST_IPADDR \
+		UINT32_C(0x10)
+	/*
+	 * This bit must be '1' for the ip_protocol field to be
+	 * configured.
+	 */
+	#define HWRM_CFA_TLS_FILTER_ALLOC_INPUT_ENABLES_IP_PROTOCOL \
+		UINT32_C(0x20)
+	/*
+	 * This bit must be '1' for the src_port field to be
+	 * configured.
+	 */
+	#define HWRM_CFA_TLS_FILTER_ALLOC_INPUT_ENABLES_SRC_PORT \
+		UINT32_C(0x40)
+	/*
+	 * This bit must be '1' for the dst_port field to be
+	 * configured.
+	 */
+	#define HWRM_CFA_TLS_FILTER_ALLOC_INPUT_ENABLES_DST_PORT \
+		UINT32_C(0x80)
+	/*
+	 * This bit must be '1' for the kid field to be
+	 * configured.
+	 */
+	#define HWRM_CFA_TLS_FILTER_ALLOC_INPUT_ENABLES_KID \
+		UINT32_C(0x100)
+	/*
+	 * This bit must be '1' for the dst_id field to be
+	 * configured.
+	 */
+	#define HWRM_CFA_TLS_FILTER_ALLOC_INPUT_ENABLES_DST_ID \
+		UINT32_C(0x200)
+	/*
+	 * This bit must be '1' for the mirror_vnic_id field to be
+	 * configured.
+	 */
+	#define HWRM_CFA_TLS_FILTER_ALLOC_INPUT_ENABLES_MIRROR_VNIC_ID \
+		UINT32_C(0x400)
+	/*
+	 * This value identifies a set of CFA data structures used for an L2
+	 * context.
+	 */
+	uint64_t	l2_filter_id;
+	uint8_t	unused_1[6];
+	/* This value indicates the ethertype in the Ethernet header. */
+	uint16_t	ethertype;
+	/*
+	 * This value indicates the type of IP address.
+	 * 4 - IPv4
+	 * 6 - IPv6
+	 * All others are invalid.
+	 */
+	uint8_t	ip_addr_type;
+	/* invalid */
+	#define HWRM_CFA_TLS_FILTER_ALLOC_INPUT_IP_ADDR_TYPE_UNKNOWN \
+		UINT32_C(0x0)
+	/* IPv4 */
+	#define HWRM_CFA_TLS_FILTER_ALLOC_INPUT_IP_ADDR_TYPE_IPV4 \
+		UINT32_C(0x4)
+	/* IPv6 */
+	#define HWRM_CFA_TLS_FILTER_ALLOC_INPUT_IP_ADDR_TYPE_IPV6 \
+		UINT32_C(0x6)
+	#define HWRM_CFA_TLS_FILTER_ALLOC_INPUT_IP_ADDR_TYPE_LAST \
+		HWRM_CFA_TLS_FILTER_ALLOC_INPUT_IP_ADDR_TYPE_IPV6
+	/*
+	 * The value of protocol filed in IP header.
+	 * Applies to UDP and TCP traffic.
+	 * 6 - TCP
+	 * 17 - UDP
+	 */
+	uint8_t	ip_protocol;
+	/* invalid */
+	#define HWRM_CFA_TLS_FILTER_ALLOC_INPUT_IP_PROTOCOL_UNKNOWN \
+		UINT32_C(0x0)
+	/* TCP */
+	#define HWRM_CFA_TLS_FILTER_ALLOC_INPUT_IP_PROTOCOL_TCP \
+		UINT32_C(0x6)
+	/* UDP */
+	#define HWRM_CFA_TLS_FILTER_ALLOC_INPUT_IP_PROTOCOL_UDP \
+		UINT32_C(0x11)
+	#define HWRM_CFA_TLS_FILTER_ALLOC_INPUT_IP_PROTOCOL_LAST \
+		HWRM_CFA_TLS_FILTER_ALLOC_INPUT_IP_PROTOCOL_UDP
+	/*
+	 * If set, this value shall represent the
+	 * Logical VNIC ID of the destination VNIC for the RX
+	 * path and network port id of the destination port for
+	 * the TX path.
+	 */
+	uint16_t	dst_id;
+	/*
+	 * Logical VNIC ID of the VNIC where traffic is
+	 * mirrored.
+	 */
+	uint16_t	mirror_vnic_id;
+	uint8_t	unused_2[2];
+	/*
+	 * The value of source IP address to be used in filtering.
+	 * For IPv4, first four bytes represent the IP address.
+	 */
+	uint32_t	src_ipaddr[4];
+	/*
+	 * The value of destination IP address to be used in filtering.
+	 * For IPv4, first four bytes represent the IP address.
+	 */
+	uint32_t	dst_ipaddr[4];
+	/*
+	 * The value of source port to be used in filtering.
+	 * Applies to UDP and TCP traffic.
+	 */
+	uint16_t	src_port;
+	/*
+	 * The value of destination port to be used in filtering.
+	 * Applies to UDP and TCP traffic.
+	 */
+	uint16_t	dst_port;
+	/*
+	 * The Key Context Identifier (KID) for use with KTLS.
+	 * KID is limited to 20-bits.
+	 */
+	uint32_t	kid;
+} __rte_packed;
+
+/* hwrm_cfa_tls_filter_alloc_output (size:192b/24B) */
+struct hwrm_cfa_tls_filter_alloc_output {
+	/* The specific error status for the command. */
+	uint16_t	error_code;
+	/* The HWRM command request type. */
+	uint16_t	req_type;
+	/* The sequence ID from the original command. */
+	uint16_t	seq_id;
+	/* The length of the response data in number of bytes. */
+	uint16_t	resp_len;
+	/* This value is an opaque id into CFA data structures. */
+	uint64_t	tls_filter_id;
+	/*
+	 * The flow id value in bit 0-29 is the actual ID of the flow
+	 * associated with this filter and it shall be used to match
+	 * and associate the flow identifier returned in completion
+	 * records. A value of 0xFFFFFFFF in the 32-bit flow_id field
+	 * shall indicate no valid flow id.
+	 */
+	uint32_t	flow_id;
+	/* Indicate the flow id value. */
+	#define HWRM_CFA_TLS_FILTER_ALLOC_OUTPUT_FLOW_ID_VALUE_MASK \
+		UINT32_C(0x3fffffff)
+	#define HWRM_CFA_TLS_FILTER_ALLOC_OUTPUT_FLOW_ID_VALUE_SFT 0
+	/* Indicate type of the flow. */
+	#define HWRM_CFA_TLS_FILTER_ALLOC_OUTPUT_FLOW_ID_TYPE \
+		UINT32_C(0x40000000)
+	/*
+	 * If this bit set to 0, then it indicates that the flow is
+	 * internal flow.
+	 */
+	#define HWRM_CFA_TLS_FILTER_ALLOC_OUTPUT_FLOW_ID_TYPE_INT \
+		(UINT32_C(0x0) << 30)
+	/*
+	 * If this bit is set to 1, then it indicates that the flow is
+	 * external flow.
+	 */
+	#define HWRM_CFA_TLS_FILTER_ALLOC_OUTPUT_FLOW_ID_TYPE_EXT \
+		(UINT32_C(0x1) << 30)
+	#define HWRM_CFA_TLS_FILTER_ALLOC_OUTPUT_FLOW_ID_TYPE_LAST \
+		HWRM_CFA_TLS_FILTER_ALLOC_OUTPUT_FLOW_ID_TYPE_EXT
+	/* Indicate the flow direction. */
+	#define HWRM_CFA_TLS_FILTER_ALLOC_OUTPUT_FLOW_ID_DIR \
+		UINT32_C(0x80000000)
+	/* If this bit set to 0, then it indicates rx flow. */
+	#define HWRM_CFA_TLS_FILTER_ALLOC_OUTPUT_FLOW_ID_DIR_RX \
+		(UINT32_C(0x0) << 31)
+	/* If this bit is set to 1, then it indicates that tx flow. */
+	#define HWRM_CFA_TLS_FILTER_ALLOC_OUTPUT_FLOW_ID_DIR_TX \
+		(UINT32_C(0x1) << 31)
+	#define HWRM_CFA_TLS_FILTER_ALLOC_OUTPUT_FLOW_ID_DIR_LAST \
+		HWRM_CFA_TLS_FILTER_ALLOC_OUTPUT_FLOW_ID_DIR_TX
+	uint8_t	unused_0[3];
+	/*
+	 * This field is used in Output records to indicate that the output
+	 * is completely written to RAM. This field should be read as '1'
+	 * to indicate that the output has been completely written.
+	 * When writing a command completion or response to an internal
+	 * processor, the order of writes has to be such that this field is
+	 * written last.
+	 */
+	uint8_t	valid;
+} __rte_packed;
+
+/****************************
+ * hwrm_cfa_tls_filter_free *
+ ****************************/
+
+
+/* hwrm_cfa_tls_filter_free_input (size:192b/24B) */
+struct hwrm_cfa_tls_filter_free_input {
+	/* The HWRM command request type. */
+	uint16_t	req_type;
+	/*
+	 * The completion ring to send the completion event on. This should
+	 * be the NQ ID returned from the `nq_alloc` HWRM command.
+	 */
+	uint16_t	cmpl_ring;
+	/*
+	 * The sequence ID is used by the driver for tracking multiple
+	 * commands. This ID is treated as opaque data by the firmware and
+	 * the value is returned in the `hwrm_resp_hdr` upon completion.
+	 */
+	uint16_t	seq_id;
+	/*
+	 * The target ID of the command:
+	 * * 0x0-0xFFF8 - The function ID
+	 * * 0xFFF8-0xFFFC, 0xFFFE - Reserved for internal processors
+	 * * 0xFFFD - Reserved for user-space HWRM interface
+	 * * 0xFFFF - HWRM
+	 */
+	uint16_t	target_id;
+	/*
+	 * A physical address pointer pointing to a host buffer that the
+	 * command's response data will be written. This can be either a host
+	 * physical address (HPA) or a guest physical address (GPA) and must
+	 * point to a physically contiguous block of memory.
+	 */
+	uint64_t	resp_addr;
+	/* This value is an opaque id into CFA data structures. */
+	uint64_t	tls_filter_id;
+} __rte_packed;
+
+/* hwrm_cfa_tls_filter_free_output (size:128b/16B) */
+struct hwrm_cfa_tls_filter_free_output {
+	/* The specific error status for the command. */
+	uint16_t	error_code;
+	/* The HWRM command request type. */
+	uint16_t	req_type;
+	/* The sequence ID from the original command. */
+	uint16_t	seq_id;
+	/* The length of the response data in number of bytes. */
+	uint16_t	resp_len;
+	uint8_t	unused_0[7];
+	/*
+	 * This field is used in Output records to indicate that the output
+	 * is completely written to RAM. This field should be read as '1'
+	 * to indicate that the output has been completely written.
+	 * When writing a command completion or response to an internal
+	 * processor, the order of writes has to be such that this field is
+	 * written last.
+	 */
+	uint8_t	valid;
+} __rte_packed;
+
 /***********
  * hwrm_tf *
  ***********/
@@ -49045,7 +52232,7 @@ struct hwrm_tf_if_tbl_get_input {
 	uint32_t	index;
 } __rte_packed;
 
-/* hwrm_tf_if_tbl_get_output (size:256b/32B) */
+/* hwrm_tf_if_tbl_get_output (size:1216b/152B) */
 struct hwrm_tf_if_tbl_get_output {
 	/* The specific error status for the command. */
 	uint16_t	error_code;
@@ -49062,7 +52249,7 @@ struct hwrm_tf_if_tbl_get_output {
 	/* unused */
 	uint16_t	unused0;
 	/* Response data. */
-	uint8_t	data[8];
+	uint8_t	data[128];
 	/* unused */
 	uint8_t	unused1[7];
 	/*
@@ -49081,7 +52268,7 @@ struct hwrm_tf_if_tbl_get_output {
  ***************************/
 
 
-/* hwrm_tf_if_tbl_set_input (size:384b/48B) */
+/* hwrm_tf_if_tbl_set_input (size:1024b/128B) */
 struct hwrm_tf_if_tbl_set_input {
 	/* The HWRM command request type. */
 	uint16_t	req_type;
@@ -49137,7 +52324,7 @@ struct hwrm_tf_if_tbl_set_input {
 	/* unused */
 	uint8_t	unused1[6];
 	/* Data to be set. */
-	uint8_t	data[8];
+	uint8_t	data[88];
 } __rte_packed;
 
 /* hwrm_tf_if_tbl_set_output (size:128b/16B) */
-- 
2.10.1


^ permalink raw reply	[flat|nested] 19+ messages in thread

* [PATCH 02/17] net/bnxt: fix device capability reporting
  2022-04-27 14:58 [dpdk-dev] [PATCH 00/17] bnxt PMD fixes Kalesh A P
  2022-04-27 14:58 ` [PATCH 01/17] net/bnxt: update HWRM structures Kalesh A P
@ 2022-04-27 14:58 ` Kalesh A P
  2022-04-27 14:58 ` [PATCH 03/17] net/bnxt: fix to remove an unused macro Kalesh A P
                   ` (15 subsequent siblings)
  17 siblings, 0 replies; 19+ messages in thread
From: Kalesh A P @ 2022-04-27 14:58 UTC (permalink / raw)
  To: dev; +Cc: ajit.khaparde

From: Kalesh AP <kalesh-anakkur.purayil@broadcom.com>

1. Added two functions bnxt_get_tx_port_offloads() and
   bnxt_get_rx_port_offloads() to report the device
   tx/rx offload capabilities to the application.
2. This avoids few duplicate code in the driver and make
   VF-rep capability the same as VF.
3. This will help in selectively reporting offload capabilities
   based on FW support.

Fixes: 0a6d2a720078 ("net/bnxt: get device infos")
Cc: stable@dpdk.org

Signed-off-by: Kalesh AP <kalesh-anakkur.purayil@broadcom.com>
Reviewed-by: Ajit Khaparde <ajit.khaparde@broadcom.com>
Reviewed-by: Somnath Kotur <somnath.kotur@broadcom.com>
---
 drivers/net/bnxt/bnxt.h        | 24 ------------------------
 drivers/net/bnxt/bnxt_ethdev.c | 10 ++--------
 drivers/net/bnxt/bnxt_reps.c   |  6 ++----
 drivers/net/bnxt/bnxt_rxq.c    | 25 +++++++++++++++++++++++++
 drivers/net/bnxt/bnxt_rxq.h    |  1 +
 drivers/net/bnxt/bnxt_txq.c    | 23 +++++++++++++++++++++++
 drivers/net/bnxt/bnxt_txq.h    |  1 +
 7 files changed, 54 insertions(+), 36 deletions(-)

diff --git a/drivers/net/bnxt/bnxt.h b/drivers/net/bnxt/bnxt.h
index 44724a9..5eddb4f 100644
--- a/drivers/net/bnxt/bnxt.h
+++ b/drivers/net/bnxt/bnxt.h
@@ -580,30 +580,6 @@ struct bnxt_rep_info {
 	RTE_ETH_RSS_NONFRAG_IPV6_UDP |	\
 	RTE_ETH_RSS_LEVEL_MASK)
 
-#define BNXT_DEV_TX_OFFLOAD_SUPPORT (RTE_ETH_TX_OFFLOAD_IPV4_CKSUM | \
-				     RTE_ETH_TX_OFFLOAD_TCP_CKSUM | \
-				     RTE_ETH_TX_OFFLOAD_UDP_CKSUM | \
-				     RTE_ETH_TX_OFFLOAD_TCP_TSO | \
-				     RTE_ETH_TX_OFFLOAD_OUTER_IPV4_CKSUM | \
-				     RTE_ETH_TX_OFFLOAD_VXLAN_TNL_TSO | \
-				     RTE_ETH_TX_OFFLOAD_GRE_TNL_TSO | \
-				     RTE_ETH_TX_OFFLOAD_IPIP_TNL_TSO | \
-				     RTE_ETH_TX_OFFLOAD_GENEVE_TNL_TSO | \
-				     RTE_ETH_TX_OFFLOAD_QINQ_INSERT | \
-				     RTE_ETH_TX_OFFLOAD_MULTI_SEGS)
-
-#define BNXT_DEV_RX_OFFLOAD_SUPPORT (RTE_ETH_RX_OFFLOAD_VLAN_FILTER | \
-				     RTE_ETH_RX_OFFLOAD_IPV4_CKSUM | \
-				     RTE_ETH_RX_OFFLOAD_UDP_CKSUM | \
-				     RTE_ETH_RX_OFFLOAD_TCP_CKSUM | \
-				     RTE_ETH_RX_OFFLOAD_OUTER_IPV4_CKSUM | \
-				     RTE_ETH_RX_OFFLOAD_OUTER_UDP_CKSUM | \
-				     RTE_ETH_RX_OFFLOAD_KEEP_CRC | \
-				     RTE_ETH_RX_OFFLOAD_VLAN_EXTEND | \
-				     RTE_ETH_RX_OFFLOAD_TCP_LRO | \
-				     RTE_ETH_RX_OFFLOAD_SCATTER | \
-				     RTE_ETH_RX_OFFLOAD_RSS_HASH)
-
 #define BNXT_HWRM_SHORT_REQ_LEN		sizeof(struct hwrm_short_input)
 
 struct bnxt_flow_stat_info {
diff --git a/drivers/net/bnxt/bnxt_ethdev.c b/drivers/net/bnxt/bnxt_ethdev.c
index 261fe0b..fac3925 100644
--- a/drivers/net/bnxt/bnxt_ethdev.c
+++ b/drivers/net/bnxt/bnxt_ethdev.c
@@ -971,16 +971,10 @@ static int bnxt_dev_info_get_op(struct rte_eth_dev *eth_dev,
 	dev_info->min_rx_bufsize = 1;
 	dev_info->max_rx_pktlen = BNXT_MAX_PKT_LEN;
 
-	dev_info->rx_offload_capa = BNXT_DEV_RX_OFFLOAD_SUPPORT;
-	if (bp->flags & BNXT_FLAG_PTP_SUPPORTED)
-		dev_info->rx_offload_capa |= RTE_ETH_RX_OFFLOAD_TIMESTAMP;
-	if (bp->vnic_cap_flags & BNXT_VNIC_CAP_VLAN_RX_STRIP)
-		dev_info->rx_offload_capa |= RTE_ETH_RX_OFFLOAD_VLAN_STRIP;
+	dev_info->rx_offload_capa = bnxt_get_rx_port_offloads(bp);
 	dev_info->tx_queue_offload_capa = RTE_ETH_TX_OFFLOAD_MBUF_FAST_FREE;
-	dev_info->tx_offload_capa = BNXT_DEV_TX_OFFLOAD_SUPPORT |
+	dev_info->tx_offload_capa = bnxt_get_tx_port_offloads(bp) |
 				    dev_info->tx_queue_offload_capa;
-	if (bp->fw_cap & BNXT_FW_CAP_VLAN_TX_INSERT)
-		dev_info->tx_offload_capa |= RTE_ETH_TX_OFFLOAD_VLAN_INSERT;
 	dev_info->flow_type_rss_offloads = BNXT_ETH_RSS_SUPPORT;
 
 	dev_info->speed_capa = bnxt_get_speed_capabilities(bp);
diff --git a/drivers/net/bnxt/bnxt_reps.c b/drivers/net/bnxt/bnxt_reps.c
index e773932..8a5b777 100644
--- a/drivers/net/bnxt/bnxt_reps.c
+++ b/drivers/net/bnxt/bnxt_reps.c
@@ -567,10 +567,8 @@ int bnxt_rep_dev_info_get_op(struct rte_eth_dev *eth_dev,
 	dev_info->min_rx_bufsize = 1;
 	dev_info->max_rx_pktlen = BNXT_MAX_PKT_LEN;
 
-	dev_info->rx_offload_capa = BNXT_DEV_RX_OFFLOAD_SUPPORT;
-	if (parent_bp->flags & BNXT_FLAG_PTP_SUPPORTED)
-		dev_info->rx_offload_capa |= RTE_ETH_RX_OFFLOAD_TIMESTAMP;
-	dev_info->tx_offload_capa = BNXT_DEV_TX_OFFLOAD_SUPPORT;
+	dev_info->rx_offload_capa = bnxt_get_rx_port_offloads(parent_bp);
+	dev_info->tx_offload_capa = bnxt_get_tx_port_offloads(parent_bp);
 	dev_info->flow_type_rss_offloads = BNXT_ETH_RSS_SUPPORT;
 
 	dev_info->switch_info.name = eth_dev->device->name;
diff --git a/drivers/net/bnxt/bnxt_rxq.c b/drivers/net/bnxt/bnxt_rxq.c
index cd3bb14..0cfd0e5 100644
--- a/drivers/net/bnxt/bnxt_rxq.c
+++ b/drivers/net/bnxt/bnxt_rxq.c
@@ -20,6 +20,31 @@
  * RX Queues
  */
 
+uint64_t bnxt_get_rx_port_offloads(struct bnxt *bp)
+{
+	uint64_t rx_offload_capa;
+
+	rx_offload_capa = RTE_ETH_RX_OFFLOAD_IPV4_CKSUM  |
+			  RTE_ETH_RX_OFFLOAD_UDP_CKSUM   |
+			  RTE_ETH_RX_OFFLOAD_TCP_CKSUM   |
+			  RTE_ETH_RX_OFFLOAD_KEEP_CRC    |
+			  RTE_ETH_RX_OFFLOAD_VLAN_FILTER |
+			  RTE_ETH_RX_OFFLOAD_VLAN_EXTEND |
+			  RTE_ETH_RX_OFFLOAD_TCP_LRO |
+			  RTE_ETH_RX_OFFLOAD_SCATTER |
+			  RTE_ETH_RX_OFFLOAD_RSS_HASH;
+
+	rx_offload_capa |= RTE_ETH_RX_OFFLOAD_OUTER_IPV4_CKSUM |
+			   RTE_ETH_RX_OFFLOAD_OUTER_UDP_CKSUM;
+
+	if (bp->flags & BNXT_FLAG_PTP_SUPPORTED)
+		rx_offload_capa |= RTE_ETH_RX_OFFLOAD_TIMESTAMP;
+	if (bp->vnic_cap_flags & BNXT_VNIC_CAP_VLAN_RX_STRIP)
+		rx_offload_capa |= RTE_ETH_RX_OFFLOAD_VLAN_STRIP;
+
+	return rx_offload_capa;
+}
+
 /* Determine whether the current configuration needs aggregation ring in HW. */
 int bnxt_need_agg_ring(struct rte_eth_dev *eth_dev)
 {
diff --git a/drivers/net/bnxt/bnxt_rxq.h b/drivers/net/bnxt/bnxt_rxq.h
index 0331c23..287df8d 100644
--- a/drivers/net/bnxt/bnxt_rxq.h
+++ b/drivers/net/bnxt/bnxt_rxq.h
@@ -65,4 +65,5 @@ int bnxt_rx_queue_stop(struct rte_eth_dev *dev,
 void bnxt_rx_queue_release_mbufs(struct bnxt_rx_queue *rxq);
 int bnxt_need_agg_ring(struct rte_eth_dev *eth_dev);
 void bnxt_free_rxq_mem(struct bnxt_rx_queue *rxq);
+uint64_t bnxt_get_rx_port_offloads(struct bnxt *bp);
 #endif
diff --git a/drivers/net/bnxt/bnxt_txq.c b/drivers/net/bnxt/bnxt_txq.c
index 72a55ea..86073a5 100644
--- a/drivers/net/bnxt/bnxt_txq.c
+++ b/drivers/net/bnxt/bnxt_txq.c
@@ -17,6 +17,29 @@
  * TX Queues
  */
 
+uint64_t bnxt_get_tx_port_offloads(struct bnxt *bp)
+{
+	uint64_t tx_offload_capa;
+
+	tx_offload_capa = RTE_ETH_TX_OFFLOAD_IPV4_CKSUM  |
+			  RTE_ETH_TX_OFFLOAD_UDP_CKSUM   |
+			  RTE_ETH_TX_OFFLOAD_TCP_CKSUM   |
+			  RTE_ETH_TX_OFFLOAD_TCP_TSO     |
+			  RTE_ETH_TX_OFFLOAD_QINQ_INSERT |
+			  RTE_ETH_TX_OFFLOAD_MULTI_SEGS;
+
+	tx_offload_capa |= RTE_ETH_TX_OFFLOAD_OUTER_IPV4_CKSUM |
+			   RTE_ETH_TX_OFFLOAD_VXLAN_TNL_TSO |
+			   RTE_ETH_TX_OFFLOAD_GRE_TNL_TSO |
+			   RTE_ETH_TX_OFFLOAD_IPIP_TNL_TSO |
+			   RTE_ETH_TX_OFFLOAD_GENEVE_TNL_TSO;
+
+	if (bp->fw_cap & BNXT_FW_CAP_VLAN_TX_INSERT)
+		tx_offload_capa |= RTE_ETH_TX_OFFLOAD_VLAN_INSERT;
+
+	return tx_offload_capa;
+}
+
 void bnxt_free_txq_stats(struct bnxt_tx_queue *txq)
 {
 	if (txq && txq->cp_ring && txq->cp_ring->hw_stats)
diff --git a/drivers/net/bnxt/bnxt_txq.h b/drivers/net/bnxt/bnxt_txq.h
index 67fd4cb..f3a0381 100644
--- a/drivers/net/bnxt/bnxt_txq.h
+++ b/drivers/net/bnxt/bnxt_txq.h
@@ -43,4 +43,5 @@ int bnxt_tx_queue_setup_op(struct rte_eth_dev *eth_dev,
 			       uint16_t nb_desc,
 			       unsigned int socket_id,
 			       const struct rte_eth_txconf *tx_conf);
+uint64_t bnxt_get_tx_port_offloads(struct bnxt *bp);
 #endif
-- 
2.10.1


^ permalink raw reply	[flat|nested] 19+ messages in thread

* [PATCH 03/17] net/bnxt: fix to remove an unused macro
  2022-04-27 14:58 [dpdk-dev] [PATCH 00/17] bnxt PMD fixes Kalesh A P
  2022-04-27 14:58 ` [PATCH 01/17] net/bnxt: update HWRM structures Kalesh A P
  2022-04-27 14:58 ` [PATCH 02/17] net/bnxt: fix device capability reporting Kalesh A P
@ 2022-04-27 14:58 ` Kalesh A P
  2022-04-27 14:58 ` [PATCH 04/17] net/bnxt: fix Rxq configure Kalesh A P
                   ` (14 subsequent siblings)
  17 siblings, 0 replies; 19+ messages in thread
From: Kalesh A P @ 2022-04-27 14:58 UTC (permalink / raw)
  To: dev; +Cc: ajit.khaparde

From: Kalesh AP <kalesh-anakkur.purayil@broadcom.com>

BNXT_FLAG_UPDATE_HASH is redundant now, remove it.

Fixes: 1ebb765090a6 ("net/bnxt: fix config RSS update")
Cc: stable@dpdk.org

Signed-off-by: Kalesh AP <kalesh-anakkur.purayil@broadcom.com>
Reviewed-by: Ajit Khaparde <ajit.khaparde@broadcom.com>
Reviewed-by: Somnath Kotur <somnath.kotur@broadcom.com>
---
 drivers/net/bnxt/bnxt.h        | 1 -
 drivers/net/bnxt/bnxt_ethdev.c | 2 --
 drivers/net/bnxt/bnxt_rxq.c    | 3 ---
 3 files changed, 6 deletions(-)

diff --git a/drivers/net/bnxt/bnxt.h b/drivers/net/bnxt/bnxt.h
index 5eddb4f..9e5ff74 100644
--- a/drivers/net/bnxt/bnxt.h
+++ b/drivers/net/bnxt/bnxt.h
@@ -648,7 +648,6 @@ struct bnxt {
 #define BNXT_FLAG_PORT_STATS		BIT(2)
 #define BNXT_FLAG_JUMBO			BIT(3)
 #define BNXT_FLAG_SHORT_CMD		BIT(4)
-#define BNXT_FLAG_UPDATE_HASH		BIT(5)
 #define BNXT_FLAG_PTP_SUPPORTED		BIT(6)
 #define BNXT_FLAG_MULTI_HOST    	BIT(7)
 #define BNXT_FLAG_EXT_RX_PORT_STATS	BIT(8)
diff --git a/drivers/net/bnxt/bnxt_ethdev.c b/drivers/net/bnxt/bnxt_ethdev.c
index fac3925..181de42 100644
--- a/drivers/net/bnxt/bnxt_ethdev.c
+++ b/drivers/net/bnxt/bnxt_ethdev.c
@@ -2125,8 +2125,6 @@ static int bnxt_rss_hash_update_op(struct rte_eth_dev *eth_dev,
 			return -EINVAL;
 	}
 
-	bp->flags |= BNXT_FLAG_UPDATE_HASH;
-
 	/* Update the default RSS VNIC(s) */
 	vnic = BNXT_GET_DEFAULT_VNIC(bp);
 	vnic->hash_type = bnxt_rte_to_hwrm_hash_types(rss_conf->rss_hf);
diff --git a/drivers/net/bnxt/bnxt_rxq.c b/drivers/net/bnxt/bnxt_rxq.c
index 0cfd0e5..3c2283b 100644
--- a/drivers/net/bnxt/bnxt_rxq.c
+++ b/drivers/net/bnxt/bnxt_rxq.c
@@ -175,9 +175,6 @@ int bnxt_mq_rx_configure(struct bnxt *bp)
 	if (dev_conf->rxmode.mq_mode & RTE_ETH_MQ_RX_RSS_FLAG) {
 		struct rte_eth_rss_conf *rss = &bp->rss_conf;
 
-		if (bp->flags & BNXT_FLAG_UPDATE_HASH)
-			bp->flags &= ~BNXT_FLAG_UPDATE_HASH;
-
 		for (i = 0; i < bp->nr_vnics; i++) {
 			uint32_t lvl = RTE_ETH_RSS_LEVEL(rss->rss_hf);
 
-- 
2.10.1


^ permalink raw reply	[flat|nested] 19+ messages in thread

* [PATCH 04/17] net/bnxt: fix Rxq configure
  2022-04-27 14:58 [dpdk-dev] [PATCH 00/17] bnxt PMD fixes Kalesh A P
                   ` (2 preceding siblings ...)
  2022-04-27 14:58 ` [PATCH 03/17] net/bnxt: fix to remove an unused macro Kalesh A P
@ 2022-04-27 14:58 ` Kalesh A P
  2022-04-27 14:58 ` [PATCH 05/17] net/bnxt: fix support for tunnel stateless offloads Kalesh A P
                   ` (13 subsequent siblings)
  17 siblings, 0 replies; 19+ messages in thread
From: Kalesh A P @ 2022-04-27 14:58 UTC (permalink / raw)
  To: dev; +Cc: ajit.khaparde

From: Kalesh AP <kalesh-anakkur.purayil@broadcom.com>

We are currently not handling RX/RSS modes correctly.
After launching testpmd with multiple RXQs, if the user tries to set
the number of RXQs to 1, driver is not updating the "hash_type"
and "hash_mode" values of the VNICs. As a result, driver issues
bnxt_vnic_rss_configure() unnecessarily and the FW command fails.

Fixed bnxt_mq_rx_configure() to update VNIC RSS fields unconditionally.

Fixes: 4191bc8f79a8 ("net/bnxt: handle multi queue mode properly")
Cc: stable@dpdk.org

Signed-off-by: Kalesh AP <kalesh-anakkur.purayil@broadcom.com>
Reviewed-by: Ajit Khaparde <ajit.khaparde@broadcom.com>
Reviewed-by: Somnath Kotur <somnath.kotur@broadcom.com>
---
 drivers/net/bnxt/bnxt_rxq.c | 37 ++++++++++++++-----------------------
 1 file changed, 14 insertions(+), 23 deletions(-)

diff --git a/drivers/net/bnxt/bnxt_rxq.c b/drivers/net/bnxt/bnxt_rxq.c
index 3c2283b..8977138 100644
--- a/drivers/net/bnxt/bnxt_rxq.c
+++ b/drivers/net/bnxt/bnxt_rxq.c
@@ -65,6 +65,7 @@ void bnxt_free_rxq_stats(struct bnxt_rx_queue *rxq)
 int bnxt_mq_rx_configure(struct bnxt *bp)
 {
 	struct rte_eth_conf *dev_conf = &bp->eth_dev->data->dev_conf;
+	struct rte_eth_rss_conf *rss = &bp->rss_conf;
 	const struct rte_eth_vmdq_rx_conf *conf =
 		    &dev_conf->rx_adv_conf.vmdq_rx_conf;
 	unsigned int i, j, nb_q_per_grp = 1, ring_idx = 0;
@@ -172,29 +173,19 @@ int bnxt_mq_rx_configure(struct bnxt *bp)
 
 	bp->rx_num_qs_per_vnic = nb_q_per_grp;
 
-	if (dev_conf->rxmode.mq_mode & RTE_ETH_MQ_RX_RSS_FLAG) {
-		struct rte_eth_rss_conf *rss = &bp->rss_conf;
-
-		for (i = 0; i < bp->nr_vnics; i++) {
-			uint32_t lvl = RTE_ETH_RSS_LEVEL(rss->rss_hf);
-
-			vnic = &bp->vnic_info[i];
-			vnic->hash_type =
-				bnxt_rte_to_hwrm_hash_types(rss->rss_hf);
-			vnic->hash_mode =
-				bnxt_rte_to_hwrm_hash_level(bp,
-							    rss->rss_hf,
-							    lvl);
-
-			/*
-			 * Use the supplied key if the key length is
-			 * acceptable and the rss_key is not NULL
-			 */
-			if (rss->rss_key &&
-			    rss->rss_key_len <= HW_HASH_KEY_SIZE)
-				memcpy(vnic->rss_hash_key,
-				       rss->rss_key, rss->rss_key_len);
-		}
+	for (i = 0; i < bp->nr_vnics; i++) {
+		uint32_t lvl = RTE_ETH_RSS_LEVEL(rss->rss_hf);
+
+		vnic = &bp->vnic_info[i];
+		vnic->hash_type = bnxt_rte_to_hwrm_hash_types(rss->rss_hf);
+		vnic->hash_mode = bnxt_rte_to_hwrm_hash_level(bp, rss->rss_hf, lvl);
+
+		/*
+		 * Use the supplied key if the key length is
+		 * acceptable and the rss_key is not NULL
+		 */
+		if (rss->rss_key && rss->rss_key_len <= HW_HASH_KEY_SIZE)
+			memcpy(vnic->rss_hash_key, rss->rss_key, rss->rss_key_len);
 	}
 
 	return rc;
-- 
2.10.1


^ permalink raw reply	[flat|nested] 19+ messages in thread

* [PATCH 05/17] net/bnxt: fix support for tunnel stateless offloads
  2022-04-27 14:58 [dpdk-dev] [PATCH 00/17] bnxt PMD fixes Kalesh A P
                   ` (3 preceding siblings ...)
  2022-04-27 14:58 ` [PATCH 04/17] net/bnxt: fix Rxq configure Kalesh A P
@ 2022-04-27 14:58 ` Kalesh A P
  2022-04-27 14:58 ` [PATCH 06/17] net/bnxt: remove support for COUNT action Kalesh A P
                   ` (12 subsequent siblings)
  17 siblings, 0 replies; 19+ messages in thread
From: Kalesh A P @ 2022-04-27 14:58 UTC (permalink / raw)
  To: dev; +Cc: ajit.khaparde

From: Kalesh AP <kalesh-anakkur.purayil@broadcom.com>

The HW only supports tunnel header parsing globally for supported tunnel
types. When a function uses one default VNIC to receive both the tunnel
and non-tunnel packets, applying the same stateless offload operation to
both tunnel and non-tunnel packets can cause problems in certain scenarios.
To workaround these problems, the firmware advertises no tunnel header
parsing capabilities to the driver using the HWRM_FUNC_QCAPS.
The driver must check this flag setting and accordingly not advertise
tunnel packet stateless offload capabilities to the stack.

If the device supports VXLAN, GRE, IPIP and GENEVE tunnel parsing,
then reports RX_OFFLOAD_OUTER_IPV4_CKSUM, RX_OFFLOAD_OUTER_UDP_CKSUM
and TX_OFFLOAD_OUTER_IPV4_CKSUM in the Rx/Tx offload capabilities of
the device.
Also, advertise tunnel TSO capabilities based on FW support.

Fixes: 0a6d2a720078 ("net/bnxt: get device infos")
Cc: stable@dpdk.org

Signed-off-by: Kalesh AP <kalesh-anakkur.purayil@broadcom.com>
Reviewed-by: Ajit Khaparde <ajit.khaparde@broadcom.com>
Reviewed-by: Somnath Kotur <somnath.kotur@broadcom.com>
---
 drivers/net/bnxt/bnxt.h      |  1 +
 drivers/net/bnxt/bnxt_hwrm.c |  5 +++++
 drivers/net/bnxt/bnxt_hwrm.h | 20 ++++++++++++++++++++
 drivers/net/bnxt/bnxt_rxq.c  |  7 ++++---
 drivers/net/bnxt/bnxt_txq.c  | 18 ++++++++++++------
 5 files changed, 42 insertions(+), 9 deletions(-)

diff --git a/drivers/net/bnxt/bnxt.h b/drivers/net/bnxt/bnxt.h
index 9e5ff74..e4e8e8e 100644
--- a/drivers/net/bnxt/bnxt.h
+++ b/drivers/net/bnxt/bnxt.h
@@ -871,6 +871,7 @@ struct bnxt {
 	uint32_t		max_mcast_addr; /* maximum number of mcast filters supported */
 
 	struct rte_eth_rss_conf	rss_conf; /* RSS configuration. */
+	uint16_t		tunnel_disable_flag; /* tunnel stateless offloads status */
 };
 
 static
diff --git a/drivers/net/bnxt/bnxt_hwrm.c b/drivers/net/bnxt/bnxt_hwrm.c
index bff73a9..178a112 100644
--- a/drivers/net/bnxt/bnxt_hwrm.c
+++ b/drivers/net/bnxt/bnxt_hwrm.c
@@ -939,6 +939,11 @@ static int __bnxt_hwrm_func_qcaps(struct bnxt *bp)
 		bp->fw_cap |= BNXT_FW_CAP_VLAN_TX_INSERT;
 		PMD_DRV_LOG(DEBUG, "VLAN acceleration for TX is enabled\n");
 	}
+
+	bp->tunnel_disable_flag = rte_le_to_cpu_16(resp->tunnel_disable_flag);
+	if (bp->tunnel_disable_flag)
+		PMD_DRV_LOG(DEBUG, "Tunnel parsing capability is disabled, flags : %#x\n",
+			    bp->tunnel_disable_flag);
 unlock:
 	HWRM_UNLOCK();
 
diff --git a/drivers/net/bnxt/bnxt_hwrm.h b/drivers/net/bnxt/bnxt_hwrm.h
index 63f8d8c..77f8521 100644
--- a/drivers/net/bnxt/bnxt_hwrm.h
+++ b/drivers/net/bnxt/bnxt_hwrm.h
@@ -121,6 +121,26 @@ struct bnxt_pf_resource_info {
 
 #define BNXT_CTX_VAL_INVAL	0xFFFF
 
+#define BNXT_TUNNELED_OFFLOADS_CAP_VXLAN_EN(bp)		\
+	(!((bp)->tunnel_disable_flag & HWRM_FUNC_QCAPS_OUTPUT_TUNNEL_DISABLE_FLAG_DISABLE_VXLAN))
+#define BNXT_TUNNELED_OFFLOADS_CAP_NGE_EN(bp)		\
+	(!((bp)->tunnel_disable_flag & HWRM_FUNC_QCAPS_OUTPUT_TUNNEL_DISABLE_FLAG_DISABLE_NGE))
+#define BNXT_TUNNELED_OFFLOADS_CAP_GRE_EN(bp)		\
+	(!((bp)->tunnel_disable_flag & HWRM_FUNC_QCAPS_OUTPUT_TUNNEL_DISABLE_FLAG_DISABLE_GRE))
+#define BNXT_TUNNELED_OFFLOADS_CAP_IPINIP_EN(bp)	\
+	(!((bp)->tunnel_disable_flag & HWRM_FUNC_QCAPS_OUTPUT_TUNNEL_DISABLE_FLAG_DISABLE_IPINIP))
+
+/*
+ * If the device supports VXLAN, GRE, IPIP and GENEVE tunnel parsing, then report
+ * RTE_ETH_RX_OFFLOAD_OUTER_IPV4_CKSUM, RTE_ETH_RX_OFFLOAD_OUTER_UDP_CKSUM and
+ * RTE_ETH_TX_OFFLOAD_OUTER_IPV4_CKSUM in the Rx/Tx offload capabilities of the device.
+ */
+#define BNXT_TUNNELED_OFFLOADS_CAP_ALL_EN(bp)			\
+	(BNXT_TUNNELED_OFFLOADS_CAP_VXLAN_EN(bp) &&		\
+	 BNXT_TUNNELED_OFFLOADS_CAP_NGE_EN(bp)   &&		\
+	 BNXT_TUNNELED_OFFLOADS_CAP_GRE_EN(bp)   &&		\
+	 BNXT_TUNNELED_OFFLOADS_CAP_IPINIP_EN(bp))
+
 int bnxt_hwrm_cfa_l2_clear_rx_mask(struct bnxt *bp,
 				   struct bnxt_vnic_info *vnic);
 int bnxt_hwrm_cfa_l2_set_rx_mask(struct bnxt *bp, struct bnxt_vnic_info *vnic,
diff --git a/drivers/net/bnxt/bnxt_rxq.c b/drivers/net/bnxt/bnxt_rxq.c
index 8977138..8147404 100644
--- a/drivers/net/bnxt/bnxt_rxq.c
+++ b/drivers/net/bnxt/bnxt_rxq.c
@@ -34,14 +34,15 @@ uint64_t bnxt_get_rx_port_offloads(struct bnxt *bp)
 			  RTE_ETH_RX_OFFLOAD_SCATTER |
 			  RTE_ETH_RX_OFFLOAD_RSS_HASH;
 
-	rx_offload_capa |= RTE_ETH_RX_OFFLOAD_OUTER_IPV4_CKSUM |
-			   RTE_ETH_RX_OFFLOAD_OUTER_UDP_CKSUM;
-
 	if (bp->flags & BNXT_FLAG_PTP_SUPPORTED)
 		rx_offload_capa |= RTE_ETH_RX_OFFLOAD_TIMESTAMP;
 	if (bp->vnic_cap_flags & BNXT_VNIC_CAP_VLAN_RX_STRIP)
 		rx_offload_capa |= RTE_ETH_RX_OFFLOAD_VLAN_STRIP;
 
+	if (BNXT_TUNNELED_OFFLOADS_CAP_ALL_EN(bp))
+		rx_offload_capa |= RTE_ETH_RX_OFFLOAD_OUTER_IPV4_CKSUM |
+					RTE_ETH_RX_OFFLOAD_OUTER_UDP_CKSUM;
+
 	return rx_offload_capa;
 }
 
diff --git a/drivers/net/bnxt/bnxt_txq.c b/drivers/net/bnxt/bnxt_txq.c
index 86073a5..c8745ad 100644
--- a/drivers/net/bnxt/bnxt_txq.c
+++ b/drivers/net/bnxt/bnxt_txq.c
@@ -28,15 +28,21 @@ uint64_t bnxt_get_tx_port_offloads(struct bnxt *bp)
 			  RTE_ETH_TX_OFFLOAD_QINQ_INSERT |
 			  RTE_ETH_TX_OFFLOAD_MULTI_SEGS;
 
-	tx_offload_capa |= RTE_ETH_TX_OFFLOAD_OUTER_IPV4_CKSUM |
-			   RTE_ETH_TX_OFFLOAD_VXLAN_TNL_TSO |
-			   RTE_ETH_TX_OFFLOAD_GRE_TNL_TSO |
-			   RTE_ETH_TX_OFFLOAD_IPIP_TNL_TSO |
-			   RTE_ETH_TX_OFFLOAD_GENEVE_TNL_TSO;
-
 	if (bp->fw_cap & BNXT_FW_CAP_VLAN_TX_INSERT)
 		tx_offload_capa |= RTE_ETH_TX_OFFLOAD_VLAN_INSERT;
 
+	if (BNXT_TUNNELED_OFFLOADS_CAP_ALL_EN(bp))
+		tx_offload_capa |= RTE_ETH_TX_OFFLOAD_OUTER_IPV4_CKSUM;
+
+	if (BNXT_TUNNELED_OFFLOADS_CAP_VXLAN_EN(bp))
+		tx_offload_capa |= RTE_ETH_TX_OFFLOAD_VXLAN_TNL_TSO;
+	if (BNXT_TUNNELED_OFFLOADS_CAP_GRE_EN(bp))
+		tx_offload_capa |= RTE_ETH_TX_OFFLOAD_GRE_TNL_TSO;
+	if (BNXT_TUNNELED_OFFLOADS_CAP_NGE_EN(bp))
+		tx_offload_capa |= RTE_ETH_TX_OFFLOAD_GENEVE_TNL_TSO;
+	if (BNXT_TUNNELED_OFFLOADS_CAP_IPINIP_EN(bp))
+		tx_offload_capa |= RTE_ETH_TX_OFFLOAD_IPIP_TNL_TSO;
+
 	return tx_offload_capa;
 }
 
-- 
2.10.1


^ permalink raw reply	[flat|nested] 19+ messages in thread

* [PATCH 06/17] net/bnxt: remove support for COUNT action
  2022-04-27 14:58 [dpdk-dev] [PATCH 00/17] bnxt PMD fixes Kalesh A P
                   ` (4 preceding siblings ...)
  2022-04-27 14:58 ` [PATCH 05/17] net/bnxt: fix support for tunnel stateless offloads Kalesh A P
@ 2022-04-27 14:58 ` Kalesh A P
  2022-04-27 14:58 ` [PATCH 07/17] net/bnxt: fix RSS action support Kalesh A P
                   ` (11 subsequent siblings)
  17 siblings, 0 replies; 19+ messages in thread
From: Kalesh A P @ 2022-04-27 14:58 UTC (permalink / raw)
  To: dev; +Cc: ajit.khaparde

From: Somnath Kotur <somnath.kotur@broadcom.com>

'Count' action was never really implemented in the legacy/AFM model.
But there was some place holder code, remove it so that the user
will see a failure when a flow with 'count' action is being
created.

Signed-off-by: Somnath Kotur <somnath.kotur@broadcom.com>
Reviewed-by: Kalesh AP <kalesh-anakkur.purayil@broadcom.com>
---
 drivers/net/bnxt/bnxt_flow.c | 17 -----------------
 1 file changed, 17 deletions(-)

diff --git a/drivers/net/bnxt/bnxt_flow.c b/drivers/net/bnxt/bnxt_flow.c
index f7c90c4..71a8edd 100644
--- a/drivers/net/bnxt/bnxt_flow.c
+++ b/drivers/net/bnxt/bnxt_flow.c
@@ -1405,23 +1405,6 @@ bnxt_validate_and_parse_flow(struct rte_eth_dev *dev,
 
 		bnxt_update_filter_flags_en(filter, filter1, use_ntuple);
 		break;
-	case RTE_FLOW_ACTION_TYPE_COUNT:
-		vnic0 = &bp->vnic_info[0];
-		filter1 = bnxt_get_l2_filter(bp, filter, vnic0);
-		if (filter1 == NULL) {
-			rte_flow_error_set(error,
-					   ENOSPC,
-					   RTE_FLOW_ERROR_TYPE_ACTION,
-					   act,
-					   "New filter not available");
-			rc = -rte_errno;
-			goto ret;
-		}
-
-		filter->fw_l2_filter_id = filter1->fw_l2_filter_id;
-		filter->flow_id = filter1->flow_id;
-		filter->flags = HWRM_CFA_NTUPLE_FILTER_ALLOC_INPUT_FLAGS_METER;
-		break;
 	case RTE_FLOW_ACTION_TYPE_VF:
 		act_vf = (const struct rte_flow_action_vf *)act->conf;
 		vf = act_vf->id;
-- 
2.10.1


^ permalink raw reply	[flat|nested] 19+ messages in thread

* [PATCH 07/17] net/bnxt: fix RSS action support
  2022-04-27 14:58 [dpdk-dev] [PATCH 00/17] bnxt PMD fixes Kalesh A P
                   ` (5 preceding siblings ...)
  2022-04-27 14:58 ` [PATCH 06/17] net/bnxt: remove support for COUNT action Kalesh A P
@ 2022-04-27 14:58 ` Kalesh A P
  2022-04-27 14:58 ` [PATCH 08/17] net/bnxt: fix to reconfigure the VNIC's default receive ring Kalesh A P
                   ` (10 subsequent siblings)
  17 siblings, 0 replies; 19+ messages in thread
From: Kalesh A P @ 2022-04-27 14:58 UTC (permalink / raw)
  To: dev; +Cc: ajit.khaparde

From: Kalesh AP <kalesh-anakkur.purayil@broadcom.com>

Specifying a subset of Rx queues created by the application in
the "flow create" command is invalid.
User must either specify all Rx queues created or no queues.

Also removed a wrong comment as RSS action will not be supported
if user or application specifies MARK or COUNT action.

Fixes: 239695f754cb ("net/bnxt: enhance RSS action support")
Cc: stable@dpdk.org

Signed-off-by: Kalesh AP <kalesh-anakkur.purayil@broadcom.com>
Reviewed-by: Somnath Kotur <somnath.kotur@broadcom.com>
---
 drivers/net/bnxt/bnxt_flow.c | 12 +++++++++++-
 1 file changed, 11 insertions(+), 1 deletion(-)

diff --git a/drivers/net/bnxt/bnxt_flow.c b/drivers/net/bnxt/bnxt_flow.c
index 71a8edd..bd96bba 100644
--- a/drivers/net/bnxt/bnxt_flow.c
+++ b/drivers/net/bnxt/bnxt_flow.c
@@ -1074,7 +1074,6 @@ bnxt_update_filter_flags_en(struct bnxt_filter_info *filter,
 		filter1, filter->fw_l2_filter_id, filter->l2_ref_cnt);
 }
 
-/* Valid actions supported along with RSS are count and mark. */
 static int
 bnxt_validate_rss_action(const struct rte_flow_action actions[])
 {
@@ -1123,6 +1122,17 @@ bnxt_vnic_rss_cfg_update(struct bnxt *bp,
 
 	rss = (const struct rte_flow_action_rss *)act->conf;
 
+	/* must specify either all the Rx queues created by application or zero queues */
+	if (rss->queue_num && vnic->rx_queue_cnt != rss->queue_num) {
+		rte_flow_error_set(error,
+				   EINVAL,
+				   RTE_FLOW_ERROR_TYPE_ACTION,
+				   act,
+				   "Incorrect RXQ count");
+		rc = -rte_errno;
+		goto ret;
+	}
+
 	/* Currently only Toeplitz hash is supported. */
 	if (rss->func != RTE_ETH_HASH_FUNCTION_DEFAULT &&
 	    rss->func != RTE_ETH_HASH_FUNCTION_TOEPLITZ) {
-- 
2.10.1


^ permalink raw reply	[flat|nested] 19+ messages in thread

* [PATCH 08/17] net/bnxt: fix to reconfigure the VNIC's default receive ring
  2022-04-27 14:58 [dpdk-dev] [PATCH 00/17] bnxt PMD fixes Kalesh A P
                   ` (6 preceding siblings ...)
  2022-04-27 14:58 ` [PATCH 07/17] net/bnxt: fix RSS action support Kalesh A P
@ 2022-04-27 14:58 ` Kalesh A P
  2022-04-27 14:58 ` [PATCH 09/17] net/bnxt: add check for dupliate queue ids Kalesh A P
                   ` (9 subsequent siblings)
  17 siblings, 0 replies; 19+ messages in thread
From: Kalesh A P @ 2022-04-27 14:58 UTC (permalink / raw)
  To: dev; +Cc: ajit.khaparde

From: Somnath Kotur <somnath.kotur@broadcom.com>

When an Rx queue is stopped and restarted, as part of that workflow,
for cards that have ring groups, we free and reallocate the ring group.
This new ring group is not communicated to the VNIC though via
HWRM_VNIC_CFG cmd.
Fix to issue HWRM_VNIC_CFG cmd on all adapters now in this scenario.

Fixes: ed0ae3502fc9 ("net/bnxt: update ring group after ring stop start")

Signed-off-by: Somnath Kotur <somnath.kotur@broadcom.com>
Reviewed-by: Ajit Khaparde <ajit.khaparde@broadcom.com>
Reviewed-by: Kalesh AP <kalesh-anakkur.purayil@broadcom.com>
---
 drivers/net/bnxt/bnxt_rxq.c | 9 +++++----
 1 file changed, 5 insertions(+), 4 deletions(-)

diff --git a/drivers/net/bnxt/bnxt_rxq.c b/drivers/net/bnxt/bnxt_rxq.c
index 8147404..9b5ff4c 100644
--- a/drivers/net/bnxt/bnxt_rxq.c
+++ b/drivers/net/bnxt/bnxt_rxq.c
@@ -488,10 +488,11 @@ int bnxt_rx_queue_start(struct rte_eth_dev *dev, uint16_t rx_queue_id)
 	if (rc)
 		return rc;
 
-	if (BNXT_CHIP_P5(bp)) {
-		/* Reconfigure default receive ring and MRU. */
-		bnxt_hwrm_vnic_cfg(bp, rxq->vnic);
-	}
+	if (BNXT_HAS_RING_GRPS(bp))
+		rxq->vnic->dflt_ring_grp = bp->grp_info[rx_queue_id].fw_grp_id;
+	/* Reconfigure default receive ring and MRU. */
+	bnxt_hwrm_vnic_cfg(bp, rxq->vnic);
+
 	PMD_DRV_LOG(INFO, "Rx queue started %d\n", rx_queue_id);
 
 	if (dev_conf->rxmode.mq_mode & RTE_ETH_MQ_RX_RSS_FLAG) {
-- 
2.10.1


^ permalink raw reply	[flat|nested] 19+ messages in thread

* [PATCH 09/17] net/bnxt: add check for dupliate queue ids
  2022-04-27 14:58 [dpdk-dev] [PATCH 00/17] bnxt PMD fixes Kalesh A P
                   ` (7 preceding siblings ...)
  2022-04-27 14:58 ` [PATCH 08/17] net/bnxt: fix to reconfigure the VNIC's default receive ring Kalesh A P
@ 2022-04-27 14:58 ` Kalesh A P
  2022-04-27 14:58 ` [PATCH 10/17] net/bnxt: fix to handle queue stop during RSS flow create Kalesh A P
                   ` (8 subsequent siblings)
  17 siblings, 0 replies; 19+ messages in thread
From: Kalesh A P @ 2022-04-27 14:58 UTC (permalink / raw)
  To: dev; +Cc: ajit.khaparde

From: Kalesh AP <kalesh-anakkur.purayil@broadcom.com>

Currently driver does not have a check for duplicate queue ids.
User must either specify all Rx queues created or no queues in the
flow create command. Repeating the queue index is invalid.

Also, moved the check for invalid queue to the beginning of the function.

Fixes: 239695f754cb ("net/bnxt: enhance RSS action support")
Cc: stable@dpdk.org

Signed-off-by: Kalesh AP <kalesh-anakkur.purayil@broadcom.com>
Reviewed-by: Ajit Khaparde <ajit.khaparde@broadcom.com>
Reviewed-by: Somnath Kotur <somnath.kotur@broadcom.com>
---
 drivers/net/bnxt/bnxt_flow.c | 49 +++++++++++++++++++++++++++++---------------
 1 file changed, 32 insertions(+), 17 deletions(-)

diff --git a/drivers/net/bnxt/bnxt_flow.c b/drivers/net/bnxt/bnxt_flow.c
index bd96bba..fc5bacf 100644
--- a/drivers/net/bnxt/bnxt_flow.c
+++ b/drivers/net/bnxt/bnxt_flow.c
@@ -1115,7 +1115,7 @@ bnxt_vnic_rss_cfg_update(struct bnxt *bp,
 			 struct rte_flow_error *error)
 {
 	const struct rte_flow_action_rss *rss;
-	unsigned int rss_idx, i;
+	unsigned int rss_idx, i, j;
 	uint16_t hash_type;
 	uint64_t types;
 	int rc;
@@ -1133,6 +1133,37 @@ bnxt_vnic_rss_cfg_update(struct bnxt *bp,
 		goto ret;
 	}
 
+	/* Validate Rx queues */
+	for (i = 0; i < rss->queue_num; i++) {
+		PMD_DRV_LOG(DEBUG, "RSS action Queue %d\n", rss->queue[i]);
+
+		if (rss->queue[i] >= bp->rx_nr_rings ||
+		    !bp->rx_queues[rss->queue[i]]) {
+			rte_flow_error_set(error,
+					   EINVAL,
+					   RTE_FLOW_ERROR_TYPE_ACTION,
+					   act,
+					   "Invalid queue ID for RSS");
+			rc = -rte_errno;
+			goto ret;
+		}
+	}
+
+	/* Duplicate queue ids are not supported. */
+	for (i = 0; i < rss->queue_num; i++) {
+		for (j = i + 1; j < rss->queue_num; j++) {
+			if (rss->queue[i] == rss->queue[j]) {
+				rte_flow_error_set(error,
+						   EINVAL,
+						   RTE_FLOW_ERROR_TYPE_ACTION,
+						   act,
+						   "Duplicate queue ID for RSS");
+				rc = -rte_errno;
+				goto ret;
+			}
+		}
+	}
+
 	/* Currently only Toeplitz hash is supported. */
 	if (rss->func != RTE_ETH_HASH_FUNCTION_DEFAULT &&
 	    rss->func != RTE_ETH_HASH_FUNCTION_TOEPLITZ) {
@@ -1200,22 +1231,6 @@ bnxt_vnic_rss_cfg_update(struct bnxt *bp,
 	if (rss->queue_num == 0)
 		goto skip_rss_table;
 
-	/* Validate Rx queues */
-	for (i = 0; i < rss->queue_num; i++) {
-		PMD_DRV_LOG(DEBUG, "RSS action Queue %d\n", rss->queue[i]);
-
-		if (rss->queue[i] >= bp->rx_nr_rings ||
-		    !bp->rx_queues[rss->queue[i]]) {
-			rte_flow_error_set(error,
-					   EINVAL,
-					   RTE_FLOW_ERROR_TYPE_ACTION,
-					   act,
-					   "Invalid queue ID for RSS");
-			rc = -rte_errno;
-			goto ret;
-		}
-	}
-
 	/* Prepare the indirection table */
 	for (rss_idx = 0; rss_idx < HW_HASH_INDEX_SIZE; rss_idx++) {
 		struct bnxt_rx_queue *rxq;
-- 
2.10.1


^ permalink raw reply	[flat|nested] 19+ messages in thread

* [PATCH 10/17] net/bnxt: fix to handle queue stop during RSS flow create
  2022-04-27 14:58 [dpdk-dev] [PATCH 00/17] bnxt PMD fixes Kalesh A P
                   ` (8 preceding siblings ...)
  2022-04-27 14:58 ` [PATCH 09/17] net/bnxt: add check for dupliate queue ids Kalesh A P
@ 2022-04-27 14:58 ` Kalesh A P
  2022-04-27 14:58 ` [PATCH 11/17] net/bnxt: avoid unnecessary endianness conversion Kalesh A P
                   ` (7 subsequent siblings)
  17 siblings, 0 replies; 19+ messages in thread
From: Kalesh A P @ 2022-04-27 14:58 UTC (permalink / raw)
  To: dev; +Cc: ajit.khaparde

From: Somnath Kotur <somnath.kotur@broadcom.com>

The programming of the RSS table was not taking into account if
any of the queues in the set were stopped prior to the flow
creation, hence leading to a vnic RSS config cmd failure thrown by
the FW.
Fix by programming only the active queues in the RSS action queue
set.

Fixes: 239695f754cb ("net/bnxt: enhance RSS action support")
Cc: stable@dpdk.org

Signed-off-by: Somnath Kotur <somnath.kotur@broadcom.com>
Reviewed-by: Ajit Khaparde <ajit.khaparde@broadcom.com>
Reviewed-by: Kalesh AP <kalesh-anakkur.purayil@broadcom.com>
---
 drivers/net/bnxt/bnxt_flow.c | 16 +++++++++++++---
 1 file changed, 13 insertions(+), 3 deletions(-)

diff --git a/drivers/net/bnxt/bnxt_flow.c b/drivers/net/bnxt/bnxt_flow.c
index fc5bacf..f8e1096 100644
--- a/drivers/net/bnxt/bnxt_flow.c
+++ b/drivers/net/bnxt/bnxt_flow.c
@@ -1115,7 +1115,7 @@ bnxt_vnic_rss_cfg_update(struct bnxt *bp,
 			 struct rte_flow_error *error)
 {
 	const struct rte_flow_action_rss *rss;
-	unsigned int rss_idx, i, j;
+	unsigned int rss_idx, i, j, fw_idx;
 	uint16_t hash_type;
 	uint64_t types;
 	int rc;
@@ -1232,11 +1232,21 @@ bnxt_vnic_rss_cfg_update(struct bnxt *bp,
 		goto skip_rss_table;
 
 	/* Prepare the indirection table */
-	for (rss_idx = 0; rss_idx < HW_HASH_INDEX_SIZE; rss_idx++) {
+	for (rss_idx = 0, fw_idx = 0; rss_idx < HW_HASH_INDEX_SIZE;
+	     rss_idx++, fw_idx++) {
+		uint8_t *rxq_state = bp->eth_dev->data->rx_queue_state;
 		struct bnxt_rx_queue *rxq;
 		uint32_t idx;
 
-		idx = rss->queue[rss_idx % rss->queue_num];
+		for (i = 0; i < bp->rx_cp_nr_rings; i++) {
+			idx = rss->queue[fw_idx % rss->queue_num];
+			if (rxq_state[idx] != RTE_ETH_QUEUE_STATE_STOPPED)
+				break;
+			fw_idx++;
+		}
+
+		if (i == bp->rx_cp_nr_rings)
+			return 0;
 
 		if (BNXT_CHIP_P5(bp)) {
 			rxq = bp->rx_queues[idx];
-- 
2.10.1


^ permalink raw reply	[flat|nested] 19+ messages in thread

* [PATCH 11/17] net/bnxt: avoid unnecessary endianness conversion
  2022-04-27 14:58 [dpdk-dev] [PATCH 00/17] bnxt PMD fixes Kalesh A P
                   ` (9 preceding siblings ...)
  2022-04-27 14:58 ` [PATCH 10/17] net/bnxt: fix to handle queue stop during RSS flow create Kalesh A P
@ 2022-04-27 14:58 ` Kalesh A P
  2022-04-27 14:58 ` [PATCH 12/17] net/bnxt: fix setting autoneg speed Kalesh A P
                   ` (6 subsequent siblings)
  17 siblings, 0 replies; 19+ messages in thread
From: Kalesh A P @ 2022-04-27 14:58 UTC (permalink / raw)
  To: dev; +Cc: ajit.khaparde

From: Kalesh AP <kalesh-anakkur.purayil@broadcom.com>

The "active_fec_signal_mode" in HWRM_PORT_PHY_QCFG response is uint8_t.
So no need of endianness conversion while parsing response.
Also, signal_mode is the first 4bits of "active_fec_signal_mode".

Fixes: c23f9ded0391 ("net/bnxt: support 200G PAM4 link")
Cc: stable@dpdk.org

Signed-off-by: Kalesh AP <kalesh-anakkur.purayil@broadcom.com>
Reviewed-by: Ajit Khaparde <ajit.khaparde@broadcom.com>
Reviewed-by: Somnath Kotur <somnath.kotur@broadcom.com>
---
 drivers/net/bnxt/bnxt_hwrm.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/drivers/net/bnxt/bnxt_hwrm.c b/drivers/net/bnxt/bnxt_hwrm.c
index 178a112..d87f0c3 100644
--- a/drivers/net/bnxt/bnxt_hwrm.c
+++ b/drivers/net/bnxt/bnxt_hwrm.c
@@ -1506,7 +1506,7 @@ static int bnxt_hwrm_port_phy_qcfg(struct bnxt *bp,
 	link_info->phy_ver[1] = resp->phy_min;
 	link_info->phy_ver[2] = resp->phy_bld;
 	link_info->link_signal_mode =
-		rte_le_to_cpu_16(resp->active_fec_signal_mode);
+		resp->active_fec_signal_mode & HWRM_PORT_PHY_QCFG_OUTPUT_SIGNAL_MODE_MASK;
 	link_info->force_pam4_link_speed =
 			rte_le_to_cpu_16(resp->force_pam4_link_speed);
 	link_info->support_pam4_speeds =
-- 
2.10.1


^ permalink raw reply	[flat|nested] 19+ messages in thread

* [PATCH 12/17] net/bnxt: fix setting autoneg speed
  2022-04-27 14:58 [dpdk-dev] [PATCH 00/17] bnxt PMD fixes Kalesh A P
                   ` (10 preceding siblings ...)
  2022-04-27 14:58 ` [PATCH 11/17] net/bnxt: avoid unnecessary endianness conversion Kalesh A P
@ 2022-04-27 14:58 ` Kalesh A P
  2022-04-27 14:58 ` [PATCH 13/17] net/bnxt: force PHY update on certain configurations Kalesh A P
                   ` (5 subsequent siblings)
  17 siblings, 0 replies; 19+ messages in thread
From: Kalesh A P @ 2022-04-27 14:58 UTC (permalink / raw)
  To: dev; +Cc: ajit.khaparde

From: Kalesh AP <kalesh-anakkur.purayil@broadcom.com>

The "active_fec_signal_mode" in HWRM_PORT_PHY_QCFG response
does not return correct value till the link is up. Driver cannot
rely on active_fec_signal_mode while setting autoneg speed.

While setting autoneg speed, driver is currently checking only
"auto_link_speed_mask". Fixed to check "auto_pam4_link_speed_mask"
as well. Also, while setting auto mode and setting speed mask,
driver will have to set both NRZ and PAM4 mask.

Fixes: c23f9ded0391 ("net/bnxt: support 200G PAM4 link")
Cc: stable@dpdk.org

Signed-off-by: Kalesh AP <kalesh-anakkur.purayil@broadcom.com>
Reviewed-by: Ajit Khaparde <ajit.khaparde@broadcom.com>
Reviewed-by: Somnath Kotur <somnath.kotur@broadcom.com>
---
 drivers/net/bnxt/bnxt_hwrm.c | 8 ++++----
 1 file changed, 4 insertions(+), 4 deletions(-)

diff --git a/drivers/net/bnxt/bnxt_hwrm.c b/drivers/net/bnxt/bnxt_hwrm.c
index d87f0c3..9eb8b8d 100644
--- a/drivers/net/bnxt/bnxt_hwrm.c
+++ b/drivers/net/bnxt/bnxt_hwrm.c
@@ -1424,17 +1424,17 @@ static int bnxt_hwrm_port_phy_cfg(struct bnxt *bp, struct bnxt_link_info *conf)
 			}
 		}
 		/* AutoNeg - Advertise speeds specified. */
-		if (conf->auto_link_speed_mask &&
+		if ((conf->auto_link_speed_mask || conf->auto_pam4_link_speed_mask) &&
 		    !(conf->phy_flags & HWRM_PORT_PHY_CFG_INPUT_FLAGS_FORCE)) {
 			req.auto_mode =
 				HWRM_PORT_PHY_CFG_INPUT_AUTO_MODE_SPEED_MASK;
-			if (conf->auto_pam4_link_speed_mask &&
-			    bp->link_info->link_signal_mode) {
+			if (conf->auto_pam4_link_speed_mask) {
 				enables |=
 				HWRM_PORT_PHY_CFG_IN_EN_AUTO_PAM4_LINK_SPD_MASK;
 				req.auto_link_pam4_speed_mask =
 				rte_cpu_to_le_16(conf->auto_pam4_link_speed_mask);
-			} else {
+			}
+			if (conf->auto_link_speed_mask) {
 				enables |=
 				HWRM_PORT_PHY_CFG_IN_EN_AUTO_LINK_SPEED_MASK;
 				req.auto_link_speed_mask =
-- 
2.10.1


^ permalink raw reply	[flat|nested] 19+ messages in thread

* [PATCH 13/17] net/bnxt: force PHY update on certain configurations
  2022-04-27 14:58 [dpdk-dev] [PATCH 00/17] bnxt PMD fixes Kalesh A P
                   ` (11 preceding siblings ...)
  2022-04-27 14:58 ` [PATCH 12/17] net/bnxt: fix setting autoneg speed Kalesh A P
@ 2022-04-27 14:58 ` Kalesh A P
  2022-04-27 14:58 ` [PATCH 14/17] net/bnxt: fix reporting link status when port is stopped Kalesh A P
                   ` (4 subsequent siblings)
  17 siblings, 0 replies; 19+ messages in thread
From: Kalesh A P @ 2022-04-27 14:58 UTC (permalink / raw)
  To: dev; +Cc: ajit.khaparde

From: Kalesh AP <kalesh-anakkur.purayil@broadcom.com>

Device is not obliged link down in certain scenarios, even
when forced. When FW does not allow any user other than the BMC
to shutdown the port, bnxt_get_hwrm_link_config() call always
returns link up. Force phy update always in that case,
else user configuration for speed/autoneg would not get applied
correctly.

Fixes: 7bc8e9a227cc ("net/bnxt: support async link notification")
Cc: stable@dpdk.org

Signed-off-by: Kalesh AP <kalesh-anakkur.purayil@broadcom.com>
Reviewed-by: Ajit Khaparde <ajit.khaparde@broadcom.com>
Reviewed-by: Somnath Kotur <somnath.kotur@broadcom.com>
---
 drivers/net/bnxt/bnxt.h        |  3 +--
 drivers/net/bnxt/bnxt_ethdev.c | 22 ++++++++++++++++++----
 2 files changed, 19 insertions(+), 6 deletions(-)

diff --git a/drivers/net/bnxt/bnxt.h b/drivers/net/bnxt/bnxt.h
index e4e8e8e..e86e51e 100644
--- a/drivers/net/bnxt/bnxt.h
+++ b/drivers/net/bnxt/bnxt.h
@@ -72,8 +72,7 @@
 #define BROADCOM_DEV_ID_58818_VF	0xd82e
 
 #define BROADCOM_DEV_957508_N2100	0x5208
-#define IS_BNXT_DEV_957508_N2100(bp)	\
-	((bp)->pdev->id.subsystem_device_id == BROADCOM_DEV_957508_N2100)
+#define BROADCOM_DEV_957414_N225	0x4145
 
 #define BNXT_MAX_MTU		9574
 #define BNXT_NUM_VLANS		2
diff --git a/drivers/net/bnxt/bnxt_ethdev.c b/drivers/net/bnxt/bnxt_ethdev.c
index 181de42..1904db9 100644
--- a/drivers/net/bnxt/bnxt_ethdev.c
+++ b/drivers/net/bnxt/bnxt_ethdev.c
@@ -659,6 +659,19 @@ static int bnxt_init_ctx_mem(struct bnxt *bp)
 	return rc;
 }
 
+static inline bool bnxt_force_link_config(struct bnxt *bp)
+{
+	uint16_t subsystem_device_id = bp->pdev->id.subsystem_device_id;
+
+	switch (subsystem_device_id) {
+	case BROADCOM_DEV_957508_N2100:
+	case BROADCOM_DEV_957414_N225:
+		return true;
+	default:
+		return false;
+	}
+}
+
 static int bnxt_update_phy_setting(struct bnxt *bp)
 {
 	struct rte_eth_link new;
@@ -671,11 +684,12 @@ static int bnxt_update_phy_setting(struct bnxt *bp)
 	}
 
 	/*
-	 * On BCM957508-N2100 adapters, FW will not allow any user other
-	 * than BMC to shutdown the port. bnxt_get_hwrm_link_config() call
-	 * always returns link up. Force phy update always in that case.
+	 * Device is not obliged link down in certain scenarios, even
+	 * when forced. When FW does not allow any user other than BMC
+	 * to shutdown the port, bnxt_get_hwrm_link_config() call always
+	 * returns link up. Force phy update always in that case.
 	 */
-	if (!new.link_status || IS_BNXT_DEV_957508_N2100(bp)) {
+	if (!new.link_status || bnxt_force_link_config(bp)) {
 		rc = bnxt_set_hwrm_link_config(bp, true);
 		if (rc) {
 			PMD_DRV_LOG(ERR, "Failed to update PHY settings\n");
-- 
2.10.1


^ permalink raw reply	[flat|nested] 19+ messages in thread

* [PATCH 14/17] net/bnxt: fix reporting link status when port is stopped
  2022-04-27 14:58 [dpdk-dev] [PATCH 00/17] bnxt PMD fixes Kalesh A P
                   ` (12 preceding siblings ...)
  2022-04-27 14:58 ` [PATCH 13/17] net/bnxt: force PHY update on certain configurations Kalesh A P
@ 2022-04-27 14:58 ` Kalesh A P
  2022-04-27 14:58 ` [PATCH 15/17] net/bnxt: recheck FW readiness if FW is in reset process Kalesh A P
                   ` (3 subsequent siblings)
  17 siblings, 0 replies; 19+ messages in thread
From: Kalesh A P @ 2022-04-27 14:58 UTC (permalink / raw)
  To: dev; +Cc: ajit.khaparde

From: Kalesh AP <kalesh-anakkur.purayil@broadcom.com>

Driver forces link down during port stop. But device is not obliged
link down in certain scenarios, even when forced. In that case,
subsequent link queries returns link as up.
Fixed to return link status as down when port is stopped.
Driver is already doing that for VF/NPAR/MH functions.

Fixes: c09f57b49c13 ("net/bnxt: add start/stop/link update operations")
Cc: stable@dpdk.org

Signed-off-by: Kalesh AP <kalesh-anakkur.purayil@broadcom.com>
Reviewed-by: Ajit Khaparde <ajit.khaparde@broadcom.com>
Reviewed-by: Somnath Kotur <somnath.kotur@broadcom.com>
---
 drivers/net/bnxt/bnxt_ethdev.c | 14 ++++++++------
 1 file changed, 8 insertions(+), 6 deletions(-)

diff --git a/drivers/net/bnxt/bnxt_ethdev.c b/drivers/net/bnxt/bnxt_ethdev.c
index 1904db9..69f1117 100644
--- a/drivers/net/bnxt/bnxt_ethdev.c
+++ b/drivers/net/bnxt/bnxt_ethdev.c
@@ -1826,6 +1826,14 @@ int bnxt_link_update_op(struct rte_eth_dev *eth_dev, int wait_to_complete)
 	if (bp->link_info == NULL)
 		goto out;
 
+	/* Only single function PF can bring the phy down.
+	 * In certain scenarios, device is not obliged link down even when forced.
+	 * When port is stopped, report link down in those cases.
+	 */
+	if (!eth_dev->data->dev_started &&
+	    (!BNXT_SINGLE_PF(bp) || bnxt_force_link_config(bp)))
+		goto out;
+
 	do {
 		/* Retrieve link info from hardware */
 		rc = bnxt_get_hwrm_link_config(bp, &new);
@@ -1843,12 +1851,6 @@ int bnxt_link_update_op(struct rte_eth_dev *eth_dev, int wait_to_complete)
 		rte_delay_ms(BNXT_LINK_WAIT_INTERVAL);
 	} while (cnt--);
 
-	/* Only single function PF can bring phy down.
-	 * When port is stopped, report link down for VF/MH/NPAR functions.
-	 */
-	if (!BNXT_SINGLE_PF(bp) && !eth_dev->data->dev_started)
-		memset(&new, 0, sizeof(new));
-
 out:
 	/* Timed out or success */
 	if (new.link_status != eth_dev->data->dev_link.link_status ||
-- 
2.10.1


^ permalink raw reply	[flat|nested] 19+ messages in thread

* [PATCH 15/17] net/bnxt: recheck FW readiness if FW is in reset process
  2022-04-27 14:58 [dpdk-dev] [PATCH 00/17] bnxt PMD fixes Kalesh A P
                   ` (13 preceding siblings ...)
  2022-04-27 14:58 ` [PATCH 14/17] net/bnxt: fix reporting link status when port is stopped Kalesh A P
@ 2022-04-27 14:58 ` Kalesh A P
  2022-04-27 14:58 ` [PATCH 16/17] net/bnxt: fix freeing of VNIC filters Kalesh A P
                   ` (2 subsequent siblings)
  17 siblings, 0 replies; 19+ messages in thread
From: Kalesh A P @ 2022-04-27 14:58 UTC (permalink / raw)
  To: dev; +Cc: ajit.khaparde

From: Kalesh AP <kalesh-anakkur.purayil@broadcom.com>

If Firmware is still in reset process and returns the error
HWRM_ERR_CODE_HOT_RESET_PROGRESS, retry VER_GET command.
We have to do it in bnxt_handle_if_change_status().

Fixes: 0b533591238f ("net/bnxt: inform firmware about IF state changes")
Cc: stable@dpdk.org

Signed-off-by: Kalesh AP <kalesh-anakkur.purayil@broadcom.com>
Reviewed-by: Somnath Kotur <somnath.kotur@broadcom.com>
Reviewed-by: Ajit Khaparde <ajit.khaparde@broadcom.com>
---
 drivers/net/bnxt/bnxt_ethdev.c | 8 +++++++-
 1 file changed, 7 insertions(+), 1 deletion(-)

diff --git a/drivers/net/bnxt/bnxt_ethdev.c b/drivers/net/bnxt/bnxt_ethdev.c
index 69f1117..abcb534 100644
--- a/drivers/net/bnxt/bnxt_ethdev.c
+++ b/drivers/net/bnxt/bnxt_ethdev.c
@@ -177,6 +177,7 @@ static int bnxt_restore_vlan_filters(struct bnxt *bp);
 static void bnxt_dev_recover(void *arg);
 static void bnxt_free_error_recovery_info(struct bnxt *bp);
 static void bnxt_free_rep_info(struct bnxt *bp);
+static int bnxt_check_fw_ready(struct bnxt *bp);
 
 int is_bnxt_in_error(struct bnxt *bp)
 {
@@ -1350,6 +1351,11 @@ static int bnxt_handle_if_change_status(struct bnxt *bp)
 
 	/* clear fatal flag so that re-init happens */
 	bp->flags &= ~BNXT_FLAG_FATAL_ERROR;
+
+	rc = bnxt_check_fw_ready(bp);
+	if (rc)
+		return rc;
+
 	rc = bnxt_init_resources(bp, true);
 
 	bp->flags &= ~BNXT_FLAG_IF_CHANGE_HOT_FW_RESET_DONE;
@@ -4324,7 +4330,7 @@ static int bnxt_restore_filters(struct bnxt *bp)
 
 static int bnxt_check_fw_ready(struct bnxt *bp)
 {
-	int timeout = bp->fw_reset_max_msecs;
+	int timeout = bp->fw_reset_max_msecs ? : BNXT_MAX_FW_RESET_TIMEOUT;
 	int rc = 0;
 
 	do {
-- 
2.10.1


^ permalink raw reply	[flat|nested] 19+ messages in thread

* [PATCH 16/17] net/bnxt: fix freeing of VNIC filters
  2022-04-27 14:58 [dpdk-dev] [PATCH 00/17] bnxt PMD fixes Kalesh A P
                   ` (14 preceding siblings ...)
  2022-04-27 14:58 ` [PATCH 15/17] net/bnxt: recheck FW readiness if FW is in reset process Kalesh A P
@ 2022-04-27 14:58 ` Kalesh A P
  2022-04-27 14:58 ` [PATCH 17/17] net/bnxt: don't wait for link up completion in dev start Kalesh A P
  2022-05-10  5:16 ` [dpdk-dev] [PATCH 00/17] bnxt PMD fixes Ajit Khaparde
  17 siblings, 0 replies; 19+ messages in thread
From: Kalesh A P @ 2022-04-27 14:58 UTC (permalink / raw)
  To: dev; +Cc: ajit.khaparde

From: Somnath Kotur <somnath.kotur@broadcom.com>

In bnxt_free_all_filters(), all the filters attached to a vnic are removed.
But each of these filters hold a backreference ptr to the vnic and they
need to be reset to NULL now. Otherwise, during a normal testpmd quit, as
part of dev_close_op(), first bnxt_free_all_filters() is invoked in
dev_stop, followed by bnxt_free_filter_mem() from bnxt_uninit_resources(),
which finds a filter with a vnic back reference ptr and now
bnxt_hwrm_clean_up_l2_filter() also tries to remove the filter from the
vnic's filter list which was already done as part of
bnxt_free_all_filters().

Fixes: f0f6b5e6cf9("net/bnxt: fix reusing L2 filter")
Cc: stable@dpdk.org

Signed-off-by: Somnath Kotur <somnath.kotur@broadcom.com>
Reviewed-by: Kalesh AP <kalesh-anakkur.purayil@broadcom.com>
---
 drivers/net/bnxt/bnxt_filter.c | 2 ++
 1 file changed, 2 insertions(+)

diff --git a/drivers/net/bnxt/bnxt_filter.c b/drivers/net/bnxt/bnxt_filter.c
index 1d08e03..b0c3bbd 100644
--- a/drivers/net/bnxt/bnxt_filter.c
+++ b/drivers/net/bnxt/bnxt_filter.c
@@ -99,6 +99,8 @@ void bnxt_free_all_filters(struct bnxt *bp)
 					bnxt_filter_info, next);
 			STAILQ_INSERT_TAIL(&bp->free_filter_list,
 					filter, next);
+			if (filter->vnic)
+				filter->vnic = NULL;
 			filter = temp_filter;
 		}
 		STAILQ_INIT(&vnic->filter);
-- 
2.10.1


^ permalink raw reply	[flat|nested] 19+ messages in thread

* [PATCH 17/17] net/bnxt: don't wait for link up completion in dev start
  2022-04-27 14:58 [dpdk-dev] [PATCH 00/17] bnxt PMD fixes Kalesh A P
                   ` (15 preceding siblings ...)
  2022-04-27 14:58 ` [PATCH 16/17] net/bnxt: fix freeing of VNIC filters Kalesh A P
@ 2022-04-27 14:58 ` Kalesh A P
  2022-05-10  5:16 ` [dpdk-dev] [PATCH 00/17] bnxt PMD fixes Ajit Khaparde
  17 siblings, 0 replies; 19+ messages in thread
From: Kalesh A P @ 2022-04-27 14:58 UTC (permalink / raw)
  To: dev; +Cc: ajit.khaparde

From: Somnath Kotur <somnath.kotur@broadcom.com>

Invoking bnxt_link_update_op() with wait_for_completion set would
result in the driver waiting for 10s in case the port link is down to
complete port initialization (dev_start_op()).
Change it by not waiting for the completion when invoking it in
dev_start_op()

Signed-off-by: Somnath Kotur <somnath.kotur@broadcom.com>
Reviewed-by: Kalesh AP <kalesh-anakkur.purayil@broadcom.com>
---
 drivers/net/bnxt/bnxt_ethdev.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/drivers/net/bnxt/bnxt_ethdev.c b/drivers/net/bnxt/bnxt_ethdev.c
index abcb534..0f0f40b 100644
--- a/drivers/net/bnxt/bnxt_ethdev.c
+++ b/drivers/net/bnxt/bnxt_ethdev.c
@@ -1608,7 +1608,7 @@ int bnxt_dev_start_op(struct rte_eth_dev *eth_dev)
 
 	eth_dev->data->dev_started = 1;
 
-	bnxt_link_update_op(eth_dev, 1);
+	bnxt_link_update_op(eth_dev, 0);
 
 	if (rx_offloads & RTE_ETH_RX_OFFLOAD_VLAN_FILTER)
 		vlan_mask |= RTE_ETH_VLAN_FILTER_MASK;
-- 
2.10.1


^ permalink raw reply	[flat|nested] 19+ messages in thread

* Re: [dpdk-dev] [PATCH 00/17] bnxt PMD fixes
  2022-04-27 14:58 [dpdk-dev] [PATCH 00/17] bnxt PMD fixes Kalesh A P
                   ` (16 preceding siblings ...)
  2022-04-27 14:58 ` [PATCH 17/17] net/bnxt: don't wait for link up completion in dev start Kalesh A P
@ 2022-05-10  5:16 ` Ajit Khaparde
  17 siblings, 0 replies; 19+ messages in thread
From: Ajit Khaparde @ 2022-05-10  5:16 UTC (permalink / raw)
  To: Kalesh A P; +Cc: dpdk-dev

[-- Attachment #1: Type: text/plain, Size: 1880 bytes --]

On Wed, Apr 27, 2022 at 7:58 AM Kalesh A P
<kalesh-anakkur.purayil@broadcom.com> wrote:
>
> From: Kalesh AP <kalesh-anakkur.purayil@broadcom.com>
>
> This patch set contains bug fixes in bnxt PMD. Please apply.
>
> Kalesh AP (12):
>   net/bnxt: update HWRM structures
>   net/bnxt: fix device capability reporting
>   net/bnxt: fix to remove an unused macro
>   net/bnxt: fix Rxq configure
>   net/bnxt: fix support for tunnel stateless offloads
>   net/bnxt: fix RSS action support
>   net/bnxt: add check for dupliate queue ids
>   net/bnxt: avoid unnecessary endianness conversion
>   net/bnxt: fix setting autoneg speed
>   net/bnxt: force PHY update on certain configurations
>   net/bnxt: fix reporting link status when port is stopped
>   net/bnxt: recheck FW readiness if FW is in reset process
>
> Somnath Kotur (5):
>   net/bnxt: remove support for COUNT action
>   net/bnxt: fix to reconfigure the VNIC's default receive ring
>   net/bnxt: fix to handle queue stop during RSS flow create
>   net/bnxt: fix freeing of VNIC filters
>   net/bnxt: don't wait for link up completion in dev start


Patches applied to dpdk-next-net-brcm. Thanks

>
>
>  drivers/net/bnxt/bnxt.h                |   29 +-
>  drivers/net/bnxt/bnxt_ethdev.c         |   58 +-
>  drivers/net/bnxt/bnxt_filter.c         |    2 +
>  drivers/net/bnxt/bnxt_flow.c           |   92 +-
>  drivers/net/bnxt/bnxt_hwrm.c           |   15 +-
>  drivers/net/bnxt/bnxt_hwrm.h           |   20 +
>  drivers/net/bnxt/bnxt_reps.c           |    6 +-
>  drivers/net/bnxt/bnxt_rxq.c            |   75 +-
>  drivers/net/bnxt/bnxt_rxq.h            |    1 +
>  drivers/net/bnxt/bnxt_txq.c            |   29 +
>  drivers/net/bnxt/bnxt_txq.h            |    1 +
>  drivers/net/bnxt/hsi_struct_def_dpdk.h | 4025 ++++++++++++++++++++++++++++----
>  12 files changed, 3809 insertions(+), 544 deletions(-)
>
> --
> 2.10.1
>

[-- Attachment #2: S/MIME Cryptographic Signature --]
[-- Type: application/pkcs7-signature, Size: 4218 bytes --]

^ permalink raw reply	[flat|nested] 19+ messages in thread

end of thread, other threads:[~2022-05-10  5:16 UTC | newest]

Thread overview: 19+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2022-04-27 14:58 [dpdk-dev] [PATCH 00/17] bnxt PMD fixes Kalesh A P
2022-04-27 14:58 ` [PATCH 01/17] net/bnxt: update HWRM structures Kalesh A P
2022-04-27 14:58 ` [PATCH 02/17] net/bnxt: fix device capability reporting Kalesh A P
2022-04-27 14:58 ` [PATCH 03/17] net/bnxt: fix to remove an unused macro Kalesh A P
2022-04-27 14:58 ` [PATCH 04/17] net/bnxt: fix Rxq configure Kalesh A P
2022-04-27 14:58 ` [PATCH 05/17] net/bnxt: fix support for tunnel stateless offloads Kalesh A P
2022-04-27 14:58 ` [PATCH 06/17] net/bnxt: remove support for COUNT action Kalesh A P
2022-04-27 14:58 ` [PATCH 07/17] net/bnxt: fix RSS action support Kalesh A P
2022-04-27 14:58 ` [PATCH 08/17] net/bnxt: fix to reconfigure the VNIC's default receive ring Kalesh A P
2022-04-27 14:58 ` [PATCH 09/17] net/bnxt: add check for dupliate queue ids Kalesh A P
2022-04-27 14:58 ` [PATCH 10/17] net/bnxt: fix to handle queue stop during RSS flow create Kalesh A P
2022-04-27 14:58 ` [PATCH 11/17] net/bnxt: avoid unnecessary endianness conversion Kalesh A P
2022-04-27 14:58 ` [PATCH 12/17] net/bnxt: fix setting autoneg speed Kalesh A P
2022-04-27 14:58 ` [PATCH 13/17] net/bnxt: force PHY update on certain configurations Kalesh A P
2022-04-27 14:58 ` [PATCH 14/17] net/bnxt: fix reporting link status when port is stopped Kalesh A P
2022-04-27 14:58 ` [PATCH 15/17] net/bnxt: recheck FW readiness if FW is in reset process Kalesh A P
2022-04-27 14:58 ` [PATCH 16/17] net/bnxt: fix freeing of VNIC filters Kalesh A P
2022-04-27 14:58 ` [PATCH 17/17] net/bnxt: don't wait for link up completion in dev start Kalesh A P
2022-05-10  5:16 ` [dpdk-dev] [PATCH 00/17] bnxt PMD fixes Ajit Khaparde

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).