DPDK patches and discussions
 help / color / mirror / Atom feed
* [dpdk-dev] [PATCH 00/50] add features for host-based flow management
@ 2020-06-12 13:28 Somnath Kotur
  2020-06-12 13:28 ` [dpdk-dev] [PATCH 01/50] net/bnxt: Basic infrastructure support for VF representors Somnath Kotur
                   ` (50 more replies)
  0 siblings, 51 replies; 271+ messages in thread
From: Somnath Kotur @ 2020-06-12 13:28 UTC (permalink / raw)
  To: dev; +Cc: ferruh.yigit

This patchset introduces support for VF representors, flow
counters and on-chip exact match flows.
Also implements the driver hook for the rte_flow_query API.

Jay Ding (5):
  net/bnxt: implement support for TCAM access
  net/bnxt: support two level priority for TCAMs
  net/bnxt: add external action alloc and free
  net/bnxt: implement IF tables set and get
  net/bnxt: add global config set and get APIs

Kishore Padmanabha (8):
  net/bnxt: integrate with the latest tf_core library
  net/bnxt: add support for if table processing
  net/bnxt: disable vector mode in tx direction when truflow is enabled
  net/bnxt: add index opcode and index operand mapper table
  net/bnxt: add support for global resource templates
  net/bnxt: add support for internal exact match entries
  net/bnxt: add support for conditional execution of mapper tables
  net/bnxt: add support for vf rep and stat templates

Lance Richardson (1):
  net/bnxt: initialize parent PF information

Michael Wildt (7):
  net/bnxt: add multi device support
  net/bnxt: update multi device design support
  net/bnxt: multiple device implementation
  net/bnxt: update identifier with remap support
  net/bnxt: update RM with residual checker
  net/bnxt: update table get to use new design
  net/bnxt: add TF register and unregister

Mike Baucom (1):
  net/bnxt: add support for internal encap records

Pete Spreadborough (6):
  net/bnxt: add support for Exact Match
  net/bnxt: modify EM insert and delete to use HWRM direct
  net/bnxt: add HCAPI interface support
  net/bnxt: support EM and TCAM lookup with table scope
  net/bnxt: update RM to support HCAPI only
  net/bnxt: remove table scope from session

Peter Spreadborough (1):
  net/bnxt: add support for EEM System memory

Randy Schacher (2):
  net/bnxt: add core changes for EM and EEM lookups
  net/bnxt: align CFA resources with RM

Shahaji Bhosle (2):
  net/bnxt: support bulk table get and mirror
  net/bnxt: support two-level priority for TCAMs

Somnath Kotur (7):
  net/bnxt: Basic infrastructure support for VF representors
  net/bnxt: Infrastructure support for VF-reps data path
  net/bnxt: add support to get FID,default vnic ID and svif of VF-Rep
    Endpoint
  net/bnxt: fix to parse representor along with other dev-args
  net/bnxt: create default flow rules for the VF-rep conduit
  net/bnxt: support for ULP Flow counter Manager
  net/bnxt: Add support for flow query with action_type COUNT

Venkat Duvvuru (10):
  net/bnxt: modify ulp_port_db_dev_port_intf_update prototype
  net/bnxt: get port & function related information
  net/bnxt: add support for bnxt_hwrm_port_phy_qcaps
  net/bnxt: modify port_db to store & retrieve more info
  net/bnxt: enable HWRM_PORT_MAC_QCFG for trusted vf
  net/bnxt: fixes for port db
  net/bnxt: fix for VF to VFR conduit
  net/bnxt: fill mapper parameters with default rules info
  net/bnxt: add ingress & egress port default rules
  net/bnxt: fill cfa_action in the tx buffer descriptor properly

 config/common_base                              |    5 +-
 drivers/net/bnxt/Makefile                       |    8 +-
 drivers/net/bnxt/bnxt.h                         |  121 +-
 drivers/net/bnxt/bnxt_ethdev.c                  |  519 ++-
 drivers/net/bnxt/bnxt_hwrm.c                    |  122 +-
 drivers/net/bnxt/bnxt_hwrm.h                    |    7 +
 drivers/net/bnxt/bnxt_reps.c                    |  773 ++++
 drivers/net/bnxt/bnxt_reps.h                    |   45 +
 drivers/net/bnxt/bnxt_rxr.c                     |   38 +-
 drivers/net/bnxt/bnxt_rxr.h                     |    1 +
 drivers/net/bnxt/bnxt_txq.h                     |    2 +
 drivers/net/bnxt/bnxt_txr.c                     |   18 +-
 drivers/net/bnxt/hcapi/Makefile                 |   10 +
 drivers/net/bnxt/hcapi/cfa_p40_hw.h             |  781 ++++
 drivers/net/bnxt/hcapi/cfa_p40_tbl.h            |  303 ++
 drivers/net/bnxt/hcapi/hcapi_cfa.h              |  273 ++
 drivers/net/bnxt/hcapi/hcapi_cfa_defs.h         |  669 +++
 drivers/net/bnxt/hcapi/hcapi_cfa_p4.c           |  411 ++
 drivers/net/bnxt/hcapi/hcapi_cfa_p4.h           |  467 ++
 drivers/net/bnxt/hsi_struct_def_dpdk.h          | 3095 ++++++++++++--
 drivers/net/bnxt/meson.build                    |   21 +-
 drivers/net/bnxt/tf_core/Makefile               |   29 +-
 drivers/net/bnxt/tf_core/bitalloc.c             |  107 +
 drivers/net/bnxt/tf_core/bitalloc.h             |    5 +
 drivers/net/bnxt/tf_core/cfa_resource_types.h   |  293 ++
 drivers/net/bnxt/tf_core/hwrm_tf.h              |  995 +----
 drivers/net/bnxt/tf_core/ll.c                   |   52 +
 drivers/net/bnxt/tf_core/ll.h                   |   46 +
 drivers/net/bnxt/tf_core/lookup3.h              |    1 -
 drivers/net/bnxt/tf_core/stack.c                |   10 +-
 drivers/net/bnxt/tf_core/stack.h                |   10 +
 drivers/net/bnxt/tf_core/tf_common.h            |   43 +
 drivers/net/bnxt/tf_core/tf_core.c              | 1495 +++++--
 drivers/net/bnxt/tf_core/tf_core.h              |  874 +++-
 drivers/net/bnxt/tf_core/tf_device.c            |  271 ++
 drivers/net/bnxt/tf_core/tf_device.h            |  650 +++
 drivers/net/bnxt/tf_core/tf_device_p4.c         |  147 +
 drivers/net/bnxt/tf_core/tf_device_p4.h         |  104 +
 drivers/net/bnxt/tf_core/tf_em.c                |  515 ---
 drivers/net/bnxt/tf_core/tf_em.h                |  492 ++-
 drivers/net/bnxt/tf_core/tf_em_common.c         | 1050 +++++
 drivers/net/bnxt/tf_core/tf_em_common.h         |  134 +
 drivers/net/bnxt/tf_core/tf_em_host.c           |  532 +++
 drivers/net/bnxt/tf_core/tf_em_internal.c       |  352 ++
 drivers/net/bnxt/tf_core/tf_em_system.c         |  538 +++
 drivers/net/bnxt/tf_core/tf_ext_flow_handle.h   |   16 +
 drivers/net/bnxt/tf_core/tf_global_cfg.c        |  199 +
 drivers/net/bnxt/tf_core/tf_global_cfg.h        |  170 +
 drivers/net/bnxt/tf_core/tf_identifier.c        |  186 +
 drivers/net/bnxt/tf_core/tf_identifier.h        |  147 +
 drivers/net/bnxt/tf_core/tf_if_tbl.c            |  178 +
 drivers/net/bnxt/tf_core/tf_if_tbl.h            |  236 +
 drivers/net/bnxt/tf_core/tf_msg.c               | 1681 ++++----
 drivers/net/bnxt/tf_core/tf_msg.h               |  409 +-
 drivers/net/bnxt/tf_core/tf_resources.h         |  531 ---
 drivers/net/bnxt/tf_core/tf_rm.c                | 3840 ++++-------------
 drivers/net/bnxt/tf_core/tf_rm.h                |  554 ++-
 drivers/net/bnxt/tf_core/tf_session.c           |  776 ++++
 drivers/net/bnxt/tf_core/tf_session.h           |  565 ++-
 drivers/net/bnxt/tf_core/tf_shadow_tbl.c        |   63 +
 drivers/net/bnxt/tf_core/tf_shadow_tbl.h        |  240 ++
 drivers/net/bnxt/tf_core/tf_shadow_tcam.c       |   63 +
 drivers/net/bnxt/tf_core/tf_shadow_tcam.h       |  239 ++
 drivers/net/bnxt/tf_core/tf_tbl.c               | 1935 ++-------
 drivers/net/bnxt/tf_core/tf_tbl.h               |  469 +-
 drivers/net/bnxt/tf_core/tf_tcam.c              |  430 ++
 drivers/net/bnxt/tf_core/tf_tcam.h              |  360 ++
 drivers/net/bnxt/tf_core/tf_util.c              |  176 +
 drivers/net/bnxt/tf_core/tf_util.h              |   98 +
 drivers/net/bnxt/tf_core/tfp.c                  |   33 +-
 drivers/net/bnxt/tf_core/tfp.h                  |  153 +-
 drivers/net/bnxt/tf_ulp/Makefile                |    2 +
 drivers/net/bnxt/tf_ulp/bnxt_tf_common.h        |   16 +
 drivers/net/bnxt/tf_ulp/bnxt_ulp.c              |  129 +-
 drivers/net/bnxt/tf_ulp/bnxt_ulp.h              |   35 +
 drivers/net/bnxt/tf_ulp/bnxt_ulp_flow.c         |   85 +-
 drivers/net/bnxt/tf_ulp/ulp_def_rules.c         |  385 ++
 drivers/net/bnxt/tf_ulp/ulp_fc_mgr.c            |  596 +++
 drivers/net/bnxt/tf_ulp/ulp_fc_mgr.h            |  163 +
 drivers/net/bnxt/tf_ulp/ulp_flow_db.c           |   42 +-
 drivers/net/bnxt/tf_ulp/ulp_mapper.c            |  482 ++-
 drivers/net/bnxt/tf_ulp/ulp_mapper.h            |    6 +-
 drivers/net/bnxt/tf_ulp/ulp_mark_mgr.c          |   10 +
 drivers/net/bnxt/tf_ulp/ulp_port_db.c           |  234 +-
 drivers/net/bnxt/tf_ulp/ulp_port_db.h           |  122 +-
 drivers/net/bnxt/tf_ulp/ulp_rte_parser.c        |   30 +-
 drivers/net/bnxt/tf_ulp/ulp_template_db_act.c   |  433 +-
 drivers/net/bnxt/tf_ulp/ulp_template_db_class.c | 5217 +++++++++++++++++------
 drivers/net/bnxt/tf_ulp/ulp_template_db_enum.h  |  537 +--
 drivers/net/bnxt/tf_ulp/ulp_template_db_field.h |  463 +-
 drivers/net/bnxt/tf_ulp/ulp_template_db_tbl.c   |   84 +-
 drivers/net/bnxt/tf_ulp/ulp_template_struct.h   |   23 +-
 drivers/net/bnxt/tf_ulp/ulp_utils.c             |    2 +-
 93 files changed, 28024 insertions(+), 11253 deletions(-)
 create mode 100644 drivers/net/bnxt/bnxt_reps.c
 create mode 100644 drivers/net/bnxt/bnxt_reps.h
 create mode 100644 drivers/net/bnxt/hcapi/Makefile
 create mode 100644 drivers/net/bnxt/hcapi/cfa_p40_hw.h
 create mode 100644 drivers/net/bnxt/hcapi/cfa_p40_tbl.h
 create mode 100644 drivers/net/bnxt/hcapi/hcapi_cfa.h
 create mode 100644 drivers/net/bnxt/hcapi/hcapi_cfa_defs.h
 create mode 100644 drivers/net/bnxt/hcapi/hcapi_cfa_p4.c
 create mode 100644 drivers/net/bnxt/hcapi/hcapi_cfa_p4.h
 create mode 100644 drivers/net/bnxt/tf_core/cfa_resource_types.h
 create mode 100644 drivers/net/bnxt/tf_core/ll.c
 create mode 100644 drivers/net/bnxt/tf_core/ll.h
 create mode 100644 drivers/net/bnxt/tf_core/tf_common.h
 create mode 100644 drivers/net/bnxt/tf_core/tf_device.c
 create mode 100644 drivers/net/bnxt/tf_core/tf_device.h
 create mode 100644 drivers/net/bnxt/tf_core/tf_device_p4.c
 create mode 100644 drivers/net/bnxt/tf_core/tf_device_p4.h
 delete mode 100644 drivers/net/bnxt/tf_core/tf_em.c
 create mode 100644 drivers/net/bnxt/tf_core/tf_em_common.c
 create mode 100644 drivers/net/bnxt/tf_core/tf_em_common.h
 create mode 100644 drivers/net/bnxt/tf_core/tf_em_host.c
 create mode 100644 drivers/net/bnxt/tf_core/tf_em_internal.c
 create mode 100644 drivers/net/bnxt/tf_core/tf_em_system.c
 create mode 100644 drivers/net/bnxt/tf_core/tf_global_cfg.c
 create mode 100644 drivers/net/bnxt/tf_core/tf_global_cfg.h
 create mode 100644 drivers/net/bnxt/tf_core/tf_identifier.c
 create mode 100644 drivers/net/bnxt/tf_core/tf_identifier.h
 create mode 100644 drivers/net/bnxt/tf_core/tf_if_tbl.c
 create mode 100644 drivers/net/bnxt/tf_core/tf_if_tbl.h
 create mode 100644 drivers/net/bnxt/tf_core/tf_session.c
 create mode 100644 drivers/net/bnxt/tf_core/tf_shadow_tbl.c
 create mode 100644 drivers/net/bnxt/tf_core/tf_shadow_tbl.h
 create mode 100644 drivers/net/bnxt/tf_core/tf_shadow_tcam.c
 create mode 100644 drivers/net/bnxt/tf_core/tf_shadow_tcam.h
 create mode 100644 drivers/net/bnxt/tf_core/tf_tcam.c
 create mode 100644 drivers/net/bnxt/tf_core/tf_tcam.h
 create mode 100644 drivers/net/bnxt/tf_core/tf_util.c
 create mode 100644 drivers/net/bnxt/tf_core/tf_util.h
 create mode 100644 drivers/net/bnxt/tf_ulp/ulp_def_rules.c
 create mode 100644 drivers/net/bnxt/tf_ulp/ulp_fc_mgr.c
 create mode 100644 drivers/net/bnxt/tf_ulp/ulp_fc_mgr.h

-- 
2.7.4


^ permalink raw reply	[flat|nested] 271+ messages in thread

* [dpdk-dev] [PATCH 01/50] net/bnxt: Basic infrastructure support for VF representors
  2020-06-12 13:28 [dpdk-dev] [PATCH 00/50] add features for host-based flow management Somnath Kotur
@ 2020-06-12 13:28 ` Somnath Kotur
  2020-06-12 13:28 ` [dpdk-dev] [PATCH 02/50] net/bnxt: Infrastructure support for VF-reps data path Somnath Kotur
                   ` (49 subsequent siblings)
  50 siblings, 0 replies; 271+ messages in thread
From: Somnath Kotur @ 2020-06-12 13:28 UTC (permalink / raw)
  To: dev; +Cc: ferruh.yigit

Defines data structures and code to init/uninit
VF representors during pci_probe and pci_remove
respectively.
Most of the dev_ops for the VF representor are just
stubs for now and will be will be filled out in next patch.

To create a representor using testpmd:
testpmd -c 0xff -wB:D.F,representor=1 -- -i
testpmd -c 0xff -w05:02.0,representor=[1] -- -i

To create a representor using ovs-dpdk:
1. Firt add the trusted VF port to a bridge
ovs-vsctl add-port ovsbr0 vf_rep1 -- set Interface vf_rep1 type=dpdk
options:dpdk-devargs=0000:06:02.0
2. Add the representor port to the bridge
ovs-vsctl add-port ovsbr0 vf_rep1 -- set Interface vf_rep1 type=dpdk
options:dpdk-devargs=0000:06:02.0,representor=1

Reviewed-by: Kalesh Anakkur Purayil <kalesh-anakkur.purayil@broadcom.com>
Signed-off-by: Somnath Kotur <somnath.kotur@broadcom.com>
Signed-off-by: Venkat Duvvuru <venkatkumar.duvvuru@broadcom.com>
---
 drivers/net/bnxt/Makefile      |   2 +
 drivers/net/bnxt/bnxt.h        |  64 ++++++++-
 drivers/net/bnxt/bnxt_ethdev.c | 225 +++++++++++++++++++++++++-------
 drivers/net/bnxt/bnxt_reps.c   | 287 +++++++++++++++++++++++++++++++++++++++++
 drivers/net/bnxt/bnxt_reps.h   |  35 +++++
 drivers/net/bnxt/meson.build   |   1 +
 6 files changed, 566 insertions(+), 48 deletions(-)
 create mode 100644 drivers/net/bnxt/bnxt_reps.c
 create mode 100644 drivers/net/bnxt/bnxt_reps.h

diff --git a/drivers/net/bnxt/Makefile b/drivers/net/bnxt/Makefile
index a375299..3656274 100644
--- a/drivers/net/bnxt/Makefile
+++ b/drivers/net/bnxt/Makefile
@@ -14,6 +14,7 @@ LIB = librte_pmd_bnxt.a
 EXPORT_MAP := rte_pmd_bnxt_version.map
 
 CFLAGS += -O3
+CFLAGS += -DALLOW_EXPERIMENTAL_API
 CFLAGS += $(WERROR_FLAGS)
 LDLIBS += -lrte_eal -lrte_mbuf -lrte_mempool -lrte_ring
 LDLIBS += -lrte_ethdev -lrte_net -lrte_kvargs
@@ -38,6 +39,7 @@ SRCS-$(CONFIG_RTE_LIBRTE_BNXT_PMD) += bnxt_txr.c
 SRCS-$(CONFIG_RTE_LIBRTE_BNXT_PMD) += bnxt_vnic.c
 SRCS-$(CONFIG_RTE_LIBRTE_BNXT_PMD) += bnxt_irq.c
 SRCS-$(CONFIG_RTE_LIBRTE_BNXT_PMD) += bnxt_util.c
+SRCS-$(CONFIG_RTE_LIBRTE_BNXT_PMD) += bnxt_reps.c
 SRCS-$(CONFIG_RTE_LIBRTE_BNXT_PMD) += rte_pmd_bnxt.c
 ifeq ($(CONFIG_RTE_ARCH_X86), y)
 SRCS-$(CONFIG_RTE_LIBRTE_BNXT_PMD) += bnxt_rxtx_vec_sse.c
diff --git a/drivers/net/bnxt/bnxt.h b/drivers/net/bnxt/bnxt.h
index d455f8d..9b7b87c 100644
--- a/drivers/net/bnxt/bnxt.h
+++ b/drivers/net/bnxt/bnxt.h
@@ -220,6 +220,7 @@ struct bnxt_child_vf_info {
 struct bnxt_pf_info {
 #define BNXT_FIRST_PF_FID	1
 #define BNXT_MAX_VFS(bp)	((bp)->pf->max_vfs)
+#define BNXT_MAX_VF_REPS	64
 #define BNXT_TOTAL_VFS(bp)	((bp)->pf->total_vfs)
 #define BNXT_FIRST_VF_FID	128
 #define BNXT_PF_RINGS_USED(bp)	bnxt_get_num_queues(bp)
@@ -492,6 +493,10 @@ struct bnxt_mark_info {
 	bool		valid;
 };
 
+struct bnxt_rep_info {
+	struct rte_eth_dev	*vfr_eth_dev;
+};
+
 /* address space location of register */
 #define BNXT_FW_STATUS_REG_TYPE_MASK	3
 /* register is located in PCIe config space */
@@ -515,6 +520,40 @@ struct bnxt_mark_info {
 #define BNXT_FW_STATUS_HEALTHY		0x8000
 #define BNXT_FW_STATUS_SHUTDOWN		0x100000
 
+#define BNXT_ETH_RSS_SUPPORT (	\
+	ETH_RSS_IPV4 |		\
+	ETH_RSS_NONFRAG_IPV4_TCP |	\
+	ETH_RSS_NONFRAG_IPV4_UDP |	\
+	ETH_RSS_IPV6 |		\
+	ETH_RSS_NONFRAG_IPV6_TCP |	\
+	ETH_RSS_NONFRAG_IPV6_UDP)
+
+#define BNXT_DEV_TX_OFFLOAD_SUPPORT (DEV_TX_OFFLOAD_VLAN_INSERT | \
+				     DEV_TX_OFFLOAD_IPV4_CKSUM | \
+				     DEV_TX_OFFLOAD_TCP_CKSUM | \
+				     DEV_TX_OFFLOAD_UDP_CKSUM | \
+				     DEV_TX_OFFLOAD_TCP_TSO | \
+				     DEV_TX_OFFLOAD_OUTER_IPV4_CKSUM | \
+				     DEV_TX_OFFLOAD_VXLAN_TNL_TSO | \
+				     DEV_TX_OFFLOAD_GRE_TNL_TSO | \
+				     DEV_TX_OFFLOAD_IPIP_TNL_TSO | \
+				     DEV_TX_OFFLOAD_GENEVE_TNL_TSO | \
+				     DEV_TX_OFFLOAD_QINQ_INSERT | \
+				     DEV_TX_OFFLOAD_MULTI_SEGS)
+
+#define BNXT_DEV_RX_OFFLOAD_SUPPORT (DEV_RX_OFFLOAD_VLAN_FILTER | \
+				     DEV_RX_OFFLOAD_VLAN_STRIP | \
+				     DEV_RX_OFFLOAD_IPV4_CKSUM | \
+				     DEV_RX_OFFLOAD_UDP_CKSUM | \
+				     DEV_RX_OFFLOAD_TCP_CKSUM | \
+				     DEV_RX_OFFLOAD_OUTER_IPV4_CKSUM | \
+				     DEV_RX_OFFLOAD_JUMBO_FRAME | \
+				     DEV_RX_OFFLOAD_KEEP_CRC | \
+				     DEV_RX_OFFLOAD_VLAN_EXTEND | \
+				     DEV_RX_OFFLOAD_TCP_LRO | \
+				     DEV_RX_OFFLOAD_SCATTER | \
+				     DEV_RX_OFFLOAD_RSS_HASH)
+
 #define BNXT_HWRM_SHORT_REQ_LEN		sizeof(struct hwrm_short_input)
 
 struct bnxt_flow_stat_info {
@@ -682,6 +721,9 @@ struct bnxt {
 #define BNXT_MAX_RINGS(bp) \
 	(RTE_MIN((((bp)->max_cp_rings - BNXT_NUM_ASYNC_CPR(bp)) / 2U), \
 		 BNXT_MAX_TX_RINGS(bp)))
+
+#define BNXT_MAX_VF_REP_RINGS	8
+
 	uint16_t		max_nq_rings;
 	uint16_t		max_l2_ctx;
 	uint16_t		max_rx_em_flows;
@@ -711,7 +753,9 @@ struct bnxt {
 
 	uint16_t		fw_reset_min_msecs;
 	uint16_t		fw_reset_max_msecs;
-
+	uint16_t		switch_domain_id;
+	uint16_t		num_reps;
+	struct bnxt_rep_info	rep_info[BNXT_MAX_VF_REPS];
 	/* Struct to hold adapter error recovery related info */
 	struct bnxt_error_recovery_info *recovery_info;
 #define BNXT_MARK_TABLE_SZ	(sizeof(struct bnxt_mark_info)  * 64 * 1024)
@@ -732,6 +776,18 @@ struct bnxt {
 
 #define BNXT_FC_TIMER	1 /* Timer freq in Sec Flow Counters */
 
+/**
+ * Structure to store private data for each VF representor instance
+ */
+struct bnxt_vf_representor {
+	uint16_t switch_domain_id;
+	uint16_t vf_id;
+	/* Private data store of associated PF/Trusted VF */
+	struct bnxt	*parent_priv;
+	uint8_t		mac_addr[RTE_ETHER_ADDR_LEN];
+	uint8_t		dflt_mac_addr[RTE_ETHER_ADDR_LEN];
+};
+
 int bnxt_mtu_set_op(struct rte_eth_dev *eth_dev, uint16_t new_mtu);
 int bnxt_link_update(struct rte_eth_dev *eth_dev, int wait_to_complete,
 		     bool exp_link_status);
@@ -744,7 +800,13 @@ void bnxt_schedule_fw_health_check(struct bnxt *bp);
 
 bool is_bnxt_supported(struct rte_eth_dev *dev);
 bool bnxt_stratus_device(struct bnxt *bp);
+void bnxt_print_link_info(struct rte_eth_dev *eth_dev);
+uint16_t bnxt_rss_hash_tbl_size(const struct bnxt *bp);
+int bnxt_link_update_op(struct rte_eth_dev *eth_dev,
+			int wait_to_complete);
+
 extern const struct rte_flow_ops bnxt_flow_ops;
+
 #define bnxt_acquire_flow_lock(bp) \
 	pthread_mutex_lock(&(bp)->flow_lock)
 
diff --git a/drivers/net/bnxt/bnxt_ethdev.c b/drivers/net/bnxt/bnxt_ethdev.c
index 7022f6d..4911745 100644
--- a/drivers/net/bnxt/bnxt_ethdev.c
+++ b/drivers/net/bnxt/bnxt_ethdev.c
@@ -18,6 +18,7 @@
 #include "bnxt_filter.h"
 #include "bnxt_hwrm.h"
 #include "bnxt_irq.h"
+#include "bnxt_reps.h"
 #include "bnxt_ring.h"
 #include "bnxt_rxq.h"
 #include "bnxt_rxr.h"
@@ -93,40 +94,6 @@ static const struct rte_pci_id bnxt_pci_id_map[] = {
 	{ .vendor_id = 0, /* sentinel */ },
 };
 
-#define BNXT_ETH_RSS_SUPPORT (	\
-	ETH_RSS_IPV4 |		\
-	ETH_RSS_NONFRAG_IPV4_TCP |	\
-	ETH_RSS_NONFRAG_IPV4_UDP |	\
-	ETH_RSS_IPV6 |		\
-	ETH_RSS_NONFRAG_IPV6_TCP |	\
-	ETH_RSS_NONFRAG_IPV6_UDP)
-
-#define BNXT_DEV_TX_OFFLOAD_SUPPORT (DEV_TX_OFFLOAD_VLAN_INSERT | \
-				     DEV_TX_OFFLOAD_IPV4_CKSUM | \
-				     DEV_TX_OFFLOAD_TCP_CKSUM | \
-				     DEV_TX_OFFLOAD_UDP_CKSUM | \
-				     DEV_TX_OFFLOAD_TCP_TSO | \
-				     DEV_TX_OFFLOAD_OUTER_IPV4_CKSUM | \
-				     DEV_TX_OFFLOAD_VXLAN_TNL_TSO | \
-				     DEV_TX_OFFLOAD_GRE_TNL_TSO | \
-				     DEV_TX_OFFLOAD_IPIP_TNL_TSO | \
-				     DEV_TX_OFFLOAD_GENEVE_TNL_TSO | \
-				     DEV_TX_OFFLOAD_QINQ_INSERT | \
-				     DEV_TX_OFFLOAD_MULTI_SEGS)
-
-#define BNXT_DEV_RX_OFFLOAD_SUPPORT (DEV_RX_OFFLOAD_VLAN_FILTER | \
-				     DEV_RX_OFFLOAD_VLAN_STRIP | \
-				     DEV_RX_OFFLOAD_IPV4_CKSUM | \
-				     DEV_RX_OFFLOAD_UDP_CKSUM | \
-				     DEV_RX_OFFLOAD_TCP_CKSUM | \
-				     DEV_RX_OFFLOAD_OUTER_IPV4_CKSUM | \
-				     DEV_RX_OFFLOAD_JUMBO_FRAME | \
-				     DEV_RX_OFFLOAD_KEEP_CRC | \
-				     DEV_RX_OFFLOAD_VLAN_EXTEND | \
-				     DEV_RX_OFFLOAD_TCP_LRO | \
-				     DEV_RX_OFFLOAD_SCATTER | \
-				     DEV_RX_OFFLOAD_RSS_HASH)
-
 #define BNXT_DEVARG_TRUFLOW	"host-based-truflow"
 #define BNXT_DEVARG_FLOW_XSTAT	"flow-xstat"
 #define BNXT_DEVARG_MAX_NUM_KFLOWS  "max-num-kflows"
@@ -163,7 +130,6 @@ static int bnxt_devarg_max_num_kflow_invalid(uint16_t max_num_kflows)
 }
 
 static int bnxt_vlan_offload_set_op(struct rte_eth_dev *dev, int mask);
-static void bnxt_print_link_info(struct rte_eth_dev *eth_dev);
 static int bnxt_dev_uninit(struct rte_eth_dev *eth_dev);
 static int bnxt_init_resources(struct bnxt *bp, bool reconfig_dev);
 static int bnxt_uninit_resources(struct bnxt *bp, bool reconfig_dev);
@@ -198,7 +164,7 @@ static uint16_t bnxt_rss_ctxts(const struct bnxt *bp)
 				    BNXT_RSS_ENTRIES_PER_CTX_THOR;
 }
 
-static uint16_t  bnxt_rss_hash_tbl_size(const struct bnxt *bp)
+uint16_t bnxt_rss_hash_tbl_size(const struct bnxt *bp)
 {
 	if (!BNXT_CHIP_THOR(bp))
 		return HW_HASH_INDEX_SIZE;
@@ -1047,7 +1013,7 @@ static int bnxt_dev_configure_op(struct rte_eth_dev *eth_dev)
 	return -ENOSPC;
 }
 
-static void bnxt_print_link_info(struct rte_eth_dev *eth_dev)
+void bnxt_print_link_info(struct rte_eth_dev *eth_dev)
 {
 	struct rte_eth_link *link = &eth_dev->data->dev_link;
 
@@ -1273,6 +1239,12 @@ static int bnxt_dev_set_link_down_op(struct rte_eth_dev *eth_dev)
 	return 0;
 }
 
+static void bnxt_free_switch_domain(struct bnxt *bp)
+{
+	if (bp->switch_domain_id)
+		rte_eth_switch_domain_free(bp->switch_domain_id);
+}
+
 /* Unload the driver, release resources */
 static void bnxt_dev_stop_op(struct rte_eth_dev *eth_dev)
 {
@@ -1341,6 +1313,8 @@ static void bnxt_dev_close_op(struct rte_eth_dev *eth_dev)
 	if (eth_dev->data->dev_started)
 		bnxt_dev_stop_op(eth_dev);
 
+	bnxt_free_switch_domain(bp);
+
 	bnxt_uninit_resources(bp, false);
 
 	bnxt_free_leds_info(bp);
@@ -1522,8 +1496,8 @@ int bnxt_link_update(struct rte_eth_dev *eth_dev, int wait_to_complete,
 	return rc;
 }
 
-static int bnxt_link_update_op(struct rte_eth_dev *eth_dev,
-			       int wait_to_complete)
+int bnxt_link_update_op(struct rte_eth_dev *eth_dev,
+			int wait_to_complete)
 {
 	return bnxt_link_update(eth_dev, wait_to_complete, ETH_LINK_UP);
 }
@@ -5477,8 +5451,26 @@ bnxt_parse_dev_args(struct bnxt *bp, struct rte_devargs *devargs)
 	rte_kvargs_free(kvlist);
 }
 
+static int bnxt_alloc_switch_domain(struct bnxt *bp)
+{
+	int rc = 0;
+
+	if (BNXT_PF(bp) || BNXT_VF_IS_TRUSTED(bp)) {
+		rc = rte_eth_switch_domain_alloc(&bp->switch_domain_id);
+		if (rc)
+			PMD_DRV_LOG(ERR,
+				    "Failed to alloc switch domain: %d\n", rc);
+		else
+			PMD_DRV_LOG(INFO,
+				    "Switch domain allocated %d\n",
+				    bp->switch_domain_id);
+	}
+
+	return rc;
+}
+
 static int
-bnxt_dev_init(struct rte_eth_dev *eth_dev)
+bnxt_dev_init(struct rte_eth_dev *eth_dev, void *params __rte_unused)
 {
 	struct rte_pci_device *pci_dev = RTE_ETH_DEV_TO_PCI(eth_dev);
 	static int version_printed;
@@ -5557,6 +5549,8 @@ bnxt_dev_init(struct rte_eth_dev *eth_dev)
 	if (rc)
 		goto error_free;
 
+	bnxt_alloc_switch_domain(bp);
+
 	/* Pass the information to the rte_eth_dev_close() that it should also
 	 * release the private port resources.
 	 */
@@ -5689,25 +5683,162 @@ bnxt_dev_uninit(struct rte_eth_dev *eth_dev)
 	return 0;
 }
 
+static int bnxt_pci_remove_dev_with_reps(struct rte_eth_dev *eth_dev)
+{
+	struct bnxt *bp = eth_dev->data->dev_private;
+	struct rte_eth_dev *vf_rep_eth_dev;
+	int ret = 0, i;
+
+	if (!bp)
+		return -EINVAL;
+
+	for (i = 0; i < bp->num_reps; i++) {
+		vf_rep_eth_dev = bp->rep_info[i].vfr_eth_dev;
+		if (!vf_rep_eth_dev)
+			continue;
+		rte_eth_dev_destroy(vf_rep_eth_dev, bnxt_vf_representor_uninit);
+	}
+	ret = rte_eth_dev_destroy(eth_dev, bnxt_dev_uninit);
+
+	return ret;
+}
+
 static int bnxt_pci_probe(struct rte_pci_driver *pci_drv __rte_unused,
 	struct rte_pci_device *pci_dev)
 {
-	return rte_eth_dev_pci_generic_probe(pci_dev, sizeof(struct bnxt),
-		bnxt_dev_init);
+	char name[RTE_ETH_NAME_MAX_LEN];
+	struct rte_eth_devargs eth_da = { .nb_representor_ports = 0 };
+	struct rte_eth_dev *backing_eth_dev, *vf_rep_eth_dev;
+	uint16_t num_rep;
+	int i, ret = 0;
+	struct bnxt *backing_bp;
+
+	if (pci_dev->device.devargs) {
+		ret = rte_eth_devargs_parse(pci_dev->device.devargs->args,
+					    &eth_da);
+		if (ret)
+			return ret;
+	}
+
+	num_rep = eth_da.nb_representor_ports;
+	PMD_DRV_LOG(DEBUG, "nb_representor_ports = %d\n",
+		    num_rep);
+
+	/* We could come here after first level of probe is already invoked
+	 * as part of an application bringup(OVS-DPDK vswitchd), so first check
+	 * for already allocated eth_dev for the backing device (PF/Trusted VF)
+	 */
+	backing_eth_dev = rte_eth_dev_allocated(pci_dev->device.name);
+	if (backing_eth_dev == NULL) {
+		ret = rte_eth_dev_create(&pci_dev->device, pci_dev->device.name,
+					 sizeof(struct bnxt),
+					 eth_dev_pci_specific_init, pci_dev,
+					 bnxt_dev_init, NULL);
+
+		if (ret || !num_rep)
+			return ret;
+	}
+
+	if (num_rep > BNXT_MAX_VF_REPS) {
+		PMD_DRV_LOG(ERR, "nb_representor_ports = %d > %d MAX VF REPS\n",
+			    eth_da.nb_representor_ports, BNXT_MAX_VF_REPS);
+		ret = -EINVAL;
+		return ret;
+	}
+
+	/* probe representor ports now */
+	if (!backing_eth_dev)
+		backing_eth_dev = rte_eth_dev_allocated(pci_dev->device.name);
+	if (backing_eth_dev == NULL) {
+		ret = -ENODEV;
+		return ret;
+	}
+	backing_bp = backing_eth_dev->data->dev_private;
+
+	if (!(BNXT_PF(backing_bp) || BNXT_VF_IS_TRUSTED(backing_bp))) {
+		PMD_DRV_LOG(ERR,
+			    "Not a PF or trusted VF. No Representor support\n");
+		/* Returning an error is not an option.
+		 * Applications are not handling this correctly
+		 */
+		return ret;
+	}
+
+	for (i = 0; i < eth_da.nb_representor_ports; i++) {
+		struct bnxt_vf_representor representor = {
+			.vf_id = eth_da.representor_ports[i],
+			.switch_domain_id = backing_bp->switch_domain_id,
+			.parent_priv = backing_bp
+		};
+
+		if (representor.vf_id >= BNXT_MAX_VF_REPS) {
+			PMD_DRV_LOG(ERR, "VF-Rep id %d >= %d MAX VF ID\n",
+				    representor.vf_id, BNXT_MAX_VF_REPS);
+			continue;
+		}
+
+		/* representor port net_bdf_port */
+		snprintf(name, sizeof(name), "net_%s_representor_%d",
+			 pci_dev->device.name, eth_da.representor_ports[i]);
+
+		ret = rte_eth_dev_create(&pci_dev->device, name,
+					 sizeof(struct bnxt_vf_representor),
+					 NULL, NULL,
+					 bnxt_vf_representor_init,
+					 &representor);
+
+		if (!ret) {
+			vf_rep_eth_dev = rte_eth_dev_allocated(name);
+			if (!vf_rep_eth_dev) {
+				PMD_DRV_LOG(ERR, "Failed to find the eth_dev"
+					    " for VF-Rep: %s.", name);
+				bnxt_pci_remove_dev_with_reps(backing_eth_dev);
+				ret = -ENODEV;
+				return ret;
+			}
+			backing_bp->rep_info[representor.vf_id].vfr_eth_dev =
+				vf_rep_eth_dev;
+			backing_bp->num_reps++;
+		} else {
+			PMD_DRV_LOG(ERR, "failed to create bnxt vf "
+				    "representor %s.", name);
+			bnxt_pci_remove_dev_with_reps(backing_eth_dev);
+		}
+	}
+
+	return ret;
 }
 
 static int bnxt_pci_remove(struct rte_pci_device *pci_dev)
 {
-	if (rte_eal_process_type() == RTE_PROC_PRIMARY)
-		return rte_eth_dev_pci_generic_remove(pci_dev,
-				bnxt_dev_uninit);
-	else
+	struct rte_eth_dev *eth_dev;
+
+	eth_dev = rte_eth_dev_allocated(pci_dev->device.name);
+	if (!eth_dev)
+		return ENODEV; /* Invoked typically only by OVS-DPDK, by the
+				* time it comes here the eth_dev is already
+				* deleted by rte_eth_dev_close(), so returning
+				* +ve value will atleast help in proper cleanup
+				*/
+
+	if (rte_eal_process_type() == RTE_PROC_PRIMARY) {
+		if (eth_dev->data->dev_flags & RTE_ETH_DEV_REPRESENTOR)
+			return rte_eth_dev_destroy(eth_dev,
+						   bnxt_vf_representor_uninit);
+		else
+			return rte_eth_dev_destroy(eth_dev,
+						   bnxt_dev_uninit);
+	} else {
 		return rte_eth_dev_pci_generic_remove(pci_dev, NULL);
+	}
 }
 
 static struct rte_pci_driver bnxt_rte_pmd = {
 	.id_table = bnxt_pci_id_map,
-	.drv_flags = RTE_PCI_DRV_NEED_MAPPING | RTE_PCI_DRV_INTR_LSC,
+	.drv_flags = RTE_PCI_DRV_NEED_MAPPING | RTE_PCI_DRV_INTR_LSC |
+			RTE_PCI_DRV_PROBE_AGAIN, /* Needed in case of VF-REPs
+						  * and OVS-DPDK
+						  */
 	.probe = bnxt_pci_probe,
 	.remove = bnxt_pci_remove,
 };
diff --git a/drivers/net/bnxt/bnxt_reps.c b/drivers/net/bnxt/bnxt_reps.c
new file mode 100644
index 0000000..7033d62
--- /dev/null
+++ b/drivers/net/bnxt/bnxt_reps.c
@@ -0,0 +1,287 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2014-2018 Broadcom
+ * All rights reserved.
+ */
+
+#include "bnxt.h"
+#include "bnxt_ring.h"
+#include "bnxt_reps.h"
+#include "hsi_struct_def_dpdk.h"
+
+static const struct eth_dev_ops bnxt_vf_rep_dev_ops = {
+	.dev_infos_get = bnxt_vf_rep_dev_info_get_op,
+	.dev_configure = bnxt_vf_rep_dev_configure_op,
+	.dev_start = bnxt_vf_rep_dev_start_op,
+	.rx_queue_setup = bnxt_vf_rep_rx_queue_setup_op,
+	.tx_queue_setup = bnxt_vf_rep_tx_queue_setup_op,
+	.link_update = bnxt_vf_rep_link_update_op,
+	.dev_close = bnxt_vf_rep_dev_close_op,
+	.dev_stop = bnxt_vf_rep_dev_stop_op
+};
+
+static uint16_t
+bnxt_vf_rep_rx_burst(__rte_unused void *rx_queue,
+		     __rte_unused struct rte_mbuf **rx_pkts,
+		     __rte_unused uint16_t nb_pkts)
+{
+	return 0;
+}
+
+static uint16_t
+bnxt_vf_rep_tx_burst(__rte_unused void *tx_queue,
+		     __rte_unused struct rte_mbuf **tx_pkts,
+		     __rte_unused uint16_t nb_pkts)
+{
+	return 0;
+}
+
+int bnxt_vf_representor_init(struct rte_eth_dev *eth_dev, void *params)
+{
+	struct bnxt_vf_representor *vf_rep_bp = eth_dev->data->dev_private;
+	struct bnxt_vf_representor *rep_params =
+				 (struct bnxt_vf_representor *)params;
+	struct rte_eth_link *link;
+	struct bnxt *parent_bp;
+
+	vf_rep_bp->vf_id = rep_params->vf_id;
+	vf_rep_bp->switch_domain_id = rep_params->switch_domain_id;
+	vf_rep_bp->parent_priv = rep_params->parent_priv;
+
+	eth_dev->data->dev_flags |= RTE_ETH_DEV_REPRESENTOR;
+	eth_dev->data->representor_id = rep_params->vf_id;
+
+	rte_eth_random_addr(vf_rep_bp->dflt_mac_addr);
+	memcpy(vf_rep_bp->mac_addr, vf_rep_bp->dflt_mac_addr,
+	       sizeof(vf_rep_bp->mac_addr));
+	eth_dev->data->mac_addrs =
+		(struct rte_ether_addr *)&vf_rep_bp->mac_addr;
+	eth_dev->dev_ops = &bnxt_vf_rep_dev_ops;
+
+	/* No data-path, but need stub Rx/Tx functions to avoid crash
+	 * when testing with ovs-dpdk
+	 */
+	eth_dev->rx_pkt_burst = bnxt_vf_rep_rx_burst;
+	eth_dev->tx_pkt_burst = bnxt_vf_rep_tx_burst;
+	/* Link state. Inherited from PF or trusted VF */
+	parent_bp = vf_rep_bp->parent_priv;
+	link = &parent_bp->eth_dev->data->dev_link;
+
+	eth_dev->data->dev_link.link_speed = link->link_speed;
+	eth_dev->data->dev_link.link_duplex = link->link_duplex;
+	eth_dev->data->dev_link.link_status = link->link_status;
+	eth_dev->data->dev_link.link_autoneg = link->link_autoneg;
+
+	PMD_DRV_LOG(INFO, "calling bnxt_print_link_info\n");
+	bnxt_print_link_info(eth_dev);
+
+	/* Pass the information to the rte_eth_dev_close() that it should also
+	 * release the private port resources.
+	 */
+	eth_dev->data->dev_flags |= RTE_ETH_DEV_CLOSE_REMOVE;
+	PMD_DRV_LOG(INFO,
+		    "Switch domain id %d: Representor Device %d init done\n",
+		    vf_rep_bp->switch_domain_id, vf_rep_bp->vf_id);
+
+	return 0;
+}
+
+int bnxt_vf_representor_uninit(struct rte_eth_dev *eth_dev)
+{
+	struct bnxt *parent_bp;
+	struct bnxt_vf_representor *rep =
+		(struct bnxt_vf_representor *)eth_dev->data->dev_private;
+
+	uint16_t vf_id;
+
+	eth_dev->data->mac_addrs = NULL;
+
+	parent_bp = rep->parent_priv;
+	if (parent_bp) {
+		parent_bp->num_reps--;
+		vf_id = rep->vf_id;
+		if (parent_bp->rep_info) {
+			memset(&parent_bp->rep_info[vf_id], 0,
+			       sizeof(parent_bp->rep_info[vf_id]));
+			/* mark that this representor has been freed */
+		}
+	}
+	eth_dev->dev_ops = NULL;
+	return 0;
+}
+
+int bnxt_vf_rep_link_update_op(struct rte_eth_dev *eth_dev, int wait_to_compl)
+{
+	struct bnxt *parent_bp;
+	struct bnxt_vf_representor *rep =
+		(struct bnxt_vf_representor *)eth_dev->data->dev_private;
+	struct rte_eth_link *link;
+	int rc;
+
+	parent_bp = rep->parent_priv;
+	rc = bnxt_link_update_op(parent_bp->eth_dev, wait_to_compl);
+
+	/* Link state. Inherited from PF or trusted VF */
+	link = &parent_bp->eth_dev->data->dev_link;
+
+	eth_dev->data->dev_link.link_speed = link->link_speed;
+	eth_dev->data->dev_link.link_duplex = link->link_duplex;
+	eth_dev->data->dev_link.link_status = link->link_status;
+	eth_dev->data->dev_link.link_autoneg = link->link_autoneg;
+	bnxt_print_link_info(eth_dev);
+
+	return rc;
+}
+
+int bnxt_vf_rep_dev_start_op(struct rte_eth_dev *eth_dev)
+{
+	bnxt_vf_rep_link_update_op(eth_dev, 1);
+
+	return 0;
+}
+
+void bnxt_vf_rep_dev_stop_op(struct rte_eth_dev *eth_dev)
+{
+	eth_dev = eth_dev;
+}
+
+void bnxt_vf_rep_dev_close_op(struct rte_eth_dev *eth_dev)
+{
+	bnxt_vf_representor_uninit(eth_dev);
+}
+
+int bnxt_vf_rep_dev_info_get_op(struct rte_eth_dev *eth_dev,
+				struct rte_eth_dev_info *dev_info)
+{
+	struct bnxt_vf_representor *rep_bp = eth_dev->data->dev_private;
+	struct bnxt *parent_bp;
+	uint16_t max_vnics, i, j, vpool, vrxq;
+	unsigned int max_rx_rings;
+	int rc = 0;
+
+	/* MAC Specifics */
+	parent_bp = rep_bp->parent_priv;
+	if (!parent_bp) {
+		PMD_DRV_LOG(ERR, "Rep parent NULL!\n");
+		return rc;
+	}
+	PMD_DRV_LOG(DEBUG, "Representor dev_info_get_op\n");
+	dev_info->max_mac_addrs = parent_bp->max_l2_ctx;
+	dev_info->max_hash_mac_addrs = 0;
+
+	max_rx_rings = BNXT_MAX_VF_REP_RINGS;
+	/* For the sake of symmetry, max_rx_queues = max_tx_queues */
+	dev_info->max_rx_queues = max_rx_rings;
+	dev_info->max_tx_queues = max_rx_rings;
+	dev_info->reta_size = bnxt_rss_hash_tbl_size(parent_bp);
+	dev_info->hash_key_size = 40;
+	max_vnics = parent_bp->max_vnics;
+
+	/* MTU specifics */
+	dev_info->min_mtu = RTE_ETHER_MIN_MTU;
+	dev_info->max_mtu = BNXT_MAX_MTU;
+
+	/* Fast path specifics */
+	dev_info->min_rx_bufsize = 1;
+	dev_info->max_rx_pktlen = BNXT_MAX_PKT_LEN;
+
+	dev_info->rx_offload_capa = BNXT_DEV_RX_OFFLOAD_SUPPORT;
+	if (parent_bp->flags & BNXT_FLAG_PTP_SUPPORTED)
+		dev_info->rx_offload_capa |= DEV_RX_OFFLOAD_TIMESTAMP;
+	dev_info->tx_offload_capa = BNXT_DEV_TX_OFFLOAD_SUPPORT;
+	dev_info->flow_type_rss_offloads = BNXT_ETH_RSS_SUPPORT;
+
+	/* *INDENT-OFF* */
+	dev_info->default_rxconf = (struct rte_eth_rxconf) {
+		.rx_thresh = {
+			.pthresh = 8,
+			.hthresh = 8,
+			.wthresh = 0,
+		},
+		.rx_free_thresh = 32,
+		/* If no descriptors available, pkts are dropped by default */
+		.rx_drop_en = 1,
+	};
+
+	dev_info->default_txconf = (struct rte_eth_txconf) {
+		.tx_thresh = {
+			.pthresh = 32,
+			.hthresh = 0,
+			.wthresh = 0,
+		},
+		.tx_free_thresh = 32,
+		.tx_rs_thresh = 32,
+	};
+	eth_dev->data->dev_conf.intr_conf.lsc = 1;
+
+	eth_dev->data->dev_conf.intr_conf.rxq = 1;
+	dev_info->rx_desc_lim.nb_min = BNXT_MIN_RING_DESC;
+	dev_info->rx_desc_lim.nb_max = BNXT_MAX_RX_RING_DESC;
+	dev_info->tx_desc_lim.nb_min = BNXT_MIN_RING_DESC;
+	dev_info->tx_desc_lim.nb_max = BNXT_MAX_TX_RING_DESC;
+
+	/* *INDENT-ON* */
+
+	/*
+	 * TODO: default_rxconf, default_txconf, rx_desc_lim, and tx_desc_lim
+	 *       need further investigation.
+	 */
+
+	/* VMDq resources */
+	vpool = 64; /* ETH_64_POOLS */
+	vrxq = 128; /* ETH_VMDQ_DCB_NUM_QUEUES */
+	for (i = 0; i < 4; vpool >>= 1, i++) {
+		if (max_vnics > vpool) {
+			for (j = 0; j < 5; vrxq >>= 1, j++) {
+				if (dev_info->max_rx_queues > vrxq) {
+					if (vpool > vrxq)
+						vpool = vrxq;
+					goto found;
+				}
+			}
+			/* Not enough resources to support VMDq */
+			break;
+		}
+	}
+	/* Not enough resources to support VMDq */
+	vpool = 0;
+	vrxq = 0;
+found:
+	dev_info->max_vmdq_pools = vpool;
+	dev_info->vmdq_queue_num = vrxq;
+
+	dev_info->vmdq_pool_base = 0;
+	dev_info->vmdq_queue_base = 0;
+
+	return 0;
+}
+
+int bnxt_vf_rep_dev_configure_op(__rte_unused struct rte_eth_dev *eth_dev)
+{
+	PMD_DRV_LOG(DEBUG, "Representor dev_configure_op\n");
+	return 0;
+}
+
+int bnxt_vf_rep_rx_queue_setup_op(struct rte_eth_dev *eth_dev,
+				  __rte_unused uint16_t queue_idx,
+				  __rte_unused uint16_t nb_desc,
+				  __rte_unused unsigned int socket_id,
+				  __rte_unused const struct rte_eth_rxconf *
+				  rx_conf,
+				  __rte_unused struct rte_mempool *mp)
+{
+	eth_dev = eth_dev;
+
+	return 0;
+}
+
+int bnxt_vf_rep_tx_queue_setup_op(struct rte_eth_dev *eth_dev,
+				  __rte_unused uint16_t queue_idx,
+				  __rte_unused uint16_t nb_desc,
+				  __rte_unused unsigned int socket_id,
+				  __rte_unused const struct rte_eth_txconf *
+				  tx_conf)
+{
+	eth_dev = eth_dev;
+
+	return 0;
+}
diff --git a/drivers/net/bnxt/bnxt_reps.h b/drivers/net/bnxt/bnxt_reps.h
new file mode 100644
index 0000000..f4c033a
--- /dev/null
+++ b/drivers/net/bnxt/bnxt_reps.h
@@ -0,0 +1,35 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2014-2018 Broadcom
+ * All rights reserved.
+ */
+
+#ifndef _BNXT_REPS_H_
+#define _BNXT_REPS_H_
+
+#include <rte_malloc.h>
+#include <rte_ethdev.h>
+
+int bnxt_vf_representor_init(struct rte_eth_dev *eth_dev, void *params);
+int bnxt_vf_representor_uninit(struct rte_eth_dev *eth_dev);
+int bnxt_vf_rep_dev_info_get_op(struct rte_eth_dev *eth_dev,
+				struct rte_eth_dev_info *dev_info);
+int bnxt_vf_rep_dev_configure_op(struct rte_eth_dev *eth_dev);
+
+int bnxt_vf_rep_link_update_op(struct rte_eth_dev *eth_dev, int wait_to_compl);
+int bnxt_vf_rep_dev_start_op(struct rte_eth_dev *eth_dev);
+int bnxt_vf_rep_rx_queue_setup_op(struct rte_eth_dev *eth_dev,
+				  __rte_unused uint16_t queue_idx,
+				  __rte_unused uint16_t nb_desc,
+				  __rte_unused unsigned int socket_id,
+				  __rte_unused const struct rte_eth_rxconf *
+				  rx_conf,
+				  __rte_unused struct rte_mempool *mp);
+int bnxt_vf_rep_tx_queue_setup_op(struct rte_eth_dev *eth_dev,
+				  __rte_unused uint16_t queue_idx,
+				  __rte_unused uint16_t nb_desc,
+				  __rte_unused unsigned int socket_id,
+				  __rte_unused const struct rte_eth_txconf *
+				  tx_conf);
+void bnxt_vf_rep_dev_stop_op(struct rte_eth_dev *eth_dev);
+void bnxt_vf_rep_dev_close_op(struct rte_eth_dev *eth_dev);
+#endif /* _BNXT_REPS_H_ */
diff --git a/drivers/net/bnxt/meson.build b/drivers/net/bnxt/meson.build
index 4306c60..5c7859c 100644
--- a/drivers/net/bnxt/meson.build
+++ b/drivers/net/bnxt/meson.build
@@ -21,6 +21,7 @@ sources = files('bnxt_cpr.c',
 	'bnxt_txr.c',
 	'bnxt_util.c',
 	'bnxt_vnic.c',
+	'bnxt_reps.c',
 
 	'tf_core/tf_core.c',
 	'tf_core/bitalloc.c',
-- 
2.7.4


^ permalink raw reply	[flat|nested] 271+ messages in thread

* [dpdk-dev] [PATCH 02/50] net/bnxt: Infrastructure support for VF-reps data path
  2020-06-12 13:28 [dpdk-dev] [PATCH 00/50] add features for host-based flow management Somnath Kotur
  2020-06-12 13:28 ` [dpdk-dev] [PATCH 01/50] net/bnxt: Basic infrastructure support for VF representors Somnath Kotur
@ 2020-06-12 13:28 ` Somnath Kotur
  2020-06-12 13:28 ` [dpdk-dev] [PATCH 03/50] net/bnxt: add support to get FID, default vnic ID and svif of VF-Rep Endpoint Somnath Kotur
                   ` (48 subsequent siblings)
  50 siblings, 0 replies; 271+ messages in thread
From: Somnath Kotur @ 2020-06-12 13:28 UTC (permalink / raw)
  To: dev; +Cc: ferruh.yigit

Added code to support Tx/Rx from a VF representor port.
The VF-reps use the RX/TX rings of the Trusted VF/PF.
For each VF-rep, the Trusted VF/PF driver issues a VFR_ALLOC FW cmd that
returns "cfa_code" and "cfa_action" values.
The FW sets up the filter tables in such a way that VF traffic by
default (in absence of other rules) gets punted to the parent function
i.e. either the Trusted VF or the PF.
The cfa_code value in the RX-compl informs the driver of the source VF.
For traffic being transmitted from the VF-rep, the TX BD is tagged with
a cfa_action value that informs the HW to punt it to the corresponding
VF.

Reviewed-by: Venkat Duvvuru <venkatkumar.duvvuru@broadcom.com>
Reviewed-by: Ajit Kumar Khaparde <ajit.khaparde@broadcom.com>
Signed-off-by: Somnath Kotur <somnath.kotur@broadcom.com>
Signed-off-by: Venkat Duvvuru <venkatkumar.duvvuru@broadcom.com>
---
 drivers/net/bnxt/bnxt.h        |  30 ++-
 drivers/net/bnxt/bnxt_ethdev.c | 150 +++++++++----
 drivers/net/bnxt/bnxt_reps.c   | 476 ++++++++++++++++++++++++++++++++++++++---
 drivers/net/bnxt/bnxt_reps.h   |  11 +
 drivers/net/bnxt/bnxt_rxr.c    |  22 +-
 drivers/net/bnxt/bnxt_rxr.h    |   1 +
 drivers/net/bnxt/bnxt_txq.h    |   1 +
 drivers/net/bnxt/bnxt_txr.c    |   4 +-
 8 files changed, 616 insertions(+), 79 deletions(-)

diff --git a/drivers/net/bnxt/bnxt.h b/drivers/net/bnxt/bnxt.h
index 9b7b87c..443d9fe 100644
--- a/drivers/net/bnxt/bnxt.h
+++ b/drivers/net/bnxt/bnxt.h
@@ -495,6 +495,7 @@ struct bnxt_mark_info {
 
 struct bnxt_rep_info {
 	struct rte_eth_dev	*vfr_eth_dev;
+	pthread_mutex_t		vfr_lock;
 };
 
 /* address space location of register */
@@ -755,7 +756,8 @@ struct bnxt {
 	uint16_t		fw_reset_max_msecs;
 	uint16_t		switch_domain_id;
 	uint16_t		num_reps;
-	struct bnxt_rep_info	rep_info[BNXT_MAX_VF_REPS];
+	struct bnxt_rep_info	*rep_info;
+	uint16_t                *cfa_code_map;
 	/* Struct to hold adapter error recovery related info */
 	struct bnxt_error_recovery_info *recovery_info;
 #define BNXT_MARK_TABLE_SZ	(sizeof(struct bnxt_mark_info)  * 64 * 1024)
@@ -780,12 +782,28 @@ struct bnxt {
  * Structure to store private data for each VF representor instance
  */
 struct bnxt_vf_representor {
-	uint16_t switch_domain_id;
-	uint16_t vf_id;
+	uint16_t		switch_domain_id;
+	uint16_t		vf_id;
+	uint16_t		tx_cfa_action;
+	uint16_t		rx_cfa_code;
 	/* Private data store of associated PF/Trusted VF */
-	struct bnxt	*parent_priv;
-	uint8_t		mac_addr[RTE_ETHER_ADDR_LEN];
-	uint8_t		dflt_mac_addr[RTE_ETHER_ADDR_LEN];
+	struct rte_eth_dev	*parent_dev;
+	uint8_t			mac_addr[RTE_ETHER_ADDR_LEN];
+	uint8_t			dflt_mac_addr[RTE_ETHER_ADDR_LEN];
+	struct bnxt_rx_queue	**rx_queues;
+	unsigned int		rx_nr_rings;
+	unsigned int		tx_nr_rings;
+	uint64_t                tx_pkts[BNXT_MAX_VF_REP_RINGS];
+	uint64_t                tx_bytes[BNXT_MAX_VF_REP_RINGS];
+	uint64_t                rx_pkts[BNXT_MAX_VF_REP_RINGS];
+	uint64_t                rx_bytes[BNXT_MAX_VF_REP_RINGS];
+	uint64_t                rx_drop_pkts[BNXT_MAX_VF_REP_RINGS];
+	uint64_t                rx_drop_bytes[BNXT_MAX_VF_REP_RINGS];
+};
+
+struct bnxt_vf_rep_tx_queue {
+	struct bnxt_tx_queue *txq;
+	struct bnxt_vf_representor *bp;
 };
 
 int bnxt_mtu_set_op(struct rte_eth_dev *eth_dev, uint16_t new_mtu);
diff --git a/drivers/net/bnxt/bnxt_ethdev.c b/drivers/net/bnxt/bnxt_ethdev.c
index 4911745..4202904 100644
--- a/drivers/net/bnxt/bnxt_ethdev.c
+++ b/drivers/net/bnxt/bnxt_ethdev.c
@@ -137,6 +137,7 @@ static void bnxt_cancel_fw_health_check(struct bnxt *bp);
 static int bnxt_restore_vlan_filters(struct bnxt *bp);
 static void bnxt_dev_recover(void *arg);
 static void bnxt_free_error_recovery_info(struct bnxt *bp);
+static void bnxt_free_rep_info(struct bnxt *bp);
 
 int is_bnxt_in_error(struct bnxt *bp)
 {
@@ -5243,7 +5244,7 @@ bnxt_init_locks(struct bnxt *bp)
 
 static int bnxt_init_resources(struct bnxt *bp, bool reconfig_dev)
 {
-	int rc;
+	int rc = 0;
 
 	rc = bnxt_init_fw(bp);
 	if (rc)
@@ -5642,6 +5643,8 @@ bnxt_uninit_locks(struct bnxt *bp)
 {
 	pthread_mutex_destroy(&bp->flow_lock);
 	pthread_mutex_destroy(&bp->def_cp_lock);
+	if (bp->rep_info)
+		pthread_mutex_destroy(&bp->rep_info->vfr_lock);
 }
 
 static int
@@ -5664,6 +5667,7 @@ bnxt_uninit_resources(struct bnxt *bp, bool reconfig_dev)
 
 	bnxt_uninit_locks(bp);
 	bnxt_free_flow_stats_info(bp);
+	bnxt_free_rep_info(bp);
 	rte_free(bp->ptp_cfg);
 	bp->ptp_cfg = NULL;
 	return rc;
@@ -5703,56 +5707,73 @@ static int bnxt_pci_remove_dev_with_reps(struct rte_eth_dev *eth_dev)
 	return ret;
 }
 
-static int bnxt_pci_probe(struct rte_pci_driver *pci_drv __rte_unused,
-	struct rte_pci_device *pci_dev)
+static void bnxt_free_rep_info(struct bnxt *bp)
 {
-	char name[RTE_ETH_NAME_MAX_LEN];
-	struct rte_eth_devargs eth_da = { .nb_representor_ports = 0 };
-	struct rte_eth_dev *backing_eth_dev, *vf_rep_eth_dev;
-	uint16_t num_rep;
-	int i, ret = 0;
-	struct bnxt *backing_bp;
+	rte_free(bp->rep_info);
+	bp->rep_info = NULL;
+	rte_free(bp->cfa_code_map);
+	bp->cfa_code_map = NULL;
+}
 
-	if (pci_dev->device.devargs) {
-		ret = rte_eth_devargs_parse(pci_dev->device.devargs->args,
-					    &eth_da);
-		if (ret)
-			return ret;
-	}
+static int bnxt_init_rep_info(struct bnxt *bp)
+{
+	int i = 0, rc;
 
-	num_rep = eth_da.nb_representor_ports;
-	PMD_DRV_LOG(DEBUG, "nb_representor_ports = %d\n",
-		    num_rep);
+	if (bp->rep_info)
+		return 0;
 
-	/* We could come here after first level of probe is already invoked
-	 * as part of an application bringup(OVS-DPDK vswitchd), so first check
-	 * for already allocated eth_dev for the backing device (PF/Trusted VF)
-	 */
-	backing_eth_dev = rte_eth_dev_allocated(pci_dev->device.name);
-	if (backing_eth_dev == NULL) {
-		ret = rte_eth_dev_create(&pci_dev->device, pci_dev->device.name,
-					 sizeof(struct bnxt),
-					 eth_dev_pci_specific_init, pci_dev,
-					 bnxt_dev_init, NULL);
+	bp->rep_info = rte_zmalloc("bnxt_rep_info",
+				   sizeof(bp->rep_info[0]) * BNXT_MAX_VF_REPS,
+				   0);
+	if (!bp->rep_info) {
+		PMD_DRV_LOG(ERR, "Failed to alloc memory for rep info\n");
+		return -ENOMEM;
+	}
+	bp->cfa_code_map = rte_zmalloc("bnxt_cfa_code_map",
+				       sizeof(*bp->cfa_code_map) *
+				       BNXT_MAX_CFA_CODE, 0);
+	if (!bp->cfa_code_map) {
+		PMD_DRV_LOG(ERR, "Failed to alloc memory for cfa_code_map\n");
+		bnxt_free_rep_info(bp);
+		return -ENOMEM;
+	}
 
-		if (ret || !num_rep)
-			return ret;
+	for (i = 0; i < BNXT_MAX_CFA_CODE; i++)
+		bp->cfa_code_map[i] = BNXT_VF_IDX_INVALID;
+
+	rc = pthread_mutex_init(&bp->rep_info->vfr_lock, NULL);
+	if (rc) {
+		PMD_DRV_LOG(ERR, "Unable to initialize vfr_lock\n");
+		bnxt_free_rep_info(bp);
+		return rc;
 	}
+	return rc;
+}
+
+static int bnxt_rep_port_probe(struct rte_pci_device *pci_dev,
+			       struct rte_eth_devargs eth_da,
+			       struct rte_eth_dev *backing_eth_dev)
+{
+	struct rte_eth_dev *vf_rep_eth_dev;
+	char name[RTE_ETH_NAME_MAX_LEN];
+	struct bnxt *backing_bp;
+	uint16_t num_rep;
+	int i, ret = 0;
 
+	num_rep = eth_da.nb_representor_ports;
 	if (num_rep > BNXT_MAX_VF_REPS) {
 		PMD_DRV_LOG(ERR, "nb_representor_ports = %d > %d MAX VF REPS\n",
-			    eth_da.nb_representor_ports, BNXT_MAX_VF_REPS);
-		ret = -EINVAL;
-		return ret;
+			    num_rep, BNXT_MAX_VF_REPS);
+		return -EINVAL;
 	}
 
-	/* probe representor ports now */
-	if (!backing_eth_dev)
-		backing_eth_dev = rte_eth_dev_allocated(pci_dev->device.name);
-	if (backing_eth_dev == NULL) {
-		ret = -ENODEV;
-		return ret;
+	if (num_rep > RTE_MAX_ETHPORTS) {
+		PMD_DRV_LOG(ERR,
+			    "nb_representor_ports = %d > %d MAX ETHPORTS\n",
+			    num_rep, RTE_MAX_ETHPORTS);
+		return -EINVAL;
 	}
+
 	backing_bp = backing_eth_dev->data->dev_private;
 
 	if (!(BNXT_PF(backing_bp) || BNXT_VF_IS_TRUSTED(backing_bp))) {
@@ -5761,14 +5782,17 @@ static int bnxt_pci_probe(struct rte_pci_driver *pci_drv __rte_unused,
 		/* Returning an error is not an option.
 		 * Applications are not handling this correctly
 		 */
-		return ret;
+		return 0;
 	}
 
-	for (i = 0; i < eth_da.nb_representor_ports; i++) {
+	if (bnxt_init_rep_info(backing_bp))
+		return 0;
+
+	for (i = 0; i < num_rep; i++) {
 		struct bnxt_vf_representor representor = {
 			.vf_id = eth_da.representor_ports[i],
 			.switch_domain_id = backing_bp->switch_domain_id,
-			.parent_priv = backing_bp
+			.parent_dev = backing_eth_dev
 		};
 
 		if (representor.vf_id >= BNXT_MAX_VF_REPS) {
@@ -5809,6 +5833,48 @@ static int bnxt_pci_probe(struct rte_pci_driver *pci_drv __rte_unused,
 	return ret;
 }
 
+static int bnxt_pci_probe(struct rte_pci_driver *pci_drv __rte_unused,
+			  struct rte_pci_device *pci_dev)
+{
+	struct rte_eth_devargs eth_da = { .nb_representor_ports = 0 };
+	struct rte_eth_dev *backing_eth_dev;
+	uint16_t num_rep;
+	int ret = 0;
+
+	if (pci_dev->device.devargs) {
+		ret = rte_eth_devargs_parse(pci_dev->device.devargs->args,
+					    &eth_da);
+		if (ret)
+			return ret;
+	}
+
+	num_rep = eth_da.nb_representor_ports;
+	PMD_DRV_LOG(DEBUG, "nb_representor_ports = %d\n",
+		    num_rep);
+
+	/* We could come here after first level of probe is already invoked
+	 * as part of an application bringup(OVS-DPDK vswitchd), so first check
+	 * for already allocated eth_dev for the backing device (PF/Trusted VF)
+	 */
+	backing_eth_dev = rte_eth_dev_allocated(pci_dev->device.name);
+	if (backing_eth_dev == NULL) {
+		ret = rte_eth_dev_create(&pci_dev->device, pci_dev->device.name,
+					 sizeof(struct bnxt),
+					 eth_dev_pci_specific_init, pci_dev,
+					 bnxt_dev_init, NULL);
+
+		if (ret || !num_rep)
+			return ret;
+
+		backing_eth_dev = rte_eth_dev_allocated(pci_dev->device.name);
+	}
+
+	/* probe representor ports now */
+	ret = bnxt_rep_port_probe(pci_dev, eth_da, backing_eth_dev);
+
+	return ret;
+}
+
 static int bnxt_pci_remove(struct rte_pci_device *pci_dev)
 {
 	struct rte_eth_dev *eth_dev;
diff --git a/drivers/net/bnxt/bnxt_reps.c b/drivers/net/bnxt/bnxt_reps.c
index 7033d62..39f3d23 100644
--- a/drivers/net/bnxt/bnxt_reps.c
+++ b/drivers/net/bnxt/bnxt_reps.c
@@ -6,6 +6,11 @@
 #include "bnxt.h"
 #include "bnxt_ring.h"
 #include "bnxt_reps.h"
+#include "bnxt_rxq.h"
+#include "bnxt_rxr.h"
+#include "bnxt_txq.h"
+#include "bnxt_txr.h"
+#include "bnxt_hwrm.h"
 #include "hsi_struct_def_dpdk.h"
 
 static const struct eth_dev_ops bnxt_vf_rep_dev_ops = {
@@ -13,25 +18,128 @@ static const struct eth_dev_ops bnxt_vf_rep_dev_ops = {
 	.dev_configure = bnxt_vf_rep_dev_configure_op,
 	.dev_start = bnxt_vf_rep_dev_start_op,
 	.rx_queue_setup = bnxt_vf_rep_rx_queue_setup_op,
+	.rx_queue_release = bnxt_vf_rep_rx_queue_release_op,
 	.tx_queue_setup = bnxt_vf_rep_tx_queue_setup_op,
+	.tx_queue_release = bnxt_vf_rep_tx_queue_release_op,
 	.link_update = bnxt_vf_rep_link_update_op,
 	.dev_close = bnxt_vf_rep_dev_close_op,
-	.dev_stop = bnxt_vf_rep_dev_stop_op
+	.dev_stop = bnxt_vf_rep_dev_stop_op,
+	.stats_get = bnxt_vf_rep_stats_get_op,
+	.stats_reset = bnxt_vf_rep_stats_reset_op,
 };
 
-static uint16_t
-bnxt_vf_rep_rx_burst(__rte_unused void *rx_queue,
-		     __rte_unused struct rte_mbuf **rx_pkts,
-		     __rte_unused uint16_t nb_pkts)
+uint16_t
+bnxt_vfr_recv(struct bnxt *bp, uint16_t cfa_code, uint16_t queue_id,
+	      struct rte_mbuf *mbuf)
 {
+	struct bnxt_sw_rx_bd *prod_rx_buf;
+	struct bnxt_rx_ring_info *rep_rxr;
+	struct bnxt_rx_queue *rep_rxq;
+	struct rte_eth_dev *vfr_eth_dev;
+	struct bnxt_vf_representor *vfr_bp;
+	uint16_t vf_id;
+	uint16_t mask;
+	uint8_t que;
+
+	vf_id = bp->cfa_code_map[cfa_code];
+	/* cfa_code is invalid OR vf_id > MAX REP. Assume normal Rx */
+	if (vf_id == BNXT_VF_IDX_INVALID || vf_id > BNXT_MAX_VF_REPS)
+		return 1;
+	vfr_eth_dev = bp->rep_info[vf_id].vfr_eth_dev;
+	if (!vfr_eth_dev)
+		return 1;
+	vfr_bp = vfr_eth_dev->data->dev_private;
+	if (vfr_bp->rx_cfa_code != cfa_code) {
+		/* cfa_code not meant for this VF rep!!?? */
+		return 1;
+	}
+	/* If rxq_id happens to be > max rep_queue, use rxq0 */
+	que = queue_id < BNXT_MAX_VF_REP_RINGS ? queue_id : 0;
+	rep_rxq = vfr_bp->rx_queues[que];
+	rep_rxr = rep_rxq->rx_ring;
+	mask = rep_rxr->rx_ring_struct->ring_mask;
+
+	/* Put this mbuf on the RxQ of the Representor */
+	prod_rx_buf =
+		&rep_rxr->rx_buf_ring[rep_rxr->rx_prod++ & mask];
+	if (!prod_rx_buf->mbuf) {
+		prod_rx_buf->mbuf = mbuf;
+		vfr_bp->rx_bytes[que] += mbuf->pkt_len;
+		vfr_bp->rx_pkts[que]++;
+	} else {
+		vfr_bp->rx_drop_bytes[que] += mbuf->pkt_len;
+		vfr_bp->rx_drop_pkts[que]++;
+		rte_free(mbuf); /* Representor Rx ring full, drop pkt */
+	}
+
 	return 0;
 }
 
 static uint16_t
-bnxt_vf_rep_tx_burst(__rte_unused void *tx_queue,
-		     __rte_unused struct rte_mbuf **tx_pkts,
+bnxt_vf_rep_rx_burst(void *rx_queue,
+		     struct rte_mbuf **rx_pkts,
+		     uint16_t nb_pkts)
+{
+	struct bnxt_rx_queue *rxq = rx_queue;
+	struct bnxt_sw_rx_bd *cons_rx_buf;
+	struct bnxt_rx_ring_info *rxr;
+	uint16_t nb_rx_pkts = 0;
+	uint16_t mask, i;
+
+	if (!rxq)
+		return 0;
+
+	rxr = rxq->rx_ring;
+	mask = rxr->rx_ring_struct->ring_mask;
+	for (i = 0; i < nb_pkts; i++) {
+		cons_rx_buf = &rxr->rx_buf_ring[rxr->rx_cons & mask];
+		if (!cons_rx_buf->mbuf)
+			return nb_rx_pkts;
+		rx_pkts[nb_rx_pkts] = cons_rx_buf->mbuf;
+		rx_pkts[nb_rx_pkts]->port = rxq->port_id;
+		cons_rx_buf->mbuf = NULL;
+		nb_rx_pkts++;
+		rxr->rx_cons++;
+	}
+
+	return nb_rx_pkts;
+}
+
+static uint16_t
+bnxt_vf_rep_tx_burst(void *tx_queue,
+		     struct rte_mbuf **tx_pkts,
 		     __rte_unused uint16_t nb_pkts)
 {
+	struct bnxt_vf_rep_tx_queue *vfr_txq = tx_queue;
+	struct bnxt_tx_queue *ptxq;
+	struct bnxt *parent;
+	struct  bnxt_vf_representor *vf_rep_bp;
+	int qid;
+	int rc;
+	int i;
+
+	if (!vfr_txq)
+		return 0;
+
+	qid = vfr_txq->txq->queue_id;
+	vf_rep_bp = vfr_txq->bp;
+	parent = vf_rep_bp->parent_dev->data->dev_private;
+	pthread_mutex_lock(&parent->rep_info->vfr_lock);
+	ptxq = parent->tx_queues[qid];
+
+	ptxq->tx_cfa_action = vf_rep_bp->tx_cfa_action;
+
+	for (i = 0; i < nb_pkts; i++) {
+		vf_rep_bp->tx_bytes[qid] += tx_pkts[i]->pkt_len;
+		vf_rep_bp->tx_pkts[qid]++;
+	}
+
+	rc = bnxt_xmit_pkts(ptxq, tx_pkts, nb_pkts);
+	ptxq->tx_cfa_action = 0;
+	pthread_mutex_unlock(&parent->rep_info->vfr_lock);
+
+	return rc;
+
 	return 0;
 }
 
@@ -45,7 +153,7 @@ int bnxt_vf_representor_init(struct rte_eth_dev *eth_dev, void *params)
 
 	vf_rep_bp->vf_id = rep_params->vf_id;
 	vf_rep_bp->switch_domain_id = rep_params->switch_domain_id;
-	vf_rep_bp->parent_priv = rep_params->parent_priv;
+	vf_rep_bp->parent_dev = rep_params->parent_dev;
 
 	eth_dev->data->dev_flags |= RTE_ETH_DEV_REPRESENTOR;
 	eth_dev->data->representor_id = rep_params->vf_id;
@@ -63,7 +171,7 @@ int bnxt_vf_representor_init(struct rte_eth_dev *eth_dev, void *params)
 	eth_dev->rx_pkt_burst = bnxt_vf_rep_rx_burst;
 	eth_dev->tx_pkt_burst = bnxt_vf_rep_tx_burst;
 	/* Link state. Inherited from PF or trusted VF */
-	parent_bp = vf_rep_bp->parent_priv;
+	parent_bp = vf_rep_bp->parent_dev->data->dev_private;
 	link = &parent_bp->eth_dev->data->dev_link;
 
 	eth_dev->data->dev_link.link_speed = link->link_speed;
@@ -94,18 +202,18 @@ int bnxt_vf_representor_uninit(struct rte_eth_dev *eth_dev)
 	uint16_t vf_id;
 
 	eth_dev->data->mac_addrs = NULL;
-
-	parent_bp = rep->parent_priv;
-	if (parent_bp) {
-		parent_bp->num_reps--;
-		vf_id = rep->vf_id;
-		if (parent_bp->rep_info) {
-			memset(&parent_bp->rep_info[vf_id], 0,
-			       sizeof(parent_bp->rep_info[vf_id]));
-			/* mark that this representor has been freed */
-		}
-	}
 	eth_dev->dev_ops = NULL;
+
+	parent_bp = rep->parent_dev->data->dev_private;
+	if (!parent_bp)
+		return 0;
+
+	parent_bp->num_reps--;
+	vf_id = rep->vf_id;
+	if (parent_bp->rep_info)
+		memset(&parent_bp->rep_info[vf_id], 0,
+		       sizeof(parent_bp->rep_info[vf_id]));
+		/* mark that this representor has been freed */
 	return 0;
 }
 
@@ -117,7 +225,7 @@ int bnxt_vf_rep_link_update_op(struct rte_eth_dev *eth_dev, int wait_to_compl)
 	struct rte_eth_link *link;
 	int rc;
 
-	parent_bp = rep->parent_priv;
+	parent_bp = rep->parent_dev->data->dev_private;
 	rc = bnxt_link_update_op(parent_bp->eth_dev, wait_to_compl);
 
 	/* Link state. Inherited from PF or trusted VF */
@@ -132,16 +240,134 @@ int bnxt_vf_rep_link_update_op(struct rte_eth_dev *eth_dev, int wait_to_compl)
 	return rc;
 }
 
+static int bnxt_vfr_alloc(struct bnxt_vf_representor *vfr)
+{
+	int rc = 0;
+	struct bnxt *parent_bp;
+
+	if (!vfr || !vfr->parent_dev) {
+		PMD_DRV_LOG(ERR,
+			    "No memory allocated for representor\n");
+		return -ENOMEM;
+	}
+
+	parent_bp = vfr->parent_dev->data->dev_private;
+
+	/* Check if representor has been already allocated in FW */
+	if (vfr->tx_cfa_action && vfr->rx_cfa_code)
+		return 0;
+
+	/*
+	 * Alloc VF rep rules in CFA after default VNIC is created.
+	 * Otherwise the FW will create the VF-rep rules with
+	 * default drop action.
+	 */
+
+	/*
+	 * This is where we need to replace invoking an HWRM cmd
+	 * with the new TFLIB ULP API to do more/less the same job
+	rc = bnxt_hwrm_cfa_vfr_alloc(parent_bp,
+				     vfr->vf_id,
+				     &vfr->tx_cfa_action,
+				     &vfr->rx_cfa_code);
+	 */
+	if (!rc) {
+		parent_bp->cfa_code_map[vfr->rx_cfa_code] = vfr->vf_id;
+		PMD_DRV_LOG(DEBUG, "allocated representor %d in FW\n",
+			    vfr->vf_id);
+	} else {
+		PMD_DRV_LOG(ERR,
+			    "Failed to alloc representor %d in FW\n",
+			    vfr->vf_id);
+	}
+
+	return rc;
+}
+
+static void bnxt_vf_rep_free_rx_mbufs(struct bnxt_vf_representor *rep_bp)
+{
+	struct bnxt_rx_queue *rxq;
+	unsigned int i;
+
+	for (i = 0; i < rep_bp->rx_nr_rings; i++) {
+		rxq = rep_bp->rx_queues[i];
+		bnxt_rx_queue_release_mbufs(rxq);
+	}
+}
+
 int bnxt_vf_rep_dev_start_op(struct rte_eth_dev *eth_dev)
 {
-	bnxt_vf_rep_link_update_op(eth_dev, 1);
+	struct bnxt_vf_representor *rep_bp = eth_dev->data->dev_private;
+	int rc;
 
-	return 0;
+	rc = bnxt_vfr_alloc(rep_bp);
+
+	if (!rc) {
+		eth_dev->rx_pkt_burst = &bnxt_vf_rep_rx_burst;
+		eth_dev->tx_pkt_burst = &bnxt_vf_rep_tx_burst;
+
+		bnxt_vf_rep_link_update_op(eth_dev, 1);
+	} else {
+		eth_dev->data->dev_link.link_status = 0;
+		bnxt_vf_rep_free_rx_mbufs(rep_bp);
+	}
+
+	return rc;
+}
+
+static int bnxt_vfr_free(struct bnxt_vf_representor *vfr)
+{
+	int rc = 0;
+	struct bnxt *parent_bp;
+
+	if (!vfr || !vfr->parent_dev) {
+		PMD_DRV_LOG(ERR,
+			    "No memory allocated for representor\n");
+		return -ENOMEM;
+	}
+
+	parent_bp = vfr->parent_dev->data->dev_private;
+
+	/* Check if representor has been already freed in FW */
+	if (!vfr->tx_cfa_action && !vfr->rx_cfa_code)
+		return 0;
+
+	/*
+	 * This is where we need to replace invoking an HWRM cmd
+	 * with the new TFLIB ULP API to do more/less the same job
+	rc = bnxt_hwrm_cfa_vfr_free(parent_bp,
+				    vfr->vf_id);
+	 */
+	if (rc) {
+		PMD_DRV_LOG(ERR,
+			    "Failed to free representor %d in FW\n",
+			    vfr->vf_id);
+		return rc;
+	}
+
+	parent_bp->cfa_code_map[vfr->rx_cfa_code] = BNXT_VF_IDX_INVALID;
+	PMD_DRV_LOG(DEBUG, "freed representor %d in FW\n",
+		    vfr->vf_id);
+	vfr->tx_cfa_action = 0;
+	vfr->rx_cfa_code = 0;
+
+	return rc;
 }
 
 void bnxt_vf_rep_dev_stop_op(struct rte_eth_dev *eth_dev)
 {
-	eth_dev = eth_dev;
+	struct bnxt_vf_representor *vfr_bp = eth_dev->data->dev_private;
+
+	/* Avoid crashes as we are about to free queues */
+	eth_dev->rx_pkt_burst = &bnxt_dummy_recv_pkts;
+	eth_dev->tx_pkt_burst = &bnxt_dummy_xmit_pkts;
+
+	bnxt_vfr_free(vfr_bp);
+
+	if (eth_dev->data->dev_started)
+		eth_dev->data->dev_link.link_status = 0;
+
+	bnxt_vf_rep_free_rx_mbufs(vfr_bp);
 }
 
 void bnxt_vf_rep_dev_close_op(struct rte_eth_dev *eth_dev)
@@ -159,7 +385,7 @@ int bnxt_vf_rep_dev_info_get_op(struct rte_eth_dev *eth_dev,
 	int rc = 0;
 
 	/* MAC Specifics */
-	parent_bp = rep_bp->parent_priv;
+	parent_bp = rep_bp->parent_dev->data->dev_private;
 	if (!parent_bp) {
 		PMD_DRV_LOG(ERR, "Rep parent NULL!\n");
 		return rc;
@@ -257,7 +483,13 @@ int bnxt_vf_rep_dev_info_get_op(struct rte_eth_dev *eth_dev,
 
 int bnxt_vf_rep_dev_configure_op(__rte_unused struct rte_eth_dev *eth_dev)
 {
+	struct bnxt_vf_representor *rep_bp = eth_dev->data->dev_private;
+
 	PMD_DRV_LOG(DEBUG, "Representor dev_configure_op\n");
+	rep_bp->rx_queues = (void *)eth_dev->data->rx_queues;
+	rep_bp->tx_nr_rings = eth_dev->data->nb_tx_queues;
+	rep_bp->rx_nr_rings = eth_dev->data->nb_rx_queues;
+
 	return 0;
 }
 
@@ -269,9 +501,94 @@ int bnxt_vf_rep_rx_queue_setup_op(struct rte_eth_dev *eth_dev,
 				  rx_conf,
 				  __rte_unused struct rte_mempool *mp)
 {
-	eth_dev = eth_dev;
+	struct bnxt_vf_representor *rep_bp = eth_dev->data->dev_private;
+	struct bnxt *parent_bp = rep_bp->parent_dev->data->dev_private;
+	struct bnxt_rx_queue *parent_rxq;
+	struct bnxt_rx_queue *rxq;
+	struct bnxt_sw_rx_bd *buf_ring;
+	int rc = 0;
+
+	if (queue_idx >= BNXT_MAX_VF_REP_RINGS) {
+		PMD_DRV_LOG(ERR,
+			    "Cannot create Rx ring %d. %d rings available\n",
+			    queue_idx, BNXT_MAX_VF_REP_RINGS);
+		return -EINVAL;
+	}
+
+	if (!nb_desc || nb_desc > MAX_RX_DESC_CNT) {
+		PMD_DRV_LOG(ERR, "nb_desc %d is invalid\n", nb_desc);
+		return -EINVAL;
+	}
+
+	parent_rxq = parent_bp->rx_queues[queue_idx];
+	if (!parent_rxq) {
+		PMD_DRV_LOG(ERR, "Parent RxQ has not been configured yet\n");
+		return -EINVAL;
+	}
+
+	if (nb_desc != parent_rxq->nb_rx_desc) {
+		PMD_DRV_LOG(ERR, "nb_desc %d do not match parent rxq", nb_desc);
+		return -EINVAL;
+	}
+
+	if (eth_dev->data->rx_queues) {
+		rxq = eth_dev->data->rx_queues[queue_idx];
+		if (rxq)
+			bnxt_rx_queue_release_op(rxq);
+	}
+
+	rxq = rte_zmalloc_socket("bnxt_vfr_rx_queue",
+				 sizeof(struct bnxt_rx_queue),
+				 RTE_CACHE_LINE_SIZE, socket_id);
+	if (!rxq) {
+		PMD_DRV_LOG(ERR, "bnxt_vfr_rx_queue allocation failed!\n");
+		return -ENOMEM;
+	}
+
+	rxq->nb_rx_desc = nb_desc;
+
+	rc = bnxt_init_rx_ring_struct(rxq, socket_id);
+	if (rc)
+		goto out;
+
+	buf_ring = rte_zmalloc_socket("bnxt_rx_vfr_buf_ring",
+				      sizeof(struct bnxt_sw_rx_bd) *
+				      rxq->rx_ring->rx_ring_struct->ring_size,
+				      RTE_CACHE_LINE_SIZE, socket_id);
+	if (!buf_ring) {
+		PMD_DRV_LOG(ERR, "bnxt_rx_vfr_buf_ring allocation failed!\n");
+		rc = -ENOMEM;
+		goto out;
+	}
+
+	rxq->rx_ring->rx_buf_ring = buf_ring;
+	rxq->queue_id = queue_idx;
+	rxq->port_id = eth_dev->data->port_id;
+	eth_dev->data->rx_queues[queue_idx] = rxq;
 
 	return 0;
+
+out:
+	if (rxq)
+		bnxt_rx_queue_release_op(rxq);
+
+	return rc;
+}
+
+void bnxt_vf_rep_rx_queue_release_op(void *rx_queue)
+{
+	struct bnxt_rx_queue *rxq = (struct bnxt_rx_queue *)rx_queue;
+
+	if (!rxq)
+		return;
+
+	bnxt_rx_queue_release_mbufs(rxq);
+
+	bnxt_free_ring(rxq->rx_ring->rx_ring_struct);
+	bnxt_free_ring(rxq->rx_ring->ag_ring_struct);
+	bnxt_free_ring(rxq->cp_ring->cp_ring_struct);
+
+	rte_free(rxq);
 }
 
 int bnxt_vf_rep_tx_queue_setup_op(struct rte_eth_dev *eth_dev,
@@ -281,7 +598,112 @@ int bnxt_vf_rep_tx_queue_setup_op(struct rte_eth_dev *eth_dev,
 				  __rte_unused const struct rte_eth_txconf *
 				  tx_conf)
 {
-	eth_dev = eth_dev;
+	struct bnxt_vf_representor *rep_bp = eth_dev->data->dev_private;
+	struct bnxt *parent_bp = rep_bp->parent_dev->data->dev_private;
+	struct bnxt_tx_queue *parent_txq, *txq;
+	struct bnxt_vf_rep_tx_queue *vfr_txq;
+
+	if (queue_idx >= BNXT_MAX_VF_REP_RINGS) {
+		PMD_DRV_LOG(ERR,
+			    "Cannot create Tx rings %d. %d rings available\n",
+			    queue_idx, BNXT_MAX_VF_REP_RINGS);
+		return -EINVAL;
+	}
+
+	if (!nb_desc || nb_desc > MAX_TX_DESC_CNT) {
+		PMD_DRV_LOG(ERR, "nb_desc %d is invalid", nb_desc);
+		return -EINVAL;
+	}
+
+	parent_txq = parent_bp->tx_queues[queue_idx];
+	if (!parent_txq) {
+		PMD_DRV_LOG(ERR, "Parent TxQ has not been configured yet\n");
+		return -EINVAL;
+	}
 
+	if (nb_desc != parent_txq->nb_tx_desc) {
+		PMD_DRV_LOG(ERR, "nb_desc %d do not match parent txq", nb_desc);
+		return -EINVAL;
+	}
+
+	if (eth_dev->data->tx_queues) {
+		vfr_txq = eth_dev->data->tx_queues[queue_idx];
+		bnxt_vf_rep_tx_queue_release_op(vfr_txq);
+		vfr_txq = NULL;
+	}
+
+	vfr_txq = rte_zmalloc_socket("bnxt_vfr_tx_queue",
+				     sizeof(struct bnxt_vf_rep_tx_queue),
+				     RTE_CACHE_LINE_SIZE, socket_id);
+	if (!vfr_txq) {
+		PMD_DRV_LOG(ERR, "bnxt_vfr_tx_queue allocation failed!");
+		return -ENOMEM;
+	}
+	txq = rte_zmalloc_socket("bnxt_tx_queue",
+				 sizeof(struct bnxt_tx_queue),
+				 RTE_CACHE_LINE_SIZE, socket_id);
+	if (!txq) {
+		PMD_DRV_LOG(ERR, "bnxt_tx_queue allocation failed!");
+		rte_free(vfr_txq);
+		return -ENOMEM;
+	}
+
+	txq->nb_tx_desc = nb_desc;
+	txq->queue_id = queue_idx;
+	txq->port_id = eth_dev->data->port_id;
+	vfr_txq->txq = txq;
+	vfr_txq->bp = rep_bp;
+	eth_dev->data->tx_queues[queue_idx] = vfr_txq;
+
+	return 0;
+}
+
+void bnxt_vf_rep_tx_queue_release_op(void *tx_queue)
+{
+	struct bnxt_vf_rep_tx_queue *vfr_txq = tx_queue;
+
+	if (!vfr_txq)
+		return;
+
+	rte_free(vfr_txq->txq);
+	rte_free(vfr_txq);
+}
+
+int bnxt_vf_rep_stats_get_op(struct rte_eth_dev *eth_dev,
+			     struct rte_eth_stats *stats)
+{
+	struct bnxt_vf_representor *rep_bp = eth_dev->data->dev_private;
+	int i;
+
+	memset(stats, 0, sizeof(*stats));
+	for (i = 0; i < BNXT_MAX_VF_REP_RINGS; i++) {
+		stats->obytes += rep_bp->tx_bytes[i];
+		stats->opackets += rep_bp->tx_pkts[i];
+		stats->ibytes += rep_bp->rx_bytes[i];
+		stats->ipackets += rep_bp->rx_pkts[i];
+		stats->imissed += rep_bp->rx_drop_pkts[i];
+
+		stats->q_ipackets[i] = rep_bp->rx_pkts[i];
+		stats->q_ibytes[i] = rep_bp->rx_bytes[i];
+		stats->q_opackets[i] = rep_bp->tx_pkts[i];
+		stats->q_obytes[i] = rep_bp->tx_bytes[i];
+		stats->q_errors[i] = rep_bp->rx_drop_pkts[i];
+	}
+
+	return 0;
+}
+
+int bnxt_vf_rep_stats_reset_op(struct rte_eth_dev *eth_dev)
+{
+	struct bnxt_vf_representor *rep_bp = eth_dev->data->dev_private;
+	int i;
+
+	for (i = 0; i < BNXT_MAX_VF_REP_RINGS; i++) {
+		rep_bp->tx_pkts[i] = 0;
+		rep_bp->tx_bytes[i] = 0;
+		rep_bp->rx_pkts[i] = 0;
+		rep_bp->rx_bytes[i] = 0;
+		rep_bp->rx_drop_pkts[i] = 0;
+	}
 	return 0;
 }
diff --git a/drivers/net/bnxt/bnxt_reps.h b/drivers/net/bnxt/bnxt_reps.h
index f4c033a..c8a3c7d 100644
--- a/drivers/net/bnxt/bnxt_reps.h
+++ b/drivers/net/bnxt/bnxt_reps.h
@@ -9,6 +9,12 @@
 #include <rte_malloc.h>
 #include <rte_ethdev.h>
 
+#define BNXT_MAX_CFA_CODE               65536
+#define BNXT_VF_IDX_INVALID             0xffff
+
+uint16_t
+bnxt_vfr_recv(struct bnxt *bp, uint16_t cfa_code, uint16_t queue_id,
+	      struct rte_mbuf *mbuf);
 int bnxt_vf_representor_init(struct rte_eth_dev *eth_dev, void *params);
 int bnxt_vf_representor_uninit(struct rte_eth_dev *eth_dev);
 int bnxt_vf_rep_dev_info_get_op(struct rte_eth_dev *eth_dev,
@@ -30,6 +36,11 @@ int bnxt_vf_rep_tx_queue_setup_op(struct rte_eth_dev *eth_dev,
 				  __rte_unused unsigned int socket_id,
 				  __rte_unused const struct rte_eth_txconf *
 				  tx_conf);
+void bnxt_vf_rep_rx_queue_release_op(void *rx_queue);
+void bnxt_vf_rep_tx_queue_release_op(void *tx_queue);
 void bnxt_vf_rep_dev_stop_op(struct rte_eth_dev *eth_dev);
 void bnxt_vf_rep_dev_close_op(struct rte_eth_dev *eth_dev);
+int bnxt_vf_rep_stats_get_op(struct rte_eth_dev *eth_dev,
+			     struct rte_eth_stats *stats);
+int bnxt_vf_rep_stats_reset_op(struct rte_eth_dev *eth_dev);
 #endif /* _BNXT_REPS_H_ */
diff --git a/drivers/net/bnxt/bnxt_rxr.c b/drivers/net/bnxt/bnxt_rxr.c
index 3bcc624..0ecf199 100644
--- a/drivers/net/bnxt/bnxt_rxr.c
+++ b/drivers/net/bnxt/bnxt_rxr.c
@@ -12,6 +12,7 @@
 #include <rte_memory.h>
 
 #include "bnxt.h"
+#include "bnxt_reps.h"
 #include "bnxt_ring.h"
 #include "bnxt_rxr.h"
 #include "bnxt_rxq.h"
@@ -538,7 +539,7 @@ void bnxt_set_mark_in_mbuf(struct bnxt *bp,
 }
 
 static int bnxt_rx_pkt(struct rte_mbuf **rx_pkt,
-			    struct bnxt_rx_queue *rxq, uint32_t *raw_cons)
+		       struct bnxt_rx_queue *rxq, uint32_t *raw_cons)
 {
 	struct bnxt_cp_ring_info *cpr = rxq->cp_ring;
 	struct bnxt_rx_ring_info *rxr = rxq->rx_ring;
@@ -734,6 +735,20 @@ static int bnxt_rx_pkt(struct rte_mbuf **rx_pkt,
 rx:
 	*rx_pkt = mbuf;
 
+	if ((BNXT_VF_IS_TRUSTED(rxq->bp) || BNXT_PF(rxq->bp)) &&
+	    rxq->bp->cfa_code_map && rxcmp1->cfa_code) {
+		if (!bnxt_vfr_recv(rxq->bp, rxcmp1->cfa_code, rxq->queue_id,
+				   mbuf)) {
+			/* Now return an error so that nb_rx_pkts is not
+			 * incremented.
+			 * This packet was meant to be given to the representor.
+			 * So no need to account the packet and give it to
+			 * parent Rx burst function.
+			 */
+			rc = -ENODEV;
+		}
+	}
+
 next_rx:
 
 	*raw_cons = tmp_raw_cons;
@@ -750,6 +765,7 @@ uint16_t bnxt_recv_pkts(void *rx_queue, struct rte_mbuf **rx_pkts,
 	uint32_t raw_cons = cpr->cp_raw_cons;
 	uint32_t cons;
 	int nb_rx_pkts = 0;
+	int nb_rep_rx_pkts = 0;
 	struct rx_pkt_cmpl *rxcmp;
 	uint16_t prod = rxr->rx_prod;
 	uint16_t ag_prod = rxr->ag_prod;
@@ -783,6 +799,8 @@ uint16_t bnxt_recv_pkts(void *rx_queue, struct rte_mbuf **rx_pkts,
 				nb_rx_pkts++;
 			if (rc == -EBUSY)	/* partial completion */
 				break;
+			if (rc == -ENODEV)	/* completion for representor */
+				nb_rep_rx_pkts++;
 		} else if (!BNXT_NUM_ASYNC_CPR(rxq->bp)) {
 			evt =
 			bnxt_event_hwrm_resp_handler(rxq->bp,
@@ -801,7 +819,7 @@ uint16_t bnxt_recv_pkts(void *rx_queue, struct rte_mbuf **rx_pkts,
 	}
 
 	cpr->cp_raw_cons = raw_cons;
-	if (!nb_rx_pkts && !evt) {
+	if (!nb_rx_pkts && !nb_rep_rx_pkts && !evt) {
 		/*
 		 * For PMD, there is no need to keep on pushing to REARM
 		 * the doorbell if there are no new completions
diff --git a/drivers/net/bnxt/bnxt_rxr.h b/drivers/net/bnxt/bnxt_rxr.h
index 811dcd8..e60c97f 100644
--- a/drivers/net/bnxt/bnxt_rxr.h
+++ b/drivers/net/bnxt/bnxt_rxr.h
@@ -188,6 +188,7 @@ struct bnxt_sw_rx_bd {
 struct bnxt_rx_ring_info {
 	uint16_t		rx_prod;
 	uint16_t		ag_prod;
+	uint16_t                rx_cons; /* Needed for representor */
 	struct bnxt_db_info     rx_db;
 	struct bnxt_db_info     ag_db;
 
diff --git a/drivers/net/bnxt/bnxt_txq.h b/drivers/net/bnxt/bnxt_txq.h
index 37a3f95..69ff89a 100644
--- a/drivers/net/bnxt/bnxt_txq.h
+++ b/drivers/net/bnxt/bnxt_txq.h
@@ -29,6 +29,7 @@ struct bnxt_tx_queue {
 	struct bnxt		*bp;
 	int			index;
 	int			tx_wake_thresh;
+	uint32_t                tx_cfa_action;
 	struct bnxt_tx_ring_info	*tx_ring;
 
 	unsigned int		cp_nr_rings;
diff --git a/drivers/net/bnxt/bnxt_txr.c b/drivers/net/bnxt/bnxt_txr.c
index 1602140..d7e193d 100644
--- a/drivers/net/bnxt/bnxt_txr.c
+++ b/drivers/net/bnxt/bnxt_txr.c
@@ -131,7 +131,7 @@ static uint16_t bnxt_start_xmit(struct rte_mbuf *tx_pkt,
 				PKT_TX_VLAN_PKT | PKT_TX_OUTER_IP_CKSUM |
 				PKT_TX_TUNNEL_GRE | PKT_TX_TUNNEL_VXLAN |
 				PKT_TX_TUNNEL_GENEVE | PKT_TX_IEEE1588_TMST |
-				PKT_TX_QINQ_PKT))
+				PKT_TX_QINQ_PKT) || txq->tx_cfa_action)
 		long_bd = true;
 
 	nr_bds = long_bd + tx_pkt->nb_segs;
@@ -184,7 +184,7 @@ static uint16_t bnxt_start_xmit(struct rte_mbuf *tx_pkt,
 	if (long_bd) {
 		txbd->flags_type |= TX_BD_LONG_TYPE_TX_BD_LONG;
 		vlan_tag_flags = 0;
-		cfa_action = 0;
+		cfa_action = txq->tx_cfa_action;
 		/* HW can accelerate only outer vlan in QinQ mode */
 		if (tx_buf->mbuf->ol_flags & PKT_TX_QINQ_PKT) {
 			vlan_tag_flags = TX_BD_LONG_CFA_META_KEY_VLAN_TAG |
-- 
2.7.4


^ permalink raw reply	[flat|nested] 271+ messages in thread

* [dpdk-dev] [PATCH 03/50] net/bnxt: add support to get FID, default vnic ID and svif of VF-Rep Endpoint
  2020-06-12 13:28 [dpdk-dev] [PATCH 00/50] add features for host-based flow management Somnath Kotur
  2020-06-12 13:28 ` [dpdk-dev] [PATCH 01/50] net/bnxt: Basic infrastructure support for VF representors Somnath Kotur
  2020-06-12 13:28 ` [dpdk-dev] [PATCH 02/50] net/bnxt: Infrastructure support for VF-reps data path Somnath Kotur
@ 2020-06-12 13:28 ` Somnath Kotur
  2020-06-12 13:28 ` [dpdk-dev] [PATCH 04/50] net/bnxt: initialize parent PF information Somnath Kotur
                   ` (47 subsequent siblings)
  50 siblings, 0 replies; 271+ messages in thread
From: Somnath Kotur @ 2020-06-12 13:28 UTC (permalink / raw)
  To: dev; +Cc: ferruh.yigit

Use 'first_vf_id' and the 'vf_id' that is input as part of adding
a representor to obtain the PCI function ID(FID) of the VF(VFR endpoint).
Use the FID as an input to FUNC_QCFG HWRM cmd to obtain the default
vnic ID of the VF.
Along with getting the default vNIC ID by supplying the FW FID of
the VF-rep endpoint to HWRM_FUNC_QCFG, obtain and store it's
function svif.

Reviewed-by: Ajit Kumar Khaparde <ajit.khaparde@broadcom.com>
Signed-off-by: Somnath Kotur <somnath.kotur@broadcom.com>
Signed-off-by: Venkat Duvvuru <venkatkumar.duvvuru@broadcom.com>
---
 drivers/net/bnxt/bnxt.h      |  3 +++
 drivers/net/bnxt/bnxt_hwrm.c | 27 +++++++++++++++++++++++++++
 drivers/net/bnxt/bnxt_hwrm.h |  4 ++++
 drivers/net/bnxt/bnxt_reps.c | 12 ++++++++++++
 4 files changed, 46 insertions(+)

diff --git a/drivers/net/bnxt/bnxt.h b/drivers/net/bnxt/bnxt.h
index 443d9fe..7afbd5c 100644
--- a/drivers/net/bnxt/bnxt.h
+++ b/drivers/net/bnxt/bnxt.h
@@ -784,6 +784,9 @@ struct bnxt {
 struct bnxt_vf_representor {
 	uint16_t		switch_domain_id;
 	uint16_t		vf_id;
+	uint16_t		fw_fid;
+	uint16_t		dflt_vnic_id;
+	uint16_t		svif;
 	uint16_t		tx_cfa_action;
 	uint16_t		rx_cfa_code;
 	/* Private data store of associated PF/Trusted VF */
diff --git a/drivers/net/bnxt/bnxt_hwrm.c b/drivers/net/bnxt/bnxt_hwrm.c
index 945bc90..ed42e58 100644
--- a/drivers/net/bnxt/bnxt_hwrm.c
+++ b/drivers/net/bnxt/bnxt_hwrm.c
@@ -3094,6 +3094,33 @@ int bnxt_hwrm_func_qcfg(struct bnxt *bp, uint16_t *mtu)
 	return rc;
 }
 
+int bnxt_hwrm_get_dflt_vnic_svif(struct bnxt *bp, uint16_t fid,
+				 uint16_t *vnic_id, uint16_t *svif)
+{
+	struct hwrm_func_qcfg_input req = {0};
+	struct hwrm_func_qcfg_output *resp = bp->hwrm_cmd_resp_addr;
+	uint16_t svif_info;
+	int rc = 0;
+
+	HWRM_PREP(&req, HWRM_FUNC_QCFG, BNXT_USE_CHIMP_MB);
+	req.fid = rte_cpu_to_le_16(fid);
+
+	rc = bnxt_hwrm_send_message(bp, &req, sizeof(req), BNXT_USE_CHIMP_MB);
+
+	HWRM_CHECK_RESULT();
+
+	if (vnic_id)
+		*vnic_id = rte_le_to_cpu_16(resp->dflt_vnic_id);
+
+	svif_info = rte_le_to_cpu_16(resp->svif_info);
+	if (svif && (svif_info & HWRM_FUNC_QCFG_OUTPUT_SVIF_INFO_SVIF_VALID))
+		*svif = svif_info & HWRM_FUNC_QCFG_OUTPUT_SVIF_INFO_SVIF_MASK;
+
+	HWRM_UNLOCK();
+
+	return rc;
+}
+
 int bnxt_hwrm_port_mac_qcfg(struct bnxt *bp)
 {
 	struct hwrm_port_mac_qcfg_input req = {0};
diff --git a/drivers/net/bnxt/bnxt_hwrm.h b/drivers/net/bnxt/bnxt_hwrm.h
index 58b414d..8d19998 100644
--- a/drivers/net/bnxt/bnxt_hwrm.h
+++ b/drivers/net/bnxt/bnxt_hwrm.h
@@ -270,4 +270,8 @@ int bnxt_hwrm_cfa_counter_qstats(struct bnxt *bp,
 				 enum bnxt_flow_dir dir,
 				 uint16_t cntr,
 				 uint16_t num_entries);
+int bnxt_hwrm_get_dflt_vnic_id(struct bnxt *bp, uint16_t fid,
+			       uint16_t *vnic_id);
+int bnxt_hwrm_get_dflt_vnic_svif(struct bnxt *bp, uint16_t fid,
+				 uint16_t *vnic_id, uint16_t *svif);
 #endif
diff --git a/drivers/net/bnxt/bnxt_reps.c b/drivers/net/bnxt/bnxt_reps.c
index 39f3d23..b6964ab 100644
--- a/drivers/net/bnxt/bnxt_reps.c
+++ b/drivers/net/bnxt/bnxt_reps.c
@@ -150,6 +150,7 @@ int bnxt_vf_representor_init(struct rte_eth_dev *eth_dev, void *params)
 				 (struct bnxt_vf_representor *)params;
 	struct rte_eth_link *link;
 	struct bnxt *parent_bp;
+	int rc = 0;
 
 	vf_rep_bp->vf_id = rep_params->vf_id;
 	vf_rep_bp->switch_domain_id = rep_params->switch_domain_id;
@@ -179,6 +180,17 @@ int bnxt_vf_representor_init(struct rte_eth_dev *eth_dev, void *params)
 	eth_dev->data->dev_link.link_status = link->link_status;
 	eth_dev->data->dev_link.link_autoneg = link->link_autoneg;
 
+	vf_rep_bp->fw_fid = rep_params->vf_id + parent_bp->first_vf_id;
+	PMD_DRV_LOG(INFO, "vf_rep->fw_fid = %d\n", vf_rep_bp->fw_fid);
+	rc = bnxt_hwrm_get_dflt_vnic_svif(parent_bp, vf_rep_bp->fw_fid,
+					  &vf_rep_bp->dflt_vnic_id,
+					  &vf_rep_bp->svif);
+	if (rc)
+		PMD_DRV_LOG(ERR, "Failed to get default vnic id of VF\n");
+	else
+		PMD_DRV_LOG(INFO, "vf_rep->dflt_vnic_id = %d\n",
+			    vf_rep_bp->dflt_vnic_id);
+
 	PMD_DRV_LOG(INFO, "calling bnxt_print_link_info\n");
 	bnxt_print_link_info(eth_dev);
 
-- 
2.7.4


^ permalink raw reply	[flat|nested] 271+ messages in thread

* [dpdk-dev] [PATCH 04/50] net/bnxt: initialize parent PF information
  2020-06-12 13:28 [dpdk-dev] [PATCH 00/50] add features for host-based flow management Somnath Kotur
                   ` (2 preceding siblings ...)
  2020-06-12 13:28 ` [dpdk-dev] [PATCH 03/50] net/bnxt: add support to get FID, default vnic ID and svif of VF-Rep Endpoint Somnath Kotur
@ 2020-06-12 13:28 ` Somnath Kotur
  2020-06-12 13:28 ` [dpdk-dev] [PATCH 05/50] net/bnxt: modify ulp_port_db_dev_port_intf_update prototype Somnath Kotur
                   ` (46 subsequent siblings)
  50 siblings, 0 replies; 271+ messages in thread
From: Somnath Kotur @ 2020-06-12 13:28 UTC (permalink / raw)
  To: dev; +Cc: ferruh.yigit

From: Lance Richardson <lance.richardson@broadcom.com>

Add support to query parent PF information (MAC address,
function ID, port ID and default VNIC) from firmware.

Current firmware returns zero for parent default vnic,
a temporary Wh+-specific workaround is included until
that can be fixed.

Signed-off-by: Lance Richardson <lance.richardson@broadcom.com>
Signed-off-by: Venkat Duvvuru <venkatkumar.duvvuru@broadcom.com>
Reviewed-by: Kalesh Anakkur Purayil <kalesh-anakkur.purayil@broadcom.com>
Reviewed-by: Somnath Kotur <somnath.kotur@broadcom.com>
Reviewed-by: Kishore Padmanabha <kishore.padmanabha@broadcom.com>
Signed-off-by: Somnath Kotur <somnath.kotur@broadcom.com>
---
 drivers/net/bnxt/bnxt.h        |  9 +++++++++
 drivers/net/bnxt/bnxt_ethdev.c | 23 +++++++++++++++++++++++
 drivers/net/bnxt/bnxt_hwrm.c   | 42 ++++++++++++++++++++++++++++++++++++++++++
 drivers/net/bnxt/bnxt_hwrm.h   |  1 +
 4 files changed, 75 insertions(+)

diff --git a/drivers/net/bnxt/bnxt.h b/drivers/net/bnxt/bnxt.h
index 7afbd5c..2b87899 100644
--- a/drivers/net/bnxt/bnxt.h
+++ b/drivers/net/bnxt/bnxt.h
@@ -217,6 +217,14 @@ struct bnxt_child_vf_info {
 	bool			persist_stats;
 };
 
+struct bnxt_parent_info {
+#define	BNXT_PF_FID_INVALID	0xFFFF
+	uint16_t		fid;
+	uint16_t		vnic;
+	uint16_t		port_id;
+	uint8_t			mac_addr[RTE_ETHER_ADDR_LEN];
+};
+
 struct bnxt_pf_info {
 #define BNXT_FIRST_PF_FID	1
 #define BNXT_MAX_VFS(bp)	((bp)->pf->max_vfs)
@@ -738,6 +746,7 @@ struct bnxt {
 #define BNXT_OUTER_TPID_BD_SHFT	16
 	uint32_t		outer_tpid_bd;
 	struct bnxt_pf_info	*pf;
+	struct bnxt_parent_info	*parent;
 	uint8_t			vxlan_port_cnt;
 	uint8_t			geneve_port_cnt;
 	uint16_t		vxlan_port;
diff --git a/drivers/net/bnxt/bnxt_ethdev.c b/drivers/net/bnxt/bnxt_ethdev.c
index 4202904..bf018be 100644
--- a/drivers/net/bnxt/bnxt_ethdev.c
+++ b/drivers/net/bnxt/bnxt_ethdev.c
@@ -97,6 +97,7 @@ static const struct rte_pci_id bnxt_pci_id_map[] = {
 #define BNXT_DEVARG_TRUFLOW	"host-based-truflow"
 #define BNXT_DEVARG_FLOW_XSTAT	"flow-xstat"
 #define BNXT_DEVARG_MAX_NUM_KFLOWS  "max-num-kflows"
+
 static const char *const bnxt_dev_args[] = {
 	BNXT_DEVARG_TRUFLOW,
 	BNXT_DEVARG_FLOW_XSTAT,
@@ -173,6 +174,11 @@ uint16_t bnxt_rss_hash_tbl_size(const struct bnxt *bp)
 	return bnxt_rss_ctxts(bp) * BNXT_RSS_ENTRIES_PER_CTX_THOR;
 }
 
+static void bnxt_free_parent_info(struct bnxt *bp)
+{
+	rte_free(bp->parent);
+}
+
 static void bnxt_free_pf_info(struct bnxt *bp)
 {
 	rte_free(bp->pf);
@@ -223,6 +229,16 @@ static void bnxt_free_mem(struct bnxt *bp, bool reconfig)
 	bp->grp_info = NULL;
 }
 
+static int bnxt_alloc_parent_info(struct bnxt *bp)
+{
+	bp->parent = rte_zmalloc("bnxt_parent_info",
+				 sizeof(struct bnxt_parent_info), 0);
+	if (bp->parent == NULL)
+		return -ENOMEM;
+
+	return 0;
+}
+
 static int bnxt_alloc_pf_info(struct bnxt *bp)
 {
 	bp->pf = rte_zmalloc("bnxt_pf_info", sizeof(struct bnxt_pf_info), 0);
@@ -1322,6 +1338,7 @@ static void bnxt_dev_close_op(struct rte_eth_dev *eth_dev)
 	bnxt_free_cos_queues(bp);
 	bnxt_free_link_info(bp);
 	bnxt_free_pf_info(bp);
+	bnxt_free_parent_info(bp);
 
 	eth_dev->dev_ops = NULL;
 	eth_dev->rx_pkt_burst = NULL;
@@ -5210,6 +5227,8 @@ static int bnxt_init_fw(struct bnxt *bp)
 
 	bnxt_hwrm_port_mac_qcfg(bp);
 
+	bnxt_hwrm_parent_pf_qcfg(bp);
+
 	rc = bnxt_hwrm_cfa_adv_flow_mgmt_qcaps(bp);
 	if (rc)
 		return rc;
@@ -5528,6 +5547,10 @@ bnxt_dev_init(struct rte_eth_dev *eth_dev, void *params __rte_unused)
 	if (rc)
 		goto error_free;
 
+	rc = bnxt_alloc_parent_info(bp);
+	if (rc)
+		goto error_free;
+
 	rc = bnxt_alloc_hwrm_resources(bp);
 	if (rc) {
 		PMD_DRV_LOG(ERR,
diff --git a/drivers/net/bnxt/bnxt_hwrm.c b/drivers/net/bnxt/bnxt_hwrm.c
index ed42e58..347e1c7 100644
--- a/drivers/net/bnxt/bnxt_hwrm.c
+++ b/drivers/net/bnxt/bnxt_hwrm.c
@@ -3094,6 +3094,48 @@ int bnxt_hwrm_func_qcfg(struct bnxt *bp, uint16_t *mtu)
 	return rc;
 }
 
+int bnxt_hwrm_parent_pf_qcfg(struct bnxt *bp)
+{
+	struct hwrm_func_qcfg_input req = {0};
+	struct hwrm_func_qcfg_output *resp = bp->hwrm_cmd_resp_addr;
+	int rc;
+
+	if (!BNXT_VF_IS_TRUSTED(bp))
+		return 0;
+
+	if (!bp->parent)
+		return -EINVAL;
+
+	bp->parent->fid = BNXT_PF_FID_INVALID;
+
+	HWRM_PREP(&req, HWRM_FUNC_QCFG, BNXT_USE_CHIMP_MB);
+
+	req.fid = rte_cpu_to_le_16(0xfffe); /* Request parent PF information. */
+
+	rc = bnxt_hwrm_send_message(bp, &req, sizeof(req), BNXT_USE_CHIMP_MB);
+
+	HWRM_CHECK_RESULT();
+
+	memcpy(bp->parent->mac_addr, resp->mac_address, RTE_ETHER_ADDR_LEN);
+	bp->parent->vnic = rte_le_to_cpu_16(resp->dflt_vnic_id);
+	bp->parent->fid = rte_le_to_cpu_16(resp->fid);
+	bp->parent->port_id = rte_le_to_cpu_16(resp->port_id);
+
+	/* FIXME: Temporary workaround - remove when firmware issue is fixed. */
+	if (bp->parent->vnic == 0) {
+		PMD_DRV_LOG(ERR, "Error: parent VNIC unavailable.\n");
+		/* Use hard-coded values appropriate for current Wh+ fw. */
+		if (bp->parent->fid == 2)
+			bp->parent->vnic = 0x100;
+		else
+			bp->parent->vnic = 1;
+	}
+
+	HWRM_UNLOCK();
+
+	return 0;
+}
+
 int bnxt_hwrm_get_dflt_vnic_svif(struct bnxt *bp, uint16_t fid,
 				 uint16_t *vnic_id, uint16_t *svif)
 {
diff --git a/drivers/net/bnxt/bnxt_hwrm.h b/drivers/net/bnxt/bnxt_hwrm.h
index 8d19998..ef89975 100644
--- a/drivers/net/bnxt/bnxt_hwrm.h
+++ b/drivers/net/bnxt/bnxt_hwrm.h
@@ -274,4 +274,5 @@ int bnxt_hwrm_get_dflt_vnic_id(struct bnxt *bp, uint16_t fid,
 			       uint16_t *vnic_id);
 int bnxt_hwrm_get_dflt_vnic_svif(struct bnxt *bp, uint16_t fid,
 				 uint16_t *vnic_id, uint16_t *svif);
+int bnxt_hwrm_parent_pf_qcfg(struct bnxt *bp);
 #endif
-- 
2.7.4


^ permalink raw reply	[flat|nested] 271+ messages in thread

* [dpdk-dev] [PATCH 05/50] net/bnxt: modify ulp_port_db_dev_port_intf_update prototype
  2020-06-12 13:28 [dpdk-dev] [PATCH 00/50] add features for host-based flow management Somnath Kotur
                   ` (3 preceding siblings ...)
  2020-06-12 13:28 ` [dpdk-dev] [PATCH 04/50] net/bnxt: initialize parent PF information Somnath Kotur
@ 2020-06-12 13:28 ` Somnath Kotur
  2020-06-12 13:28 ` [dpdk-dev] [PATCH 06/50] net/bnxt: get port & function related information Somnath Kotur
                   ` (45 subsequent siblings)
  50 siblings, 0 replies; 271+ messages in thread
From: Somnath Kotur @ 2020-06-12 13:28 UTC (permalink / raw)
  To: dev; +Cc: ferruh.yigit

From: Venkat Duvvuru <venkatkumar.duvvuru@broadcom.com>

Modify ulp_port_db_dev_port_intf_update prototype to take
"struct rte_eth_dev *" as the second parameter.

Signed-off-by: Venkat Duvvuru <venkatkumar.duvvuru@broadcom.com>
Reviewed-by: Somnath Kotur <somnath.kotur@broadcom.com>
Reviewed-by: Kishore Padmanabha <kishore.padmanabha@broadcom.com>
Signed-off-by: Somnath Kotur <somnath.kotur@broadcom.com>
---
 drivers/net/bnxt/tf_ulp/bnxt_ulp.c    | 4 ++--
 drivers/net/bnxt/tf_ulp/ulp_port_db.c | 5 +++--
 drivers/net/bnxt/tf_ulp/ulp_port_db.h | 2 +-
 3 files changed, 6 insertions(+), 5 deletions(-)

diff --git a/drivers/net/bnxt/tf_ulp/bnxt_ulp.c b/drivers/net/bnxt/tf_ulp/bnxt_ulp.c
index 0c3c638..c7281ab 100644
--- a/drivers/net/bnxt/tf_ulp/bnxt_ulp.c
+++ b/drivers/net/bnxt/tf_ulp/bnxt_ulp.c
@@ -548,7 +548,7 @@ bnxt_ulp_init(struct bnxt *bp)
 		}
 
 		/* update the port database */
-		rc = ulp_port_db_dev_port_intf_update(bp->ulp_ctx, bp);
+		rc = ulp_port_db_dev_port_intf_update(bp->ulp_ctx, bp->eth_dev);
 		if (rc) {
 			BNXT_TF_DBG(ERR,
 				    "Failed to update port database\n");
@@ -584,7 +584,7 @@ bnxt_ulp_init(struct bnxt *bp)
 	}
 
 	/* update the port database */
-	rc = ulp_port_db_dev_port_intf_update(bp->ulp_ctx, bp);
+	rc = ulp_port_db_dev_port_intf_update(bp->ulp_ctx, bp->eth_dev);
 	if (rc) {
 		BNXT_TF_DBG(ERR, "Failed to update port database\n");
 		goto jump_to_error;
diff --git a/drivers/net/bnxt/tf_ulp/ulp_port_db.c b/drivers/net/bnxt/tf_ulp/ulp_port_db.c
index e3b9242..66b5840 100644
--- a/drivers/net/bnxt/tf_ulp/ulp_port_db.c
+++ b/drivers/net/bnxt/tf_ulp/ulp_port_db.c
@@ -104,10 +104,11 @@ int32_t	ulp_port_db_deinit(struct bnxt_ulp_context *ulp_ctxt)
  * Returns 0 on success or negative number on failure.
  */
 int32_t	ulp_port_db_dev_port_intf_update(struct bnxt_ulp_context *ulp_ctxt,
-					 struct bnxt *bp)
+					 struct rte_eth_dev *eth_dev)
 {
 	struct bnxt_ulp_port_db *port_db;
-	uint32_t port_id = bp->eth_dev->data->port_id;
+	struct bnxt *bp = eth_dev->data->dev_private;
+	uint32_t port_id = eth_dev->data->port_id;
 	uint32_t ifindex;
 	struct ulp_interface_info *intf;
 	int32_t rc;
diff --git a/drivers/net/bnxt/tf_ulp/ulp_port_db.h b/drivers/net/bnxt/tf_ulp/ulp_port_db.h
index 271c29a..929a5a5 100644
--- a/drivers/net/bnxt/tf_ulp/ulp_port_db.h
+++ b/drivers/net/bnxt/tf_ulp/ulp_port_db.h
@@ -71,7 +71,7 @@ int32_t	ulp_port_db_deinit(struct bnxt_ulp_context *ulp_ctxt);
  * Returns 0 on success or negative number on failure.
  */
 int32_t	ulp_port_db_dev_port_intf_update(struct bnxt_ulp_context *ulp_ctxt,
-					 struct bnxt *bp);
+					 struct rte_eth_dev *eth_dev);
 
 /*
  * Api to get the ulp ifindex for a given device port.
-- 
2.7.4


^ permalink raw reply	[flat|nested] 271+ messages in thread

* [dpdk-dev] [PATCH 06/50] net/bnxt: get port & function related information
  2020-06-12 13:28 [dpdk-dev] [PATCH 00/50] add features for host-based flow management Somnath Kotur
                   ` (4 preceding siblings ...)
  2020-06-12 13:28 ` [dpdk-dev] [PATCH 05/50] net/bnxt: modify ulp_port_db_dev_port_intf_update prototype Somnath Kotur
@ 2020-06-12 13:28 ` Somnath Kotur
  2020-06-12 13:28 ` [dpdk-dev] [PATCH 07/50] net/bnxt: add support for bnxt_hwrm_port_phy_qcaps Somnath Kotur
                   ` (44 subsequent siblings)
  50 siblings, 0 replies; 271+ messages in thread
From: Somnath Kotur @ 2020-06-12 13:28 UTC (permalink / raw)
  To: dev; +Cc: ferruh.yigit

From: Venkat Duvvuru <venkatkumar.duvvuru@broadcom.com>

add helper functions to get port & function related information
like parif, physical port id & vport id.

Signed-off-by: Venkat Duvvuru <venkatkumar.duvvuru@broadcom.com>
Reviewed-by: Kalesh Anakkur Purayil <kalesh-anakkur.purayil@broadcom.com>
Reviewed-by: Somnath Kotur <somnath.kotur@broadcom.com>
Reviewed-by: Kishore Padmanabha <kishore.padmanabha@broadcom.com>
Signed-off-by: Somnath Kotur <somnath.kotur@broadcom.com>
---
 drivers/net/bnxt/bnxt.h                  |  8 +++++
 drivers/net/bnxt/bnxt_ethdev.c           | 58 ++++++++++++++++++++++++++++++++
 drivers/net/bnxt/tf_ulp/bnxt_tf_common.h | 10 ++++++
 drivers/net/bnxt/tf_ulp/ulp_port_db.h    | 10 ------
 4 files changed, 76 insertions(+), 10 deletions(-)

diff --git a/drivers/net/bnxt/bnxt.h b/drivers/net/bnxt/bnxt.h
index 2b87899..0bdf8f5 100644
--- a/drivers/net/bnxt/bnxt.h
+++ b/drivers/net/bnxt/bnxt.h
@@ -855,6 +855,9 @@ extern const struct rte_flow_ops bnxt_flow_ops;
 	} \
 } while (0)
 
+#define	BNXT_ETH_DEV_IS_REPRESENTOR(eth_dev)	\
+		((eth_dev)->data->dev_flags & RTE_ETH_DEV_REPRESENTOR)
+
 extern int bnxt_logtype_driver;
 #define PMD_DRV_LOG_RAW(level, fmt, args...) \
 	rte_log(RTE_LOG_ ## level, bnxt_logtype_driver, "%s(): " fmt, \
@@ -870,6 +873,11 @@ void bnxt_ulp_deinit(struct bnxt *bp);
 uint16_t bnxt_get_vnic_id(uint16_t port);
 uint16_t bnxt_get_svif(uint16_t port_id, bool func_svif);
 uint16_t bnxt_get_fw_func_id(uint16_t port);
+uint16_t bnxt_get_parif(uint16_t port);
+uint16_t bnxt_get_phy_port_id(uint16_t port);
+uint16_t bnxt_get_vport(uint16_t port);
+enum bnxt_ulp_intf_type
+bnxt_get_interface_type(uint16_t port);
 
 void bnxt_cancel_fc_thread(struct bnxt *bp);
 void bnxt_flow_cnt_alarm_cb(void *arg);
diff --git a/drivers/net/bnxt/bnxt_ethdev.c b/drivers/net/bnxt/bnxt_ethdev.c
index bf018be..af88b36 100644
--- a/drivers/net/bnxt/bnxt_ethdev.c
+++ b/drivers/net/bnxt/bnxt_ethdev.c
@@ -28,6 +28,7 @@
 #include "bnxt_vnic.h"
 #include "hsi_struct_def_dpdk.h"
 #include "bnxt_nvm_defs.h"
+#include "bnxt_tf_common.h"
 
 #define DRV_MODULE_NAME		"bnxt"
 static const char bnxt_version[] =
@@ -5101,6 +5102,63 @@ bnxt_get_fw_func_id(uint16_t port)
 	return bp->fw_fid;
 }
 
+enum bnxt_ulp_intf_type
+bnxt_get_interface_type(uint16_t port)
+{
+	struct rte_eth_dev *eth_dev;
+	struct bnxt *bp;
+
+	eth_dev = &rte_eth_devices[port];
+	if (BNXT_ETH_DEV_IS_REPRESENTOR(eth_dev))
+		return BNXT_ULP_INTF_TYPE_VF_REP;
+
+	bp = eth_dev->data->dev_private;
+	return BNXT_PF(bp) ? BNXT_ULP_INTF_TYPE_PF
+			   : BNXT_ULP_INTF_TYPE_VF;
+}
+
+uint16_t
+bnxt_get_phy_port_id(uint16_t port_id)
+{
+	struct bnxt_vf_representor *vfr;
+	struct rte_eth_dev *eth_dev;
+	struct bnxt *bp;
+
+	eth_dev = &rte_eth_devices[port_id];
+	if (BNXT_ETH_DEV_IS_REPRESENTOR(eth_dev)) {
+		vfr = eth_dev->data->dev_private;
+		eth_dev = vfr->parent_dev;
+	}
+
+	bp = eth_dev->data->dev_private;
+
+	return BNXT_PF(bp) ? bp->pf->port_id : bp->parent->port_id;
+}
+
+uint16_t
+bnxt_get_parif(uint16_t port_id)
+{
+	struct bnxt_vf_representor *vfr;
+	struct rte_eth_dev *eth_dev;
+	struct bnxt *bp;
+
+	eth_dev = &rte_eth_devices[port_id];
+	if (BNXT_ETH_DEV_IS_REPRESENTOR(eth_dev)) {
+		vfr = eth_dev->data->dev_private;
+		eth_dev = vfr->parent_dev;
+	}
+
+	bp = eth_dev->data->dev_private;
+
+	return BNXT_PF(bp) ? bp->fw_fid - 1 : bp->parent->fid - 1;
+}
+
+uint16_t
+bnxt_get_vport(uint16_t port_id)
+{
+	return (1 << bnxt_get_phy_port_id(port_id));
+}
+
 static void bnxt_alloc_error_recovery_info(struct bnxt *bp)
 {
 	struct bnxt_error_recovery_info *info = bp->recovery_info;
diff --git a/drivers/net/bnxt/tf_ulp/bnxt_tf_common.h b/drivers/net/bnxt/tf_ulp/bnxt_tf_common.h
index f417579..f772d49 100644
--- a/drivers/net/bnxt/tf_ulp/bnxt_tf_common.h
+++ b/drivers/net/bnxt/tf_ulp/bnxt_tf_common.h
@@ -44,6 +44,16 @@ enum ulp_direction_type {
 	ULP_DIR_EGRESS,
 };
 
+/* enumeration of the interface types */
+enum bnxt_ulp_intf_type {
+	BNXT_ULP_INTF_TYPE_INVALID = 0,
+	BNXT_ULP_INTF_TYPE_PF,
+	BNXT_ULP_INTF_TYPE_VF,
+	BNXT_ULP_INTF_TYPE_PF_REP,
+	BNXT_ULP_INTF_TYPE_VF_REP,
+	BNXT_ULP_INTF_TYPE_LAST
+};
+
 struct bnxt_ulp_mark_tbl *
 bnxt_ulp_cntxt_ptr2_mark_db_get(struct bnxt_ulp_context *ulp_ctx);
 
diff --git a/drivers/net/bnxt/tf_ulp/ulp_port_db.h b/drivers/net/bnxt/tf_ulp/ulp_port_db.h
index 929a5a5..604c438 100644
--- a/drivers/net/bnxt/tf_ulp/ulp_port_db.h
+++ b/drivers/net/bnxt/tf_ulp/ulp_port_db.h
@@ -10,16 +10,6 @@
 
 #define BNXT_PORT_DB_MAX_INTF_LIST		256
 
-/* enumeration of the interface types */
-enum bnxt_ulp_intf_type {
-	BNXT_ULP_INTF_TYPE_INVALID = 0,
-	BNXT_ULP_INTF_TYPE_PF = 1,
-	BNXT_ULP_INTF_TYPE_VF,
-	BNXT_ULP_INTF_TYPE_PF_REP,
-	BNXT_ULP_INTF_TYPE_VF_REP,
-	BNXT_ULP_INTF_TYPE_LAST
-};
-
 /* Structure for the Port database resource information. */
 struct ulp_interface_info {
 	enum bnxt_ulp_intf_type	type;
-- 
2.7.4


^ permalink raw reply	[flat|nested] 271+ messages in thread

* [dpdk-dev] [PATCH 07/50] net/bnxt: add support for bnxt_hwrm_port_phy_qcaps
  2020-06-12 13:28 [dpdk-dev] [PATCH 00/50] add features for host-based flow management Somnath Kotur
                   ` (5 preceding siblings ...)
  2020-06-12 13:28 ` [dpdk-dev] [PATCH 06/50] net/bnxt: get port & function related information Somnath Kotur
@ 2020-06-12 13:28 ` Somnath Kotur
  2020-06-12 13:28 ` [dpdk-dev] [PATCH 08/50] net/bnxt: modify port_db to store & retrieve more info Somnath Kotur
                   ` (43 subsequent siblings)
  50 siblings, 0 replies; 271+ messages in thread
From: Somnath Kotur @ 2020-06-12 13:28 UTC (permalink / raw)
  To: dev; +Cc: ferruh.yigit

From: Venkat Duvvuru <venkatkumar.duvvuru@broadcom.com>

Issue HWRM_PORT_PHY_QCAPS to the firmware to get the physical
port count of the device.

Signed-off-by: Venkat Duvvuru <venkatkumar.duvvuru@broadcom.com>
Reviewed-by: Somnath Kotur <somnath.kotur@broadcom.com>
Reviewed-by: Kalesh Anakkur Purayil <kalesh-anakkur.purayil@broadcom.com>
Reviewed-by: Kishore Padmanabha <kishore.padmanabha@broadcom.com>
Signed-off-by: Somnath Kotur <somnath.kotur@broadcom.com>
---
 drivers/net/bnxt/bnxt.h        |  1 +
 drivers/net/bnxt/bnxt_ethdev.c |  2 ++
 drivers/net/bnxt/bnxt_hwrm.c   | 22 ++++++++++++++++++++++
 drivers/net/bnxt/bnxt_hwrm.h   |  1 +
 4 files changed, 26 insertions(+)

diff --git a/drivers/net/bnxt/bnxt.h b/drivers/net/bnxt/bnxt.h
index 0bdf8f5..65862ab 100644
--- a/drivers/net/bnxt/bnxt.h
+++ b/drivers/net/bnxt/bnxt.h
@@ -747,6 +747,7 @@ struct bnxt {
 	uint32_t		outer_tpid_bd;
 	struct bnxt_pf_info	*pf;
 	struct bnxt_parent_info	*parent;
+	uint8_t			port_cnt;
 	uint8_t			vxlan_port_cnt;
 	uint8_t			geneve_port_cnt;
 	uint16_t		vxlan_port;
diff --git a/drivers/net/bnxt/bnxt_ethdev.c b/drivers/net/bnxt/bnxt_ethdev.c
index af88b36..697cd66 100644
--- a/drivers/net/bnxt/bnxt_ethdev.c
+++ b/drivers/net/bnxt/bnxt_ethdev.c
@@ -5287,6 +5287,8 @@ static int bnxt_init_fw(struct bnxt *bp)
 
 	bnxt_hwrm_parent_pf_qcfg(bp);
 
+	bnxt_hwrm_port_phy_qcaps(bp);
+
 	rc = bnxt_hwrm_cfa_adv_flow_mgmt_qcaps(bp);
 	if (rc)
 		return rc;
diff --git a/drivers/net/bnxt/bnxt_hwrm.c b/drivers/net/bnxt/bnxt_hwrm.c
index 347e1c7..e6a28d0 100644
--- a/drivers/net/bnxt/bnxt_hwrm.c
+++ b/drivers/net/bnxt/bnxt_hwrm.c
@@ -1330,6 +1330,28 @@ static int bnxt_hwrm_port_phy_qcfg(struct bnxt *bp,
 	return rc;
 }
 
+int bnxt_hwrm_port_phy_qcaps(struct bnxt *bp)
+{
+	int rc = 0;
+	struct hwrm_port_phy_qcaps_input req = {0};
+	struct hwrm_port_phy_qcaps_output *resp = bp->hwrm_cmd_resp_addr;
+
+	if (BNXT_VF(bp) && !BNXT_VF_IS_TRUSTED(bp))
+		return 0;
+
+	HWRM_PREP(&req, HWRM_PORT_PHY_QCAPS, BNXT_USE_CHIMP_MB);
+
+	rc = bnxt_hwrm_send_message(bp, &req, sizeof(req), BNXT_USE_CHIMP_MB);
+
+	HWRM_CHECK_RESULT();
+
+	bp->port_cnt = resp->port_cnt;
+
+	HWRM_UNLOCK();
+
+	return 0;
+}
+
 static bool bnxt_find_lossy_profile(struct bnxt *bp)
 {
 	int i = 0;
diff --git a/drivers/net/bnxt/bnxt_hwrm.h b/drivers/net/bnxt/bnxt_hwrm.h
index ef89975..87cd407 100644
--- a/drivers/net/bnxt/bnxt_hwrm.h
+++ b/drivers/net/bnxt/bnxt_hwrm.h
@@ -275,4 +275,5 @@ int bnxt_hwrm_get_dflt_vnic_id(struct bnxt *bp, uint16_t fid,
 int bnxt_hwrm_get_dflt_vnic_svif(struct bnxt *bp, uint16_t fid,
 				 uint16_t *vnic_id, uint16_t *svif);
 int bnxt_hwrm_parent_pf_qcfg(struct bnxt *bp);
+int bnxt_hwrm_port_phy_qcaps(struct bnxt *bp);
 #endif
-- 
2.7.4


^ permalink raw reply	[flat|nested] 271+ messages in thread

* [dpdk-dev] [PATCH 08/50] net/bnxt: modify port_db to store & retrieve more info
  2020-06-12 13:28 [dpdk-dev] [PATCH 00/50] add features for host-based flow management Somnath Kotur
                   ` (6 preceding siblings ...)
  2020-06-12 13:28 ` [dpdk-dev] [PATCH 07/50] net/bnxt: add support for bnxt_hwrm_port_phy_qcaps Somnath Kotur
@ 2020-06-12 13:28 ` Somnath Kotur
  2020-06-12 13:28 ` [dpdk-dev] [PATCH 09/50] net/bnxt: add support for Exact Match Somnath Kotur
                   ` (42 subsequent siblings)
  50 siblings, 0 replies; 271+ messages in thread
From: Somnath Kotur @ 2020-06-12 13:28 UTC (permalink / raw)
  To: dev; +Cc: ferruh.yigit

From: Venkat Duvvuru <venkatkumar.duvvuru@broadcom.com>

Apart from func_svif, func_id & vnic, port_db now stores and
retrieves func_spif, func_parif, phy_port_id, port_svif, port_spif,
port_parif, port_vport. New helper functions have been added to
support the same.

Signed-off-by: Venkat Duvvuru <venkatkumar.duvvuru@broadcom.com>
Reviewed-by: Somnath Kotur <somnath.kotur@broadcom.com>
Reviewed-by: Kishore Padmanabha <kishore.padmanabha@broadcom.com>
Signed-off-by: Somnath Kotur <somnath.kotur@broadcom.com>
---
 drivers/net/bnxt/tf_ulp/ulp_port_db.c | 145 ++++++++++++++++++++++++++++------
 drivers/net/bnxt/tf_ulp/ulp_port_db.h |  72 +++++++++++++----
 2 files changed, 179 insertions(+), 38 deletions(-)

diff --git a/drivers/net/bnxt/tf_ulp/ulp_port_db.c b/drivers/net/bnxt/tf_ulp/ulp_port_db.c
index 66b5840..ea27ef4 100644
--- a/drivers/net/bnxt/tf_ulp/ulp_port_db.c
+++ b/drivers/net/bnxt/tf_ulp/ulp_port_db.c
@@ -106,13 +106,12 @@ int32_t	ulp_port_db_deinit(struct bnxt_ulp_context *ulp_ctxt)
 int32_t	ulp_port_db_dev_port_intf_update(struct bnxt_ulp_context *ulp_ctxt,
 					 struct rte_eth_dev *eth_dev)
 {
-	struct bnxt_ulp_port_db *port_db;
-	struct bnxt *bp = eth_dev->data->dev_private;
 	uint32_t port_id = eth_dev->data->port_id;
-	uint32_t ifindex;
+	struct ulp_phy_port_info *port_data;
+	struct bnxt_ulp_port_db *port_db;
 	struct ulp_interface_info *intf;
+	uint32_t ifindex;
 	int32_t rc;
-	struct bnxt_vnic_info *vnic;
 
 	port_db = bnxt_ulp_cntxt_ptr2_port_db_get(ulp_ctxt);
 	if (!port_db) {
@@ -133,22 +132,22 @@ int32_t	ulp_port_db_dev_port_intf_update(struct bnxt_ulp_context *ulp_ctxt,
 
 	/* update the interface details */
 	intf = &port_db->ulp_intf_list[ifindex];
-	if (BNXT_PF(bp) || BNXT_VF(bp)) {
-		if (BNXT_PF(bp)) {
-			intf->type = BNXT_ULP_INTF_TYPE_PF;
-			intf->port_svif = bp->port_svif;
-		} else {
-			intf->type = BNXT_ULP_INTF_TYPE_VF;
-		}
-		intf->func_id = bp->fw_fid;
-		intf->func_svif = bp->func_svif;
-		vnic = BNXT_GET_DEFAULT_VNIC(bp);
-		if (vnic)
-			intf->default_vnic = vnic->fw_vnic_id;
-		intf->bp = bp;
-		memcpy(intf->mac_addr, bp->mac_addr, sizeof(intf->mac_addr));
-	} else {
-		BNXT_TF_DBG(ERR, "Invalid interface type\n");
+
+	intf->type = bnxt_get_interface_type(port_id);
+
+	intf->func_id = bnxt_get_fw_func_id(port_id);
+	intf->func_svif = bnxt_get_svif(port_id, 1);
+	intf->func_spif = bnxt_get_phy_port_id(port_id);
+	intf->func_parif = bnxt_get_parif(port_id);
+	intf->default_vnic = bnxt_get_vnic_id(port_id);
+	intf->phy_port_id = bnxt_get_phy_port_id(port_id);
+
+	if (intf->type == BNXT_ULP_INTF_TYPE_PF) {
+		port_data = &port_db->phy_port_list[intf->phy_port_id];
+		port_data->port_svif = bnxt_get_svif(port_id, 0);
+		port_data->port_spif = bnxt_get_phy_port_id(port_id);
+		port_data->port_parif = bnxt_get_parif(port_id);
+		port_data->port_vport = bnxt_get_vport(port_id);
 	}
 
 	return 0;
@@ -209,7 +208,7 @@ ulp_port_db_function_id_get(struct bnxt_ulp_context *ulp_ctxt,
 }
 
 /*
- * Api to get the svid for a given ulp ifindex.
+ * Api to get the svif for a given ulp ifindex.
  *
  * ulp_ctxt [in] Ptr to ulp context
  * ifindex [in] ulp ifindex
@@ -225,16 +224,88 @@ ulp_port_db_svif_get(struct bnxt_ulp_context *ulp_ctxt,
 		     uint16_t *svif)
 {
 	struct bnxt_ulp_port_db *port_db;
+	uint16_t phy_port_id;
 
 	port_db = bnxt_ulp_cntxt_ptr2_port_db_get(ulp_ctxt);
 	if (!port_db || ifindex >= port_db->ulp_intf_list_size || !ifindex) {
 		BNXT_TF_DBG(ERR, "Invalid Arguments\n");
 		return -EINVAL;
 	}
-	if (dir == ULP_DIR_EGRESS)
+	if (dir == ULP_DIR_EGRESS) {
 		*svif = port_db->ulp_intf_list[ifindex].func_svif;
-	else
-		*svif = port_db->ulp_intf_list[ifindex].port_svif;
+	} else {
+		phy_port_id = port_db->ulp_intf_list[ifindex].phy_port_id;
+		*svif = port_db->phy_port_list[phy_port_id].port_svif;
+	}
+
+	return 0;
+}
+
+/*
+ * Api to get the spif for a given ulp ifindex.
+ *
+ * ulp_ctxt [in] Ptr to ulp context
+ * ifindex [in] ulp ifindex
+ * dir [in] the direction for the flow.
+ * spif [out] the spif of the given ifindex.
+ *
+ * Returns 0 on success or negative number on failure.
+ */
+int32_t
+ulp_port_db_spif_get(struct bnxt_ulp_context *ulp_ctxt,
+		     uint32_t ifindex,
+		     uint32_t dir,
+		     uint16_t *spif)
+{
+	struct bnxt_ulp_port_db *port_db;
+	uint16_t phy_port_id;
+
+	port_db = bnxt_ulp_cntxt_ptr2_port_db_get(ulp_ctxt);
+	if (!port_db || ifindex >= port_db->ulp_intf_list_size || !ifindex) {
+		BNXT_TF_DBG(ERR, "Invalid Arguments\n");
+		return -EINVAL;
+	}
+	if (dir == ULP_DIR_EGRESS) {
+		*spif = port_db->ulp_intf_list[ifindex].func_spif;
+	} else {
+		phy_port_id = port_db->ulp_intf_list[ifindex].phy_port_id;
+		*spif = port_db->phy_port_list[phy_port_id].port_spif;
+	}
+
+	return 0;
+}
+
+/*
+ * Api to get the parif for a given ulp ifindex.
+ *
+ * ulp_ctxt [in] Ptr to ulp context
+ * ifindex [in] ulp ifindex
+ * dir [in] the direction for the flow.
+ * parif [out] the parif of the given ifindex.
+ *
+ * Returns 0 on success or negative number on failure.
+ */
+int32_t
+ulp_port_db_parif_get(struct bnxt_ulp_context *ulp_ctxt,
+		     uint32_t ifindex,
+		     uint32_t dir,
+		     uint16_t *parif)
+{
+	struct bnxt_ulp_port_db *port_db;
+	uint16_t phy_port_id;
+
+	port_db = bnxt_ulp_cntxt_ptr2_port_db_get(ulp_ctxt);
+	if (!port_db || ifindex >= port_db->ulp_intf_list_size || !ifindex) {
+		BNXT_TF_DBG(ERR, "Invalid Arguments\n");
+		return -EINVAL;
+	}
+	if (dir == ULP_DIR_EGRESS) {
+		*parif = port_db->ulp_intf_list[ifindex].func_parif;
+	} else {
+		phy_port_id = port_db->ulp_intf_list[ifindex].phy_port_id;
+		*parif = port_db->phy_port_list[phy_port_id].port_parif;
+	}
+
 	return 0;
 }
 
@@ -262,3 +333,29 @@ ulp_port_db_default_vnic_get(struct bnxt_ulp_context *ulp_ctxt,
 	*vnic = port_db->ulp_intf_list[ifindex].default_vnic;
 	return 0;
 }
+
+/*
+ * Api to get the vport id for a given ulp ifindex.
+ *
+ * ulp_ctxt [in] Ptr to ulp context
+ * ifindex [in] ulp ifindex
+ * vport [out] the port of the given ifindex.
+ *
+ * Returns 0 on success or negative number on failure.
+ */
+int32_t
+ulp_port_db_vport_get(struct bnxt_ulp_context *ulp_ctxt,
+		      uint32_t ifindex, uint16_t *vport)
+{
+	struct bnxt_ulp_port_db *port_db;
+	uint16_t phy_port_id;
+
+	port_db = bnxt_ulp_cntxt_ptr2_port_db_get(ulp_ctxt);
+	if (!port_db || ifindex >= port_db->ulp_intf_list_size || !ifindex) {
+		BNXT_TF_DBG(ERR, "Invalid Arguments\n");
+		return -EINVAL;
+	}
+	phy_port_id = port_db->ulp_intf_list[ifindex].phy_port_id;
+	*vport = port_db->phy_port_list[phy_port_id].port_vport;
+	return 0;
+}
diff --git a/drivers/net/bnxt/tf_ulp/ulp_port_db.h b/drivers/net/bnxt/tf_ulp/ulp_port_db.h
index 604c438..87de3bc 100644
--- a/drivers/net/bnxt/tf_ulp/ulp_port_db.h
+++ b/drivers/net/bnxt/tf_ulp/ulp_port_db.h
@@ -15,11 +15,17 @@ struct ulp_interface_info {
 	enum bnxt_ulp_intf_type	type;
 	uint16_t		func_id;
 	uint16_t		func_svif;
-	uint16_t		port_svif;
+	uint16_t		func_spif;
+	uint16_t		func_parif;
 	uint16_t		default_vnic;
-	uint8_t			mac_addr[RTE_ETHER_ADDR_LEN];
-	/* back pointer to the bnxt driver, it is null for rep ports */
-	struct bnxt		*bp;
+	uint16_t		phy_port_id;
+};
+
+struct ulp_phy_port_info {
+	uint16_t	port_svif;
+	uint16_t	port_spif;
+	uint16_t	port_parif;
+	uint16_t	port_vport;
 };
 
 /* Structure for the Port database */
@@ -29,6 +35,7 @@ struct bnxt_ulp_port_db {
 
 	/* dpdk device external port list */
 	uint16_t			dev_port_list[RTE_MAX_ETHPORTS];
+	struct ulp_phy_port_info	phy_port_list[RTE_MAX_ETHPORTS];
 };
 
 /*
@@ -74,8 +81,7 @@ int32_t	ulp_port_db_dev_port_intf_update(struct bnxt_ulp_context *ulp_ctxt,
  */
 int32_t
 ulp_port_db_dev_port_to_ulp_index(struct bnxt_ulp_context *ulp_ctxt,
-				  uint32_t port_id,
-				  uint32_t *ifindex);
+				  uint32_t port_id, uint32_t *ifindex);
 
 /*
  * Api to get the function id for a given ulp ifindex.
@@ -88,11 +94,10 @@ ulp_port_db_dev_port_to_ulp_index(struct bnxt_ulp_context *ulp_ctxt,
  */
 int32_t
 ulp_port_db_function_id_get(struct bnxt_ulp_context *ulp_ctxt,
-			    uint32_t ifindex,
-			    uint16_t *func_id);
+			    uint32_t ifindex, uint16_t *func_id);
 
 /*
- * Api to get the svid for a given ulp ifindex.
+ * Api to get the svif for a given ulp ifindex.
  *
  * ulp_ctxt [in] Ptr to ulp context
  * ifindex [in] ulp ifindex
@@ -103,9 +108,36 @@ ulp_port_db_function_id_get(struct bnxt_ulp_context *ulp_ctxt,
  */
 int32_t
 ulp_port_db_svif_get(struct bnxt_ulp_context *ulp_ctxt,
-		     uint32_t ifindex,
-		     uint32_t dir,
-		     uint16_t *svif);
+		     uint32_t ifindex, uint32_t dir, uint16_t *svif);
+
+/*
+ * Api to get the spif for a given ulp ifindex.
+ *
+ * ulp_ctxt [in] Ptr to ulp context
+ * ifindex [in] ulp ifindex
+ * dir [in] the direction for the flow.
+ * spif [out] the spif of the given ifindex.
+ *
+ * Returns 0 on success or negative number on failure.
+ */
+int32_t
+ulp_port_db_spif_get(struct bnxt_ulp_context *ulp_ctxt,
+		     uint32_t ifindex, uint32_t dir, uint16_t *spif);
+
+
+/*
+ * Api to get the parif for a given ulp ifindex.
+ *
+ * ulp_ctxt [in] Ptr to ulp context
+ * ifindex [in] ulp ifindex
+ * dir [in] the direction for the flow.
+ * parif [out] the parif of the given ifindex.
+ *
+ * Returns 0 on success or negative number on failure.
+ */
+int32_t
+ulp_port_db_parif_get(struct bnxt_ulp_context *ulp_ctxt,
+		      uint32_t ifindex, uint32_t dir, uint16_t *parif);
 
 /*
  * Api to get the vnic id for a given ulp ifindex.
@@ -118,7 +150,19 @@ ulp_port_db_svif_get(struct bnxt_ulp_context *ulp_ctxt,
  */
 int32_t
 ulp_port_db_default_vnic_get(struct bnxt_ulp_context *ulp_ctxt,
-			     uint32_t ifindex,
-			     uint16_t *vnic);
+			     uint32_t ifindex, uint16_t *vnic);
+
+/*
+ * Api to get the vport id for a given ulp ifindex.
+ *
+ * ulp_ctxt [in] Ptr to ulp context
+ * ifindex [in] ulp ifindex
+ * vport [out] the port of the given ifindex.
+ *
+ * Returns 0 on success or negative number on failure.
+ */
+int32_t
+ulp_port_db_vport_get(struct bnxt_ulp_context *ulp_ctxt,
+		      uint32_t ifindex,	uint16_t *vport);
 
 #endif /* _ULP_PORT_DB_H_ */
-- 
2.7.4


^ permalink raw reply	[flat|nested] 271+ messages in thread

* [dpdk-dev] [PATCH 09/50] net/bnxt: add support for Exact Match
  2020-06-12 13:28 [dpdk-dev] [PATCH 00/50] add features for host-based flow management Somnath Kotur
                   ` (7 preceding siblings ...)
  2020-06-12 13:28 ` [dpdk-dev] [PATCH 08/50] net/bnxt: modify port_db to store & retrieve more info Somnath Kotur
@ 2020-06-12 13:28 ` Somnath Kotur
  2020-06-12 13:28 ` [dpdk-dev] [PATCH 10/50] net/bnxt: modify EM insert and delete to use HWRM direct Somnath Kotur
                   ` (41 subsequent siblings)
  50 siblings, 0 replies; 271+ messages in thread
From: Somnath Kotur @ 2020-06-12 13:28 UTC (permalink / raw)
  To: dev; +Cc: ferruh.yigit

From: Pete Spreadborough <peter.spreadborough@broadcom.com>

- Add Exact Match support
- Create EM table pool of memory indices
- Insert exact match internal entry API
- Sends EM internal insert and delete request to firmware

Signed-off-by: Pete Spreadborough <peter.spreadborough@broadcom.com>
Reviewed-by: Randy Schacher <stuart.schacher@broadcom.com>
Reviewed-by: Ajit Kumar Khaparde <ajit.khaparde@broadcom.com>
Signed-off-by: Venkat Duvvuru <venkatkumar.duvvuru@broadcom.com>
---
 drivers/net/bnxt/hsi_struct_def_dpdk.h        | 3095 +++++++++++++++++++++----
 drivers/net/bnxt/tf_core/hwrm_tf.h            |    9 +
 drivers/net/bnxt/tf_core/lookup3.h            |    1 -
 drivers/net/bnxt/tf_core/stack.c              |    8 +
 drivers/net/bnxt/tf_core/stack.h              |   10 +
 drivers/net/bnxt/tf_core/tf_core.c            |  144 +-
 drivers/net/bnxt/tf_core/tf_core.h            |  383 +--
 drivers/net/bnxt/tf_core/tf_em.c              |   98 +-
 drivers/net/bnxt/tf_core/tf_em.h              |   31 +
 drivers/net/bnxt/tf_core/tf_ext_flow_handle.h |   16 +
 drivers/net/bnxt/tf_core/tf_msg.c             |   86 +-
 drivers/net/bnxt/tf_core/tf_msg.h             |   13 +
 drivers/net/bnxt/tf_core/tf_session.h         |   18 +
 drivers/net/bnxt/tf_core/tf_tbl.c             |  123 +-
 drivers/net/bnxt/tf_core/tf_tbl.h             |   57 +-
 drivers/net/bnxt/tf_core/tfp.h                |  123 +-
 16 files changed, 3511 insertions(+), 704 deletions(-)

diff --git a/drivers/net/bnxt/hsi_struct_def_dpdk.h b/drivers/net/bnxt/hsi_struct_def_dpdk.h
index 7e30c9f..e51f42f 100644
--- a/drivers/net/bnxt/hsi_struct_def_dpdk.h
+++ b/drivers/net/bnxt/hsi_struct_def_dpdk.h
@@ -611,6 +611,10 @@ struct cmd_nums {
 	#define HWRM_FUNC_VF_BW_QCFG                      UINT32_C(0x196)
 	/* Queries pf ids belong to specified host(s) */
 	#define HWRM_FUNC_HOST_PF_IDS_QUERY               UINT32_C(0x197)
+	/* Queries extended stats per function */
+	#define HWRM_FUNC_QSTATS_EXT                      UINT32_C(0x198)
+	/* Queries exteded statitics context */
+	#define HWRM_STAT_EXT_CTX_QUERY                   UINT32_C(0x199)
 	/* Experimental */
 	#define HWRM_SELFTEST_QLIST                       UINT32_C(0x200)
 	/* Experimental */
@@ -647,41 +651,49 @@ struct cmd_nums {
 	/* Experimental */
 	#define HWRM_TF_SESSION_ATTACH                    UINT32_C(0x2c7)
 	/* Experimental */
-	#define HWRM_TF_SESSION_CLOSE                     UINT32_C(0x2c8)
+	#define HWRM_TF_SESSION_REGISTER                  UINT32_C(0x2c8)
 	/* Experimental */
-	#define HWRM_TF_SESSION_QCFG                      UINT32_C(0x2c9)
+	#define HWRM_TF_SESSION_UNREGISTER                UINT32_C(0x2c9)
 	/* Experimental */
-	#define HWRM_TF_SESSION_RESC_QCAPS                UINT32_C(0x2ca)
+	#define HWRM_TF_SESSION_CLOSE                     UINT32_C(0x2ca)
 	/* Experimental */
-	#define HWRM_TF_SESSION_RESC_ALLOC                UINT32_C(0x2cb)
+	#define HWRM_TF_SESSION_QCFG                      UINT32_C(0x2cb)
 	/* Experimental */
-	#define HWRM_TF_SESSION_RESC_FREE                 UINT32_C(0x2cc)
+	#define HWRM_TF_SESSION_RESC_QCAPS                UINT32_C(0x2cc)
 	/* Experimental */
-	#define HWRM_TF_SESSION_RESC_FLUSH                UINT32_C(0x2cd)
+	#define HWRM_TF_SESSION_RESC_ALLOC                UINT32_C(0x2cd)
 	/* Experimental */
-	#define HWRM_TF_TBL_TYPE_GET                      UINT32_C(0x2d0)
+	#define HWRM_TF_SESSION_RESC_FREE                 UINT32_C(0x2ce)
 	/* Experimental */
-	#define HWRM_TF_TBL_TYPE_SET                      UINT32_C(0x2d1)
+	#define HWRM_TF_SESSION_RESC_FLUSH                UINT32_C(0x2cf)
 	/* Experimental */
-	#define HWRM_TF_CTXT_MEM_RGTR                     UINT32_C(0x2da)
+	#define HWRM_TF_TBL_TYPE_GET                      UINT32_C(0x2da)
 	/* Experimental */
-	#define HWRM_TF_CTXT_MEM_UNRGTR                   UINT32_C(0x2db)
+	#define HWRM_TF_TBL_TYPE_SET                      UINT32_C(0x2db)
 	/* Experimental */
-	#define HWRM_TF_EXT_EM_QCAPS                      UINT32_C(0x2dc)
+	#define HWRM_TF_CTXT_MEM_RGTR                     UINT32_C(0x2e4)
 	/* Experimental */
-	#define HWRM_TF_EXT_EM_OP                         UINT32_C(0x2dd)
+	#define HWRM_TF_CTXT_MEM_UNRGTR                   UINT32_C(0x2e5)
 	/* Experimental */
-	#define HWRM_TF_EXT_EM_CFG                        UINT32_C(0x2de)
+	#define HWRM_TF_EXT_EM_QCAPS                      UINT32_C(0x2e6)
 	/* Experimental */
-	#define HWRM_TF_EXT_EM_QCFG                       UINT32_C(0x2df)
+	#define HWRM_TF_EXT_EM_OP                         UINT32_C(0x2e7)
 	/* Experimental */
-	#define HWRM_TF_TCAM_SET                          UINT32_C(0x2ee)
+	#define HWRM_TF_EXT_EM_CFG                        UINT32_C(0x2e8)
 	/* Experimental */
-	#define HWRM_TF_TCAM_GET                          UINT32_C(0x2ef)
+	#define HWRM_TF_EXT_EM_QCFG                       UINT32_C(0x2e9)
 	/* Experimental */
-	#define HWRM_TF_TCAM_MOVE                         UINT32_C(0x2f0)
+	#define HWRM_TF_EM_INSERT                         UINT32_C(0x2ea)
 	/* Experimental */
-	#define HWRM_TF_TCAM_FREE                         UINT32_C(0x2f1)
+	#define HWRM_TF_EM_DELETE                         UINT32_C(0x2eb)
+	/* Experimental */
+	#define HWRM_TF_TCAM_SET                          UINT32_C(0x2f8)
+	/* Experimental */
+	#define HWRM_TF_TCAM_GET                          UINT32_C(0x2f9)
+	/* Experimental */
+	#define HWRM_TF_TCAM_MOVE                         UINT32_C(0x2fa)
+	/* Experimental */
+	#define HWRM_TF_TCAM_FREE                         UINT32_C(0x2fb)
 	/* Experimental */
 	#define HWRM_SV                                   UINT32_C(0x400)
 	/* Experimental */
@@ -715,6 +727,13 @@ struct cmd_nums {
 	#define HWRM_DBG_CRASHDUMP_ERASE                  UINT32_C(0xff1e)
 	/* Send driver debug information to firmware */
 	#define HWRM_DBG_DRV_TRACE                        UINT32_C(0xff1f)
+	/* Query debug capabilities of firmware */
+	#define HWRM_DBG_QCAPS                            UINT32_C(0xff20)
+	/* Retrieve debug settings of firmware */
+	#define HWRM_DBG_QCFG                             UINT32_C(0xff21)
+	/* Set destination parameters for crashdump medium */
+	#define HWRM_DBG_CRASHDUMP_MEDIUM_CFG             UINT32_C(0xff22)
+	#define HWRM_NVM_REQ_ARBITRATION                  UINT32_C(0xffed)
 	/* Experimental */
 	#define HWRM_NVM_FACTORY_DEFAULTS                 UINT32_C(0xffee)
 	#define HWRM_NVM_VALIDATE_OPTION                  UINT32_C(0xffef)
@@ -914,8 +933,8 @@ struct hwrm_err_output {
 #define HWRM_VERSION_MINOR 10
 #define HWRM_VERSION_UPDATE 1
 /* non-zero means beta version */
-#define HWRM_VERSION_RSVD 30
-#define HWRM_VERSION_STR "1.10.1.30"
+#define HWRM_VERSION_RSVD 45
+#define HWRM_VERSION_STR "1.10.1.45"
 
 /****************
  * hwrm_ver_get *
@@ -2293,6 +2312,35 @@ struct cmpl_base {
 	 */
 	#define CMPL_BASE_TYPE_TX_L2             UINT32_C(0x0)
 	/*
+	 * NO-OP completion:
+	 * Completion of NO-OP. Length = 16B
+	 */
+	#define CMPL_BASE_TYPE_NO_OP             UINT32_C(0x1)
+	/*
+	 * TX L2 coalesced completion:
+	 * Completion of coalesced TX packet. Length = 16B
+	 */
+	#define CMPL_BASE_TYPE_TX_L2_COAL        UINT32_C(0x2)
+	/*
+	 * TX L2 PTP completion:
+	 * Completion of PTP TX packet. Length = 32B
+	 */
+	#define CMPL_BASE_TYPE_TX_L2_PTP         UINT32_C(0x3)
+	/*
+	 * RX L2 TPA Start V2 Completion:
+	 * Completion of and L2 RX packet. Length = 32B
+	 * This is the new version of the RX_TPA_START completion used
+	 * in SR2 and later chips.
+	 */
+	#define CMPL_BASE_TYPE_RX_TPA_START_V2   UINT32_C(0xd)
+	/*
+	 * RX L2 V2 completion:
+	 * Completion of and L2 RX packet. Length = 32B
+	 * This is the new version of the RX_L2 completion used in SR2
+	 * and later chips.
+	 */
+	#define CMPL_BASE_TYPE_RX_L2_V2          UINT32_C(0xf)
+	/*
 	 * RX L2 completion:
 	 * Completion of and L2 RX packet. Length = 32B
 	 */
@@ -2322,6 +2370,24 @@ struct cmpl_base {
 	 */
 	#define CMPL_BASE_TYPE_STAT_EJECT        UINT32_C(0x1a)
 	/*
+	 * VEE Flush Completion:
+	 * This completion is inserted manually by
+	 * the Primate and processed by the VEE hardware to ensure that
+	 * all completions on a VEE function have been processed by the
+	 * VEE hardware before FLR process is completed.
+	 */
+	#define CMPL_BASE_TYPE_VEE_FLUSH         UINT32_C(0x1c)
+	/*
+	 * Mid Path Short Completion :
+	 * Completion of a Mid Path Command. Length = 16B
+	 */
+	#define CMPL_BASE_TYPE_MID_PATH_SHORT    UINT32_C(0x1e)
+	/*
+	 * Mid Path Long Completion :
+	 * Completion of a Mid Path Command. Length = 32B
+	 */
+	#define CMPL_BASE_TYPE_MID_PATH_LONG     UINT32_C(0x1f)
+	/*
 	 * HWRM Command Completion:
 	 * Completion of an HWRM command.
 	 */
@@ -2398,7 +2464,9 @@ struct tx_cmpl {
 	uint16_t	unused_0;
 	/*
 	 * This is a copy of the opaque field from the first TX BD of this
-	 * transmitted packet.
+	 * transmitted packet. Note that, if the packet was described by a short
+	 * CSO or short CSO inline BD, then the 16-bit opaque field from the
+	 * short CSO BD will appear in the bottom 16 bits of this field.
 	 */
 	uint32_t	opaque;
 	uint16_t	errors_v;
@@ -2407,58 +2475,352 @@ struct tx_cmpl {
 	 * for each pass through the completion queue. The even passes
 	 * will write 1. The odd passes will write 0.
 	 */
-	#define TX_CMPL_V                              UINT32_C(0x1)
-	#define TX_CMPL_ERRORS_MASK                    UINT32_C(0xfffe)
-	#define TX_CMPL_ERRORS_SFT                     1
+	#define TX_CMPL_V                                  UINT32_C(0x1)
+	#define TX_CMPL_ERRORS_MASK                        UINT32_C(0xfffe)
+	#define TX_CMPL_ERRORS_SFT                         1
 	/*
 	 * This error indicates that there was some sort of problem
 	 * with the BDs for the packet.
 	 */
-	#define TX_CMPL_ERRORS_BUFFER_ERROR_MASK        UINT32_C(0xe)
-	#define TX_CMPL_ERRORS_BUFFER_ERROR_SFT         1
+	#define TX_CMPL_ERRORS_BUFFER_ERROR_MASK            UINT32_C(0xe)
+	#define TX_CMPL_ERRORS_BUFFER_ERROR_SFT             1
 	/* No error */
-	#define TX_CMPL_ERRORS_BUFFER_ERROR_NO_ERROR      (UINT32_C(0x0) << 1)
+	#define TX_CMPL_ERRORS_BUFFER_ERROR_NO_ERROR \
+		(UINT32_C(0x0) << 1)
 	/*
 	 * Bad Format:
 	 * BDs were not formatted correctly.
 	 */
-	#define TX_CMPL_ERRORS_BUFFER_ERROR_BAD_FMT       (UINT32_C(0x2) << 1)
+	#define TX_CMPL_ERRORS_BUFFER_ERROR_BAD_FMT \
+		(UINT32_C(0x2) << 1)
 	#define TX_CMPL_ERRORS_BUFFER_ERROR_LAST \
 		TX_CMPL_ERRORS_BUFFER_ERROR_BAD_FMT
 	/*
 	 * When this bit is '1', it indicates that the length of
 	 * the packet was zero. No packet was transmitted.
 	 */
-	#define TX_CMPL_ERRORS_ZERO_LENGTH_PKT          UINT32_C(0x10)
+	#define TX_CMPL_ERRORS_ZERO_LENGTH_PKT              UINT32_C(0x10)
 	/*
 	 * When this bit is '1', it indicates that the packet
 	 * was longer than the programmed limit in TDI. No
 	 * packet was transmitted.
 	 */
-	#define TX_CMPL_ERRORS_EXCESSIVE_BD_LENGTH      UINT32_C(0x20)
+	#define TX_CMPL_ERRORS_EXCESSIVE_BD_LENGTH          UINT32_C(0x20)
 	/*
 	 * When this bit is '1', it indicates that one or more of the
 	 * BDs associated with this packet generated a PCI error.
 	 * This probably means the address was not valid.
 	 */
-	#define TX_CMPL_ERRORS_DMA_ERROR                UINT32_C(0x40)
+	#define TX_CMPL_ERRORS_DMA_ERROR                    UINT32_C(0x40)
 	/*
 	 * When this bit is '1', it indicates that the packet was longer
 	 * than indicated by the hint. No packet was transmitted.
 	 */
-	#define TX_CMPL_ERRORS_HINT_TOO_SHORT           UINT32_C(0x80)
+	#define TX_CMPL_ERRORS_HINT_TOO_SHORT               UINT32_C(0x80)
 	/*
 	 * When this bit is '1', it indicates that the packet was
 	 * dropped due to Poison TLP error on one or more of the
 	 * TLPs in the PXP completion.
 	 */
-	#define TX_CMPL_ERRORS_POISON_TLP_ERROR         UINT32_C(0x100)
+	#define TX_CMPL_ERRORS_POISON_TLP_ERROR             UINT32_C(0x100)
+	/*
+	 * When this bit is '1', it indicates that the packet was dropped
+	 * due to a transient internal error in TDC. The packet or LSO can
+	 * be retried and may transmit successfully on a subsequent attempt.
+	 */
+	#define TX_CMPL_ERRORS_INTERNAL_ERROR               UINT32_C(0x200)
+	/*
+	 * When this bit is '1', it was not possible to collect a a timestamp
+	 * for a PTP completion, in which case the timestamp_hi and
+	 * timestamp_lo fields are invalid. When this bit is '0' for a PTP
+	 * completion, the timestamp_hi and timestamp_lo fields are valid.
+	 * RJRN will copy the value of this bit into the field of the same
+	 * name in all TX completions, regardless of whether such completions
+	 * are PTP completions or other TX completions.
+	 */
+	#define TX_CMPL_ERRORS_TIMESTAMP_INVALID_ERROR      UINT32_C(0x400)
 	/* unused2 is 16 b */
 	uint16_t	unused_1;
 	/* unused3 is 32 b */
 	uint32_t	unused_2;
 } __rte_packed;
 
+/* tx_cmpl_coal (size:128b/16B) */
+struct tx_cmpl_coal {
+	uint16_t	flags_type;
+	/*
+	 * This field indicates the exact type of the completion.
+	 * By convention, the LSB identifies the length of the
+	 * record in 16B units. Even values indicate 16B
+	 * records. Odd values indicate 32B
+	 * records.
+	 */
+	#define TX_CMPL_COAL_TYPE_MASK       UINT32_C(0x3f)
+	#define TX_CMPL_COAL_TYPE_SFT        0
+	/*
+	 * TX L2 coalesced completion:
+	 * Completion of TX packet. Length = 16B
+	 */
+	#define TX_CMPL_COAL_TYPE_TX_L2_COAL   UINT32_C(0x2)
+	#define TX_CMPL_COAL_TYPE_LAST        TX_CMPL_COAL_TYPE_TX_L2_COAL
+	#define TX_CMPL_COAL_FLAGS_MASK      UINT32_C(0xffc0)
+	#define TX_CMPL_COAL_FLAGS_SFT       6
+	/*
+	 * When this bit is '1', it indicates a packet that has an
+	 * error of some type. Type of error is indicated in
+	 * error_flags.
+	 */
+	#define TX_CMPL_COAL_FLAGS_ERROR      UINT32_C(0x40)
+	/*
+	 * When this bit is '1', it indicates that the packet completed
+	 * was transmitted using the push acceleration data provided
+	 * by the driver. When this bit is '0', it indicates that the
+	 * packet had not push acceleration data written or was executed
+	 * as a normal packet even though push data was provided.
+	 */
+	#define TX_CMPL_COAL_FLAGS_PUSH       UINT32_C(0x80)
+	/* unused1 is 16 b */
+	uint16_t	unused_0;
+	/*
+	 * This is a copy of the opaque field from the first TX BD of the packet
+	 * which corresponds with the reported sq_cons_idx. Note that, with
+	 * coalesced completions, completions are generated for only some of the
+	 * packets. The driver will see the opaque field for only those packets.
+	 * Note that, if the packet was described by a short CSO or short CSO
+	 * inline BD, then the 16-bit opaque field from the short CSO BD will
+	 * appear in the bottom 16 bits of this field. For TX rings with
+	 * completion coalescing enabled (which would use the coalesced
+	 * completion record), it is suggested that the driver populate the
+	 * opaque field to indicate the specific TX ring with which the
+	 * completion is associated, then utilize the opaque and sq_cons_idx
+	 * fields in the coalesced completion record to determine the specific
+	 * packets that are to be completed on that ring.
+	 */
+	uint32_t	opaque;
+	uint16_t	errors_v;
+	/*
+	 * This value is written by the NIC such that it will be different
+	 * for each pass through the completion queue. The even passes
+	 * will write 1. The odd passes will write 0.
+	 */
+	#define TX_CMPL_COAL_V                                  UINT32_C(0x1)
+	#define TX_CMPL_COAL_ERRORS_MASK \
+		UINT32_C(0xfffe)
+	#define TX_CMPL_COAL_ERRORS_SFT                         1
+	/*
+	 * This error indicates that there was some sort of problem
+	 * with the BDs for the packet.
+	 */
+	#define TX_CMPL_COAL_ERRORS_BUFFER_ERROR_MASK            UINT32_C(0xe)
+	#define TX_CMPL_COAL_ERRORS_BUFFER_ERROR_SFT             1
+	/* No error */
+	#define TX_CMPL_COAL_ERRORS_BUFFER_ERROR_NO_ERROR \
+		(UINT32_C(0x0) << 1)
+	/*
+	 * Bad Format:
+	 * BDs were not formatted correctly.
+	 */
+	#define TX_CMPL_COAL_ERRORS_BUFFER_ERROR_BAD_FMT \
+		(UINT32_C(0x2) << 1)
+	#define TX_CMPL_COAL_ERRORS_BUFFER_ERROR_LAST \
+		TX_CMPL_COAL_ERRORS_BUFFER_ERROR_BAD_FMT
+	/*
+	 * When this bit is '1', it indicates that the length of
+	 * the packet was zero. No packet was transmitted.
+	 */
+	#define TX_CMPL_COAL_ERRORS_ZERO_LENGTH_PKT              UINT32_C(0x10)
+	/*
+	 * When this bit is '1', it indicates that the packet
+	 * was longer than the programmed limit in TDI. No
+	 * packet was transmitted.
+	 */
+	#define TX_CMPL_COAL_ERRORS_EXCESSIVE_BD_LENGTH          UINT32_C(0x20)
+	/*
+	 * When this bit is '1', it indicates that one or more of the
+	 * BDs associated with this packet generated a PCI error.
+	 * This probably means the address was not valid.
+	 */
+	#define TX_CMPL_COAL_ERRORS_DMA_ERROR                    UINT32_C(0x40)
+	/*
+	 * When this bit is '1', it indicates that the packet was longer
+	 * than indicated by the hint. No packet was transmitted.
+	 */
+	#define TX_CMPL_COAL_ERRORS_HINT_TOO_SHORT               UINT32_C(0x80)
+	/*
+	 * When this bit is '1', it indicates that the packet was
+	 * dropped due to Poison TLP error on one or more of the
+	 * TLPs in the PXP completion.
+	 */
+	#define TX_CMPL_COAL_ERRORS_POISON_TLP_ERROR \
+		UINT32_C(0x100)
+	/*
+	 * When this bit is '1', it indicates that the packet was dropped
+	 * due to a transient internal error in TDC. The packet or LSO can
+	 * be retried and may transmit successfully on a subsequent attempt.
+	 */
+	#define TX_CMPL_COAL_ERRORS_INTERNAL_ERROR \
+		UINT32_C(0x200)
+	/*
+	 * When this bit is '1', it was not possible to collect a a timestamp
+	 * for a PTP completion, in which case the timestamp_hi and
+	 * timestamp_lo fields are invalid. When this bit is '0' for a PTP
+	 * completion, the timestamp_hi and timestamp_lo fields are valid.
+	 * RJRN will copy the value of this bit into the field of the same
+	 * name in all TX completions, regardless of whether such
+	 * completions are PTP completions or other TX completions.
+	 */
+	#define TX_CMPL_COAL_ERRORS_TIMESTAMP_INVALID_ERROR \
+		UINT32_C(0x400)
+	/* unused2 is 16 b */
+	uint16_t	unused_1;
+	uint32_t	sq_cons_idx;
+	/*
+	 * This value is SQ index for the start of the packet following the
+	 * last completed packet.
+	 */
+	#define TX_CMPL_COAL_SQ_CONS_IDX_MASK UINT32_C(0xffffff)
+	#define TX_CMPL_COAL_SQ_CONS_IDX_SFT 0
+} __rte_packed;
+
+/* tx_cmpl_ptp (size:128b/16B) */
+struct tx_cmpl_ptp {
+	uint16_t	flags_type;
+	/*
+	 * This field indicates the exact type of the completion.
+	 * By convention, the LSB identifies the length of the
+	 * record in 16B units. Even values indicate 16B
+	 * records. Odd values indicate 32B
+	 * records.
+	 */
+	#define TX_CMPL_PTP_TYPE_MASK       UINT32_C(0x3f)
+	#define TX_CMPL_PTP_TYPE_SFT        0
+	/*
+	 * TX L2 PTP completion:
+	 * Completion of TX packet. Length = 32B
+	 */
+	#define TX_CMPL_PTP_TYPE_TX_L2_PTP    UINT32_C(0x2)
+	#define TX_CMPL_PTP_TYPE_LAST        TX_CMPL_PTP_TYPE_TX_L2_PTP
+	#define TX_CMPL_PTP_FLAGS_MASK      UINT32_C(0xffc0)
+	#define TX_CMPL_PTP_FLAGS_SFT       6
+	/*
+	 * When this bit is '1', it indicates a packet that has an
+	 * error of some type. Type of error is indicated in
+	 * error_flags.
+	 */
+	#define TX_CMPL_PTP_FLAGS_ERROR      UINT32_C(0x40)
+	/*
+	 * When this bit is '1', it indicates that the packet completed
+	 * was transmitted using the push acceleration data provided
+	 * by the driver. When this bit is '0', it indicates that the
+	 * packet had not push acceleration data written or was executed
+	 * as a normal packet even though push data was provided.
+	 */
+	#define TX_CMPL_PTP_FLAGS_PUSH       UINT32_C(0x80)
+	/* unused1 is 16 b */
+	uint16_t	unused_0;
+	/*
+	 * This is a copy of the opaque field from the first TX BD of this
+	 * transmitted packet. Note that, if the packet was described by a short
+	 * CSO or short CSO inline BD, then the 16-bit opaque field from the
+	 * short CSO BD will appear in the bottom 16 bits of this field.
+	 */
+	uint32_t	opaque;
+	uint16_t	errors_v;
+	/*
+	 * This value is written by the NIC such that it will be different
+	 * for each pass through the completion queue. The even passes
+	 * will write 1. The odd passes will write 0.
+	 */
+	#define TX_CMPL_PTP_V                                  UINT32_C(0x1)
+	#define TX_CMPL_PTP_ERRORS_MASK                        UINT32_C(0xfffe)
+	#define TX_CMPL_PTP_ERRORS_SFT                         1
+	/*
+	 * This error indicates that there was some sort of problem
+	 * with the BDs for the packet.
+	 */
+	#define TX_CMPL_PTP_ERRORS_BUFFER_ERROR_MASK            UINT32_C(0xe)
+	#define TX_CMPL_PTP_ERRORS_BUFFER_ERROR_SFT             1
+	/* No error */
+	#define TX_CMPL_PTP_ERRORS_BUFFER_ERROR_NO_ERROR \
+		(UINT32_C(0x0) << 1)
+	/*
+	 * Bad Format:
+	 * BDs were not formatted correctly.
+	 */
+	#define TX_CMPL_PTP_ERRORS_BUFFER_ERROR_BAD_FMT \
+		(UINT32_C(0x2) << 1)
+	#define TX_CMPL_PTP_ERRORS_BUFFER_ERROR_LAST \
+		TX_CMPL_PTP_ERRORS_BUFFER_ERROR_BAD_FMT
+	/*
+	 * When this bit is '1', it indicates that the length of
+	 * the packet was zero. No packet was transmitted.
+	 */
+	#define TX_CMPL_PTP_ERRORS_ZERO_LENGTH_PKT              UINT32_C(0x10)
+	/*
+	 * When this bit is '1', it indicates that the packet
+	 * was longer than the programmed limit in TDI. No
+	 * packet was transmitted.
+	 */
+	#define TX_CMPL_PTP_ERRORS_EXCESSIVE_BD_LENGTH          UINT32_C(0x20)
+	/*
+	 * When this bit is '1', it indicates that one or more of the
+	 * BDs associated with this packet generated a PCI error.
+	 * This probably means the address was not valid.
+	 */
+	#define TX_CMPL_PTP_ERRORS_DMA_ERROR                    UINT32_C(0x40)
+	/*
+	 * When this bit is '1', it indicates that the packet was longer
+	 * than indicated by the hint. No packet was transmitted.
+	 */
+	#define TX_CMPL_PTP_ERRORS_HINT_TOO_SHORT               UINT32_C(0x80)
+	/*
+	 * When this bit is '1', it indicates that the packet was
+	 * dropped due to Poison TLP error on one or more of the
+	 * TLPs in the PXP completion.
+	 */
+	#define TX_CMPL_PTP_ERRORS_POISON_TLP_ERROR             UINT32_C(0x100)
+	/*
+	 * When this bit is '1', it indicates that the packet was dropped due
+	 * to a transient internal error in TDC. The packet or LSO can be
+	 * retried and may transmit successfully on a subsequent attempt.
+	 */
+	#define TX_CMPL_PTP_ERRORS_INTERNAL_ERROR               UINT32_C(0x200)
+	/*
+	 * When this bit is '1', it was not possible to collect a a timestamp
+	 * for a PTP completion, in which case the timestamp_hi and
+	 * timestamp_lo fields are invalid. When this bit is '0' for a PTP
+	 * completion, the timestamp_hi and timestamp_lo fields are valid.
+	 * RJRN will copy the value of this bit into the field of the same
+	 * name in all TX completions, regardless of whether such
+	 * completions are PTP completions or other TX completions.
+	 */
+	#define TX_CMPL_PTP_ERRORS_TIMESTAMP_INVALID_ERROR      UINT32_C(0x400)
+	/* unused2 is 16 b */
+	uint16_t	unused_1;
+	/*
+	 * This is timestamp value (lower 32bits) read from PM for the PTP
+	 * timestamp enabled packet.
+	 */
+	uint32_t	timestamp_lo;
+} __rte_packed;
+
+/* tx_cmpl_ptp_hi (size:128b/16B) */
+struct tx_cmpl_ptp_hi {
+	/*
+	 * This is timestamp value (lower 32bits) read from PM for the PTP
+	 * timestamp enabled packet.
+	 */
+	uint16_t	timestamp_hi[3];
+	uint16_t	reserved16;
+	uint64_t	v2;
+	/*
+	 * This value is written by the NIC such that it will be different for
+	 * each pass through the completion queue.The even passes will write 1.
+	 * The odd passes will write 0
+	 */
+	#define TX_CMPL_PTP_HI_V2     UINT32_C(0x1)
+} __rte_packed;
+
 /* rx_pkt_cmpl (size:128b/16B) */
 struct rx_pkt_cmpl {
 	uint16_t	flags_type;
@@ -3003,12 +3365,8 @@ struct rx_pkt_cmpl_hi {
 	#define RX_PKT_CMPL_REORDER_SFT 0
 } __rte_packed;
 
-/*
- * This TPA completion structure is used on devices where the
- * `hwrm_vnic_qcaps.max_aggs_supported` value is 0.
- */
-/* rx_tpa_start_cmpl (size:128b/16B) */
-struct rx_tpa_start_cmpl {
+/* rx_pkt_v2_cmpl (size:128b/16B) */
+struct rx_pkt_v2_cmpl {
 	uint16_t	flags_type;
 	/*
 	 * This field indicates the exact type of the completion.
@@ -3017,84 +3375,143 @@ struct rx_tpa_start_cmpl {
 	 * records. Odd values indicate 32B
 	 * records.
 	 */
-	#define RX_TPA_START_CMPL_TYPE_MASK                UINT32_C(0x3f)
-	#define RX_TPA_START_CMPL_TYPE_SFT                 0
+	#define RX_PKT_V2_CMPL_TYPE_MASK                      UINT32_C(0x3f)
+	#define RX_PKT_V2_CMPL_TYPE_SFT                       0
 	/*
-	 * RX L2 TPA Start Completion:
-	 * Completion at the beginning of a TPA operation.
-	 * Length = 32B
+	 * RX L2 V2 completion:
+	 * Completion of and L2 RX packet. Length = 32B
+	 * This is the new version of the RX_L2 completion used in SR2
+	 * and later chips.
 	 */
-	#define RX_TPA_START_CMPL_TYPE_RX_TPA_START          UINT32_C(0x13)
-	#define RX_TPA_START_CMPL_TYPE_LAST \
-		RX_TPA_START_CMPL_TYPE_RX_TPA_START
-	#define RX_TPA_START_CMPL_FLAGS_MASK               UINT32_C(0xffc0)
-	#define RX_TPA_START_CMPL_FLAGS_SFT                6
-	/* This bit will always be '0' for TPA start completions. */
-	#define RX_TPA_START_CMPL_FLAGS_ERROR               UINT32_C(0x40)
+	#define RX_PKT_V2_CMPL_TYPE_RX_L2_V2                    UINT32_C(0xf)
+	#define RX_PKT_V2_CMPL_TYPE_LAST \
+		RX_PKT_V2_CMPL_TYPE_RX_L2_V2
+	#define RX_PKT_V2_CMPL_FLAGS_MASK                     UINT32_C(0xffc0)
+	#define RX_PKT_V2_CMPL_FLAGS_SFT                      6
+	/*
+	 * When this bit is '1', it indicates a packet that has an
+	 * error of some type. Type of error is indicated in
+	 * error_flags.
+	 */
+	#define RX_PKT_V2_CMPL_FLAGS_ERROR                     UINT32_C(0x40)
 	/* This field indicates how the packet was placed in the buffer. */
-	#define RX_TPA_START_CMPL_FLAGS_PLACEMENT_MASK      UINT32_C(0x380)
-	#define RX_TPA_START_CMPL_FLAGS_PLACEMENT_SFT       7
+	#define RX_PKT_V2_CMPL_FLAGS_PLACEMENT_MASK            UINT32_C(0x380)
+	#define RX_PKT_V2_CMPL_FLAGS_PLACEMENT_SFT             7
+	/*
+	 * Normal:
+	 * Packet was placed using normal algorithm.
+	 */
+	#define RX_PKT_V2_CMPL_FLAGS_PLACEMENT_NORMAL \
+		(UINT32_C(0x0) << 7)
 	/*
 	 * Jumbo:
-	 * TPA Packet was placed using jumbo algorithm. This means
-	 * that the first buffer will be filled with data before
-	 * moving to aggregation buffers. Each aggregation buffer
-	 * will be filled before moving to the next aggregation
-	 * buffer.
+	 * Packet was placed using jumbo algorithm.
 	 */
-	#define RX_TPA_START_CMPL_FLAGS_PLACEMENT_JUMBO \
+	#define RX_PKT_V2_CMPL_FLAGS_PLACEMENT_JUMBO \
 		(UINT32_C(0x1) << 7)
 	/*
 	 * Header/Data Separation:
 	 * Packet was placed using Header/Data separation algorithm.
 	 * The separation location is indicated by the itype field.
 	 */
-	#define RX_TPA_START_CMPL_FLAGS_PLACEMENT_HDS \
+	#define RX_PKT_V2_CMPL_FLAGS_PLACEMENT_HDS \
 		(UINT32_C(0x2) << 7)
 	/*
-	 * GRO/Jumbo:
-	 * Packet will be placed using GRO/Jumbo where the first
-	 * packet is filled with data. Subsequent packets will be
-	 * placed such that any one packet does not span two
-	 * aggregation buffers unless it starts at the beginning of
-	 * an aggregation buffer.
+	 * Truncation:
+	 * Packet was placed using truncation algorithm. The
+	 * placed (truncated) length is indicated in the payload_offset
+	 * field. The original length is indicated in the len field.
 	 */
-	#define RX_TPA_START_CMPL_FLAGS_PLACEMENT_GRO_JUMBO \
-		(UINT32_C(0x5) << 7)
+	#define RX_PKT_V2_CMPL_FLAGS_PLACEMENT_TRUNCATION \
+		(UINT32_C(0x3) << 7)
+	#define RX_PKT_V2_CMPL_FLAGS_PLACEMENT_LAST \
+		RX_PKT_V2_CMPL_FLAGS_PLACEMENT_TRUNCATION
+	/* This bit is '1' if the RSS field in this completion is valid. */
+	#define RX_PKT_V2_CMPL_FLAGS_RSS_VALID                 UINT32_C(0x400)
 	/*
-	 * GRO/Header-Data Separation:
-	 * Packet will be placed using GRO/HDS where the header
-	 * is in the first packet.
-	 * Payload of each packet will be
-	 * placed such that any one packet does not span two
-	 * aggregation buffers unless it starts at the beginning of
-	 * an aggregation buffer.
+	 * This bit is '1' if metadata has been added to the end of the
+	 * packet in host memory. Metadata starts at the first 32B boundary
+	 * after the end of the packet for regular and jumbo placement.
+	 * It starts at the first 32B boundary after the end of the header
+	 * for HDS placement. The length of the metadata is indicated in the
+	 * metadata itself.
 	 */
-	#define RX_TPA_START_CMPL_FLAGS_PLACEMENT_GRO_HDS \
-		(UINT32_C(0x6) << 7)
-	#define RX_TPA_START_CMPL_FLAGS_PLACEMENT_LAST \
-		RX_TPA_START_CMPL_FLAGS_PLACEMENT_GRO_HDS
-	/* This bit is '1' if the RSS field in this completion is valid. */
-	#define RX_TPA_START_CMPL_FLAGS_RSS_VALID           UINT32_C(0x400)
-	/* unused is 1 b */
-	#define RX_TPA_START_CMPL_FLAGS_UNUSED              UINT32_C(0x800)
+	#define RX_PKT_V2_CMPL_FLAGS_PKT_METADATA_PRESENT      UINT32_C(0x800)
 	/*
 	 * This value indicates what the inner packet determined for the
 	 * packet was.
 	 */
-	#define RX_TPA_START_CMPL_FLAGS_ITYPE_MASK          UINT32_C(0xf000)
-	#define RX_TPA_START_CMPL_FLAGS_ITYPE_SFT           12
+	#define RX_PKT_V2_CMPL_FLAGS_ITYPE_MASK                UINT32_C(0xf000)
+	#define RX_PKT_V2_CMPL_FLAGS_ITYPE_SFT                 12
+	/*
+	 * Not Known:
+	 * Indicates that the packet type was not known.
+	 */
+	#define RX_PKT_V2_CMPL_FLAGS_ITYPE_NOT_KNOWN \
+		(UINT32_C(0x0) << 12)
+	/*
+	 * IP Packet:
+	 * Indicates that the packet was an IP packet, but further
+	 * classification was not possible.
+	 */
+	#define RX_PKT_V2_CMPL_FLAGS_ITYPE_IP \
+		(UINT32_C(0x1) << 12)
 	/*
 	 * TCP Packet:
 	 * Indicates that the packet was IP and TCP.
+	 * This indicates that the payload_offset field is valid.
 	 */
-	#define RX_TPA_START_CMPL_FLAGS_ITYPE_TCP \
+	#define RX_PKT_V2_CMPL_FLAGS_ITYPE_TCP \
 		(UINT32_C(0x2) << 12)
-	#define RX_TPA_START_CMPL_FLAGS_ITYPE_LAST \
-		RX_TPA_START_CMPL_FLAGS_ITYPE_TCP
 	/*
-	 * This value indicates the amount of packet data written to the
-	 * buffer the opaque field in this completion corresponds to.
+	 * UDP Packet:
+	 * Indicates that the packet was IP and UDP.
+	 * This indicates that the payload_offset field is valid.
+	 */
+	#define RX_PKT_V2_CMPL_FLAGS_ITYPE_UDP \
+		(UINT32_C(0x3) << 12)
+	/*
+	 * FCoE Packet:
+	 * Indicates that the packet was recognized as a FCoE.
+	 * This also indicates that the payload_offset field is valid.
+	 */
+	#define RX_PKT_V2_CMPL_FLAGS_ITYPE_FCOE \
+		(UINT32_C(0x4) << 12)
+	/*
+	 * RoCE Packet:
+	 * Indicates that the packet was recognized as a RoCE.
+	 * This also indicates that the payload_offset field is valid.
+	 */
+	#define RX_PKT_V2_CMPL_FLAGS_ITYPE_ROCE \
+		(UINT32_C(0x5) << 12)
+	/*
+	 * ICMP Packet:
+	 * Indicates that the packet was recognized as ICMP.
+	 * This indicates that the payload_offset field is valid.
+	 */
+	#define RX_PKT_V2_CMPL_FLAGS_ITYPE_ICMP \
+		(UINT32_C(0x7) << 12)
+	/*
+	 * PtP packet wo/timestamp:
+	 * Indicates that the packet was recognized as a PtP
+	 * packet.
+	 */
+	#define RX_PKT_V2_CMPL_FLAGS_ITYPE_PTP_WO_TIMESTAMP \
+		(UINT32_C(0x8) << 12)
+	/*
+	 * PtP packet w/timestamp:
+	 * Indicates that the packet was recognized as a PtP
+	 * packet and that a timestamp was taken for the packet.
+	 */
+	#define RX_PKT_V2_CMPL_FLAGS_ITYPE_PTP_W_TIMESTAMP \
+		(UINT32_C(0x9) << 12)
+	#define RX_PKT_V2_CMPL_FLAGS_ITYPE_LAST \
+		RX_PKT_V2_CMPL_FLAGS_ITYPE_PTP_W_TIMESTAMP
+	/*
+	 * This is the length of the data for the packet stored in the
+	 * buffer(s) identified by the opaque value. This includes
+	 * the packet BD and any associated buffer BDs. This does not include
+	 * the length of any data places in aggregation BDs.
 	 */
 	uint16_t	len;
 	/*
@@ -3102,19 +3519,597 @@ struct rx_tpa_start_cmpl {
 	 * corresponds to.
 	 */
 	uint32_t	opaque;
+	uint8_t	agg_bufs_v1;
 	/*
 	 * This value is written by the NIC such that it will be different
 	 * for each pass through the completion queue. The even passes
 	 * will write 1. The odd passes will write 0.
 	 */
-	uint8_t	v1;
+	#define RX_PKT_V2_CMPL_V1           UINT32_C(0x1)
 	/*
-	 * This value is written by the NIC such that it will be different
-	 * for each pass through the completion queue. The even passes
-	 * will write 1. The odd passes will write 0.
+	 * This value is the number of aggregation buffers that follow this
+	 * entry in the completion ring that are a part of this packet.
+	 * If the value is zero, then the packet is completely contained
+	 * in the buffer space provided for the packet in the RX ring.
 	 */
-	#define RX_TPA_START_CMPL_V1 UINT32_C(0x1)
-	#define RX_TPA_START_CMPL_LAST RX_TPA_START_CMPL_V1
+	#define RX_PKT_V2_CMPL_AGG_BUFS_MASK UINT32_C(0x3e)
+	#define RX_PKT_V2_CMPL_AGG_BUFS_SFT 1
+	/* unused1 is 2 b */
+	#define RX_PKT_V2_CMPL_UNUSED1_MASK UINT32_C(0xc0)
+	#define RX_PKT_V2_CMPL_UNUSED1_SFT  6
+	/*
+	 * This is the RSS hash type for the packet. The value is packed
+	 * {tuple_extrac_op[1:0],rss_profile_id[4:0],tuple_extrac_op[2]}.
+	 *
+	 * The value of tuple_extrac_op provides the information about
+	 * what fields the hash was computed on.
+	 * * 0: The RSS hash was computed over source IP address,
+	 * destination IP address, source port, and destination port of inner
+	 * IP and TCP or UDP headers. Note: For non-tunneled packets,
+	 * the packet headers are considered inner packet headers for the RSS
+	 * hash computation purpose.
+	 * * 1: The RSS hash was computed over source IP address and destination
+	 * IP address of inner IP header. Note: For non-tunneled packets,
+	 * the packet headers are considered inner packet headers for the RSS
+	 * hash computation purpose.
+	 * * 2: The RSS hash was computed over source IP address,
+	 * destination IP address, source port, and destination port of
+	 * IP and TCP or UDP headers of outer tunnel headers.
+	 * Note: For non-tunneled packets, this value is not applicable.
+	 * * 3: The RSS hash was computed over source IP address and
+	 * destination IP address of IP header of outer tunnel headers.
+	 * Note: For non-tunneled packets, this value is not applicable.
+	 *
+	 * Note that 4-tuples values listed above are applicable
+	 * for layer 4 protocols supported and enabled for RSS in the hardware,
+	 * HWRM firmware, and drivers. For example, if RSS hash is supported and
+	 * enabled for TCP traffic only, then the values of tuple_extract_op
+	 * corresponding to 4-tuples are only valid for TCP traffic.
+	 */
+	uint8_t	rss_hash_type;
+	uint16_t	metadata1_payload_offset;
+	/*
+	 * This is data from the CFA as indicated by the meta_format field.
+	 * If truncation placement is not used, this value indicates the offset
+	 * in bytes from the beginning of the packet where the inner payload
+	 * starts. This value is valid for TCP, UDP, FCoE, and RoCE packets. If
+	 * truncation placement is used, this value represents the placed
+	 * (truncated) length of the packet.
+	 */
+	#define RX_PKT_V2_CMPL_PAYLOAD_OFFSET_MASK    UINT32_C(0x1ff)
+	#define RX_PKT_V2_CMPL_PAYLOAD_OFFSET_SFT     0
+	/* This is data from the CFA as indicated by the meta_format field. */
+	#define RX_PKT_V2_CMPL_METADATA1_MASK         UINT32_C(0xf000)
+	#define RX_PKT_V2_CMPL_METADATA1_SFT          12
+	/* When meta_format != 0, this value is the VLAN TPID_SEL. */
+	#define RX_PKT_V2_CMPL_METADATA1_TPID_SEL_MASK UINT32_C(0x7000)
+	#define RX_PKT_V2_CMPL_METADATA1_TPID_SEL_SFT  12
+	/* When meta_format != 0, this value is the VLAN TPID_SEL. */
+	#define RX_PKT_V2_CMPL_METADATA1_VALID         UINT32_C(0x8000)
+	/*
+	 * This value is the RSS hash value calculated for the packet
+	 * based on the mode bits and key value in the VNIC. When vee_cmpl_mode
+	 * is set in VNIC context, this is the lower 32b of the host address
+	 * from the first BD used to place the packet.
+	 */
+	uint32_t	rss_hash;
+} __rte_packed;
+
+/* Last 16 bytes of RX Packet V2 Completion Record */
+/* rx_pkt_v2_cmpl_hi (size:128b/16B) */
+struct rx_pkt_v2_cmpl_hi {
+	uint32_t	flags2;
+	/*
+	 * When this bit is '0', the cs_ok field has the following definition:-
+	 * ip_cs_ok[2:0] = The number of header groups with a valid IP checksum
+	 * in the delivered packet, counted from the outer-most header group to
+	 * the inner-most header group, stopping at the first error. -
+	 * l4_cs_ok[5:3] = The number of header groups with a valid L4 checksum
+	 * in the delivered packet, counted from the outer-most header group to
+	 * the inner-most header group, stopping at the first error. When this
+	 * bit is '1', the cs_ok field has the following definition: -
+	 * hdr_cnt[2:0] = The number of header groups that were parsed by the
+	 * chip and passed in the delivered packet. - ip_cs_all_ok[3] =This bit
+	 * will be '1' if all the parsed header groups with an IP checksum are
+	 * valid. - l4_cs_all_ok[4] = This bit will be '1' if all the parsed
+	 * header groups with an L4 checksum are valid.
+	 */
+	#define RX_PKT_V2_CMPL_HI_FLAGS2_CS_ALL_OK_MODE \
+		UINT32_C(0x8)
+	/* This value indicates what format the metadata field is. */
+	#define RX_PKT_V2_CMPL_HI_FLAGS2_META_FORMAT_MASK \
+		UINT32_C(0xf0)
+	#define RX_PKT_V2_CMPL_HI_FLAGS2_META_FORMAT_SFT            4
+	/* There is no metadata information. Values are zero. */
+	#define RX_PKT_V2_CMPL_HI_FLAGS2_META_FORMAT_NONE \
+		(UINT32_C(0x0) << 4)
+	/*
+	 * The {metadata1, metadata0} fields contain the vtag
+	 * information: - vtag[19:0] = {valid, tpid_sel[2:0], pri[2:0],
+	 * de, vid[11:0]} The metadata2 field contains the table scope
+	 * and action record pointer. - metadata2[25:0] contains the
+	 * action record pointer. - metadata2[31:26] contains the table
+	 * scope.
+	 */
+	#define RX_PKT_V2_CMPL_HI_FLAGS2_META_FORMAT_ACT_REC_PTR \
+		(UINT32_C(0x1) << 4)
+	/*
+	 * The {metadata1, metadata0} fields contain the vtag
+	 * information:
+	 * - vtag[19:0] = {valid, tpid_sel[2:0], pri[2:0], de, vid[11:0]}
+	 * The metadata2 field contains the Tunnel ID
+	 * value, justified to LSB. i
+	 * - VXLAN = VNI[23:0] -> VXLAN Network ID
+	 * - Geneve (NGE) = VNI[23:0] a-> Virtual Network Identifier
+	 * - NVGRE = TNI[23:0] -> Tenant Network ID
+	 * - GRE = KEY[31:0] -> key field with bit mask. zero if K=0
+	 * - IPv4 = 0 (not populated)
+	 * - IPv6 = Flow Label[19:0]
+	 * - PPPoE = sessionID[15:0]
+	 * - MPLs = Outer label[19:0]
+	 * - UPAR = Selected[31:0] with bit mask
+	 */
+	#define RX_PKT_V2_CMPL_HI_FLAGS2_META_FORMAT_TUNNEL_ID \
+		(UINT32_C(0x2) << 4)
+	/*
+	 * The {metadata1, metadata0} fields contain the vtag
+	 * information:
+	 * - vtag[19:0] = {valid, tpid_sel[2:0], pri[2:0],de, vid[11:0]}
+	 * The metadata2 field contains the 32b metadata from the prepended
+	 * header (chdr_data).
+	 */
+	#define RX_PKT_V2_CMPL_HI_FLAGS2_META_FORMAT_CHDR_DATA \
+		(UINT32_C(0x3) << 4)
+	/*
+	 * The {metadata1, metadata0} fields contain the vtag
+	 * information:
+	 * - vtag[19:0] = {valid, tpid_sel[2:0], pri[2:0], de, vid[11:0]}
+	 * The metadata2 field contains the outer_l3_offset,
+	 * inner_l2_offset, inner_l3_offset, and inner_l4_size.
+	 * - metadata2[8:0] contains the outer_l3_offset.
+	 * - metadata2[17:9] contains the inner_l2_offset.
+	 * - metadata2[26:18] contains the inner_l3_offset.
+	 * - metadata2[31:27] contains the inner_l4_size.
+	 */
+	#define RX_PKT_V2_CMPL_HI_FLAGS2_META_FORMAT_HDR_OFFSET \
+		(UINT32_C(0x4) << 4)
+	#define RX_PKT_V2_CMPL_HI_FLAGS2_META_FORMAT_LAST \
+		RX_PKT_V2_CMPL_HI_FLAGS2_META_FORMAT_HDR_OFFSET
+	/*
+	 * This field indicates the IP type for the inner-most IP header.
+	 * A value of '0' indicates IPv4. A value of '1' indicates IPv6.
+	 * This value is only valid if itype indicates a packet
+	 * with an IP header.
+	 */
+	#define RX_PKT_V2_CMPL_HI_FLAGS2_IP_TYPE \
+		UINT32_C(0x100)
+	/*
+	 * This indicates that the complete 1's complement checksum was
+	 * calculated for the packet.
+	 */
+	#define RX_PKT_V2_CMPL_HI_FLAGS2_COMPLETE_CHECKSUM_CALC \
+		UINT32_C(0x200)
+	/*
+	 * This field indicates the status of IP and L4 CS calculations done
+	 * by the chip. The format of this field is indicated by the
+	 * cs_all_ok_mode bit.
+	 */
+	#define RX_PKT_V2_CMPL_HI_FLAGS2_CS_OK_MASK \
+		UINT32_C(0xfc00)
+	#define RX_PKT_V2_CMPL_HI_FLAGS2_CS_OK_SFT                  10
+	/*
+	 * This value is the complete 1's complement checksum calculated from
+	 * the start of the outer L3 header to the end of the packet (not
+	 * including the ethernet crc). It is valid when the
+	 * 'complete_checksum_calc' flag is set.
+	 */
+	#define RX_PKT_V2_CMPL_HI_FLAGS2_COMPLETE_CHECKSUM_MASK \
+		UINT32_C(0xffff0000)
+	#define RX_PKT_V2_CMPL_HI_FLAGS2_COMPLETE_CHECKSUM_SFT      16
+	/*
+	 * This is data from the CFA block as indicated by the meta_format
+	 * field.
+	 * - meta_format 0 - none - metadata2 = 0 - not valid/not stripped
+	 * - meta_format 1 - act_rec_ptr - metadata2 = {table_scope[5:0],
+	 *   act_rec_ptr[25:0]}
+	 * - meta_format 2 - tunnel_id - metadata2 = tunnel_id[31:0]
+	 * - meta_format 3 - chdr_data - metadata2 = updated_chdr_data[31:0]
+	 * - meta_format 4 - hdr_offsets - metadata2 = hdr_offsets[31:0]
+	 * When vee_cmpl_mode is set in VNIC context, this is the upper 32b
+	 * of the host address from the first BD used to place the packet.
+	 */
+	uint32_t	metadata2;
+	uint16_t	errors_v2;
+	/*
+	 * This value is written by the NIC such that it will be different
+	 * for each pass through the completion queue. The even passes
+	 * will write 1. The odd passes will write 0.
+	 */
+	#define RX_PKT_V2_CMPL_HI_V2 \
+		UINT32_C(0x1)
+	#define RX_PKT_V2_CMPL_HI_ERRORS_MASK \
+		UINT32_C(0xfffe)
+	#define RX_PKT_V2_CMPL_HI_ERRORS_SFT                               1
+	/*
+	 * This error indicates that there was some sort of problem with
+	 * the BDs for the packet that was found after part of the
+	 * packet was already placed. The packet should be treated as
+	 * invalid.
+	 */
+	#define RX_PKT_V2_CMPL_HI_ERRORS_BUFFER_ERROR_MASK \
+		UINT32_C(0xe)
+	#define RX_PKT_V2_CMPL_HI_ERRORS_BUFFER_ERROR_SFT                   1
+	/* No buffer error */
+	#define RX_PKT_V2_CMPL_HI_ERRORS_BUFFER_ERROR_NO_BUFFER \
+		(UINT32_C(0x0) << 1)
+	/*
+	 * Did Not Fit: Packet did not fit into packet buffer provided.
+	 * For regular placement, this means the packet did not fit in
+	 * the buffer provided. For HDS and jumbo placement, this means
+	 * that the packet could not be placed into 8 physical buffers
+	 * (if fixed-size buffers are used), or that the packet could
+	 * not be placed in the number of physical buffers configured
+	 * for the VNIC (if variable-size buffers are used)
+	 */
+	#define RX_PKT_V2_CMPL_HI_ERRORS_BUFFER_ERROR_DID_NOT_FIT \
+		(UINT32_C(0x1) << 1)
+	/*
+	 * Not On Chip: All BDs needed for the packet were not on-chip
+	 * when the packet arrived. For regular placement, this error is
+	 * not valid. For HDS and jumbo placement, this means that not
+	 * enough agg BDs were posted to place the packet.
+	 */
+	#define RX_PKT_V2_CMPL_HI_ERRORS_BUFFER_ERROR_NOT_ON_CHIP \
+		(UINT32_C(0x2) << 1)
+	/*
+	 * Bad Format:
+	 * BDs were not formatted correctly.
+	 */
+	#define RX_PKT_V2_CMPL_HI_ERRORS_BUFFER_ERROR_BAD_FORMAT \
+		(UINT32_C(0x3) << 1)
+	/*
+	 * Flush:
+	 * There was a bad_format error on the previous operation
+	 */
+	#define RX_PKT_V2_CMPL_HI_ERRORS_BUFFER_ERROR_FLUSH \
+		(UINT32_C(0x5) << 1)
+	#define RX_PKT_V2_CMPL_HI_ERRORS_BUFFER_ERROR_LAST \
+		RX_PKT_V2_CMPL_HI_ERRORS_BUFFER_ERROR_FLUSH
+	/*
+	 * This indicates that there was an error in the outer tunnel
+	 * portion of the packet when this field is non-zero.
+	 */
+	#define RX_PKT_V2_CMPL_HI_ERRORS_OT_PKT_ERROR_MASK \
+		UINT32_C(0x70)
+	#define RX_PKT_V2_CMPL_HI_ERRORS_OT_PKT_ERROR_SFT                   4
+	/*
+	 * No additional error occurred on the outer tunnel portion
+	 * of the packet or the packet does not have a outer tunnel.
+	 */
+	#define RX_PKT_V2_CMPL_HI_ERRORS_OT_PKT_ERROR_NO_ERROR \
+		(UINT32_C(0x0) << 4)
+	/*
+	 * Indicates that IP header version does not match expectation
+	 * from L2 Ethertype for IPv4 and IPv6 in the outer tunnel header.
+	 */
+	#define RX_PKT_V2_CMPL_HI_ERRORS_OT_PKT_ERROR_OT_L3_BAD_VERSION \
+		(UINT32_C(0x1) << 4)
+	/*
+	 * Indicates that header length is out of range in the outer
+	 * tunnel header. Valid for IPv4.
+	 */
+	#define RX_PKT_V2_CMPL_HI_ERRORS_OT_PKT_ERROR_OT_L3_BAD_HDR_LEN \
+		(UINT32_C(0x2) << 4)
+	/*
+	 * Indicates that physical packet is shorter than that claimed
+	 * by the outer tunnel l3 header length. Valid for IPv4, or
+	 * IPv6 outer tunnel packets.
+	 */
+	#define RX_PKT_V2_CMPL_HI_ERRORS_OT_PKT_ERROR_OT_IP_TOTAL_ERROR \
+		(UINT32_C(0x3) << 4)
+	/*
+	 * Indicates that the physical packet is shorter than that
+	 * claimed by the outer tunnel UDP header length for a outer
+	 * tunnel UDP packet that is not fragmented.
+	 */
+	#define RX_PKT_V2_CMPL_HI_ERRORS_OT_PKT_ERROR_OT_UDP_TOTAL_ERROR \
+		(UINT32_C(0x4) << 4)
+	/*
+	 * Indicates that the IPv4 TTL or IPv6 hop limit check have
+	 * failed (e.g. TTL = 0) in the outer tunnel header. Valid for
+	 * IPv4, and IPv6.
+	 */
+	#define RX_PKT_V2_CMPL_HI_ERRORS_OT_PKT_ERROR_OT_L3_BAD_TTL \
+		(UINT32_C(0x5) << 4)
+	/*
+	 * Indicates that the IP checksum failed its check in the outer
+	 * tunnel header.
+	 */
+	#define RX_PKT_V2_CMPL_HI_ERRORS_OT_PKT_ERROR_OT_IP_CS_ERROR \
+		(UINT32_C(0x6) << 4)
+	/*
+	 * Indicates that the L4 checksum failed its check in the outer
+	 * tunnel header.
+	 */
+	#define RX_PKT_V2_CMPL_HI_ERRORS_OT_PKT_ERROR_OT_L4_CS_ERROR \
+		(UINT32_C(0x7) << 4)
+	#define RX_PKT_V2_CMPL_HI_ERRORS_OT_PKT_ERROR_LAST \
+		RX_PKT_V2_CMPL_HI_ERRORS_OT_PKT_ERROR_OT_L4_CS_ERROR
+	/*
+	 * This indicates that there was a CRC error on either an FCoE
+	 * or RoCE packet. The itype indicates the packet type.
+	 */
+	#define RX_PKT_V2_CMPL_HI_ERRORS_CRC_ERROR \
+		UINT32_C(0x100)
+	/*
+	 * This indicates that there was an error in the tunnel portion
+	 * of the packet when this field is non-zero.
+	 */
+	#define RX_PKT_V2_CMPL_HI_ERRORS_T_PKT_ERROR_MASK \
+		UINT32_C(0xe00)
+	#define RX_PKT_V2_CMPL_HI_ERRORS_T_PKT_ERROR_SFT                    9
+	/*
+	 * No additional error occurred on the tunnel portion
+	 * of the packet or the packet does not have a tunnel.
+	 */
+	#define RX_PKT_V2_CMPL_HI_ERRORS_T_PKT_ERROR_NO_ERROR \
+		(UINT32_C(0x0) << 9)
+	/*
+	 * Indicates that IP header version does not match expectation
+	 * from L2 Ethertype for IPv4 and IPv6 in the tunnel header.
+	 */
+	#define RX_PKT_V2_CMPL_HI_ERRORS_T_PKT_ERROR_T_L3_BAD_VERSION \
+		(UINT32_C(0x1) << 9)
+	/*
+	 * Indicates that header length is out of range in the tunnel
+	 * header. Valid for IPv4.
+	 */
+	#define RX_PKT_V2_CMPL_HI_ERRORS_T_PKT_ERROR_T_L3_BAD_HDR_LEN \
+		(UINT32_C(0x2) << 9)
+	/*
+	 * Indicates that physical packet is shorter than that claimed
+	 * by the tunnel l3 header length. Valid for IPv4, or IPv6 tunnel
+	 * packet packets.
+	 */
+	#define RX_PKT_V2_CMPL_HI_ERRORS_T_PKT_ERROR_T_IP_TOTAL_ERROR \
+		(UINT32_C(0x3) << 9)
+	/*
+	 * Indicates that the physical packet is shorter than that claimed
+	 * by the tunnel UDP header length for a tunnel UDP packet that is
+	 * not fragmented.
+	 */
+	#define RX_PKT_V2_CMPL_HI_ERRORS_T_PKT_ERROR_T_UDP_TOTAL_ERROR \
+		(UINT32_C(0x4) << 9)
+	/*
+	 * Indicates that the IPv4 TTL or IPv6 hop limit check have failed
+	 * (e.g. TTL = 0) in the tunnel header. Valid for IPv4, and IPv6.
+	 */
+	#define RX_PKT_V2_CMPL_HI_ERRORS_T_PKT_ERROR_T_L3_BAD_TTL \
+		(UINT32_C(0x5) << 9)
+	/*
+	 * Indicates that the IP checksum failed its check in the tunnel
+	 * header.
+	 */
+	#define RX_PKT_V2_CMPL_HI_ERRORS_T_PKT_ERROR_T_IP_CS_ERROR \
+		(UINT32_C(0x6) << 9)
+	/*
+	 * Indicates that the L4 checksum failed its check in the tunnel
+	 * header.
+	 */
+	#define RX_PKT_V2_CMPL_HI_ERRORS_T_PKT_ERROR_T_L4_CS_ERROR \
+		(UINT32_C(0x7) << 9)
+	#define RX_PKT_V2_CMPL_HI_ERRORS_T_PKT_ERROR_LAST \
+		RX_PKT_V2_CMPL_HI_ERRORS_T_PKT_ERROR_T_L4_CS_ERROR
+	/*
+	 * This indicates that there was an error in the inner
+	 * portion of the packet when this
+	 * field is non-zero.
+	 */
+	#define RX_PKT_V2_CMPL_HI_ERRORS_PKT_ERROR_MASK \
+		UINT32_C(0xf000)
+	#define RX_PKT_V2_CMPL_HI_ERRORS_PKT_ERROR_SFT                      12
+	/*
+	 * No additional error occurred on the tunnel portion
+	 * or the packet of the packet does not have a tunnel.
+	 */
+	#define RX_PKT_V2_CMPL_HI_ERRORS_PKT_ERROR_NO_ERROR \
+		(UINT32_C(0x0) << 12)
+	/*
+	 * Indicates that IP header version does not match
+	 * expectation from L2 Ethertype for IPv4 and IPv6 or that
+	 * option other than VFT was parsed on
+	 * FCoE packet.
+	 */
+	#define RX_PKT_V2_CMPL_HI_ERRORS_PKT_ERROR_L3_BAD_VERSION \
+		(UINT32_C(0x1) << 12)
+	/*
+	 * indicates that header length is out of range. Valid for
+	 * IPv4 and RoCE
+	 */
+	#define RX_PKT_V2_CMPL_HI_ERRORS_PKT_ERROR_L3_BAD_HDR_LEN \
+		(UINT32_C(0x2) << 12)
+	/*
+	 * indicates that the IPv4 TTL or IPv6 hop limit check
+	 * have failed (e.g. TTL = 0). Valid for IPv4, and IPv6
+	 */
+	#define RX_PKT_V2_CMPL_HI_ERRORS_PKT_ERROR_L3_BAD_TTL \
+		(UINT32_C(0x3) << 12)
+	/*
+	 * Indicates that physical packet is shorter than that
+	 * claimed by the l3 header length. Valid for IPv4,
+	 * IPv6 packet or RoCE packets.
+	 */
+	#define RX_PKT_V2_CMPL_HI_ERRORS_PKT_ERROR_IP_TOTAL_ERROR \
+		(UINT32_C(0x4) << 12)
+	/*
+	 * Indicates that the physical packet is shorter than that
+	 * claimed by the UDP header length for a UDP packet that is
+	 * not fragmented.
+	 */
+	#define RX_PKT_V2_CMPL_HI_ERRORS_PKT_ERROR_UDP_TOTAL_ERROR \
+		(UINT32_C(0x5) << 12)
+	/*
+	 * Indicates that TCP header length > IP payload. Valid for
+	 * TCP packets only.
+	 */
+	#define RX_PKT_V2_CMPL_HI_ERRORS_PKT_ERROR_L4_BAD_HDR_LEN \
+		(UINT32_C(0x6) << 12)
+	/* Indicates that TCP header length < 5. Valid for TCP. */
+	#define RX_PKT_V2_CMPL_HI_ERRORS_PKT_ERROR_L4_BAD_HDR_LEN_TOO_SMALL \
+		(UINT32_C(0x7) << 12)
+	/*
+	 * Indicates that TCP option headers result in a TCP header
+	 * size that does not match data offset in TCP header. Valid
+	 * for TCP.
+	 */
+	#define RX_PKT_V2_CMPL_HI_ERRORS_PKT_ERROR_L4_BAD_OPT_LEN \
+		(UINT32_C(0x8) << 12)
+	/*
+	 * Indicates that the IP checksum failed its check in the
+	 * inner header.
+	 */
+	#define RX_PKT_V2_CMPL_HI_ERRORS_PKT_ERROR_IP_CS_ERROR \
+		(UINT32_C(0x9) << 12)
+	/*
+	 * Indicates that the L4 checksum failed its check in the
+	 * inner header.
+	 */
+	#define RX_PKT_V2_CMPL_HI_ERRORS_PKT_ERROR_L4_CS_ERROR \
+		(UINT32_C(0xa) << 12)
+	#define RX_PKT_V2_CMPL_HI_ERRORS_PKT_ERROR_LAST \
+		RX_PKT_V2_CMPL_HI_ERRORS_PKT_ERROR_L4_CS_ERROR
+	/*
+	 * This is data from the CFA block as indicated by the meta_format
+	 * field.
+	 */
+	uint16_t	metadata0;
+	/* When meta_format=1, this value is the VLAN VID. */
+	#define RX_PKT_V2_CMPL_HI_METADATA0_VID_MASK UINT32_C(0xfff)
+	#define RX_PKT_V2_CMPL_HI_METADATA0_VID_SFT 0
+	/* When meta_format=1, this value is the VLAN DE. */
+	#define RX_PKT_V2_CMPL_HI_METADATA0_DE      UINT32_C(0x1000)
+	/* When meta_format=1, this value is the VLAN PRI. */
+	#define RX_PKT_V2_CMPL_HI_METADATA0_PRI_MASK UINT32_C(0xe000)
+	#define RX_PKT_V2_CMPL_HI_METADATA0_PRI_SFT 13
+	/*
+	 * The timestamp field contains the 32b timestamp for the packet from
+	 * the MAC.
+	 */
+	uint32_t	timestamp;
+} __rte_packed;
+
+/*
+ * This TPA completion structure is used on devices where the
+ * `hwrm_vnic_qcaps.max_aggs_supported` value is 0.
+ */
+/* rx_tpa_start_cmpl (size:128b/16B) */
+struct rx_tpa_start_cmpl {
+	uint16_t	flags_type;
+	/*
+	 * This field indicates the exact type of the completion.
+	 * By convention, the LSB identifies the length of the
+	 * record in 16B units. Even values indicate 16B
+	 * records. Odd values indicate 32B
+	 * records.
+	 */
+	#define RX_TPA_START_CMPL_TYPE_MASK                UINT32_C(0x3f)
+	#define RX_TPA_START_CMPL_TYPE_SFT                 0
+	/*
+	 * RX L2 TPA Start Completion:
+	 * Completion at the beginning of a TPA operation.
+	 * Length = 32B
+	 */
+	#define RX_TPA_START_CMPL_TYPE_RX_TPA_START          UINT32_C(0x13)
+	#define RX_TPA_START_CMPL_TYPE_LAST \
+		RX_TPA_START_CMPL_TYPE_RX_TPA_START
+	#define RX_TPA_START_CMPL_FLAGS_MASK               UINT32_C(0xffc0)
+	#define RX_TPA_START_CMPL_FLAGS_SFT                6
+	/* This bit will always be '0' for TPA start completions. */
+	#define RX_TPA_START_CMPL_FLAGS_ERROR               UINT32_C(0x40)
+	/* This field indicates how the packet was placed in the buffer. */
+	#define RX_TPA_START_CMPL_FLAGS_PLACEMENT_MASK      UINT32_C(0x380)
+	#define RX_TPA_START_CMPL_FLAGS_PLACEMENT_SFT       7
+	/*
+	 * Jumbo:
+	 * TPA Packet was placed using jumbo algorithm. This means
+	 * that the first buffer will be filled with data before
+	 * moving to aggregation buffers. Each aggregation buffer
+	 * will be filled before moving to the next aggregation
+	 * buffer.
+	 */
+	#define RX_TPA_START_CMPL_FLAGS_PLACEMENT_JUMBO \
+		(UINT32_C(0x1) << 7)
+	/*
+	 * Header/Data Separation:
+	 * Packet was placed using Header/Data separation algorithm.
+	 * The separation location is indicated by the itype field.
+	 */
+	#define RX_TPA_START_CMPL_FLAGS_PLACEMENT_HDS \
+		(UINT32_C(0x2) << 7)
+	/*
+	 * GRO/Jumbo:
+	 * Packet will be placed using GRO/Jumbo where the first
+	 * packet is filled with data. Subsequent packets will be
+	 * placed such that any one packet does not span two
+	 * aggregation buffers unless it starts at the beginning of
+	 * an aggregation buffer.
+	 */
+	#define RX_TPA_START_CMPL_FLAGS_PLACEMENT_GRO_JUMBO \
+		(UINT32_C(0x5) << 7)
+	/*
+	 * GRO/Header-Data Separation:
+	 * Packet will be placed using GRO/HDS where the header
+	 * is in the first packet.
+	 * Payload of each packet will be
+	 * placed such that any one packet does not span two
+	 * aggregation buffers unless it starts at the beginning of
+	 * an aggregation buffer.
+	 */
+	#define RX_TPA_START_CMPL_FLAGS_PLACEMENT_GRO_HDS \
+		(UINT32_C(0x6) << 7)
+	#define RX_TPA_START_CMPL_FLAGS_PLACEMENT_LAST \
+		RX_TPA_START_CMPL_FLAGS_PLACEMENT_GRO_HDS
+	/* This bit is '1' if the RSS field in this completion is valid. */
+	#define RX_TPA_START_CMPL_FLAGS_RSS_VALID           UINT32_C(0x400)
+	/* unused is 1 b */
+	#define RX_TPA_START_CMPL_FLAGS_UNUSED              UINT32_C(0x800)
+	/*
+	 * This value indicates what the inner packet determined for the
+	 * packet was.
+	 */
+	#define RX_TPA_START_CMPL_FLAGS_ITYPE_MASK          UINT32_C(0xf000)
+	#define RX_TPA_START_CMPL_FLAGS_ITYPE_SFT           12
+	/*
+	 * TCP Packet:
+	 * Indicates that the packet was IP and TCP.
+	 */
+	#define RX_TPA_START_CMPL_FLAGS_ITYPE_TCP \
+		(UINT32_C(0x2) << 12)
+	#define RX_TPA_START_CMPL_FLAGS_ITYPE_LAST \
+		RX_TPA_START_CMPL_FLAGS_ITYPE_TCP
+	/*
+	 * This value indicates the amount of packet data written to the
+	 * buffer the opaque field in this completion corresponds to.
+	 */
+	uint16_t	len;
+	/*
+	 * This is a copy of the opaque field from the RX BD this completion
+	 * corresponds to.
+	 */
+	uint32_t	opaque;
+	/*
+	 * This value is written by the NIC such that it will be different
+	 * for each pass through the completion queue. The even passes
+	 * will write 1. The odd passes will write 0.
+	 */
+	uint8_t	v1;
+	/*
+	 * This value is written by the NIC such that it will be different
+	 * for each pass through the completion queue. The even passes
+	 * will write 1. The odd passes will write 0.
+	 */
+	#define RX_TPA_START_CMPL_V1 UINT32_C(0x1)
+	#define RX_TPA_START_CMPL_LAST RX_TPA_START_CMPL_V1
 	/*
 	 * This is the RSS hash type for the packet. The value is packed
 	 * {tuple_extrac_op[1:0],rss_profile_id[4:0],tuple_extrac_op[2]}.
@@ -3288,6 +4283,430 @@ struct rx_tpa_start_cmpl_hi {
 /*
  * This TPA completion structure is used on devices where the
  * `hwrm_vnic_qcaps.max_aggs_supported` value is 0.
+ * RX L2 TPA Start V2 Completion Record (32 bytes split to 2 16-byte
+ * struct)
+ */
+/* rx_tpa_start_v2_cmpl (size:128b/16B) */
+struct rx_tpa_start_v2_cmpl {
+	uint16_t	flags_type;
+	/*
+	 * This field indicates the exact type of the completion.
+	 * By convention, the LSB identifies the length of the
+	 * record in 16B units. Even values indicate 16B
+	 * records. Odd values indicate 32B
+	 * records.
+	 */
+	#define RX_TPA_START_V2_CMPL_TYPE_MASK \
+		UINT32_C(0x3f)
+	#define RX_TPA_START_V2_CMPL_TYPE_SFT                       0
+	/*
+	 * RX L2 TPA Start V2 Completion:
+	 * Completion at the beginning of a TPA operation.
+	 * Length = 32B
+	 * This is the new version of the RX_TPA_START completion used
+	 * in SR2 and later chips.
+	 */
+	#define RX_TPA_START_V2_CMPL_TYPE_RX_TPA_START_V2 \
+		UINT32_C(0xd)
+	#define RX_TPA_START_V2_CMPL_TYPE_LAST \
+		RX_TPA_START_V2_CMPL_TYPE_RX_TPA_START_V2
+	#define RX_TPA_START_V2_CMPL_FLAGS_MASK \
+		UINT32_C(0xffc0)
+	#define RX_TPA_START_V2_CMPL_FLAGS_SFT                      6
+	/*
+	 * When this bit is '1', it indicates a packet that has an error
+	 * of some type. Type of error is indicated in error_flags.
+	 */
+	#define RX_TPA_START_V2_CMPL_FLAGS_ERROR \
+		UINT32_C(0x40)
+	/* This field indicates how the packet was placed in the buffer. */
+	#define RX_TPA_START_V2_CMPL_FLAGS_PLACEMENT_MASK \
+		UINT32_C(0x380)
+	#define RX_TPA_START_V2_CMPL_FLAGS_PLACEMENT_SFT             7
+	/*
+	 * Jumbo:
+	 * TPA Packet was placed using jumbo algorithm. This means
+	 * that the first buffer will be filled with data before
+	 * moving to aggregation buffers. Each aggregation buffer
+	 * will be filled before moving to the next aggregation
+	 * buffer.
+	 */
+	#define RX_TPA_START_V2_CMPL_FLAGS_PLACEMENT_JUMBO \
+		(UINT32_C(0x1) << 7)
+	/*
+	 * Header/Data Separation:
+	 * Packet was placed using Header/Data separation algorithm.
+	 * The separation location is indicated by the itype field.
+	 */
+	#define RX_TPA_START_V2_CMPL_FLAGS_PLACEMENT_HDS \
+		(UINT32_C(0x2) << 7)
+	/*
+	 * IOC/Jumbo:
+	 * Packet will be placed using In-Order Completion/Jumbo where
+	 * the first packet of the aggregation is placed using Jumbo
+	 * Placement. Subsequent packets will be placed such that each
+	 * packet starts at the beginning of an aggregation buffer.
+	 */
+	#define RX_TPA_START_V2_CMPL_FLAGS_PLACEMENT_IOC_JUMBO \
+		(UINT32_C(0x4) << 7)
+	/*
+	 * GRO/Jumbo:
+	 * Packet will be placed using GRO/Jumbo where the first
+	 * packet is filled with data. Subsequent packets will be
+	 * placed such that any one packet does not span two
+	 * aggregation buffers unless it starts at the beginning of
+	 * an aggregation buffer.
+	 */
+	#define RX_TPA_START_V2_CMPL_FLAGS_PLACEMENT_GRO_JUMBO \
+		(UINT32_C(0x5) << 7)
+	/*
+	 * GRO/Header-Data Separation:
+	 * Packet will be placed using GRO/HDS where the header
+	 * is in the first packet.
+	 * Payload of each packet will be
+	 * placed such that any one packet does not span two
+	 * aggregation buffers unless it starts at the beginning of
+	 * an aggregation buffer.
+	 */
+	#define RX_TPA_START_V2_CMPL_FLAGS_PLACEMENT_GRO_HDS \
+		(UINT32_C(0x6) << 7)
+	/*
+	 * IOC/Header-Data Separation:
+	 * Packet will be placed using In-Order Completion/HDS where
+	 * the header is in the first packet buffer. Payload of each
+	 * packet will be placed such that each packet starts at the
+	 * beginning of an aggregation buffer.
+	 */
+	#define RX_TPA_START_V2_CMPL_FLAGS_PLACEMENT_IOC_HDS \
+		(UINT32_C(0x7) << 7)
+	#define RX_TPA_START_V2_CMPL_FLAGS_PLACEMENT_LAST \
+		RX_TPA_START_V2_CMPL_FLAGS_PLACEMENT_IOC_HDS
+	/* This bit is '1' if the RSS field in this completion is valid. */
+	#define RX_TPA_START_V2_CMPL_FLAGS_RSS_VALID \
+		UINT32_C(0x400)
+	/*
+	 * This bit is '1' if metadata has been added to the end of the
+	 * packet in host memory. Metadata starts at the first 32B boundary
+	 * after the end of the packet for regular and jumbo placement. It
+	 * starts at the first 32B boundary after the end of the header for
+	 * HDS placement. The length of the metadata is indicated in the
+	 * metadata itself.
+	 */
+	#define RX_TPA_START_V2_CMPL_FLAGS_PKT_METADATA_PRESENT \
+		UINT32_C(0x800)
+	/*
+	 * This value indicates what the inner packet determined for the
+	 * packet was.
+	 */
+	#define RX_TPA_START_V2_CMPL_FLAGS_ITYPE_MASK \
+		UINT32_C(0xf000)
+	#define RX_TPA_START_V2_CMPL_FLAGS_ITYPE_SFT                 12
+	/*
+	 * TCP Packet:
+	 * Indicates that the packet was IP and TCP.
+	 */
+	#define RX_TPA_START_V2_CMPL_FLAGS_ITYPE_TCP \
+		(UINT32_C(0x2) << 12)
+	#define RX_TPA_START_V2_CMPL_FLAGS_ITYPE_LAST \
+		RX_TPA_START_V2_CMPL_FLAGS_ITYPE_TCP
+	/*
+	 * This value indicates the amount of packet data written to the
+	 * buffer the opaque field in this completion corresponds to.
+	 */
+	uint16_t	len;
+	/*
+	 * This is a copy of the opaque field from the RX BD this completion
+	 * corresponds to. If the VNIC is configured to not use an Rx BD for
+	 * the TPA Start completion, then this is a copy of the opaque field
+	 * from the first BD used to place the TPA Start packet.
+	 */
+	uint32_t	opaque;
+	/*
+	 * This value is written by the NIC such that it will be different
+	 * for each pass through the completion queue. The even passes
+	 * will write 1. The odd passes will write 0.
+	 */
+	uint8_t	v1;
+	/*
+	 * This value is written by the NIC such that it will be different
+	 * for each pass through the completion queue. The even passes
+	 * will write 1. The odd passes will write 0.
+	 */
+	#define RX_TPA_START_V2_CMPL_V1 UINT32_C(0x1)
+	#define RX_TPA_START_V2_CMPL_LAST RX_TPA_START_V2_CMPL_V1
+	/*
+	 * This is the RSS hash type for the packet. The value is packed
+	 * {tuple_extrac_op[1:0],rss_profile_id[4:0],tuple_extrac_op[2]}.
+	 *
+	 * The value of tuple_extrac_op provides the information about
+	 * what fields the hash was computed on.
+	 * * 0: The RSS hash was computed over source IP address,
+	 * destination IP address, source port, and destination port of inner
+	 * IP and TCP or UDP headers. Note: For non-tunneled packets,
+	 * the packet headers are considered inner packet headers for the RSS
+	 * hash computation purpose.
+	 * * 1: The RSS hash was computed over source IP address and destination
+	 * IP address of inner IP header. Note: For non-tunneled packets,
+	 * the packet headers are considered inner packet headers for the RSS
+	 * hash computation purpose.
+	 * * 2: The RSS hash was computed over source IP address,
+	 * destination IP address, source port, and destination port of
+	 * IP and TCP or UDP headers of outer tunnel headers.
+	 * Note: For non-tunneled packets, this value is not applicable.
+	 * * 3: The RSS hash was computed over source IP address and
+	 * destination IP address of IP header of outer tunnel headers.
+	 * Note: For non-tunneled packets, this value is not applicable.
+	 *
+	 * Note that 4-tuples values listed above are applicable
+	 * for layer 4 protocols supported and enabled for RSS in the hardware,
+	 * HWRM firmware, and drivers. For example, if RSS hash is supported and
+	 * enabled for TCP traffic only, then the values of tuple_extract_op
+	 * corresponding to 4-tuples are only valid for TCP traffic.
+	 */
+	uint8_t	rss_hash_type;
+	/*
+	 * This is the aggregation ID that the completion is associated
+	 * with. Use this number to correlate the TPA start completion
+	 * with the TPA end completion.
+	 */
+	uint16_t	agg_id;
+	/*
+	 * This is the aggregation ID that the completion is associated
+	 * with. Use this number to correlate the TPA start completion
+	 * with the TPA end completion.
+	 */
+	#define RX_TPA_START_V2_CMPL_AGG_ID_MASK            UINT32_C(0xfff)
+	#define RX_TPA_START_V2_CMPL_AGG_ID_SFT             0
+	#define RX_TPA_START_V2_CMPL_METADATA1_MASK         UINT32_C(0xf000)
+	#define RX_TPA_START_V2_CMPL_METADATA1_SFT          12
+	/* When meta_format != 0, this value is the VLAN TPID_SEL. */
+	#define RX_TPA_START_V2_CMPL_METADATA1_TPID_SEL_MASK UINT32_C(0x7000)
+	#define RX_TPA_START_V2_CMPL_METADATA1_TPID_SEL_SFT  12
+	/* When meta_format != 0, this value is the VLAN valid. */
+	#define RX_TPA_START_V2_CMPL_METADATA1_VALID         UINT32_C(0x8000)
+	/*
+	 * This value is the RSS hash value calculated for the packet
+	 * based on the mode bits and key value in the VNIC.
+	 * When vee_cmpl_mode is set in VNIC context, this is the lower
+	 * 32b of the host address from the first BD used to place the packet.
+	 */
+	uint32_t	rss_hash;
+} __rte_packed;
+
+/*
+ * Last 16 bytes of RX L2 TPA Start V2 Completion Record
+ *
+ * This TPA completion structure is used on devices where the
+ * `hwrm_vnic_qcaps.max_aggs_supported` value is 0.
+ */
+/* rx_tpa_start_v2_cmpl_hi (size:128b/16B) */
+struct rx_tpa_start_v2_cmpl_hi {
+	uint32_t	flags2;
+	/* This indicates that the aggregation was done using GRO rules. */
+	#define RX_TPA_START_V2_CMPL_FLAGS2_AGG_GRO \
+		UINT32_C(0x4)
+	/*
+	 * When this bit is '0', the cs_ok field has the following definition:-
+	 * ip_cs_ok[2:0] = The number of header groups with a valid IP checksum
+	 * in the delivered packet, counted from the outer-most header group to
+	 * the inner-most header group, stopping at the first error. -
+	 * l4_cs_ok[5:3] = The number of header groups with a valid L4 checksum
+	 * in the delivered packet, counted from the outer-most header group to
+	 * the inner-most header group, stopping at the first error. When this
+	 * bit is '1', the cs_ok field has the following definition: -
+	 * hdr_cnt[2:0] = The number of header groups that were parsed by the
+	 * chip and passed in the delivered packet. - ip_cs_all_ok[3] =This bit
+	 * will be '1' if all the parsed header groups with an IP checksum are
+	 * valid. - l4_cs_all_ok[4] = This bit will be '1' if all the parsed
+	 * header groups with an L4 checksum are valid.
+	 */
+	#define RX_TPA_START_V2_CMPL_FLAGS2_CS_ALL_OK_MODE \
+		UINT32_C(0x8)
+	/* This value indicates what format the metadata field is. */
+	#define RX_TPA_START_V2_CMPL_FLAGS2_META_FORMAT_MASK \
+		UINT32_C(0xf0)
+	#define RX_TPA_START_V2_CMPL_FLAGS2_META_FORMAT_SFT            4
+	/* There is no metadata information. Values are zero. */
+	#define RX_TPA_START_V2_CMPL_FLAGS2_META_FORMAT_NONE \
+		(UINT32_C(0x0) << 4)
+	/*
+	 * The {metadata1, metadata0} fields contain the vtag
+	 * information: - vtag[19:0] = {valid, tpid_sel[2:0], pri[2:0],
+	 * de, vid[11:0]} The metadata2 field contains the table scope
+	 * and action record pointer. - metadata2[25:0] contains the
+	 * action record pointer. - metadata2[31:26] contains the table
+	 * scope.
+	 */
+	#define RX_TPA_START_V2_CMPL_FLAGS2_META_FORMAT_ACT_REC_PTR \
+		(UINT32_C(0x1) << 4)
+	/*
+	 * The {metadata1, metadata0} fields contain the vtag
+	 * information:
+	 * - vtag[19:0] = {valid, tpid_sel[2:0], pri[2:0], de, vid[11:0]}
+	 * The metadata2 field contains the Tunnel ID
+	 * value, justified to LSB. i
+	 * - VXLAN = VNI[23:0] -> VXLAN Network ID
+	 * - Geneve (NGE) = VNI[23:0] a-> Virtual Network Identifier
+	 * - NVGRE = TNI[23:0] -> Tenant Network ID
+	 * - GRE = KEY[31:0] -> key field with bit mask. zero if K=0
+	 * - IPv4 = 0 (not populated)
+	 * - IPv6 = Flow Label[19:0]
+	 * - PPPoE = sessionID[15:0]
+	 * - MPLs = Outer label[19:0]
+	 * - UPAR = Selected[31:0] with bit mask
+	 */
+	#define RX_TPA_START_V2_CMPL_FLAGS2_META_FORMAT_TUNNEL_ID \
+		(UINT32_C(0x2) << 4)
+	/*
+	 * The {metadata1, metadata0} fields contain the vtag
+	 * information:
+	 * - vtag[19:0] = {valid, tpid_sel[2:0], pri[2:0],de, vid[11:0]}
+	 * The metadata2 field contains the 32b metadata from the prepended
+	 * header (chdr_data).
+	 */
+	#define RX_TPA_START_V2_CMPL_FLAGS2_META_FORMAT_CHDR_DATA \
+		(UINT32_C(0x3) << 4)
+	/*
+	 * The {metadata1, metadata0} fields contain the vtag
+	 * information:
+	 * - vtag[19:0] = {valid, tpid_sel[2:0], pri[2:0], de, vid[11:0]}
+	 * The metadata2 field contains the outer_l3_offset,
+	 * inner_l2_offset, inner_l3_offset, and inner_l4_size.
+	 * - metadata2[8:0] contains the outer_l3_offset.
+	 * - metadata2[17:9] contains the inner_l2_offset.
+	 * - metadata2[26:18] contains the inner_l3_offset.
+	 * - metadata2[31:27] contains the inner_l4_size.
+	 */
+	#define RX_TPA_START_V2_CMPL_FLAGS2_META_FORMAT_HDR_OFFSET \
+		(UINT32_C(0x4) << 4)
+	#define RX_TPA_START_V2_CMPL_FLAGS2_META_FORMAT_LAST \
+		RX_TPA_START_V2_CMPL_FLAGS2_META_FORMAT_HDR_OFFSET
+	/*
+	 * This field indicates the IP type for the inner-most IP header.
+	 * A value of '0' indicates IPv4. A value of '1' indicates IPv6.
+	 * This value is only valid if itype indicates a packet
+	 * with an IP header.
+	 */
+	#define RX_TPA_START_V2_CMPL_FLAGS2_IP_TYPE \
+		UINT32_C(0x100)
+	/*
+	 * This indicates that the complete 1's complement checksum was
+	 * calculated for the packet in the affregation.
+	 */
+	#define RX_TPA_START_V2_CMPL_FLAGS2_COMPLETE_CHECKSUM_CALC \
+		UINT32_C(0x200)
+	/*
+	 * This field indicates the status of IP and L4 CS calculations done
+	 * by the chip. The format of this field is indicated by the
+	 * cs_all_ok_mode bit.
+	 * CS status for TPA packets is always valid. This means that "all_ok"
+	 * status will always be set. The ok count status will be set
+	 * appropriately for the packet header, such that all existing CS
+	 * values are ok.
+	 */
+	#define RX_TPA_START_V2_CMPL_FLAGS2_CS_OK_MASK \
+		UINT32_C(0xfc00)
+	#define RX_TPA_START_V2_CMPL_FLAGS2_CS_OK_SFT                  10
+	/*
+	 * This value is the complete 1's complement checksum calculated from
+	 * the start of the outer L3 header to the end of the packet (not
+	 * including the ethernet crc). It is valid when the
+	 * 'complete_checksum_calc' flag is set. For TPA Start completions,
+	 * the complete checksum is calculated for the first packet in the
+	 * aggregation only.
+	 */
+	#define RX_TPA_START_V2_CMPL_FLAGS2_COMPLETE_CHECKSUM_MASK \
+		UINT32_C(0xffff0000)
+	#define RX_TPA_START_V2_CMPL_FLAGS2_COMPLETE_CHECKSUM_SFT      16
+	/*
+	 * This is data from the CFA block as indicated by the meta_format
+	 * field.
+	 * - meta_format 0 - none - metadata2 = 0 - not valid/not stripped
+	 * - meta_format 1 - act_rec_ptr - metadata2 = {table_scope[5:0],
+	 *   act_rec_ptr[25:0]}
+	 * - meta_format 2 - tunnel_id - metadata2 = tunnel_id[31:0]
+	 * - meta_format 3 - chdr_data - metadata2 = updated_chdr_data[31:0]
+	 * - meta_format 4 - hdr_offsets - metadata2 = hdr_offsets[31:0]
+	 * When vee_cmpl_mode is set in VNIC context, this is the upper 32b
+	 * of the host address from the first BD used to place the packet.
+	 */
+	uint32_t	metadata2;
+	uint16_t	errors_v2;
+	/*
+	 * This value is written by the NIC such that it will be different
+	 * for each pass through the completion queue. The even passes
+	 * will write 1. The odd passes will write 0.
+	 */
+	#define RX_TPA_START_V2_CMPL_V2 \
+		UINT32_C(0x1)
+	#define RX_TPA_START_V2_CMPL_ERRORS_MASK \
+		UINT32_C(0xfffe)
+	#define RX_TPA_START_V2_CMPL_ERRORS_SFT                     1
+	/*
+	 * This error indicates that there was some sort of problem with
+	 * the BDs for the packetThe packet should be treated as
+	 * invalid.
+	 */
+	#define RX_TPA_START_V2_CMPL_ERRORS_BUFFER_ERROR_MASK \
+		UINT32_C(0xe)
+	#define RX_TPA_START_V2_CMPL_ERRORS_BUFFER_ERROR_SFT         1
+	/* No buffer error */
+	#define RX_TPA_START_V2_CMPL_ERRORS_BUFFER_ERROR_NO_BUFFER \
+		(UINT32_C(0x0) << 1)
+	/*
+	 * Did Not Fit:
+	 * Packet did not fit into packet buffer provided. This means
+	 * that the TPA Start packet was too big to be placed into the
+	 * per-packet maximum number of physical buffers configured for
+	 * the VNIC, or that it was too big to be placed into the
+	 * per-aggregation maximum number of physical buffers configured
+	 * for the VNIC. This error only occurs when the VNIC is
+	 * configured for variable size receive buffers.
+	 */
+	#define RX_TPA_START_V2_CMPL_ERRORS_BUFFER_ERROR_DID_NOT_FIT \
+		(UINT32_C(0x1) << 1)
+	/*
+	 * Bad Format:
+	 * BDs were not formatted correctly.
+	 */
+	#define RX_TPA_START_V2_CMPL_ERRORS_BUFFER_ERROR_BAD_FORMAT \
+		(UINT32_C(0x3) << 1)
+	/*
+	 * Flush:
+	 * There was a bad_format error on the previous operation
+	 */
+	#define RX_TPA_START_V2_CMPL_ERRORS_BUFFER_ERROR_FLUSH \
+		(UINT32_C(0x5) << 1)
+	#define RX_TPA_START_V2_CMPL_ERRORS_BUFFER_ERROR_LAST \
+		RX_TPA_START_V2_CMPL_ERRORS_BUFFER_ERROR_FLUSH
+	/*
+	 * This is data from the CFA block as indicated by the meta_format
+	 * field.
+	 */
+	uint16_t	metadata0;
+	/* When meta_format != 0, this value is the VLAN VID. */
+	#define RX_TPA_START_V2_CMPL_METADATA0_VID_MASK UINT32_C(0xfff)
+	#define RX_TPA_START_V2_CMPL_METADATA0_VID_SFT 0
+	/* When meta_format != 0, this value is the VLAN DE. */
+	#define RX_TPA_START_V2_CMPL_METADATA0_DE      UINT32_C(0x1000)
+	/* When meta_format != 0, this value is the VLAN PRI. */
+	#define RX_TPA_START_V2_CMPL_METADATA0_PRI_MASK UINT32_C(0xe000)
+	#define RX_TPA_START_V2_CMPL_METADATA0_PRI_SFT 13
+	/*
+	 * This field contains the outer_l3_offset, inner_l2_offset,
+	 * inner_l3_offset, and inner_l4_size.
+	 *
+	 * hdr_offsets[8:0] contains the outer_l3_offset.
+	 * hdr_offsets[17:9] contains the inner_l2_offset.
+	 * hdr_offsets[26:18] contains the inner_l3_offset.
+	 * hdr_offsets[31:27] contains the inner_l4_size.
+	 */
+	uint32_t	hdr_offsets;
+} __rte_packed;
+
+/*
+ * This TPA completion structure is used on devices where the
+ * `hwrm_vnic_qcaps.max_aggs_supported` value is 0.
  */
 /* rx_tpa_end_cmpl (size:128b/16B) */
 struct rx_tpa_end_cmpl {
@@ -3299,27 +4718,27 @@ struct rx_tpa_end_cmpl {
 	 * records. Odd values indicate 32B
 	 * records.
 	 */
-	#define RX_TPA_END_CMPL_TYPE_MASK                UINT32_C(0x3f)
-	#define RX_TPA_END_CMPL_TYPE_SFT                 0
+	#define RX_TPA_END_CMPL_TYPE_MASK                      UINT32_C(0x3f)
+	#define RX_TPA_END_CMPL_TYPE_SFT                       0
 	/*
 	 * RX L2 TPA End Completion:
 	 * Completion at the end of a TPA operation.
 	 * Length = 32B
 	 */
-	#define RX_TPA_END_CMPL_TYPE_RX_TPA_END            UINT32_C(0x15)
+	#define RX_TPA_END_CMPL_TYPE_RX_TPA_END                  UINT32_C(0x15)
 	#define RX_TPA_END_CMPL_TYPE_LAST \
 		RX_TPA_END_CMPL_TYPE_RX_TPA_END
-	#define RX_TPA_END_CMPL_FLAGS_MASK               UINT32_C(0xffc0)
-	#define RX_TPA_END_CMPL_FLAGS_SFT                6
+	#define RX_TPA_END_CMPL_FLAGS_MASK                     UINT32_C(0xffc0)
+	#define RX_TPA_END_CMPL_FLAGS_SFT                      6
 	/*
 	 * When this bit is '1', it indicates a packet that has an
 	 * error of some type. Type of error is indicated in
 	 * error_flags.
 	 */
-	#define RX_TPA_END_CMPL_FLAGS_ERROR               UINT32_C(0x40)
+	#define RX_TPA_END_CMPL_FLAGS_ERROR                     UINT32_C(0x40)
 	/* This field indicates how the packet was placed in the buffer. */
-	#define RX_TPA_END_CMPL_FLAGS_PLACEMENT_MASK      UINT32_C(0x380)
-	#define RX_TPA_END_CMPL_FLAGS_PLACEMENT_SFT       7
+	#define RX_TPA_END_CMPL_FLAGS_PLACEMENT_MASK            UINT32_C(0x380)
+	#define RX_TPA_END_CMPL_FLAGS_PLACEMENT_SFT             7
 	/*
 	 * Jumbo:
 	 * TPA Packet was placed using jumbo algorithm. This means
@@ -3338,6 +4757,15 @@ struct rx_tpa_end_cmpl {
 	#define RX_TPA_END_CMPL_FLAGS_PLACEMENT_HDS \
 		(UINT32_C(0x2) << 7)
 	/*
+	 * IOC/Jumbo:
+	 * Packet will be placed using In-Order Completion/Jumbo where
+	 * the first packet of the aggregation is placed using Jumbo
+	 * Placement. Subsequent packets will be placed such that each
+	 * packet starts at the beginning of an aggregation buffer.
+	 */
+	#define RX_TPA_END_CMPL_FLAGS_PLACEMENT_IOC_JUMBO \
+		(UINT32_C(0x4) << 7)
+	/*
 	 * GRO/Jumbo:
 	 * Packet will be placed using GRO/Jumbo where the first
 	 * packet is filled with data. Subsequent packets will be
@@ -3358,11 +4786,28 @@ struct rx_tpa_end_cmpl {
 	 */
 	#define RX_TPA_END_CMPL_FLAGS_PLACEMENT_GRO_HDS \
 		(UINT32_C(0x6) << 7)
+	/*
+	 * IOC/Header-Data Separation:
+	 * Packet will be placed using In-Order Completion/HDS where
+	 * the header is in the first packet buffer. Payload of each
+	 * packet will be placed such that each packet starts at the
+	 * beginning of an aggregation buffer.
+	 */
+	#define RX_TPA_END_CMPL_FLAGS_PLACEMENT_IOC_HDS \
+		(UINT32_C(0x7) << 7)
 	#define RX_TPA_END_CMPL_FLAGS_PLACEMENT_LAST \
-		RX_TPA_END_CMPL_FLAGS_PLACEMENT_GRO_HDS
-	/* unused is 2 b */
-	#define RX_TPA_END_CMPL_FLAGS_UNUSED_MASK         UINT32_C(0xc00)
-	#define RX_TPA_END_CMPL_FLAGS_UNUSED_SFT          10
+		RX_TPA_END_CMPL_FLAGS_PLACEMENT_IOC_HDS
+	/* unused is 1 b */
+	#define RX_TPA_END_CMPL_FLAGS_UNUSED                    UINT32_C(0x400)
+	/*
+	 * This bit is '1' if metadata has been added to the end of the
+	 * packet in host memory. Metadata starts at the first 32B boundary
+	 * after the end of the packet for regular and jumbo placement.
+	 * It starts at the first 32B boundary after the end of the header
+	 * for HDS placement. The length of the metadata is indicated in the
+	 * metadata itself.
+	 */
+	#define RX_TPA_END_CMPL_FLAGS_PKT_METADATA_PRESENT      UINT32_C(0x800)
 	/*
 	 * This value indicates what the inner packet determined for the
 	 * packet was.
@@ -3372,8 +4817,9 @@ struct rx_tpa_end_cmpl {
 	 *     field is valid and contains the TCP checksum.
 	 *     This also indicates that the payload_offset field is valid.
 	 */
-	#define RX_TPA_END_CMPL_FLAGS_ITYPE_MASK          UINT32_C(0xf000)
-	#define RX_TPA_END_CMPL_FLAGS_ITYPE_SFT           12
+	#define RX_TPA_END_CMPL_FLAGS_ITYPE_MASK \
+		UINT32_C(0xf000)
+	#define RX_TPA_END_CMPL_FLAGS_ITYPE_SFT                 12
 	/*
 	 * This value is zero for TPA End completions.
 	 * There is no data in the buffer that corresponds to the opaque
@@ -4243,6 +5689,52 @@ struct rx_abuf_cmpl {
 	uint32_t	unused_2;
 } __rte_packed;
 
+/* VEE FLUSH Completion Record (16 bytes) */
+/* vee_flush (size:128b/16B) */
+struct vee_flush {
+	uint32_t	downstream_path_type;
+	/*
+	 * This field indicates the exact type of the completion.
+	 * By convention, the LSB identifies the length of the
+	 * record in 16B units. Even values indicate 16B
+	 * records. Odd values indicate 32B
+	 * records.
+	 */
+	#define VEE_FLUSH_TYPE_MASK           UINT32_C(0x3f)
+	#define VEE_FLUSH_TYPE_SFT            0
+	/*
+	 * VEE Flush Completion:
+	 * This completion is inserted manually by the Primate and processed
+	 * by the VEE hardware to ensure that all completions on a VEE
+	 * function have been processed by the VEE hardware before FLR
+	 * process is completed.
+	 */
+	#define VEE_FLUSH_TYPE_VEE_FLUSH        UINT32_C(0x1c)
+	#define VEE_FLUSH_TYPE_LAST            VEE_FLUSH_TYPE_VEE_FLUSH
+	/* downstream_path is 1 b */
+	#define VEE_FLUSH_DOWNSTREAM_PATH     UINT32_C(0x40)
+	/* This completion is associated with VEE Transmit */
+	#define VEE_FLUSH_DOWNSTREAM_PATH_TX    (UINT32_C(0x0) << 6)
+	/* This completion is associated with VEE Receive */
+	#define VEE_FLUSH_DOWNSTREAM_PATH_RX    (UINT32_C(0x1) << 6)
+	#define VEE_FLUSH_DOWNSTREAM_PATH_LAST VEE_FLUSH_DOWNSTREAM_PATH_RX
+	/*
+	 * This is an opaque value that is passed through the completion
+	 * to the VEE handler SW and is used to indicate what VEE VQ or
+	 * function has completed FLR processing.
+	 */
+	uint32_t	opaque;
+	uint32_t	v;
+	/*
+	 * This value is written by the NIC such that it will be different
+	 * for each pass through the completion queue. The even passes will
+	 * write 1. The odd passes will write 0.
+	 */
+	#define VEE_FLUSH_V     UINT32_C(0x1)
+	/* unused3 is 32 b */
+	uint32_t	unused_3;
+} __rte_packed;
+
 /* eject_cmpl (size:128b/16B) */
 struct eject_cmpl {
 	uint16_t	type;
@@ -6562,7 +8054,7 @@ struct hwrm_async_event_cmpl_deferred_response {
 	/*
 	 * The PF's mailbox is clear to issue another command.
 	 * A command with this seq_id is still in progress
-	 * and will return a regular HWRM completion when done.
+	 * and will return a regualr HWRM completion when done.
 	 * 'event_data1' field, if non-zero, contains the estimated
 	 * execution time for the command.
 	 */
@@ -7476,6 +8968,8 @@ struct hwrm_func_qcaps_input {
 	 * Function ID of the function that is being queried.
 	 * 0xFF... (All Fs) if the query is for the requesting
 	 * function.
+	 * 0xFFFE (REQUESTING_PARENT_FID) This is a special FID
+	 * to be used by a trusted VF to query its parent PF.
 	 */
 	uint16_t	fid;
 	uint8_t	unused_0[6];
@@ -7730,6 +9224,12 @@ struct hwrm_func_qcaps_output {
 	#define HWRM_FUNC_QCAPS_OUTPUT_FLAGS_PFC_WD_STATS_SUPPORTED \
 		UINT32_C(0x40000000)
 	/*
+	 * When this bit is '1', it indicates that core firmware supports
+	 * DBG_QCAPS command
+	 */
+	#define HWRM_FUNC_QCAPS_OUTPUT_FLAGS_DBG_QCAPS_CMD_SUPPORTED \
+		UINT32_C(0x80000000)
+	/*
 	 * This value is current MAC address configured for this
 	 * function. A value of 00-00-00-00-00-00 indicates no
 	 * MAC address is currently configured.
@@ -7854,6 +9354,19 @@ struct hwrm_func_qcaps_output {
 	 */
 	#define HWRM_FUNC_QCAPS_OUTPUT_FLAGS_EXT_ECN_STATS_SUPPORTED \
 		UINT32_C(0x2)
+	/*
+	 * If 1, the device can report extended hw statistics (including
+	 * additional tpa statistics).
+	 */
+	#define HWRM_FUNC_QCAPS_OUTPUT_FLAGS_EXT_EXT_HW_STATS_SUPPORTED \
+		UINT32_C(0x4)
+	/*
+	 * If set to 1, then the core firmware has support to enable/
+	 * disable hot reset support for interface dynamically through
+	 * HWRM_FUNC_CFG.
+	 */
+	#define HWRM_FUNC_QCAPS_OUTPUT_FLAGS_EXT_HOT_RESET_IF_SUPPORT \
+		UINT32_C(0x8)
 	uint8_t	unused_1[3];
 	/*
 	 * This field is used in Output records to indicate that the output
@@ -7904,6 +9417,8 @@ struct hwrm_func_qcfg_input {
 	 * Function ID of the function that is being queried.
 	 * 0xFF... (All Fs) if the query is for the requesting
 	 * function.
+	 * 0xFFFE (REQUESTING_PARENT_FID) This is a special FID
+	 * to be used by a trusted VF to query its parent PF.
 	 */
 	uint16_t	fid;
 	uint8_t	unused_0[6];
@@ -8014,6 +9529,15 @@ struct hwrm_func_qcfg_output {
 	#define HWRM_FUNC_QCFG_OUTPUT_FLAGS_PREBOOT_LEGACY_L2_RINGS \
 		UINT32_C(0x100)
 	/*
+	 * If set to 1, then the firmware and all currently registered driver
+	 * instances support hot reset. The hot reset support will be updated
+	 * dynamically based on the driver interface advertisement.
+	 * If set to 0, then the adapter is not currently able to initiate
+	 * hot reset.
+	 */
+	#define HWRM_FUNC_QCFG_OUTPUT_FLAGS_HOT_RESET_ALLOWED \
+		UINT32_C(0x200)
+	/*
 	 * This value is current MAC address configured for this
 	 * function. A value of 00-00-00-00-00-00 indicates no
 	 * MAC address is currently configured.
@@ -8565,6 +10089,17 @@ struct hwrm_func_cfg_input {
 	 */
 	#define HWRM_FUNC_CFG_INPUT_FLAGS_PREBOOT_LEGACY_L2_RINGS \
 		UINT32_C(0x2000000)
+	/*
+	 * If this bit is set to 0, then the interface does not support hot
+	 * reset capability which it advertised with the hot_reset_support
+	 * flag in HWRM_FUNC_DRV_RGTR. If any of the function has set this
+	 * flag to 0, adapter cannot do the hot reset. In this state, if the
+	 * firmware receives a hot reset request, firmware must fail the
+	 * request. If this bit is set to 1, then interface is renabling the
+	 * hot reset capability.
+	 */
+	#define HWRM_FUNC_CFG_INPUT_FLAGS_HOT_RESET_IF_EN_DIS \
+		UINT32_C(0x4000000)
 	uint32_t	enables;
 	/*
 	 * This bit must be '1' for the mtu field to be
@@ -8705,6 +10240,12 @@ struct hwrm_func_cfg_input {
 	#define HWRM_FUNC_CFG_INPUT_ENABLES_ADMIN_LINK_STATE \
 		UINT32_C(0x400000)
 	/*
+	 * This bit must be '1' for the hot_reset_if_en_dis field to be
+	 * configured.
+	 */
+	#define HWRM_FUNC_CFG_INPUT_ENABLES_HOT_RESET_IF_SUPPORT \
+		UINT32_C(0x800000)
+	/*
 	 * The maximum transmission unit of the function.
 	 * The HWRM should make sure that the mtu of
 	 * the function does not exceed the mtu of the physical
@@ -9036,15 +10577,21 @@ struct hwrm_func_qstats_input {
 	/* This flags indicates the type of statistics request. */
 	uint8_t	flags;
 	/* This value is not used to avoid backward compatibility issues. */
-	#define HWRM_FUNC_QSTATS_INPUT_FLAGS_UNUSED    UINT32_C(0x0)
+	#define HWRM_FUNC_QSTATS_INPUT_FLAGS_UNUSED       UINT32_C(0x0)
 	/*
 	 * flags should be set to 1 when request is for only RoCE statistics.
 	 * This will be honored only if the caller_fid is a privileged PF.
 	 * In all other cases FID and caller_fid should be the same.
 	 */
-	#define HWRM_FUNC_QSTATS_INPUT_FLAGS_ROCE_ONLY UINT32_C(0x1)
+	#define HWRM_FUNC_QSTATS_INPUT_FLAGS_ROCE_ONLY    UINT32_C(0x1)
+	/*
+	 * flags should be set to 2 when request is for the counter mask,
+	 * representing the width of each of the stats counters, rather
+	 * than counters themselves.
+	 */
+	#define HWRM_FUNC_QSTATS_INPUT_FLAGS_COUNTER_MASK UINT32_C(0x2)
 	#define HWRM_FUNC_QSTATS_INPUT_FLAGS_LAST \
-		HWRM_FUNC_QSTATS_INPUT_FLAGS_ROCE_ONLY
+		HWRM_FUNC_QSTATS_INPUT_FLAGS_COUNTER_MASK
 	uint8_t	unused_0[5];
 } __rte_packed;
 
@@ -9130,6 +10677,132 @@ struct hwrm_func_qstats_output {
 	uint8_t	valid;
 } __rte_packed;
 
+/************************
+ * hwrm_func_qstats_ext *
+ ************************/
+
+
+/* hwrm_func_qstats_ext_input (size:192b/24B) */
+struct hwrm_func_qstats_ext_input {
+	/* The HWRM command request type. */
+	uint16_t	req_type;
+	/*
+	 * The completion ring to send the completion event on. This should
+	 * be the NQ ID returned from the `nq_alloc` HWRM command.
+	 */
+	uint16_t	cmpl_ring;
+	/*
+	 * The sequence ID is used by the driver for tracking multiple
+	 * commands. This ID is treated as opaque data by the firmware and
+	 * the value is returned in the `hwrm_resp_hdr` upon completion.
+	 */
+	uint16_t	seq_id;
+	/*
+	 * The target ID of the command:
+	 * * 0x0-0xFFF8 - The function ID
+	 * * 0xFFF8-0xFFFC, 0xFFFE - Reserved for internal processors
+	 * * 0xFFFD - Reserved for user-space HWRM interface
+	 * * 0xFFFF - HWRM
+	 */
+	uint16_t	target_id;
+	/*
+	 * A physical address pointer pointing to a host buffer that the
+	 * command's response data will be written. This can be either a host
+	 * physical address (HPA) or a guest physical address (GPA) and must
+	 * point to a physically contiguous block of memory.
+	 */
+	uint64_t	resp_addr;
+	/*
+	 * Function ID of the function that is being queried.
+	 * 0xFF... (All Fs) if the query is for the requesting
+	 * function.
+	 * A privileged PF can query for other function's statistics.
+	 */
+	uint16_t	fid;
+	/* This flags indicates the type of statistics request. */
+	uint8_t	flags;
+	/* This value is not used to avoid backward compatibility issues. */
+	#define HWRM_FUNC_QSTATS_EXT_INPUT_FLAGS_UNUSED       UINT32_C(0x0)
+	/*
+	 * flags should be set to 1 when request is for only RoCE statistics.
+	 * This will be honored only if the caller_fid is a privileged PF.
+	 * In all other cases FID and caller_fid should be the same.
+	 */
+	#define HWRM_FUNC_QSTATS_EXT_INPUT_FLAGS_ROCE_ONLY    UINT32_C(0x1)
+	/*
+	 * flags should be set to 2 when request is for the counter mask
+	 * representing the width of each of the stats counters, rather
+	 * than counters themselves.
+	 */
+	#define HWRM_FUNC_QSTATS_EXT_INPUT_FLAGS_COUNTER_MASK UINT32_C(0x2)
+	#define HWRM_FUNC_QSTATS_EXT_INPUT_FLAGS_LAST \
+		HWRM_FUNC_QSTATS_EXT_INPUT_FLAGS_COUNTER_MASK
+	uint8_t	unused_0[5];
+} __rte_packed;
+
+/* hwrm_func_qstats_ext_output (size:1472b/184B) */
+struct hwrm_func_qstats_ext_output {
+	/* The specific error status for the command. */
+	uint16_t	error_code;
+	/* The HWRM command request type. */
+	uint16_t	req_type;
+	/* The sequence ID from the original command. */
+	uint16_t	seq_id;
+	/* The length of the response data in number of bytes. */
+	uint16_t	resp_len;
+	/* Number of received unicast packets */
+	uint64_t	rx_ucast_pkts;
+	/* Number of received multicast packets */
+	uint64_t	rx_mcast_pkts;
+	/* Number of received broadcast packets */
+	uint64_t	rx_bcast_pkts;
+	/* Number of discarded packets on received path */
+	uint64_t	rx_discard_pkts;
+	/* Number of packets on receive path with error */
+	uint64_t	rx_error_pkts;
+	/* Number of received bytes for unicast traffic */
+	uint64_t	rx_ucast_bytes;
+	/* Number of received bytes for multicast traffic */
+	uint64_t	rx_mcast_bytes;
+	/* Number of received bytes for broadcast traffic */
+	uint64_t	rx_bcast_bytes;
+	/* Number of transmitted unicast packets */
+	uint64_t	tx_ucast_pkts;
+	/* Number of transmitted multicast packets */
+	uint64_t	tx_mcast_pkts;
+	/* Number of transmitted broadcast packets */
+	uint64_t	tx_bcast_pkts;
+	/* Number of packets on transmit path with error */
+	uint64_t	tx_error_pkts;
+	/* Number of discarded packets on transmit path */
+	uint64_t	tx_discard_pkts;
+	/* Number of transmitted bytes for unicast traffic */
+	uint64_t	tx_ucast_bytes;
+	/* Number of transmitted bytes for multicast traffic */
+	uint64_t	tx_mcast_bytes;
+	/* Number of transmitted bytes for broadcast traffic */
+	uint64_t	tx_bcast_bytes;
+	/* Number of TPA eligible packets */
+	uint64_t	rx_tpa_eligible_pkt;
+	/* Number of TPA eligible bytes */
+	uint64_t	rx_tpa_eligible_bytes;
+	/* Number of TPA packets */
+	uint64_t	rx_tpa_pkt;
+	/* Number of TPA bytes */
+	uint64_t	rx_tpa_bytes;
+	/* Number of TPA errors */
+	uint64_t	rx_tpa_errors;
+	uint8_t	unused_0[7];
+	/*
+	 * This field is used in Output records to indicate that the output
+	 * is completely written to RAM.  This field should be read as '1'
+	 * to indicate that the output has been completely written.
+	 * When writing a command completion or response to an internal processor,
+	 * the order of writes has to be such that this field is written last.
+	 */
+	uint8_t	valid;
+} __rte_packed;
+
 /***********************
  * hwrm_func_clr_stats *
  ***********************/
@@ -10116,7 +11789,7 @@ struct hwrm_func_backing_store_qcaps_output {
 	 *
 	 * TQM slowpath rings should be sized as follows:
 	 *
-	 * num_entries = num_vnics + num_l2_tx_rings + num_roce_qps + tqm_min_size
+	 * num_entries = num_vnics + num_l2_tx_rings + 2 * num_roce_qps + tqm_min_size
 	 *
 	 * Where:
 	 *   num_vnics is the number of VNICs allocated in the VNIC backing store
@@ -11039,7 +12712,7 @@ struct hwrm_func_backing_store_cfg_input {
 	 *
 	 * TQM slowpath rings should be sized as follows:
 	 *
-	 * num_entries = num_vnics + num_l2_tx_rings + num_roce_qps + tqm_min_size
+	 * num_entries = num_vnics + num_l2_tx_rings + 2 * num_roce_qps + tqm_min_size
 	 *
 	 * Where:
 	 *   num_vnics is the number of VNICs allocated in the VNIC backing store
@@ -14403,7 +16076,7 @@ struct hwrm_port_phy_qcfg_output {
 	/* Module is not inserted. */
 	#define HWRM_PORT_PHY_QCFG_OUTPUT_MODULE_STATUS_NOTINSERTED \
 		UINT32_C(0x4)
-	/* Module is powered down because of over current fault. */
+	/* Module is powered down becuase of over current fault. */
 	#define HWRM_PORT_PHY_QCFG_OUTPUT_MODULE_STATUS_CURRENTFAULT \
 		UINT32_C(0x5)
 	/* Module status is not applicable. */
@@ -16149,7 +17822,18 @@ struct hwrm_port_qstats_input {
 	uint64_t	resp_addr;
 	/* Port ID of port that is being queried. */
 	uint16_t	port_id;
-	uint8_t	unused_0[6];
+	uint8_t	flags;
+	/* This value is not used to avoid backward compatibility issues. */
+	#define HWRM_PORT_QSTATS_INPUT_FLAGS_UNUSED       UINT32_C(0x0)
+	/*
+	 * This bit is set to 1 when request is for a counter mask,
+	 * representing the width of each of the stats counters, rather
+	 * than counters themselves.
+	 */
+	#define HWRM_PORT_QSTATS_INPUT_FLAGS_COUNTER_MASK UINT32_C(0x1)
+	#define HWRM_PORT_QSTATS_INPUT_FLAGS_LAST \
+		HWRM_PORT_QSTATS_INPUT_FLAGS_COUNTER_MASK
+	uint8_t	unused_0[5];
 	/*
 	 * This is the host address where
 	 * Tx port statistics will be stored
@@ -16382,7 +18066,7 @@ struct rx_port_stats_ext {
  * Port Rx Statistics extended PFC WatchDog Format.
  * StormDetect and StormRevert event determination is based
  * on an integration period and a percentage threshold.
- * StormDetect event - when percentage of XOFF frames received
+ * StormDetect event - when percentage of XOFF frames receieved
  * within an integration period exceeds the configured threshold.
  * StormRevert event - when percentage of XON frames received
  * within an integration period exceeds the configured threshold.
@@ -16843,7 +18527,18 @@ struct hwrm_port_qstats_ext_input {
 	 * statistics block in bytes
 	 */
 	uint16_t	rx_stat_size;
-	uint8_t	unused_0[2];
+	uint8_t	flags;
+	/* This value is not used to avoid backward compatibility issues. */
+	#define HWRM_PORT_QSTATS_EXT_INPUT_FLAGS_UNUSED       UINT32_C(0x0)
+	/*
+	 * This bit is set to 1 when request is for the counter mask,
+	 * representing width of each of the stats counters, rather than
+	 * counters themselves.
+	 */
+	#define HWRM_PORT_QSTATS_EXT_INPUT_FLAGS_COUNTER_MASK UINT32_C(0x1)
+	#define HWRM_PORT_QSTATS_EXT_INPUT_FLAGS_LAST \
+		HWRM_PORT_QSTATS_EXT_INPUT_FLAGS_COUNTER_MASK
+	uint8_t	unused_0;
 	/*
 	 * This is the host address where
 	 * Tx port statistics will be stored
@@ -19283,7 +20978,7 @@ struct hwrm_port_phy_mdio_bus_acquire_input {
 	 * Timeout in milli seconds, MDIO BUS will be released automatically
 	 * after this time, if another mdio acquire command is not received
 	 * within the timeout window from the same client.
-	 * A 0xFFFF will hold the bus until this bus is released.
+	 * A 0xFFFF will hold the bus untill this bus is released.
 	 */
 	uint16_t	mdio_bus_timeout;
 	uint8_t	unused_0[2];
@@ -25312,95 +27007,104 @@ struct hwrm_ring_free_input {
 	/* Ring Type. */
 	uint8_t	ring_type;
 	/* L2 Completion Ring (CR) */
-	#define HWRM_RING_FREE_INPUT_RING_TYPE_L2_CMPL   UINT32_C(0x0)
+	#define HWRM_RING_FREE_INPUT_RING_TYPE_L2_CMPL   UINT32_C(0x0)
+	/* TX Ring (TR) */
+	#define HWRM_RING_FREE_INPUT_RING_TYPE_TX        UINT32_C(0x1)
+	/* RX Ring (RR) */
+	#define HWRM_RING_FREE_INPUT_RING_TYPE_RX        UINT32_C(0x2)
+	/* RoCE Notification Completion Ring (ROCE_CR) */
+	#define HWRM_RING_FREE_INPUT_RING_TYPE_ROCE_CMPL UINT32_C(0x3)
+	/* RX Aggregation Ring */
+	#define HWRM_RING_FREE_INPUT_RING_TYPE_RX_AGG    UINT32_C(0x4)
+	/* Notification Queue */
+	#define HWRM_RING_FREE_INPUT_RING_TYPE_NQ        UINT32_C(0x5)
+	#define HWRM_RING_FREE_INPUT_RING_TYPE_LAST \
+		HWRM_RING_FREE_INPUT_RING_TYPE_NQ
+	uint8_t	unused_0;
+	/* Physical number of ring allocated. */
+	uint16_t	ring_id;
+	uint8_t	unused_1[4];
+} __rte_packed;
+
+/* hwrm_ring_free_output (size:128b/16B) */
+struct hwrm_ring_free_output {
+	/* The specific error status for the command. */
+	uint16_t	error_code;
+	/* The HWRM command request type. */
+	uint16_t	req_type;
+	/* The sequence ID from the original command. */
+	uint16_t	seq_id;
+	/* The length of the response data in number of bytes. */
+	uint16_t	resp_len;
+	uint8_t	unused_0[7];
+	/*
+	 * This field is used in Output records to indicate that the output
+	 * is completely written to RAM.  This field should be read as '1'
+	 * to indicate that the output has been completely written.
+	 * When writing a command completion or response to an internal processor,
+	 * the order of writes has to be such that this field is written last.
+	 */
+	uint8_t	valid;
+} __rte_packed;
+
+/*******************
+ * hwrm_ring_reset *
+ *******************/
+
+
+/* hwrm_ring_reset_input (size:192b/24B) */
+struct hwrm_ring_reset_input {
+	/* The HWRM command request type. */
+	uint16_t	req_type;
+	/*
+	 * The completion ring to send the completion event on. This should
+	 * be the NQ ID returned from the `nq_alloc` HWRM command.
+	 */
+	uint16_t	cmpl_ring;
+	/*
+	 * The sequence ID is used by the driver for tracking multiple
+	 * commands. This ID is treated as opaque data by the firmware and
+	 * the value is returned in the `hwrm_resp_hdr` upon completion.
+	 */
+	uint16_t	seq_id;
+	/*
+	 * The target ID of the command:
+	 * * 0x0-0xFFF8 - The function ID
+	 * * 0xFFF8-0xFFFC, 0xFFFE - Reserved for internal processors
+	 * * 0xFFFD - Reserved for user-space HWRM interface
+	 * * 0xFFFF - HWRM
+	 */
+	uint16_t	target_id;
+	/*
+	 * A physical address pointer pointing to a host buffer that the
+	 * command's response data will be written. This can be either a host
+	 * physical address (HPA) or a guest physical address (GPA) and must
+	 * point to a physically contiguous block of memory.
+	 */
+	uint64_t	resp_addr;
+	/* Ring Type. */
+	uint8_t	ring_type;
+	/* L2 Completion Ring (CR) */
+	#define HWRM_RING_RESET_INPUT_RING_TYPE_L2_CMPL     UINT32_C(0x0)
 	/* TX Ring (TR) */
-	#define HWRM_RING_FREE_INPUT_RING_TYPE_TX        UINT32_C(0x1)
+	#define HWRM_RING_RESET_INPUT_RING_TYPE_TX          UINT32_C(0x1)
 	/* RX Ring (RR) */
-	#define HWRM_RING_FREE_INPUT_RING_TYPE_RX        UINT32_C(0x2)
+	#define HWRM_RING_RESET_INPUT_RING_TYPE_RX          UINT32_C(0x2)
 	/* RoCE Notification Completion Ring (ROCE_CR) */
-	#define HWRM_RING_FREE_INPUT_RING_TYPE_ROCE_CMPL UINT32_C(0x3)
-	/* RX Aggregation Ring */
-	#define HWRM_RING_FREE_INPUT_RING_TYPE_RX_AGG    UINT32_C(0x4)
-	/* Notification Queue */
-	#define HWRM_RING_FREE_INPUT_RING_TYPE_NQ        UINT32_C(0x5)
-	#define HWRM_RING_FREE_INPUT_RING_TYPE_LAST \
-		HWRM_RING_FREE_INPUT_RING_TYPE_NQ
-	uint8_t	unused_0;
-	/* Physical number of ring allocated. */
-	uint16_t	ring_id;
-	uint8_t	unused_1[4];
-} __rte_packed;
-
-/* hwrm_ring_free_output (size:128b/16B) */
-struct hwrm_ring_free_output {
-	/* The specific error status for the command. */
-	uint16_t	error_code;
-	/* The HWRM command request type. */
-	uint16_t	req_type;
-	/* The sequence ID from the original command. */
-	uint16_t	seq_id;
-	/* The length of the response data in number of bytes. */
-	uint16_t	resp_len;
-	uint8_t	unused_0[7];
+	#define HWRM_RING_RESET_INPUT_RING_TYPE_ROCE_CMPL   UINT32_C(0x3)
 	/*
-	 * This field is used in Output records to indicate that the output
-	 * is completely written to RAM.  This field should be read as '1'
-	 * to indicate that the output has been completely written.
-	 * When writing a command completion or response to an internal processor,
-	 * the order of writes has to be such that this field is written last.
+	 * Rx Ring Group.  This is to reset rx and aggregation in an atomic
+	 * operation. Completion ring associated with this ring group is
+	 * not reset.
 	 */
-	uint8_t	valid;
-} __rte_packed;
-
-/*******************
- * hwrm_ring_reset *
- *******************/
-
-
-/* hwrm_ring_reset_input (size:192b/24B) */
-struct hwrm_ring_reset_input {
-	/* The HWRM command request type. */
-	uint16_t	req_type;
-	/*
-	 * The completion ring to send the completion event on. This should
-	 * be the NQ ID returned from the `nq_alloc` HWRM command.
-	 */
-	uint16_t	cmpl_ring;
-	/*
-	 * The sequence ID is used by the driver for tracking multiple
-	 * commands. This ID is treated as opaque data by the firmware and
-	 * the value is returned in the `hwrm_resp_hdr` upon completion.
-	 */
-	uint16_t	seq_id;
-	/*
-	 * The target ID of the command:
-	 * * 0x0-0xFFF8 - The function ID
-	 * * 0xFFF8-0xFFFC, 0xFFFE - Reserved for internal processors
-	 * * 0xFFFD - Reserved for user-space HWRM interface
-	 * * 0xFFFF - HWRM
-	 */
-	uint16_t	target_id;
-	/*
-	 * A physical address pointer pointing to a host buffer that the
-	 * command's response data will be written. This can be either a host
-	 * physical address (HPA) or a guest physical address (GPA) and must
-	 * point to a physically contiguous block of memory.
-	 */
-	uint64_t	resp_addr;
-	/* Ring Type. */
-	uint8_t	ring_type;
-	/* L2 Completion Ring (CR) */
-	#define HWRM_RING_RESET_INPUT_RING_TYPE_L2_CMPL   UINT32_C(0x0)
-	/* TX Ring (TR) */
-	#define HWRM_RING_RESET_INPUT_RING_TYPE_TX        UINT32_C(0x1)
-	/* RX Ring (RR) */
-	#define HWRM_RING_RESET_INPUT_RING_TYPE_RX        UINT32_C(0x2)
-	/* RoCE Notification Completion Ring (ROCE_CR) */
-	#define HWRM_RING_RESET_INPUT_RING_TYPE_ROCE_CMPL UINT32_C(0x3)
+	#define HWRM_RING_RESET_INPUT_RING_TYPE_RX_RING_GRP UINT32_C(0x6)
 	#define HWRM_RING_RESET_INPUT_RING_TYPE_LAST \
-		HWRM_RING_RESET_INPUT_RING_TYPE_ROCE_CMPL
+		HWRM_RING_RESET_INPUT_RING_TYPE_RX_RING_GRP
 	uint8_t	unused_0;
-	/* Physical number of the ring. */
+	/*
+	 * Physical number of the ring. When ring type is rx_ring_grp, ring id
+	 * actually refers to ring group id.
+	 */
 	uint16_t	ring_id;
 	uint8_t	unused_1[4];
 } __rte_packed;
@@ -25615,7 +27319,18 @@ struct hwrm_ring_cmpl_ring_qaggint_params_input {
 	uint64_t	resp_addr;
 	/* Physical number of completion ring. */
 	uint16_t	ring_id;
-	uint8_t	unused_0[6];
+	uint16_t	flags;
+	#define HWRM_RING_CMPL_RING_QAGGINT_PARAMS_INPUT_FLAGS_UNUSED_0_MASK \
+		UINT32_C(0x3)
+	#define HWRM_RING_CMPL_RING_QAGGINT_PARAMS_INPUT_FLAGS_UNUSED_0_SFT 0
+	/*
+	 * Set this flag to 1 when querying parameters on a notification
+	 * queue. Set this flag to 0 when querying parameters on a
+	 * completion queue or completion ring.
+	 */
+	#define HWRM_RING_CMPL_RING_QAGGINT_PARAMS_INPUT_FLAGS_IS_NQ \
+		UINT32_C(0x4)
+	uint8_t	unused_0[4];
 } __rte_packed;
 
 /* hwrm_ring_cmpl_ring_qaggint_params_output (size:256b/32B) */
@@ -25652,19 +27367,19 @@ struct hwrm_ring_cmpl_ring_qaggint_params_output {
 	 */
 	uint16_t	num_cmpl_dma_aggr_during_int;
 	/*
-	 * Timer in unit of 80-nsec used to aggregate completions before
+	 * Timer used to aggregate completions before
 	 * DMA during the normal mode (not in interrupt mode).
 	 */
 	uint16_t	cmpl_aggr_dma_tmr;
 	/*
-	 * Timer in unit of 80-nsec used to aggregate completions before
-	 * DMA during the interrupt mode.
+	 * Timer used to aggregate completions before
+	 * DMA when in interrupt mode.
 	 */
 	uint16_t	cmpl_aggr_dma_tmr_during_int;
-	/* Minimum time (in unit of 80-nsec) between two interrupts. */
+	/* Minimum time between two interrupts. */
 	uint16_t	int_lat_tmr_min;
 	/*
-	 * Maximum wait time (in unit of 80-nsec) spent aggregating
+	 * Maximum wait time spent aggregating
 	 * completions before signaling the interrupt after the
 	 * interrupt is enabled.
 	 */
@@ -25738,7 +27453,7 @@ struct hwrm_ring_cmpl_ring_cfg_aggint_params_input {
 	/*
 	 * Set this flag to 1 when configuring parameters on a
 	 * notification queue. Set this flag to 0 when configuring
-	 * parameters on a completion queue.
+	 * parameters on a completion queue or completion ring.
 	 */
 	#define HWRM_RING_CMPL_RING_CFG_AGGINT_PARAMS_INPUT_FLAGS_IS_NQ \
 		UINT32_C(0x4)
@@ -25753,20 +27468,20 @@ struct hwrm_ring_cmpl_ring_cfg_aggint_params_input {
 	 */
 	uint16_t	num_cmpl_dma_aggr_during_int;
 	/*
-	 * Timer in unit of 80-nsec used to aggregate completions before
+	 * Timer used to aggregate completions before
 	 * DMA during the normal mode (not in interrupt mode).
 	 */
 	uint16_t	cmpl_aggr_dma_tmr;
 	/*
-	 * Timer in unit of 80-nsec used to aggregate completions before
-	 * DMA during the interrupt mode.
+	 * Timer used to aggregate completions before
+	 * DMA while in interrupt mode.
 	 */
 	uint16_t	cmpl_aggr_dma_tmr_during_int;
-	/* Minimum time (in unit of 80-nsec) between two interrupts. */
+	/* Minimum time between two interrupts. */
 	uint16_t	int_lat_tmr_min;
 	/*
-	 * Maximum wait time (in unit of 80-nsec) spent aggregating
-	 * cmpls before signaling the interrupt after the
+	 * Maximum wait time spent aggregating
+	 * completions before signaling the interrupt after the
 	 * interrupt is enabled.
 	 */
 	uint16_t	int_lat_tmr_max;
@@ -33339,78 +35054,246 @@ struct hwrm_tf_version_get_input {
 	 * point to a physically contiguous block of memory.
 	 */
 	uint64_t	resp_addr;
-} __rte_packed;
-
-/* hwrm_tf_version_get_output (size:128b/16B) */
-struct hwrm_tf_version_get_output {
-	/* The specific error status for the command. */
-	uint16_t	error_code;
-	/* The HWRM command request type. */
-	uint16_t	req_type;
-	/* The sequence ID from the original command. */
-	uint16_t	seq_id;
-	/* The length of the response data in number of bytes. */
-	uint16_t	resp_len;
-	/* Version Major number. */
-	uint8_t	major;
-	/* Version Minor number. */
-	uint8_t	minor;
-	/* Version Update number. */
-	uint8_t	update;
-	/* unused. */
-	uint8_t	unused0[4];
+} __rte_packed;
+
+/* hwrm_tf_version_get_output (size:128b/16B) */
+struct hwrm_tf_version_get_output {
+	/* The specific error status for the command. */
+	uint16_t	error_code;
+	/* The HWRM command request type. */
+	uint16_t	req_type;
+	/* The sequence ID from the original command. */
+	uint16_t	seq_id;
+	/* The length of the response data in number of bytes. */
+	uint16_t	resp_len;
+	/* Version Major number. */
+	uint8_t	major;
+	/* Version Minor number. */
+	uint8_t	minor;
+	/* Version Update number. */
+	uint8_t	update;
+	/* unused. */
+	uint8_t	unused0[4];
+	/*
+	 * This field is used in Output records to indicate that the output
+	 * is completely written to RAM. This field should be read as '1'
+	 * to indicate that the output has been completely written.
+	 * When writing a command completion or response to an internal
+	 * processor, the order of writes has to be such that this field is
+	 * written last.
+	 */
+	uint8_t	valid;
+} __rte_packed;
+
+/************************
+ * hwrm_tf_session_open *
+ ************************/
+
+
+/* hwrm_tf_session_open_input (size:640b/80B) */
+struct hwrm_tf_session_open_input {
+	/* The HWRM command request type. */
+	uint16_t	req_type;
+	/*
+	 * The completion ring to send the completion event on. This should
+	 * be the NQ ID returned from the `nq_alloc` HWRM command.
+	 */
+	uint16_t	cmpl_ring;
+	/*
+	 * The sequence ID is used by the driver for tracking multiple
+	 * commands. This ID is treated as opaque data by the firmware and
+	 * the value is returned in the `hwrm_resp_hdr` upon completion.
+	 */
+	uint16_t	seq_id;
+	/*
+	 * The target ID of the command:
+	 * * 0x0-0xFFF8 - The function ID
+	 * * 0xFFF8-0xFFFC, 0xFFFE - Reserved for internal processors
+	 * * 0xFFFD - Reserved for user-space HWRM interface
+	 * * 0xFFFF - HWRM
+	 */
+	uint16_t	target_id;
+	/*
+	 * A physical address pointer pointing to a host buffer that the
+	 * command's response data will be written. This can be either a host
+	 * physical address (HPA) or a guest physical address (GPA) and must
+	 * point to a physically contiguous block of memory.
+	 */
+	uint64_t	resp_addr;
+	/* Name of the session. */
+	uint8_t	session_name[64];
+} __rte_packed;
+
+/* hwrm_tf_session_open_output (size:192b/24B) */
+struct hwrm_tf_session_open_output {
+	/* The specific error status for the command. */
+	uint16_t	error_code;
+	/* The HWRM command request type. */
+	uint16_t	req_type;
+	/* The sequence ID from the original command. */
+	uint16_t	seq_id;
+	/* The length of the response data in number of bytes. */
+	uint16_t	resp_len;
+	/*
+	 * Unique session identifier for the session created by the
+	 * firmware.
+	 */
+	uint32_t	fw_session_id;
+	/*
+	 * Unique session client identifier for the first client on
+	 * the newly created session.
+	 */
+	uint32_t	fw_session_client_id;
+	/* unused. */
+	uint32_t	unused0;
+	/* unused. */
+	uint8_t	unused1[3];
+	/*
+	 * This field is used in Output records to indicate that the output
+	 * is completely written to RAM. This field should be read as '1'
+	 * to indicate that the output has been completely written.
+	 * When writing a command completion or response to an internal
+	 * processor, the order of writes has to be such that this field is
+	 * written last.
+	 */
+	uint8_t	valid;
+} __rte_packed;
+
+/**************************
+ * hwrm_tf_session_attach *
+ **************************/
+
+
+/* hwrm_tf_session_attach_input (size:704b/88B) */
+struct hwrm_tf_session_attach_input {
+	/* The HWRM command request type. */
+	uint16_t	req_type;
+	/*
+	 * The completion ring to send the completion event on. This should
+	 * be the NQ ID returned from the `nq_alloc` HWRM command.
+	 */
+	uint16_t	cmpl_ring;
+	/*
+	 * The sequence ID is used by the driver for tracking multiple
+	 * commands. This ID is treated as opaque data by the firmware and
+	 * the value is returned in the `hwrm_resp_hdr` upon completion.
+	 */
+	uint16_t	seq_id;
+	/*
+	 * The target ID of the command:
+	 * * 0x0-0xFFF8 - The function ID
+	 * * 0xFFF8-0xFFFC, 0xFFFE - Reserved for internal processors
+	 * * 0xFFFD - Reserved for user-space HWRM interface
+	 * * 0xFFFF - HWRM
+	 */
+	uint16_t	target_id;
+	/*
+	 * A physical address pointer pointing to a host buffer that the
+	 * command's response data will be written. This can be either a host
+	 * physical address (HPA) or a guest physical address (GPA) and must
+	 * point to a physically contiguous block of memory.
+	 */
+	uint64_t	resp_addr;
+	/*
+	 * Unique session identifier for the session that the attach
+	 * request want to attach to. This value originates from the
+	 * shared session memory that the attach request opened by
+	 * way of the 'attach name' that was passed in to the core
+	 * attach API.
+	 * The fw_session_id of the attach session includes PCIe bus
+	 * info to distinguish the PF and session info to identify
+	 * the associated TruFlow session.
+	 */
+	uint32_t	attach_fw_session_id;
+	/* unused. */
+	uint32_t	unused0;
+	/* Name of the session it self. */
+	uint8_t	session_name[64];
+} __rte_packed;
+
+/* hwrm_tf_session_attach_output (size:128b/16B) */
+struct hwrm_tf_session_attach_output {
+	/* The specific error status for the command. */
+	uint16_t	error_code;
+	/* The HWRM command request type. */
+	uint16_t	req_type;
+	/* The sequence ID from the original command. */
+	uint16_t	seq_id;
+	/* The length of the response data in number of bytes. */
+	uint16_t	resp_len;
+	/*
+	 * Unique session identifier for the session created by the
+	 * firmware. It includes PCIe bus info to distinguish the PF
+	 * and session info to identify the associated TruFlow
+	 * session. This fw_session_id is unique to the attach
+	 * request.
+	 */
+	uint32_t	fw_session_id;
+	/* unused. */
+	uint8_t	unused0[3];
+	/*
+	 * This field is used in Output records to indicate that the output
+	 * is completely written to RAM. This field should be read as '1'
+	 * to indicate that the output has been completely written.
+	 * When writing a command completion or response to an internal
+	 * processor, the order of writes has to be such that this field is
+	 * written last.
+	 */
+	uint8_t	valid;
+} __rte_packed;
+
+/****************************
+ * hwrm_tf_session_register *
+ ****************************/
+
+
+/* hwrm_tf_session_register_input (size:704b/88B) */
+struct hwrm_tf_session_register_input {
+	/* The HWRM command request type. */
+	uint16_t	req_type;
+	/*
+	 * The completion ring to send the completion event on. This should
+	 * be the NQ ID returned from the `nq_alloc` HWRM command.
+	 */
+	uint16_t	cmpl_ring;
+	/*
+	 * The sequence ID is used by the driver for tracking multiple
+	 * commands. This ID is treated as opaque data by the firmware and
+	 * the value is returned in the `hwrm_resp_hdr` upon completion.
+	 */
+	uint16_t	seq_id;
+	/*
+	 * The target ID of the command:
+	 * * 0x0-0xFFF8 - The function ID
+	 * * 0xFFF8-0xFFFC, 0xFFFE - Reserved for internal processors
+	 * * 0xFFFD - Reserved for user-space HWRM interface
+	 * * 0xFFFF - HWRM
+	 */
+	uint16_t	target_id;
+	/*
+	 * A physical address pointer pointing to a host buffer that the
+	 * command's response data will be written. This can be either a host
+	 * physical address (HPA) or a guest physical address (GPA) and must
+	 * point to a physically contiguous block of memory.
+	 */
+	uint64_t	resp_addr;
 	/*
-	 * This field is used in Output records to indicate that the output
-	 * is completely written to RAM. This field should be read as '1'
-	 * to indicate that the output has been completely written.
-	 * When writing a command completion or response to an internal
-	 * processor, the order of writes has to be such that this field is
-	 * written last.
-	 */
-	uint8_t	valid;
-} __rte_packed;
-
-/************************
- * hwrm_tf_session_open *
- ************************/
-
-
-/* hwrm_tf_session_open_input (size:640b/80B) */
-struct hwrm_tf_session_open_input {
-	/* The HWRM command request type. */
-	uint16_t	req_type;
-	/*
-	 * The completion ring to send the completion event on. This should
-	 * be the NQ ID returned from the `nq_alloc` HWRM command.
-	 */
-	uint16_t	cmpl_ring;
-	/*
-	 * The sequence ID is used by the driver for tracking multiple
-	 * commands. This ID is treated as opaque data by the firmware and
-	 * the value is returned in the `hwrm_resp_hdr` upon completion.
-	 */
-	uint16_t	seq_id;
-	/*
-	 * The target ID of the command:
-	 * * 0x0-0xFFF8 - The function ID
-	 * * 0xFFF8-0xFFFC, 0xFFFE - Reserved for internal processors
-	 * * 0xFFFD - Reserved for user-space HWRM interface
-	 * * 0xFFFF - HWRM
-	 */
-	uint16_t	target_id;
-	/*
-	 * A physical address pointer pointing to a host buffer that the
-	 * command's response data will be written. This can be either a host
-	 * physical address (HPA) or a guest physical address (GPA) and must
-	 * point to a physically contiguous block of memory.
+	 * Unique session identifier for the session that the
+	 * register request want to create a new client on. This
+	 * value originates from the first open request.
+	 * The fw_session_id of the attach session includes PCIe bus
+	 * info to distinguish the PF and session info to identify
+	 * the associated TruFlow session.
 	 */
-	uint64_t	resp_addr;
-	/* Name of the session. */
-	uint8_t	session_name[64];
+	uint32_t	fw_session_id;
+	/* unused. */
+	uint32_t	unused0;
+	/* Name of the session client. */
+	uint8_t	session_client_name[64];
 } __rte_packed;
 
-/* hwrm_tf_session_open_output (size:128b/16B) */
-struct hwrm_tf_session_open_output {
+/* hwrm_tf_session_register_output (size:128b/16B) */
+struct hwrm_tf_session_register_output {
 	/* The specific error status for the command. */
 	uint16_t	error_code;
 	/* The HWRM command request type. */
@@ -33420,12 +35303,11 @@ struct hwrm_tf_session_open_output {
 	/* The length of the response data in number of bytes. */
 	uint16_t	resp_len;
 	/*
-	 * Unique session identifier for the session created by the
-	 * firmware. It includes PCIe bus info to distinguish the PF
-	 * and session info to identify the associated TruFlow
-	 * session.
+	 * Unique session client identifier for the session created
+	 * by the firmware. It includes the session the client it
+	 * attached to and session client info.
 	 */
-	uint32_t	fw_session_id;
+	uint32_t	fw_session_client_id;
 	/* unused. */
 	uint8_t	unused0[3];
 	/*
@@ -33439,13 +35321,13 @@ struct hwrm_tf_session_open_output {
 	uint8_t	valid;
 } __rte_packed;
 
-/**************************
- * hwrm_tf_session_attach *
- **************************/
+/******************************
+ * hwrm_tf_session_unregister *
+ ******************************/
 
 
-/* hwrm_tf_session_attach_input (size:704b/88B) */
-struct hwrm_tf_session_attach_input {
+/* hwrm_tf_session_unregister_input (size:192b/24B) */
+struct hwrm_tf_session_unregister_input {
 	/* The HWRM command request type. */
 	uint16_t	req_type;
 	/*
@@ -33475,24 +35357,19 @@ struct hwrm_tf_session_attach_input {
 	 */
 	uint64_t	resp_addr;
 	/*
-	 * Unique session identifier for the session that the attach
-	 * request want to attach to. This value originates from the
-	 * shared session memory that the attach request opened by
-	 * way of the 'attach name' that was passed in to the core
-	 * attach API.
-	 * The fw_session_id of the attach session includes PCIe bus
-	 * info to distinguish the PF and session info to identify
-	 * the associated TruFlow session.
+	 * Unique session identifier for the session that the
+	 * unregister request want to close a session client on.
 	 */
-	uint32_t	attach_fw_session_id;
-	/* unused. */
-	uint32_t	unused0;
-	/* Name of the session it self. */
-	uint8_t	session_name[64];
+	uint32_t	fw_session_id;
+	/*
+	 * Unique session client identifier for the session that the
+	 * unregister request want to close.
+	 */
+	uint32_t	fw_session_client_id;
 } __rte_packed;
 
-/* hwrm_tf_session_attach_output (size:128b/16B) */
-struct hwrm_tf_session_attach_output {
+/* hwrm_tf_session_unregister_output (size:128b/16B) */
+struct hwrm_tf_session_unregister_output {
 	/* The specific error status for the command. */
 	uint16_t	error_code;
 	/* The HWRM command request type. */
@@ -33501,16 +35378,8 @@ struct hwrm_tf_session_attach_output {
 	uint16_t	seq_id;
 	/* The length of the response data in number of bytes. */
 	uint16_t	resp_len;
-	/*
-	 * Unique session identifier for the session created by the
-	 * firmware. It includes PCIe bus info to distinguish the PF
-	 * and session info to identify the associated TruFlow
-	 * session. This fw_session_id is unique to the attach
-	 * request.
-	 */
-	uint32_t	fw_session_id;
 	/* unused. */
-	uint8_t	unused0[3];
+	uint8_t	unused0[7];
 	/*
 	 * This field is used in Output records to indicate that the output
 	 * is completely written to RAM. This field should be read as '1'
@@ -33746,15 +35615,17 @@ struct hwrm_tf_session_resc_qcaps_input {
 	#define HWRM_TF_SESSION_RESC_QCAPS_INPUT_FLAGS_DIR_LAST \
 		HWRM_TF_SESSION_RESC_QCAPS_INPUT_FLAGS_DIR_TX
 	/*
-	 * Defines the size, in bytes, of the provided qcaps_addr
+	 * Defines the size of the provided qcaps_addr array
 	 * buffer. The size should be set to the Resource Manager
-	 * provided max qcaps value that is device specific. This is
-	 * the max size possible.
+	 * provided max number of qcaps entries which is device
+	 * specific. Resource Manager gets the max size from HCAPI
+	 * RM.
 	 */
-	uint16_t	size;
+	uint16_t	qcaps_size;
 	/*
-	 * This is the DMA address for the qcaps output data
-	 * array. Array is of tf_rm_cap type and is device specific.
+	 * This is the DMA address for the qcaps output data array
+	 * buffer. Array is of tf_rm_resc_req_entry type and is
+	 * device specific.
 	 */
 	uint64_t	qcaps_addr;
 } __rte_packed;
@@ -33772,29 +35643,28 @@ struct hwrm_tf_session_resc_qcaps_output {
 	/* Control flags. */
 	uint32_t	flags;
 	/* Session reservation strategy. */
-	#define HWRM_TF_SESSION_RESC_QCAPS_OUTPUT_FLAGS_SESS_RES_STRATEGY_MASK \
+	#define HWRM_TF_SESSION_RESC_QCAPS_OUTPUT_FLAGS_SESS_RESV_STRATEGY_MASK \
 		UINT32_C(0x3)
-	#define HWRM_TF_SESSION_RESC_QCAPS_OUTPUT_FLAGS_SESS_RES_STRATEGY_SFT \
+	#define HWRM_TF_SESSION_RESC_QCAPS_OUTPUT_FLAGS_SESS_RESV_STRATEGY_SFT \
 		0
 	/* Static partitioning. */
-	#define HWRM_TF_SESSION_RESC_QCAPS_OUTPUT_FLAGS_SESS_RES_STRATEGY_STATIC \
+	#define HWRM_TF_SESSION_RESC_QCAPS_OUTPUT_FLAGS_SESS_RESV_STRATEGY_STATIC \
 		UINT32_C(0x0)
 	/* Strategy 1. */
-	#define HWRM_TF_SESSION_RESC_QCAPS_OUTPUT_FLAGS_SESS_RES_STRATEGY_1 \
+	#define HWRM_TF_SESSION_RESC_QCAPS_OUTPUT_FLAGS_SESS_RESV_STRATEGY_1 \
 		UINT32_C(0x1)
 	/* Strategy 2. */
-	#define HWRM_TF_SESSION_RESC_QCAPS_OUTPUT_FLAGS_SESS_RES_STRATEGY_2 \
+	#define HWRM_TF_SESSION_RESC_QCAPS_OUTPUT_FLAGS_SESS_RESV_STRATEGY_2 \
 		UINT32_C(0x2)
 	/* Strategy 3. */
-	#define HWRM_TF_SESSION_RESC_QCAPS_OUTPUT_FLAGS_SESS_RES_STRATEGY_3 \
+	#define HWRM_TF_SESSION_RESC_QCAPS_OUTPUT_FLAGS_SESS_RESV_STRATEGY_3 \
 		UINT32_C(0x3)
-	#define HWRM_TF_SESSION_RESC_QCAPS_OUTPUT_FLAGS_SESS_RES_STRATEGY_LAST \
-		HWRM_TF_SESSION_RESC_QCAPS_OUTPUT_FLAGS_SESS_RES_STRATEGY_3
+	#define HWRM_TF_SESSION_RESC_QCAPS_OUTPUT_FLAGS_SESS_RESV_STRATEGY_LAST \
+		HWRM_TF_SESSION_RESC_QCAPS_OUTPUT_FLAGS_SESS_RESV_STRATEGY_3
 	/*
-	 * Size of the returned tf_rm_cap data array. The value
-	 * cannot exceed the size defined by the input msg. The data
-	 * array is returned using the qcaps_addr specified DMA
-	 * address also provided by the input msg.
+	 * Size of the returned qcaps_addr data array buffer. The
+	 * value cannot exceed the size defined by the input msg,
+	 * qcaps_size.
 	 */
 	uint16_t	size;
 	/* unused. */
@@ -33817,7 +35687,7 @@ struct hwrm_tf_session_resc_qcaps_output {
  ******************************/
 
 
-/* hwrm_tf_session_resc_alloc_input (size:256b/32B) */
+/* hwrm_tf_session_resc_alloc_input (size:320b/40B) */
 struct hwrm_tf_session_resc_alloc_input {
 	/* The HWRM command request type. */
 	uint16_t	req_type;
@@ -33860,16 +35730,25 @@ struct hwrm_tf_session_resc_alloc_input {
 	#define HWRM_TF_SESSION_RESC_ALLOC_INPUT_FLAGS_DIR_LAST \
 		HWRM_TF_SESSION_RESC_ALLOC_INPUT_FLAGS_DIR_TX
 	/*
-	 * Defines the size, in bytes, of the provided num_addr
-	 * buffer.
+	 * Defines the array size of the provided req_addr and
+	 * resv_addr array buffers. Should be set to the number of
+	 * request entries.
 	 */
-	uint16_t	size;
+	uint16_t	req_size;
+	/*
+	 * This is the DMA address for the request input data array
+	 * buffer. Array is of tf_rm_resc_req_entry type. Size of the
+	 * array buffer is provided by the 'req_size' field in this
+	 * message.
+	 */
+	uint64_t	req_addr;
 	/*
-	 * This is the DMA address for the num input data array
-	 * buffer. Array is of tf_rm_num type. Size of the buffer is
-	 * provided by the 'size' field in this message.
+	 * This is the DMA address for the resc output data array
+	 * buffer. Array is of tf_rm_resc_entry type. Size of the array
+	 * buffer is provided by the 'req_size' field in this
+	 * message.
 	 */
-	uint64_t	num_addr;
+	uint64_t	resc_addr;
 } __rte_packed;
 
 /* hwrm_tf_session_resc_alloc_output (size:128b/16B) */
@@ -33882,8 +35761,15 @@ struct hwrm_tf_session_resc_alloc_output {
 	uint16_t	seq_id;
 	/* The length of the response data in number of bytes. */
 	uint16_t	resp_len;
+	/*
+	 * Size of the returned tf_rm_resc_entry data array. The value
+	 * cannot exceed the req_size defined by the input msg. The data
+	 * array is returned using the resv_addr specified DMA
+	 * address also provided by the input msg.
+	 */
+	uint16_t	size;
 	/* unused. */
-	uint8_t	unused0[7];
+	uint8_t	unused0[5];
 	/*
 	 * This field is used in Output records to indicate that the output
 	 * is completely written to RAM. This field should be read as '1'
@@ -33946,11 +35832,12 @@ struct hwrm_tf_session_resc_free_input {
 	 * Defines the size, in bytes, of the provided free_addr
 	 * buffer.
 	 */
-	uint16_t	size;
+	uint16_t	free_size;
 	/*
 	 * This is the DMA address for the free input data array
-	 * buffer.  Array of tf_rm_res type. Size of the buffer is
-	 * provided by the 'size field of this message.
+	 * buffer.  Array is of tf_rm_resc_entry type. Size of the
+	 * buffer is provided by the 'free_size' field of this
+	 * message.
 	 */
 	uint64_t	free_addr;
 } __rte_packed;
@@ -34029,11 +35916,12 @@ struct hwrm_tf_session_resc_flush_input {
 	 * Defines the size, in bytes, of the provided flush_addr
 	 * buffer.
 	 */
-	uint16_t	size;
+	uint16_t	flush_size;
 	/*
 	 * This is the DMA address for the flush input data array
-	 * buffer.  Array of tf_rm_res type. Size of the buffer is
-	 * provided by the 'size' field in this message.
+	 * buffer.  Array of tf_rm_resc_entry type. Size of the
+	 * buffer is provided by the 'flush_size' field in this
+	 * message.
 	 */
 	uint64_t	flush_addr;
 } __rte_packed;
@@ -34062,12 +35950,9 @@ struct hwrm_tf_session_resc_flush_output {
 } __rte_packed;
 
 /* TruFlow RM capability of a resource. */
-/* tf_rm_cap (size:64b/8B) */
-struct tf_rm_cap {
-	/*
-	 * Type of the resource, defined globally in the
-	 * hwrm_tf_resc_type enum.
-	 */
+/* tf_rm_resc_req_entry (size:64b/8B) */
+struct tf_rm_resc_req_entry {
+	/* Type of the resource, defined globally in HCAPI RM. */
 	uint32_t	type;
 	/* Minimum value. */
 	uint16_t	min;
@@ -34075,25 +35960,10 @@ struct tf_rm_cap {
 	uint16_t	max;
 } __rte_packed;
 
-/* TruFlow RM number of a resource. */
-/* tf_rm_num (size:64b/8B) */
-struct tf_rm_num {
-	/*
-	 * Type of the resource, defined globally in the
-	 * hwrm_tf_resc_type enum.
-	 */
-	uint32_t	type;
-	/* Number of resources. */
-	uint32_t	num;
-} __rte_packed;
-
 /* TruFlow RM reservation information. */
-/* tf_rm_res (size:64b/8B) */
-struct tf_rm_res {
-	/*
-	 * Type of the resource, defined globally in the
-	 * hwrm_tf_resc_type enum.
-	 */
+/* tf_rm_resc_entry (size:64b/8B) */
+struct tf_rm_resc_entry {
+	/* Type of the resource, defined globally in HCAPI RM. */
 	uint32_t	type;
 	/* Start offset. */
 	uint16_t	start;
@@ -34925,6 +36795,162 @@ struct hwrm_tf_ext_em_qcfg_output {
 	uint8_t	valid;
 } __rte_packed;
 
+/*********************
+ * hwrm_tf_em_insert *
+ *********************/
+
+
+/* hwrm_tf_em_insert_input (size:832b/104B) */
+struct hwrm_tf_em_insert_input {
+	/* The HWRM command request type. */
+	uint16_t	req_type;
+	/*
+	 * The completion ring to send the completion event on. This should
+	 * be the NQ ID returned from the `nq_alloc` HWRM command.
+	 */
+	uint16_t	cmpl_ring;
+	/*
+	 * The sequence ID is used by the driver for tracking multiple
+	 * commands. This ID is treated as opaque data by the firmware and
+	 * the value is returned in the `hwrm_resp_hdr` upon completion.
+	 */
+	uint16_t	seq_id;
+	/*
+	 * The target ID of the command:
+	 * * 0x0-0xFFF8 - The function ID
+	 * * 0xFFF8-0xFFFC, 0xFFFE - Reserved for internal processors
+	 * * 0xFFFD - Reserved for user-space HWRM interface
+	 * * 0xFFFF - HWRM
+	 */
+	uint16_t	target_id;
+	/*
+	 * A physical address pointer pointing to a host buffer that the
+	 * command's response data will be written. This can be either a host
+	 * physical address (HPA) or a guest physical address (GPA) and must
+	 * point to a physically contiguous block of memory.
+	 */
+	uint64_t	resp_addr;
+	/* Firmware Session Id. */
+	uint32_t	fw_session_id;
+	/* Control Flags. */
+	uint16_t	flags;
+	/* Indicates the flow direction. */
+	#define HWRM_TF_EM_INSERT_INPUT_FLAGS_DIR     UINT32_C(0x1)
+	/* If this bit set to 0, then it indicates rx flow. */
+	#define HWRM_TF_EM_INSERT_INPUT_FLAGS_DIR_RX    UINT32_C(0x0)
+	/* If this bit is set to 1, then it indicates that tx flow. */
+	#define HWRM_TF_EM_INSERT_INPUT_FLAGS_DIR_TX    UINT32_C(0x1)
+	#define HWRM_TF_EM_INSERT_INPUT_FLAGS_DIR_LAST \
+		HWRM_TF_EM_INSERT_INPUT_FLAGS_DIR_TX
+	/* Reported match strength. */
+	uint16_t	strength;
+	/* Index to action. */
+	uint32_t	action_ptr;
+	/* Index of EM record. */
+	uint32_t	em_record_idx;
+	/* EM Key value. */
+	uint64_t	em_key[8];
+	/* Number of bits in em_key. */
+	uint16_t	em_key_bitlen;
+	/* unused. */
+	uint16_t	unused0[3];
+} __rte_packed;
+
+/* hwrm_tf_em_insert_output (size:128b/16B) */
+struct hwrm_tf_em_insert_output {
+	/* The specific error status for the command. */
+	uint16_t	error_code;
+	/* The HWRM command request type. */
+	uint16_t	req_type;
+	/* The sequence ID from the original command. */
+	uint16_t	seq_id;
+	/* The length of the response data in number of bytes. */
+	uint16_t	resp_len;
+	/* EM record pointer index. */
+	uint16_t	rptr_index;
+	/* EM record offset 0~3. */
+	uint8_t	rptr_entry;
+	/* Number of word entries consumed by the key. */
+	uint8_t	num_of_entries;
+	/* unused. */
+	uint32_t	unused0;
+} __rte_packed;
+
+/*********************
+ * hwrm_tf_em_delete *
+ *********************/
+
+
+/* hwrm_tf_em_delete_input (size:832b/104B) */
+struct hwrm_tf_em_delete_input {
+	/* The HWRM command request type. */
+	uint16_t	req_type;
+	/*
+	 * The completion ring to send the completion event on. This should
+	 * be the NQ ID returned from the `nq_alloc` HWRM command.
+	 */
+	uint16_t	cmpl_ring;
+	/*
+	 * The sequence ID is used by the driver for tracking multiple
+	 * commands. This ID is treated as opaque data by the firmware and
+	 * the value is returned in the `hwrm_resp_hdr` upon completion.
+	 */
+	uint16_t	seq_id;
+	/*
+	 * The target ID of the command:
+	 * * 0x0-0xFFF8 - The function ID
+	 * * 0xFFF8-0xFFFC, 0xFFFE - Reserved for internal processors
+	 * * 0xFFFD - Reserved for user-space HWRM interface
+	 * * 0xFFFF - HWRM
+	 */
+	uint16_t	target_id;
+	/*
+	 * A physical address pointer pointing to a host buffer that the
+	 * command's response data will be written. This can be either a host
+	 * physical address (HPA) or a guest physical address (GPA) and must
+	 * point to a physically contiguous block of memory.
+	 */
+	uint64_t	resp_addr;
+	/* Session Id. */
+	uint32_t	fw_session_id;
+	/* Control flags. */
+	uint16_t	flags;
+	/* Indicates the flow direction. */
+	#define HWRM_TF_EM_DELETE_INPUT_FLAGS_DIR     UINT32_C(0x1)
+	/* If this bit set to 0, then it indicates rx flow. */
+	#define HWRM_TF_EM_DELETE_INPUT_FLAGS_DIR_RX    UINT32_C(0x0)
+	/* If this bit is set to 1, then it indicates that tx flow. */
+	#define HWRM_TF_EM_DELETE_INPUT_FLAGS_DIR_TX    UINT32_C(0x1)
+	#define HWRM_TF_EM_DELETE_INPUT_FLAGS_DIR_LAST \
+		HWRM_TF_EM_DELETE_INPUT_FLAGS_DIR_TX
+	/* Unused0 */
+	uint16_t	unused0;
+	/* EM internal flow hanndle. */
+	uint64_t	flow_handle;
+	/* EM Key value */
+	uint64_t	em_key[8];
+	/* Number of bits in em_key. */
+	uint16_t	em_key_bitlen;
+	/* unused. */
+	uint16_t	unused1[3];
+} __rte_packed;
+
+/* hwrm_tf_em_delete_output (size:128b/16B) */
+struct hwrm_tf_em_delete_output {
+	/* The specific error status for the command. */
+	uint16_t	error_code;
+	/* The HWRM command request type. */
+	uint16_t	req_type;
+	/* The sequence ID from the original command. */
+	uint16_t	seq_id;
+	/* The length of the response data in number of bytes. */
+	uint16_t	resp_len;
+	/* Original stack allocation index. */
+	uint16_t	em_index;
+	/* unused. */
+	uint16_t	unused0[3];
+} __rte_packed;
+
 /********************
  * hwrm_tf_tcam_set *
  ********************/
@@ -35582,10 +37608,10 @@ struct ctx_hw_stats {
 	uint64_t	rx_mcast_pkts;
 	/* Number of received broadcast packets */
 	uint64_t	rx_bcast_pkts;
-	/* Number of discarded packets on received path */
+	/* Number of discarded packets on receive path */
 	uint64_t	rx_discard_pkts;
-	/* Number of dropped packets on received path */
-	uint64_t	rx_drop_pkts;
+	/* Number of packets on receive path with error */
+	uint64_t	rx_error_pkts;
 	/* Number of received bytes for unicast traffic */
 	uint64_t	rx_ucast_bytes;
 	/* Number of received bytes for multicast traffic */
@@ -35598,10 +37624,10 @@ struct ctx_hw_stats {
 	uint64_t	tx_mcast_pkts;
 	/* Number of transmitted broadcast packets */
 	uint64_t	tx_bcast_pkts;
+	/* Number of packets on transmit path with error */
+	uint64_t	tx_error_pkts;
 	/* Number of discarded packets on transmit path */
 	uint64_t	tx_discard_pkts;
-	/* Number of dropped packets on transmit path */
-	uint64_t	tx_drop_pkts;
 	/* Number of transmitted bytes for unicast traffic */
 	uint64_t	tx_ucast_bytes;
 	/* Number of transmitted bytes for multicast traffic */
@@ -35618,7 +37644,11 @@ struct ctx_hw_stats {
 	uint64_t	tpa_aborts;
 } __rte_packed;
 
-/* Periodic statistics context DMA to host. */
+/*
+ * Extended periodic statistics context DMA to host. On cards that
+ * support TPA v2, additional TPA related stats exist and can be retrieved
+ * by DMA of ctx_hw_stats_ext, rather than legacy ctx_hw_stats structure.
+ */
 /* ctx_hw_stats_ext (size:1344b/168B) */
 struct ctx_hw_stats_ext {
 	/* Number of received unicast packets */
@@ -35627,10 +37657,10 @@ struct ctx_hw_stats_ext {
 	uint64_t	rx_mcast_pkts;
 	/* Number of received broadcast packets */
 	uint64_t	rx_bcast_pkts;
-	/* Number of discarded packets on received path */
+	/* Number of discarded packets on receive path */
 	uint64_t	rx_discard_pkts;
-	/* Number of dropped packets on received path */
-	uint64_t	rx_drop_pkts;
+	/* Number of packets on receive path with error */
+	uint64_t	rx_error_pkts;
 	/* Number of received bytes for unicast traffic */
 	uint64_t	rx_ucast_bytes;
 	/* Number of received bytes for multicast traffic */
@@ -35643,10 +37673,10 @@ struct ctx_hw_stats_ext {
 	uint64_t	tx_mcast_pkts;
 	/* Number of transmitted broadcast packets */
 	uint64_t	tx_bcast_pkts;
+	/* Number of packets on transmit path with error */
+	uint64_t	tx_error_pkts;
 	/* Number of discarded packets on transmit path */
 	uint64_t	tx_discard_pkts;
-	/* Number of dropped packets on transmit path */
-	uint64_t	tx_drop_pkts;
 	/* Number of transmitted bytes for unicast traffic */
 	uint64_t	tx_ucast_bytes;
 	/* Number of transmitted bytes for multicast traffic */
@@ -35912,7 +37942,14 @@ struct hwrm_stat_ctx_query_input {
 	uint64_t	resp_addr;
 	/* ID of the statistics context that is being queried. */
 	uint32_t	stat_ctx_id;
-	uint8_t	unused_0[4];
+	uint8_t	flags;
+	/*
+	 * This bit is set to 1 when request is for a counter mask,
+	 * representing the width of each of the stats counters, rather
+	 * than counters themselves.
+	 */
+	#define HWRM_STAT_CTX_QUERY_INPUT_FLAGS_COUNTER_MASK     UINT32_C(0x1)
+	uint8_t	unused_0[3];
 } __rte_packed;
 
 /* hwrm_stat_ctx_query_output (size:1408b/176B) */
@@ -35949,7 +37986,7 @@ struct hwrm_stat_ctx_query_output {
 	uint64_t	rx_bcast_pkts;
 	/* Number of received packets with error */
 	uint64_t	rx_err_pkts;
-	/* Number of dropped packets on received path */
+	/* Number of dropped packets on receive path */
 	uint64_t	rx_drop_pkts;
 	/* Number of received bytes for unicast traffic */
 	uint64_t	rx_ucast_bytes;
@@ -35977,6 +38014,117 @@ struct hwrm_stat_ctx_query_output {
 } __rte_packed;
 
 /***************************
+ * hwrm_stat_ext_ctx_query *
+ ***************************/
+
+
+/* hwrm_stat_ext_ctx_query_input (size:192b/24B) */
+struct hwrm_stat_ext_ctx_query_input {
+	/* The HWRM command request type. */
+	uint16_t	req_type;
+	/*
+	 * The completion ring to send the completion event on. This should
+	 * be the NQ ID returned from the `nq_alloc` HWRM command.
+	 */
+	uint16_t	cmpl_ring;
+	/*
+	 * The sequence ID is used by the driver for tracking multiple
+	 * commands. This ID is treated as opaque data by the firmware and
+	 * the value is returned in the `hwrm_resp_hdr` upon completion.
+	 */
+	uint16_t	seq_id;
+	/*
+	 * The target ID of the command:
+	 * * 0x0-0xFFF8 - The function ID
+	 * * 0xFFF8-0xFFFC, 0xFFFE - Reserved for internal processors
+	 * * 0xFFFD - Reserved for user-space HWRM interface
+	 * * 0xFFFF - HWRM
+	 */
+	uint16_t	target_id;
+	/*
+	 * A physical address pointer pointing to a host buffer that the
+	 * command's response data will be written. This can be either a host
+	 * physical address (HPA) or a guest physical address (GPA) and must
+	 * point to a physically contiguous block of memory.
+	 */
+	uint64_t	resp_addr;
+	/* ID of the extended statistics context that is being queried. */
+	uint32_t	stat_ctx_id;
+	uint8_t	flags;
+	/*
+	 * This bit is set to 1 when request is for a counter mask,
+	 * representing the width of each of the stats counters, rather
+	 * than counters themselves.
+	 */
+	#define HWRM_STAT_EXT_CTX_QUERY_INPUT_FLAGS_COUNTER_MASK \
+		UINT32_C(0x1)
+	uint8_t	unused_0[3];
+} __rte_packed;
+
+/* hwrm_stat_ext_ctx_query_output (size:1472b/184B) */
+struct hwrm_stat_ext_ctx_query_output {
+	/* The specific error status for the command. */
+	uint16_t	error_code;
+	/* The HWRM command request type. */
+	uint16_t	req_type;
+	/* The sequence ID from the original command. */
+	uint16_t	seq_id;
+	/* The length of the response data in number of bytes. */
+	uint16_t	resp_len;
+	/* Number of received unicast packets */
+	uint64_t	rx_ucast_pkts;
+	/* Number of received multicast packets */
+	uint64_t	rx_mcast_pkts;
+	/* Number of received broadcast packets */
+	uint64_t	rx_bcast_pkts;
+	/* Number of discarded packets on receive path */
+	uint64_t	rx_discard_pkts;
+	/* Number of packets on receive path with error */
+	uint64_t	rx_error_pkts;
+	/* Number of received bytes for unicast traffic */
+	uint64_t	rx_ucast_bytes;
+	/* Number of received bytes for multicast traffic */
+	uint64_t	rx_mcast_bytes;
+	/* Number of received bytes for broadcast traffic */
+	uint64_t	rx_bcast_bytes;
+	/* Number of transmitted unicast packets */
+	uint64_t	tx_ucast_pkts;
+	/* Number of transmitted multicast packets */
+	uint64_t	tx_mcast_pkts;
+	/* Number of transmitted broadcast packets */
+	uint64_t	tx_bcast_pkts;
+	/* Number of packets on transmit path with error */
+	uint64_t	tx_error_pkts;
+	/* Number of discarded packets on transmit path */
+	uint64_t	tx_discard_pkts;
+	/* Number of transmitted bytes for unicast traffic */
+	uint64_t	tx_ucast_bytes;
+	/* Number of transmitted bytes for multicast traffic */
+	uint64_t	tx_mcast_bytes;
+	/* Number of transmitted bytes for broadcast traffic */
+	uint64_t	tx_bcast_bytes;
+	/* Number of TPA eligible packets */
+	uint64_t	rx_tpa_eligible_pkt;
+	/* Number of TPA eligible bytes */
+	uint64_t	rx_tpa_eligible_bytes;
+	/* Number of TPA packets */
+	uint64_t	rx_tpa_pkt;
+	/* Number of TPA bytes */
+	uint64_t	rx_tpa_bytes;
+	/* Number of TPA errors */
+	uint64_t	rx_tpa_errors;
+	uint8_t	unused_0[7];
+	/*
+	 * This field is used in Output records to indicate that the output
+	 * is completely written to RAM.  This field should be read as '1'
+	 * to indicate that the output has been completely written.
+	 * When writing a command completion or response to an internal processor,
+	 * the order of writes has to be such that this field is written last.
+	 */
+	uint8_t	valid;
+} __rte_packed;
+
+/***************************
  * hwrm_stat_ctx_eng_query *
  ***************************/
 
@@ -37565,6 +39713,13 @@ struct hwrm_nvm_install_update_input {
 	 */
 	#define HWRM_NVM_INSTALL_UPDATE_INPUT_FLAGS_ALLOWED_TO_DEFRAG \
 		UINT32_C(0x4)
+	/*
+	 * If set to 1, FW will verify the package in the "UPDATE" NVM item
+	 * without installing it. This flag is for FW internal use only.
+	 * Users should not set this flag. The request will otherwise fail.
+	 */
+	#define HWRM_NVM_INSTALL_UPDATE_INPUT_FLAGS_VERIFY_ONLY \
+		UINT32_C(0x8)
 	uint8_t	unused_0[2];
 } __rte_packed;
 
@@ -38115,6 +40270,72 @@ struct hwrm_nvm_validate_option_cmd_err {
 	uint8_t	unused_0[7];
 } __rte_packed;
 
+/****************
+ * hwrm_oem_cmd *
+ ****************/
+
+
+/* hwrm_oem_cmd_input (size:1024b/128B) */
+struct hwrm_oem_cmd_input {
+	/* The HWRM command request type. */
+	uint16_t	req_type;
+	/*
+	 * The completion ring to send the completion event on. This should
+	 * be the NQ ID returned from the `nq_alloc` HWRM command.
+	 */
+	uint16_t	cmpl_ring;
+	/*
+	 * The sequence ID is used by the driver for tracking multiple
+	 * commands. This ID is treated as opaque data by the firmware and
+	 * the value is returned in the `hwrm_resp_hdr` upon completion.
+	 */
+	uint16_t	seq_id;
+	/*
+	 * The target ID of the command:
+	 * * 0x0-0xFFF8 - The function ID
+	 * * 0xFFF8-0xFFFC, 0xFFFE - Reserved for internal processors
+	 * * 0xFFFD - Reserved for user-space HWRM interface
+	 * * 0xFFFF - HWRM
+	 */
+	uint16_t	target_id;
+	/*
+	 * A physical address pointer pointing to a host buffer that the
+	 * command's response data will be written. This can be either a host
+	 * physical address (HPA) or a guest physical address (GPA) and must
+	 * point to a physically contiguous block of memory.
+	 */
+	uint64_t	resp_addr;
+	uint32_t	IANA;
+	uint32_t	unused_0;
+	/* This field contains the vendor specific command data. */
+	uint32_t	oem_data[26];
+} __rte_packed;
+
+/* hwrm_oem_cmd_output (size:768b/96B) */
+struct hwrm_oem_cmd_output {
+	/* The specific error status for the command. */
+	uint16_t	error_code;
+	/* The HWRM command request type. */
+	uint16_t	req_type;
+	/* The sequence ID from the original command. */
+	uint16_t	seq_id;
+	/* The length of the response data in number of bytes. */
+	uint16_t	resp_len;
+	uint32_t	IANA;
+	uint32_t	unused_0;
+	/* This field contains the vendor specific response data. */
+	uint32_t	oem_data[18];
+	uint8_t	unused_1[7];
+	/*
+	 * This field is used in Output records to indicate that the output
+	 * is completely written to RAM.  This field should be read as '1'
+	 * to indicate that the output has been completely written.
+	 * When writing a command completion or response to an internal processor,
+	 * the order of writes has to be such that this field is written last.
+	 */
+	uint8_t	valid;
+} __rte_packed;
+
 /*****************
  * hwrm_fw_reset *
  ******************/
@@ -38338,6 +40559,55 @@ struct hwrm_port_ts_query_output {
 	uint8_t		valid;
 } __rte_packed;
 
+/*
+ * This structure is fixed at the beginning of the ChiMP SRAM (GRC
+ * offset: 0x31001F0). Host software is expected to read from this
+ * location for a defined signature. If it exists, the software can
+ * assume the presence of this structure and the validity of the
+ * FW_STATUS location in the next field.
+ */
+/* hcomm_status (size:64b/8B) */
+struct hcomm_status {
+	uint32_t	sig_ver;
+	/*
+	 * This field defines the version of the structure. The latest
+	 * version value is 1.
+	 */
+	#define HCOMM_STATUS_VER_MASK		UINT32_C(0xff)
+	#define HCOMM_STATUS_VER_SFT		0
+	#define HCOMM_STATUS_VER_LATEST		UINT32_C(0x1)
+	#define HCOMM_STATUS_VER_LAST		HCOMM_STATUS_VER_LATEST
+	/*
+	 * This field is to store the signature value to indicate the
+	 * presence of the structure.
+	 */
+	#define HCOMM_STATUS_SIGNATURE_MASK	UINT32_C(0xffffff00)
+	#define HCOMM_STATUS_SIGNATURE_SFT	8
+	#define HCOMM_STATUS_SIGNATURE_VAL	(UINT32_C(0x484353) << 8)
+	#define HCOMM_STATUS_SIGNATURE_LAST	HCOMM_STATUS_SIGNATURE_VAL
+	uint32_t	fw_status_loc;
+	#define HCOMM_STATUS_TRUE_ADDR_SPACE_MASK	UINT32_C(0x3)
+	#define HCOMM_STATUS_TRUE_ADDR_SPACE_SFT	0
+	/* PCIE configuration space */
+	#define HCOMM_STATUS_FW_STATUS_LOC_ADDR_SPACE_PCIE_CFG	UINT32_C(0x0)
+	/* GRC space */
+	#define HCOMM_STATUS_FW_STATUS_LOC_ADDR_SPACE_GRC	UINT32_C(0x1)
+	/* BAR0 space */
+	#define HCOMM_STATUS_FW_STATUS_LOC_ADDR_SPACE_BAR0	UINT32_C(0x2)
+	/* BAR1 space */
+	#define HCOMM_STATUS_FW_STATUS_LOC_ADDR_SPACE_BAR1	UINT32_C(0x3)
+	#define HCOMM_STATUS_FW_STATUS_LOC_ADDR_SPACE_LAST	\
+		HCOMM_STATUS_FW_STATUS_LOC_ADDR_SPACE_BAR1
+	/*
+	 * This offset where the fw_status register is located. The value
+	 * is generally 4-byte aligned.
+	 */
+	#define HCOMM_STATUS_TRUE_OFFSET_MASK		UINT32_C(0xfffffffc)
+	#define HCOMM_STATUS_TRUE_OFFSET_SFT		2
+} __rte_packed;
+/* This is the GRC offset where the hcomm_status struct resides. */
+#define HCOMM_STATUS_STRUCT_LOC		0x31001F0UL
+
 /**************************
  * hwrm_cfa_counter_qcaps *
  **************************/
@@ -38622,53 +40892,4 @@ struct hwrm_cfa_counter_qstats_output {
 	uint8_t	valid;
 } __rte_packed;
 
-/*
- * This structure is fixed at the beginning of the ChiMP SRAM (GRC
- * offset: 0x31001F0). Host software is expected to read from this
- * location for a defined signature. If it exists, the software can
- * assume the presence of this structure and the validity of the
- * FW_STATUS location in the next field.
- */
-/* hcomm_status (size:64b/8B) */
-struct hcomm_status {
-	uint32_t	sig_ver;
-	/*
-	 * This field defines the version of the structure. The latest
-	 * version value is 1.
-	 */
-	#define HCOMM_STATUS_VER_MASK		UINT32_C(0xff)
-	#define HCOMM_STATUS_VER_SFT		0
-	#define HCOMM_STATUS_VER_LATEST		UINT32_C(0x1)
-	#define HCOMM_STATUS_VER_LAST		HCOMM_STATUS_VER_LATEST
-	/*
-	 * This field is to store the signature value to indicate the
-	 * presence of the structure.
-	 */
-	#define HCOMM_STATUS_SIGNATURE_MASK	UINT32_C(0xffffff00)
-	#define HCOMM_STATUS_SIGNATURE_SFT	8
-	#define HCOMM_STATUS_SIGNATURE_VAL	(UINT32_C(0x484353) << 8)
-	#define HCOMM_STATUS_SIGNATURE_LAST	HCOMM_STATUS_SIGNATURE_VAL
-	uint32_t	fw_status_loc;
-	#define HCOMM_STATUS_TRUE_ADDR_SPACE_MASK	UINT32_C(0x3)
-	#define HCOMM_STATUS_TRUE_ADDR_SPACE_SFT	0
-	/* PCIE configuration space */
-	#define HCOMM_STATUS_FW_STATUS_LOC_ADDR_SPACE_PCIE_CFG	UINT32_C(0x0)
-	/* GRC space */
-	#define HCOMM_STATUS_FW_STATUS_LOC_ADDR_SPACE_GRC	UINT32_C(0x1)
-	/* BAR0 space */
-	#define HCOMM_STATUS_FW_STATUS_LOC_ADDR_SPACE_BAR0	UINT32_C(0x2)
-	/* BAR1 space */
-	#define HCOMM_STATUS_FW_STATUS_LOC_ADDR_SPACE_BAR1	UINT32_C(0x3)
-	#define HCOMM_STATUS_FW_STATUS_LOC_ADDR_SPACE_LAST	\
-		HCOMM_STATUS_FW_STATUS_LOC_ADDR_SPACE_BAR1
-	/*
-	 * This offset where the fw_status register is located. The value
-	 * is generally 4-byte aligned.
-	 */
-	#define HCOMM_STATUS_TRUE_OFFSET_MASK		UINT32_C(0xfffffffc)
-	#define HCOMM_STATUS_TRUE_OFFSET_SFT		2
-} __rte_packed;
-/* This is the GRC offset where the hcomm_status struct resides. */
-#define HCOMM_STATUS_STRUCT_LOC		0x31001F0UL
-
 #endif /* _HSI_STRUCT_DEF_DPDK_H_ */
diff --git a/drivers/net/bnxt/tf_core/hwrm_tf.h b/drivers/net/bnxt/tf_core/hwrm_tf.h
index 3419095..439950e 100644
--- a/drivers/net/bnxt/tf_core/hwrm_tf.h
+++ b/drivers/net/bnxt/tf_core/hwrm_tf.h
@@ -86,6 +86,7 @@ struct tf_tbl_type_get_output;
 struct tf_em_internal_insert_input;
 struct tf_em_internal_insert_output;
 struct tf_em_internal_delete_input;
+struct tf_em_internal_delete_output;
 /* Input params for session attach */
 typedef struct tf_session_attach_input {
 	/* Firmware session id returned when HWRM_TF_SESSION_OPEN is sent */
@@ -949,6 +950,8 @@ typedef struct tf_em_internal_insert_output {
 	uint16_t			 rptr_index;
 	/* EM record offset 0~3 */
 	uint8_t			  rptr_entry;
+	/* Number of word entries consumed by the key */
+	uint8_t			  num_of_entries;
 } tf_em_internal_insert_output_t, *ptf_em_internal_insert_output_t;
 
 /* Input params for EM INTERNAL rule delete */
@@ -969,4 +972,10 @@ typedef struct tf_em_internal_delete_input {
 	uint16_t			 em_key_bitlen;
 } tf_em_internal_delete_input_t, *ptf_em_internal_delete_input_t;
 
+/* Input params for EM INTERNAL rule delete */
+typedef struct tf_em_internal_delete_output {
+	/* Original stack allocation index */
+	uint16_t			 em_index;
+} tf_em_internal_delete_output_t, *ptf_em_internal_delete_output_t;
+
 #endif /* _HWRM_TF_H_ */
diff --git a/drivers/net/bnxt/tf_core/lookup3.h b/drivers/net/bnxt/tf_core/lookup3.h
index e5abcc2..b1fd2cd 100644
--- a/drivers/net/bnxt/tf_core/lookup3.h
+++ b/drivers/net/bnxt/tf_core/lookup3.h
@@ -152,7 +152,6 @@ static inline uint32_t hashword(const uint32_t *k,
 		final(a, b, c);
 		/* Falls through. */
 	case 0:	    /* case 0: nothing left to add */
-		/* FALLTHROUGH */
 		break;
 	}
 	/*------------------------------------------------- report the result */
diff --git a/drivers/net/bnxt/tf_core/stack.c b/drivers/net/bnxt/tf_core/stack.c
index 9cfbd24..9548063 100644
--- a/drivers/net/bnxt/tf_core/stack.c
+++ b/drivers/net/bnxt/tf_core/stack.c
@@ -27,6 +27,14 @@ stack_init(int num_entries, uint32_t *items, struct stack *st)
 	return 0;
 }
 
+/*
+ * Return the address of the items
+ */
+uint32_t *stack_items(struct stack *st)
+{
+	return st->items;
+}
+
 /* Return the size of the stack
  */
 int32_t
diff --git a/drivers/net/bnxt/tf_core/stack.h b/drivers/net/bnxt/tf_core/stack.h
index ebd0555..6732e03 100644
--- a/drivers/net/bnxt/tf_core/stack.h
+++ b/drivers/net/bnxt/tf_core/stack.h
@@ -36,6 +36,16 @@ int stack_init(int num_entries,
 	       uint32_t *items,
 	       struct stack *st);
 
+/** Return the address of the stack contents
+ *
+ *  [in] st
+ *    pointer to the stack
+ *
+ *  return
+ *    pointer to the stack contents
+ */
+uint32_t *stack_items(struct stack *st);
+
 /** Return the size of the stack
  *
  *  [in] st
diff --git a/drivers/net/bnxt/tf_core/tf_core.c b/drivers/net/bnxt/tf_core/tf_core.c
index cf9f36a..ba54df6 100644
--- a/drivers/net/bnxt/tf_core/tf_core.c
+++ b/drivers/net/bnxt/tf_core/tf_core.c
@@ -45,6 +45,100 @@ static void tf_seeds_init(struct tf_session *session)
 	}
 }
 
+/**
+ * Create EM Tbl pool of memory indexes.
+ *
+ * [in] session
+ *   Pointer to session
+ * [in] dir
+ *   direction
+ * [in] num_entries
+ *   number of entries to write
+ *
+ * Return:
+ *  0       - Success, entry allocated - no search support
+ *  -ENOMEM -EINVAL -EOPNOTSUPP
+ *          - Failure, entry not allocated, out of resources
+ */
+static int
+tf_create_em_pool(struct tf_session *session,
+		  enum tf_dir dir,
+		  uint32_t num_entries)
+{
+	struct tfp_calloc_parms parms;
+	uint32_t i, j;
+	int rc = 0;
+	struct stack *pool = &session->em_pool[dir];
+
+	parms.nitems = num_entries;
+	parms.size = sizeof(uint32_t);
+	parms.alignment = 0;
+
+	if (tfp_calloc(&parms) != 0) {
+		TFP_DRV_LOG(ERR, "EM pool allocation failure %s\n",
+			    strerror(-ENOMEM));
+		return -ENOMEM;
+	}
+
+	/* Create empty stack
+	 */
+	rc = stack_init(num_entries, parms.mem_va, pool);
+
+	if (rc != 0) {
+		TFP_DRV_LOG(ERR, "EM pool stack init failure %s\n",
+			    strerror(-rc));
+		goto cleanup;
+	}
+
+	/* Fill pool with indexes
+	 */
+	j = num_entries - 1;
+
+	for (i = 0; i < num_entries; i++) {
+		rc = stack_push(pool, j);
+		if (rc != 0) {
+			TFP_DRV_LOG(ERR, "EM pool stack push failure %s\n",
+				    strerror(-rc));
+			goto cleanup;
+		}
+		j--;
+	}
+
+	if (!stack_is_full(pool)) {
+		rc = -EINVAL;
+		TFP_DRV_LOG(ERR, "EM pool stack failure %s\n",
+			    strerror(-rc));
+		goto cleanup;
+	}
+
+	return 0;
+cleanup:
+	tfp_free((void *)parms.mem_va);
+	return rc;
+}
+
+/**
+ * Create EM Tbl pool of memory indexes.
+ *
+ * [in] session
+ *   Pointer to session
+ * [in] dir
+ *   direction
+ *
+ * Return:
+ */
+static void
+tf_free_em_pool(struct tf_session *session,
+		enum tf_dir dir)
+{
+	struct stack *pool = &session->em_pool[dir];
+	uint32_t *ptr;
+
+	ptr = stack_items(pool);
+
+	tfp_free(ptr);
+}
+
 int
 tf_open_session(struct tf                    *tfp,
 		struct tf_open_session_parms *parms)
@@ -54,6 +148,7 @@ tf_open_session(struct tf                    *tfp,
 	struct tfp_calloc_parms alloc_parms;
 	unsigned int domain, bus, slot, device;
 	uint8_t fw_session_id;
+	int dir;
 
 	if (tfp == NULL || parms == NULL)
 		return -EINVAL;
@@ -110,7 +205,7 @@ tf_open_session(struct tf                    *tfp,
 		goto cleanup;
 	}
 
-	tfp->session = (struct tf_session_info *)alloc_parms.mem_va;
+	tfp->session = alloc_parms.mem_va;
 
 	/* Allocate core data for the session */
 	alloc_parms.nitems = 1;
@@ -175,6 +270,16 @@ tf_open_session(struct tf                    *tfp,
 	/* Setup hash seeds */
 	tf_seeds_init(session);
 
+	/* Initialize EM pool */
+	for (dir = 0; dir < TF_DIR_MAX; dir++) {
+		rc = tf_create_em_pool(session, dir, TF_SESSION_EM_POOL_SIZE);
+		if (rc) {
+			TFP_DRV_LOG(ERR,
+				    "EM Pool initialization failed\n");
+			goto cleanup_close;
+		}
+	}
+
 	session->ref_count++;
 
 	/* Return session ID */
@@ -239,6 +344,7 @@ tf_close_session(struct tf *tfp)
 	int rc_close = 0;
 	struct tf_session *tfs;
 	union tf_session_id session_id;
+	int dir;
 
 	if (tfp == NULL || tfp->session == NULL)
 		return -EINVAL;
@@ -268,6 +374,10 @@ tf_close_session(struct tf *tfp)
 
 	/* Final cleanup as we're last user of the session */
 	if (tfs->ref_count == 0) {
+		/* Free EM pool */
+		for (dir = 0; dir < TF_DIR_MAX; dir++)
+			tf_free_em_pool(tfs, dir);
+
 		tfp_free(tfp->session->core_data);
 		tfp_free(tfp->session);
 		tfp->session = NULL;
@@ -301,16 +411,25 @@ int tf_insert_em_entry(struct tf *tfp,
 	if (tfp == NULL || parms == NULL)
 		return -EINVAL;
 
-	tbl_scope_cb =
-		tbl_scope_cb_find((struct tf_session *)tfp->session->core_data,
-				  parms->tbl_scope_id);
+	tbl_scope_cb = tbl_scope_cb_find(
+		(struct tf_session *)(tfp->session->core_data),
+		parms->tbl_scope_id);
 	if (tbl_scope_cb == NULL)
 		return -EINVAL;
 
 	/* Process the EM entry per Table Scope type */
-	return tf_insert_eem_entry((struct tf_session *)tfp->session->core_data,
-				   tbl_scope_cb,
-				   parms);
+	if (parms->mem == TF_MEM_EXTERNAL) {
+		/* External EEM */
+		return tf_insert_eem_entry
+			((struct tf_session *)(tfp->session->core_data),
+			tbl_scope_cb,
+			parms);
+	} else if (parms->mem == TF_MEM_INTERNAL) {
+		/* Internal EM */
+		return tf_insert_em_internal_entry(tfp,	parms);
+	}
+
+	return -EINVAL;
 }
 
 /** Delete EM hash entry API
@@ -327,13 +446,16 @@ int tf_delete_em_entry(struct tf *tfp,
 	if (tfp == NULL || parms == NULL)
 		return -EINVAL;
 
-	tbl_scope_cb =
-		tbl_scope_cb_find((struct tf_session *)tfp->session->core_data,
-				  parms->tbl_scope_id);
+	tbl_scope_cb = tbl_scope_cb_find(
+		(struct tf_session *)(tfp->session->core_data),
+		parms->tbl_scope_id);
 	if (tbl_scope_cb == NULL)
 		return -EINVAL;
 
-	return tf_delete_eem_entry(tfp, parms);
+	if (parms->mem == TF_MEM_EXTERNAL)
+		return tf_delete_eem_entry(tfp, parms);
+	else
+		return tf_delete_em_internal_entry(tfp, parms);
 }
 
 /** allocate identifier resource
diff --git a/drivers/net/bnxt/tf_core/tf_core.h b/drivers/net/bnxt/tf_core/tf_core.h
index 1eedd80..81ff760 100644
--- a/drivers/net/bnxt/tf_core/tf_core.h
+++ b/drivers/net/bnxt/tf_core/tf_core.h
@@ -44,44 +44,7 @@ enum tf_mem {
 };
 
 /**
- * The size of the external action record (Wh+/Brd2)
- *
- * Currently set to 512.
- *
- * AR (16B) + encap (256B) + stats_ptrs (8) + resvd (8)
- * + stats (16) = 304 aligned on a 16B boundary
- *
- * Theoretically, the size should be smaller. ~304B
- */
-#define TF_ACTION_RECORD_SZ 512
-
-/**
- * External pool size
- *
- * Defines a single pool of external action records of
- * fixed size.  Currently, this is an index.
- */
-#define TF_EXT_POOL_ENTRY_SZ_BYTES 1
-
-/**
- *  External pool entry count
- *
- *  Defines the number of entries in the external action pool
- */
-#define TF_EXT_POOL_ENTRY_CNT (1 * 1024)
-
-/**
- * Number of external pools
- */
-#define TF_EXT_POOL_CNT_MAX 1
-
-/**
- * External pool Id
- */
-#define TF_EXT_POOL_0      0 /**< matches TF_TBL_TYPE_EXT   */
-#define TF_EXT_POOL_1      1 /**< matches TF_TBL_TYPE_EXT_0 */
-
-/** EEM record AR helper
+ * EEM record AR helper
  *
  * Helper to handle the Action Record Pointer in the EEM Record Entry.
  *
@@ -109,7 +72,8 @@ enum tf_mem {
  */
 
 
-/** Session Version defines
+/**
+ * Session Version defines
  *
  * The version controls the format of the tf_session and
  * tf_session_info structure. This is to assure upgrade between
@@ -119,7 +83,8 @@ enum tf_mem {
 #define TF_SESSION_VER_MINOR  0   /**< Minor Version */
 #define TF_SESSION_VER_UPDATE 0   /**< Update Version */
 
-/** Session Name
+/**
+ * Session Name
  *
  * Name of the TruFlow control channel interface.  Expects
  * format to be RTE Name specific, i.e. rte_eth_dev_get_name_by_port()
@@ -128,7 +93,8 @@ enum tf_mem {
 
 #define TF_FW_SESSION_ID_INVALID  0xFF  /**< Invalid FW Session ID define */
 
-/** Session Identifier
+/**
+ * Session Identifier
  *
  * Unique session identifier which includes PCIe bus info to
  * distinguish the PF and session info to identify the associated
@@ -146,7 +112,8 @@ union tf_session_id {
 	} internal;
 };
 
-/** Session Version
+/**
+ * Session Version
  *
  * The version controls the format of the tf_session and
  * tf_session_info structure. This is to assure upgrade between
@@ -160,8 +127,8 @@ struct tf_session_version {
 	uint8_t update;
 };
 
-/** Session supported device types
- *
+/**
+ * Session supported device types
  */
 enum tf_device_type {
 	TF_DEVICE_TYPE_WH = 0, /**< Whitney+  */
@@ -171,6 +138,147 @@ enum tf_device_type {
 	TF_DEVICE_TYPE_MAX     /**< Maximum   */
 };
 
+/** Identifier resource types
+ */
+enum tf_identifier_type {
+	/** The L2 Context is returned from the L2 Ctxt TCAM lookup
+	 *  and can be used in WC TCAM or EM keys to virtualize further
+	 *  lookups.
+	 */
+	TF_IDENT_TYPE_L2_CTXT,
+	/** The WC profile func is returned from the L2 Ctxt TCAM lookup
+	 *  to enable virtualization of the profile TCAM.
+	 */
+	TF_IDENT_TYPE_PROF_FUNC,
+	/** The WC profile ID is included in the WC lookup key
+	 *  to enable virtualization of the WC TCAM hardware.
+	 */
+	TF_IDENT_TYPE_WC_PROF,
+	/** The EM profile ID is included in the EM lookup key
+	 *  to enable virtualization of the EM hardware. (not required for SR2
+	 *  as it has table scope)
+	 */
+	TF_IDENT_TYPE_EM_PROF,
+	/** The L2 func is included in the ILT result and from recycling to
+	 *  enable virtualization of further lookups.
+	 */
+	TF_IDENT_TYPE_L2_FUNC,
+	TF_IDENT_TYPE_MAX
+};
+
+/**
+ * Enumeration of TruFlow table types. A table type is used to identify a
+ * resource object.
+ *
+ * NOTE: The table type TF_TBL_TYPE_EXT is unique in that it is
+ * the only table type that is connected with a table scope.
+ */
+enum tf_tbl_type {
+	/* Internal */
+
+	/** Wh+/SR Action Record */
+	TF_TBL_TYPE_FULL_ACT_RECORD,
+	/** Wh+/SR/Th Multicast Groups */
+	TF_TBL_TYPE_MCAST_GROUPS,
+	/** Wh+/SR Action Encap 8 Bytes */
+	TF_TBL_TYPE_ACT_ENCAP_8B,
+	/** Wh+/SR Action Encap 16 Bytes */
+	TF_TBL_TYPE_ACT_ENCAP_16B,
+	/** Action Encap 32 Bytes */
+	TF_TBL_TYPE_ACT_ENCAP_32B,
+	/** Wh+/SR Action Encap 64 Bytes */
+	TF_TBL_TYPE_ACT_ENCAP_64B,
+	/** Action Source Properties SMAC */
+	TF_TBL_TYPE_ACT_SP_SMAC,
+	/** Wh+/SR Action Source Properties SMAC IPv4 */
+	TF_TBL_TYPE_ACT_SP_SMAC_IPV4,
+	/** Action Source Properties SMAC IPv6 */
+	TF_TBL_TYPE_ACT_SP_SMAC_IPV6,
+	/** Wh+/SR Action Statistics 64 Bits */
+	TF_TBL_TYPE_ACT_STATS_64,
+	/** Wh+/SR Action Modify L4 Src Port */
+	TF_TBL_TYPE_ACT_MODIFY_SPORT,
+	/** Wh+/SR Action Modify L4 Dest Port */
+	TF_TBL_TYPE_ACT_MODIFY_DPORT,
+	/** Wh+/SR Action Modify IPv4 Source */
+	TF_TBL_TYPE_ACT_MODIFY_IPV4_SRC,
+	/** Wh+/SR Action _Modify L4 Dest Port */
+	TF_TBL_TYPE_ACT_MODIFY_IPV4_DEST,
+	/** Action Modify IPv6 Source */
+	TF_TBL_TYPE_ACT_MODIFY_IPV6_SRC,
+	/** Action Modify IPv6 Destination */
+	TF_TBL_TYPE_ACT_MODIFY_IPV6_DEST,
+	/** Meter Profiles */
+	TF_TBL_TYPE_METER_PROF,
+	/** Meter Instance */
+	TF_TBL_TYPE_METER_INST,
+	/** Mirror Config */
+	TF_TBL_TYPE_MIRROR_CONFIG,
+	/** UPAR */
+	TF_TBL_TYPE_UPAR,
+	/** SR2 Epoch 0 table */
+	TF_TBL_TYPE_EPOCH0,
+	/** SR2 Epoch 1 table  */
+	TF_TBL_TYPE_EPOCH1,
+	/** SR2 Metadata  */
+	TF_TBL_TYPE_METADATA,
+	/** SR2 CT State  */
+	TF_TBL_TYPE_CT_STATE,
+	/** SR2 Range Profile  */
+	TF_TBL_TYPE_RANGE_PROF,
+	/** SR2 Range Entry  */
+	TF_TBL_TYPE_RANGE_ENTRY,
+	/** SR2 LAG Entry  */
+	TF_TBL_TYPE_LAG,
+	/** SR2 VNIC/SVIF Table */
+	TF_TBL_TYPE_VNIC_SVIF,
+	/** Th/SR2 EM Flexible Key builder */
+	TF_TBL_TYPE_EM_FKB,
+	/** Th/SR2 WC Flexible Key builder */
+	TF_TBL_TYPE_WC_FKB,
+
+	/* External */
+
+	/** External table type - initially 1 poolsize entries.
+	 * All External table types are associated with a table
+	 * scope. Internal types are not.
+	 */
+	TF_TBL_TYPE_EXT,
+	TF_TBL_TYPE_MAX
+};
+
+/**
+ * TCAM table type
+ */
+enum tf_tcam_tbl_type {
+	/** L2 Context TCAM */
+	TF_TCAM_TBL_TYPE_L2_CTXT_TCAM,
+	/** Profile TCAM */
+	TF_TCAM_TBL_TYPE_PROF_TCAM,
+	/** Wildcard TCAM */
+	TF_TCAM_TBL_TYPE_WC_TCAM,
+	/** Source Properties TCAM */
+	TF_TCAM_TBL_TYPE_SP_TCAM,
+	/** Connection Tracking Rule TCAM */
+	TF_TCAM_TBL_TYPE_CT_RULE_TCAM,
+	/** Virtual Edge Bridge TCAM */
+	TF_TCAM_TBL_TYPE_VEB_TCAM,
+	TF_TCAM_TBL_TYPE_MAX
+};
+
+/**
+ * EM Resources
+ * These defines are provisioned during
+ * tf_open_session()
+ */
+enum tf_em_tbl_type {
+	/** The number of internal EM records for the session */
+	TF_EM_TBL_TYPE_EM_RECORD,
+	/** The number of table scopes reequested */
+	TF_EM_TBL_TYPE_TBL_SCOPE,
+	TF_EM_TBL_TYPE_MAX
+};
+
 /** TruFlow Session Information
  *
  * Structure defining a TruFlow Session, also known as a Management
@@ -309,6 +417,30 @@ struct tf_open_session_parms {
 	 * Device type is passed, one of Wh+, Brd2, Brd3, Brd4
 	 */
 	enum tf_device_type device_type;
+	/** [in] Requested Identifier Resources
+	 *
+	 * The number of identifier resources requested for the session.
+	 * The index used is tf_identifier_type.
+	 */
+	uint16_t identifer_cnt[TF_IDENT_TYPE_MAX];
+	/** [in] Requested Index Table resource counts
+	 *
+	 * The number of index table resources requested for the session.
+	 * The index used is tf_tbl_type.
+	 */
+	uint16_t tbl_cnt[TF_TBL_TYPE_MAX];
+	/** [in] Requested TCAM Table resource counts
+	 *
+	 * The number of TCAM table resources requested for the session.
+	 * The index used is tf_tcam_tbl_type.
+	 */
+	uint16_t tcam_tbl_cnt[TF_TCAM_TBL_TYPE_MAX];
+	/** [in] Requested EM resource counts
+	 *
+	 * The number of internal EM table resources requested for the session
+	 * The index used is tf_em_tbl_type.
+	 */
+	uint16_t em_tbl_cnt[TF_EM_TBL_TYPE_MAX];
 };
 
 /**
@@ -417,31 +549,6 @@ int tf_close_session(struct tf *tfp);
  *
  * @ref tf_free_identifier
  */
-enum tf_identifier_type {
-	/** The L2 Context is returned from the L2 Ctxt TCAM lookup
-	 *  and can be used in WC TCAM or EM keys to virtualize further
-	 *  lookups.
-	 */
-	TF_IDENT_TYPE_L2_CTXT,
-	/** The WC profile func is returned from the L2 Ctxt TCAM lookup
-	 *  to enable virtualization of the profile TCAM.
-	 */
-	TF_IDENT_TYPE_PROF_FUNC,
-	/** The WC profile ID is included in the WC lookup key
-	 *  to enable virtualization of the WC TCAM hardware.
-	 */
-	TF_IDENT_TYPE_WC_PROF,
-	/** The EM profile ID is included in the EM lookup key
-	 *  to enable virtualization of the EM hardware. (not required for Brd4
-	 *  as it has table scope)
-	 */
-	TF_IDENT_TYPE_EM_PROF,
-	/** The L2 func is included in the ILT result and from recycling to
-	 *  enable virtualization of further lookups.
-	 */
-	TF_IDENT_TYPE_L2_FUNC
-};
-
 /** tf_alloc_identifier parameter definition
  */
 struct tf_alloc_identifier_parms {
@@ -631,19 +738,6 @@ int tf_alloc_tbl_scope(struct tf *tfp,
 int tf_free_tbl_scope(struct tf *tfp,
 		      struct tf_free_tbl_scope_parms *parms);
 
-/**
- * TCAM table type
- */
-enum tf_tcam_tbl_type {
-	TF_TCAM_TBL_TYPE_L2_CTXT_TCAM,
-	TF_TCAM_TBL_TYPE_PROF_TCAM,
-	TF_TCAM_TBL_TYPE_WC_TCAM,
-	TF_TCAM_TBL_TYPE_SP_TCAM,
-	TF_TCAM_TBL_TYPE_CT_RULE_TCAM,
-	TF_TCAM_TBL_TYPE_VEB_TCAM,
-	TF_TCAM_TBL_TYPE_MAX
-
-};
 
 /**
  * @page tcam TCAM Access
@@ -813,7 +907,8 @@ struct tf_get_tcam_entry_parms {
 	uint16_t result_sz_in_bits;
 };
 
-/** get TCAM entry
+/*
+ * get TCAM entry
  *
  * Program a TCAM table entry for a TruFlow session.
  *
@@ -824,7 +919,8 @@ struct tf_get_tcam_entry_parms {
 int tf_get_tcam_entry(struct tf *tfp,
 		      struct tf_get_tcam_entry_parms *parms);
 
-/** tf_free_tcam_entry parameter definition
+/*
+ * tf_free_tcam_entry parameter definition
  */
 struct tf_free_tcam_entry_parms {
 	/**
@@ -845,8 +941,7 @@ struct tf_free_tcam_entry_parms {
 	uint16_t ref_cnt;
 };
 
-/** free TCAM entry
- *
+/*
  * Free TCAM entry.
  *
  * Firmware checks to ensure the TCAM entries are owned by the TruFlow
@@ -873,84 +968,7 @@ int tf_free_tcam_entry(struct tf *tfp,
  */
 
 /**
- * Enumeration of TruFlow table types. A table type is used to identify a
- * resource object.
- *
- * NOTE: The table type TF_TBL_TYPE_EXT is unique in that it is
- * the only table type that is connected with a table scope.
- */
-enum tf_tbl_type {
-	/** Wh+/Brd2 Action Record */
-	TF_TBL_TYPE_FULL_ACT_RECORD,
-	/** Multicast Groups */
-	TF_TBL_TYPE_MCAST_GROUPS,
-	/** Action Encap 8 Bytes */
-	TF_TBL_TYPE_ACT_ENCAP_8B,
-	/** Action Encap 16 Bytes */
-	TF_TBL_TYPE_ACT_ENCAP_16B,
-	/** Action Encap 64 Bytes */
-	TF_TBL_TYPE_ACT_ENCAP_32B,
-	/** Action Encap 64 Bytes */
-	TF_TBL_TYPE_ACT_ENCAP_64B,
-	/** Action Source Properties SMAC */
-	TF_TBL_TYPE_ACT_SP_SMAC,
-	/** Action Source Properties SMAC IPv4 */
-	TF_TBL_TYPE_ACT_SP_SMAC_IPV4,
-	/** Action Source Properties SMAC IPv6 */
-	TF_TBL_TYPE_ACT_SP_SMAC_IPV6,
-	/** Action Statistics 64 Bits */
-	TF_TBL_TYPE_ACT_STATS_64,
-	/** Action Modify L4 Src Port */
-	TF_TBL_TYPE_ACT_MODIFY_SPORT,
-	/** Action Modify L4 Dest Port */
-	TF_TBL_TYPE_ACT_MODIFY_DPORT,
-	/** Action Modify IPv4 Source */
-	TF_TBL_TYPE_ACT_MODIFY_IPV4_SRC,
-	/** Action _Modify L4 Dest Port */
-	TF_TBL_TYPE_ACT_MODIFY_IPV4_DEST,
-	/** Action Modify IPv6 Source */
-	TF_TBL_TYPE_ACT_MODIFY_IPV6_SRC,
-	/** Action Modify IPv6 Destination */
-	TF_TBL_TYPE_ACT_MODIFY_IPV6_DEST,
-
-	/* HW */
-
-	/** Meter Profiles */
-	TF_TBL_TYPE_METER_PROF,
-	/** Meter Instance */
-	TF_TBL_TYPE_METER_INST,
-	/** Mirror Config */
-	TF_TBL_TYPE_MIRROR_CONFIG,
-	/** UPAR */
-	TF_TBL_TYPE_UPAR,
-	/** Brd4 Epoch 0 table */
-	TF_TBL_TYPE_EPOCH0,
-	/** Brd4 Epoch 1 table  */
-	TF_TBL_TYPE_EPOCH1,
-	/** Brd4 Metadata  */
-	TF_TBL_TYPE_METADATA,
-	/** Brd4 CT State  */
-	TF_TBL_TYPE_CT_STATE,
-	/** Brd4 Range Profile  */
-	TF_TBL_TYPE_RANGE_PROF,
-	/** Brd4 Range Entry  */
-	TF_TBL_TYPE_RANGE_ENTRY,
-	/** Brd4 LAG Entry  */
-	TF_TBL_TYPE_LAG,
-	/** Brd4 only VNIC/SVIF Table */
-	TF_TBL_TYPE_VNIC_SVIF,
-
-	/* External */
-
-	/** External table type - initially 1 poolsize entries.
-	 * All External table types are associated with a table
-	 * scope. Internal types are not.
-	 */
-	TF_TBL_TYPE_EXT,
-	TF_TBL_TYPE_MAX
-};
-
-/** tf_alloc_tbl_entry parameter definition
+ * tf_alloc_tbl_entry parameter definition
  */
 struct tf_alloc_tbl_entry_parms {
 	/**
@@ -993,7 +1011,8 @@ struct tf_alloc_tbl_entry_parms {
 	uint32_t idx;
 };
 
-/** allocate index table entries
+/**
+ * allocate index table entries
  *
  * Internal types:
  *
@@ -1023,7 +1042,8 @@ struct tf_alloc_tbl_entry_parms {
 int tf_alloc_tbl_entry(struct tf *tfp,
 		       struct tf_alloc_tbl_entry_parms *parms);
 
-/** tf_free_tbl_entry parameter definition
+/**
+ * tf_free_tbl_entry parameter definition
  */
 struct tf_free_tbl_entry_parms {
 	/**
@@ -1049,7 +1069,8 @@ struct tf_free_tbl_entry_parms {
 	uint16_t ref_cnt;
 };
 
-/** free index table entry
+/**
+ * free index table entry
  *
  * Used to free a previously allocated table entry.
  *
@@ -1075,7 +1096,8 @@ struct tf_free_tbl_entry_parms {
 int tf_free_tbl_entry(struct tf *tfp,
 		      struct tf_free_tbl_entry_parms *parms);
 
-/** tf_set_tbl_entry parameter definition
+/**
+ * tf_set_tbl_entry parameter definition
  */
 struct tf_set_tbl_entry_parms {
 	/**
@@ -1104,7 +1126,8 @@ struct tf_set_tbl_entry_parms {
 	uint32_t idx;
 };
 
-/** set index table entry
+/**
+ * set index table entry
  *
  * Used to insert an application programmed index table entry into a
  * previous allocated table location.  A shadow copy of the table
@@ -1115,7 +1138,8 @@ struct tf_set_tbl_entry_parms {
 int tf_set_tbl_entry(struct tf *tfp,
 		     struct tf_set_tbl_entry_parms *parms);
 
-/** tf_get_tbl_entry parameter definition
+/**
+ * tf_get_tbl_entry parameter definition
  */
 struct tf_get_tbl_entry_parms {
 	/**
@@ -1140,7 +1164,8 @@ struct tf_get_tbl_entry_parms {
 	uint32_t idx;
 };
 
-/** get index table entry
+/**
+ * get index table entry
  *
  * Used to retrieve a previous set index table entry.
  *
@@ -1163,7 +1188,8 @@ int tf_get_tbl_entry(struct tf *tfp,
  * @ref tf_search_em_entry
  *
  */
-/** tf_insert_em_entry parameter definition
+/**
+ * tf_insert_em_entry parameter definition
  */
 struct tf_insert_em_entry_parms {
 	/**
@@ -1240,6 +1266,10 @@ struct tf_delete_em_entry_parms {
 	 */
 	uint16_t *epochs;
 	/**
+	 * [out] The index of the entry
+	 */
+	uint16_t index;
+	/**
 	 * [in] structure containing flow delete handle information
 	 */
 	uint64_t flow_handle;
@@ -1291,7 +1321,8 @@ struct tf_search_em_entry_parms {
 	uint64_t flow_handle;
 };
 
-/** insert em hash entry in internal table memory
+/**
+ * insert em hash entry in internal table memory
  *
  * Internal:
  *
@@ -1328,7 +1359,8 @@ struct tf_search_em_entry_parms {
 int tf_insert_em_entry(struct tf *tfp,
 		       struct tf_insert_em_entry_parms *parms);
 
-/** delete em hash entry table memory
+/**
+ * delete em hash entry table memory
  *
  * Internal:
  *
@@ -1353,7 +1385,8 @@ int tf_insert_em_entry(struct tf *tfp,
 int tf_delete_em_entry(struct tf *tfp,
 		       struct tf_delete_em_entry_parms *parms);
 
-/** search em hash entry table memory
+/**
+ * search em hash entry table memory
  *
  * Internal:
 
diff --git a/drivers/net/bnxt/tf_core/tf_em.c b/drivers/net/bnxt/tf_core/tf_em.c
index bd8e2ba..fd1797e 100644
--- a/drivers/net/bnxt/tf_core/tf_em.c
+++ b/drivers/net/bnxt/tf_core/tf_em.c
@@ -287,7 +287,7 @@ static int tf_em_entry_exists(struct tf_tbl_scope_cb *tbl_scope_cb,
 }
 
 static void tf_em_create_key_entry(struct tf_eem_entry_hdr *result,
-				    uint8_t	       *in_key,
+				    uint8_t *in_key,
 				    struct tf_eem_64b_entry *key_entry)
 {
 	key_entry->hdr.word1 = result->word1;
@@ -308,7 +308,7 @@ static void tf_em_create_key_entry(struct tf_eem_entry_hdr *result,
  * EEXIST  - Key does exist in table at "index" in table "table".
  * TF_ERR     - Something went horribly wrong.
  */
-static int tf_em_select_inject_table(struct tf_tbl_scope_cb	*tbl_scope_cb,
+static int tf_em_select_inject_table(struct tf_tbl_scope_cb *tbl_scope_cb,
 					  enum tf_dir dir,
 					  struct tf_eem_64b_entry *entry,
 					  uint32_t key0_hash,
@@ -368,8 +368,8 @@ static int tf_em_select_inject_table(struct tf_tbl_scope_cb	*tbl_scope_cb,
  *   0
  *   TF_ERR_EM_DUP  - key is already in table
  */
-int tf_insert_eem_entry(struct tf_session	   *session,
-			struct tf_tbl_scope_cb	   *tbl_scope_cb,
+int tf_insert_eem_entry(struct tf_session *session,
+			struct tf_tbl_scope_cb *tbl_scope_cb,
 			struct tf_insert_em_entry_parms *parms)
 {
 	uint32_t	   mask;
@@ -457,6 +457,96 @@ int tf_insert_eem_entry(struct tf_session	   *session,
 	return -EINVAL;
 }
 
+/**
+ * Insert EM internal entry API
+ *
+ *  returns:
+ *     0 - Success
+ */
+int tf_insert_em_internal_entry(struct tf *tfp,
+				struct tf_insert_em_entry_parms *parms)
+{
+	int       rc;
+	uint32_t  gfid;
+	uint16_t  rptr_index = 0;
+	uint8_t   rptr_entry = 0;
+	uint8_t   num_of_entries = 0;
+	struct tf_session *session =
+		(struct tf_session *)(tfp->session->core_data);
+	struct stack *pool = &session->em_pool[parms->dir];
+	uint32_t index;
+
+	rc = stack_pop(pool, &index);
+
+	if (rc != 0) {
+		PMD_DRV_LOG
+		   (ERR,
+		   "dir:%d, EM entry index allocation failed\n",
+		   parms->dir);
+		return rc;
+	}
+
+	rptr_index = index * TF_SESSION_EM_ENTRY_SIZE;
+	rc = tf_msg_insert_em_internal_entry(tfp,
+					     parms,
+					     &rptr_index,
+					     &rptr_entry,
+					     &num_of_entries);
+	if (rc != 0)
+		return -1;
+
+	PMD_DRV_LOG
+		   (ERR,
+		   "Internal entry @ Index:%d rptr_index:0x%x rptr_entry:0x%x num_of_entries:%d\n",
+		   index * TF_SESSION_EM_ENTRY_SIZE,
+		   rptr_index,
+		   rptr_entry,
+		   num_of_entries);
+
+	TF_SET_GFID(gfid,
+		    ((rptr_index << TF_EM_INTERNAL_INDEX_SHIFT) |
+		     rptr_entry),
+		    0); /* N/A for internal table */
+
+	TF_SET_FLOW_ID(parms->flow_id,
+		       gfid,
+		       TF_GFID_TABLE_INTERNAL,
+		       parms->dir);
+
+	TF_SET_FIELDS_IN_FLOW_HANDLE(parms->flow_handle,
+				     num_of_entries,
+				     0,
+				     0,
+				     rptr_index,
+				     rptr_entry,
+				     0);
+	return 0;
+}
+
+/** Delete EM internal entry API
+ *
+ * returns:
+ * 0
+ * -EINVAL
+ */
+int tf_delete_em_internal_entry(struct tf *tfp,
+				struct tf_delete_em_entry_parms *parms)
+{
+	int rc;
+	struct tf_session *session =
+		(struct tf_session *)(tfp->session->core_data);
+	struct stack *pool = &session->em_pool[parms->dir];
+
+	rc = tf_msg_delete_em_entry(tfp, parms);
+
+	/* Return resource to pool */
+	if (rc == 0)
+		stack_push(pool, parms->index / TF_SESSION_EM_ENTRY_SIZE);
+
+	return rc;
+}
+
+
 /** delete EEM hash entry API
  *
  * returns:
diff --git a/drivers/net/bnxt/tf_core/tf_em.h b/drivers/net/bnxt/tf_core/tf_em.h
index 8a3584f..c1805df 100644
--- a/drivers/net/bnxt/tf_core/tf_em.h
+++ b/drivers/net/bnxt/tf_core/tf_em.h
@@ -12,6 +12,20 @@
 #define TF_HW_EM_KEY_MAX_SIZE 52
 #define TF_EM_KEY_RECORD_SIZE 64
 
+/*
+ * Used to build GFID:
+ *
+ *   15           2  0
+ *  +--------------+--+
+ *  |   Index      |E |
+ *  +--------------+--+
+ *
+ * E = Entry (bucket inndex)
+ */
+#define TF_EM_INTERNAL_INDEX_SHIFT 2
+#define TF_EM_INTERNAL_INDEX_MASK 0xFFFC
+#define TF_EM_INTERNAL_ENTRY_MASK  0x3
+
 /** EEM Entry header
  *
  */
@@ -53,6 +67,17 @@ struct tf_eem_64b_entry {
 	struct tf_eem_entry_hdr hdr;
 };
 
+/** EM Entry
+ *  Each EM entry is 512-bit (64-bytes) but ordered differently to
+ *  EEM.
+ */
+struct tf_em_64b_entry {
+	/** Header is 8 bytes long */
+	struct tf_eem_entry_hdr hdr;
+	/** Key is 448 bits - 56 bytes */
+	uint8_t key[TF_EM_KEY_RECORD_SIZE - sizeof(struct tf_eem_entry_hdr)];
+};
+
 /**
  * Allocates EEM Table scope
  *
@@ -106,9 +131,15 @@ int tf_insert_eem_entry(struct tf_session *session,
 			struct tf_tbl_scope_cb *tbl_scope_cb,
 			struct tf_insert_em_entry_parms *parms);
 
+int tf_insert_em_internal_entry(struct tf *tfp,
+				struct tf_insert_em_entry_parms *parms);
+
 int tf_delete_eem_entry(struct tf *tfp,
 			struct tf_delete_em_entry_parms *parms);
 
+int tf_delete_em_internal_entry(struct tf                       *tfp,
+				struct tf_delete_em_entry_parms *parms);
+
 void *tf_em_get_table_page(struct tf_tbl_scope_cb *tbl_scope_cb,
 			   enum tf_dir dir,
 			   uint32_t offset,
diff --git a/drivers/net/bnxt/tf_core/tf_ext_flow_handle.h b/drivers/net/bnxt/tf_core/tf_ext_flow_handle.h
index 417a99c..399f7d1 100644
--- a/drivers/net/bnxt/tf_core/tf_ext_flow_handle.h
+++ b/drivers/net/bnxt/tf_core/tf_ext_flow_handle.h
@@ -90,6 +90,22 @@ do {									\
 		     TF_HASH_TYPE_FLOW_HANDLE_SFT);			\
 } while (0)
 
+#define TF_GET_NUM_KEY_ENTRIES_FROM_FLOW_HANDLE(flow_handle,		\
+					  num_key_entries)		\
+do {									\
+	num_key_entries =						\
+		(((flow_handle) & TF_NUM_KEY_ENTRIES_FLOW_HANDLE_MASK) >> \
+		     TF_NUM_KEY_ENTRIES_FLOW_HANDLE_SFT);		\
+} while (0)
+
+#define TF_GET_ENTRY_NUM_FROM_FLOW_HANDLE(flow_handle,		\
+					  entry_num)		\
+do {									\
+	entry_num =						\
+		(((flow_handle) & TF_ENTRY_NUM_FLOW_HANDLE_MASK) >> \
+		     TF_ENTRY_NUM_FLOW_HANDLE_SFT);		\
+} while (0)
+
 /*
  * 32 bit Flow ID handlers
  */
diff --git a/drivers/net/bnxt/tf_core/tf_msg.c b/drivers/net/bnxt/tf_core/tf_msg.c
index beecafd..554a849 100644
--- a/drivers/net/bnxt/tf_core/tf_msg.c
+++ b/drivers/net/bnxt/tf_core/tf_msg.c
@@ -16,6 +16,7 @@
 #include "tf_msg.h"
 #include "hsi_struct_def_dpdk.h"
 #include "hwrm_tf.h"
+#include "tf_em.h"
 
 /**
  * Endian converts min and max values from the HW response to the query
@@ -1014,14 +1015,93 @@ int tf_msg_em_cfg(struct tf *tfp,
 }
 
 /**
+ * Sends EM internal insert request to Firmware
+ */
+int tf_msg_insert_em_internal_entry(struct tf *tfp,
+				struct tf_insert_em_entry_parms *em_parms,
+				uint16_t *rptr_index,
+				uint8_t *rptr_entry,
+				uint8_t *num_of_entries)
+{
+	int rc;
+	struct tfp_send_msg_parms parms = { 0 };
+	struct tf_em_internal_insert_input req = { 0 };
+	struct tf_em_internal_insert_output resp = { 0 };
+	struct tf_session *tfs = (struct tf_session *)(tfp->session->core_data);
+	struct tf_em_64b_entry *em_result =
+		(struct tf_em_64b_entry *)em_parms->em_record;
+
+	req.fw_session_id =
+		tfp_cpu_to_le_32(tfs->session_id.internal.fw_session_id);
+	memcpy(req.em_key, em_parms->key, ((em_parms->key_sz_in_bits + 7) / 8));
+	req.flags = tfp_cpu_to_le_16(em_parms->dir);
+	req.strength = (em_result->hdr.word1 & TF_LKUP_RECORD_STRENGTH_MASK) >>
+		TF_LKUP_RECORD_STRENGTH_SHIFT;
+	req.em_key_bitlen = em_parms->key_sz_in_bits;
+	req.action_ptr = em_result->hdr.pointer;
+	req.em_record_idx = *rptr_index;
+
+	MSG_PREP(parms,
+		 TF_KONG_MB,
+		 HWRM_TF,
+		 HWRM_TFT_EM_RULE_INSERT,
+		 req,
+		 resp);
+
+	rc = tfp_send_msg_tunneled(tfp, &parms);
+	if (rc)
+		return rc;
+
+	*rptr_entry = resp.rptr_entry;
+	*rptr_index = resp.rptr_index;
+	*num_of_entries = resp.num_of_entries;
+
+	return tfp_le_to_cpu_32(parms.tf_resp_code);
+}
+
+/**
+ * Sends EM delete insert request to Firmware
+ */
+int tf_msg_delete_em_entry(struct tf *tfp,
+			   struct tf_delete_em_entry_parms *em_parms)
+{
+	int rc;
+	struct tfp_send_msg_parms parms = { 0 };
+	struct tf_em_internal_delete_input req = { 0 };
+	struct tf_em_internal_delete_output resp = { 0 };
+	struct tf_session *tfs =
+		(struct tf_session *)(tfp->session->core_data);
+
+	req.tf_session_id =
+		tfp_cpu_to_le_32(tfs->session_id.internal.fw_session_id);
+	req.flags = tfp_cpu_to_le_16(em_parms->dir);
+	req.flow_handle = tfp_cpu_to_le_64(em_parms->flow_handle);
+
+	MSG_PREP(parms,
+		 TF_KONG_MB,
+		 HWRM_TF,
+		 HWRM_TFT_EM_RULE_DELETE,
+		 req,
+		resp);
+
+	rc = tfp_send_msg_tunneled(tfp, &parms);
+	if (rc)
+		return rc;
+
+	em_parms->index = tfp_le_to_cpu_16(resp.em_index);
+
+	return tfp_le_to_cpu_32(parms.tf_resp_code);
+}
+
+/**
  * Sends EM operation request to Firmware
  */
 int tf_msg_em_op(struct tf *tfp,
-		 int        dir,
-		 uint16_t   op)
+		 int dir,
+		 uint16_t op)
 {
 	int rc;
-	struct hwrm_tf_ext_em_op_input  req = {0};
+	struct hwrm_tf_ext_em_op_input req = {0};
 	struct hwrm_tf_ext_em_op_output resp = {0};
 	uint32_t flags;
 	struct tfp_send_msg_parms parms = { 0 };
diff --git a/drivers/net/bnxt/tf_core/tf_msg.h b/drivers/net/bnxt/tf_core/tf_msg.h
index 030d188..89f7370 100644
--- a/drivers/net/bnxt/tf_core/tf_msg.h
+++ b/drivers/net/bnxt/tf_core/tf_msg.h
@@ -122,6 +122,19 @@ int tf_msg_session_sram_resc_flush(struct tf *tfp,
 				   struct tf_rm_entry *sram_entry);
 
 /**
+ * Sends EM internal insert request to Firmware
+ */
+int tf_msg_insert_em_internal_entry(struct tf *tfp,
+				    struct tf_insert_em_entry_parms *params,
+				    uint16_t *rptr_index,
+				    uint8_t *rptr_entry,
+				    uint8_t *num_of_entries);
+/**
+ * Sends EM internal delete request to Firmware
+ */
+int tf_msg_delete_em_entry(struct tf *tfp,
+			   struct tf_delete_em_entry_parms *em_parms);
+/**
  * Sends EM mem register request to Firmware
  */
 int tf_msg_em_mem_rgtr(struct tf *tfp,
diff --git a/drivers/net/bnxt/tf_core/tf_session.h b/drivers/net/bnxt/tf_core/tf_session.h
index 50ef2d5..c9f4f8f 100644
--- a/drivers/net/bnxt/tf_core/tf_session.h
+++ b/drivers/net/bnxt/tf_core/tf_session.h
@@ -13,12 +13,25 @@
 #include "tf_core.h"
 #include "tf_rm.h"
 #include "tf_tbl.h"
+#include "stack.h"
 
 /** Session defines
  */
 #define TF_SESSIONS_MAX	          1          /** max # sessions */
 #define TF_SESSION_ID_INVALID     0xFFFFFFFF /** Invalid Session ID define */
 
+/**
+ * Number of EM entries. Static for now will be removed
+ * when parameter added at a later date. At this stage we
+ * are using fixed size entries so that each stack entry
+ * represents 4 RT (f/n)blocks. So we take the total block
+ * allocation for truflow and divide that by 4.
+ */
+#define TF_SESSION_TOTAL_FN_BLOCKS (1024 * 8) /* 8K blocks */
+#define TF_SESSION_EM_ENTRY_SIZE 4 /* 4 blocks per entry */
+#define TF_SESSION_EM_POOL_SIZE \
+	(TF_SESSION_TOTAL_FN_BLOCKS / TF_SESSION_EM_ENTRY_SIZE)
+
 /** Session
  *
  * Shared memory containing private TruFlow session information.
@@ -289,6 +302,11 @@ struct tf_session {
 
 	/** Table scope array */
 	struct tf_tbl_scope_cb tbl_scopes[TF_NUM_TBL_SCOPE];
+
+	/**
+	 * EM Pools
+	 */
+	struct stack em_pool[TF_DIR_MAX];
 };
 
 #endif /* _TF_SESSION_H_ */
diff --git a/drivers/net/bnxt/tf_core/tf_tbl.c b/drivers/net/bnxt/tf_core/tf_tbl.c
index d900c9c..b9c71d4 100644
--- a/drivers/net/bnxt/tf_core/tf_tbl.c
+++ b/drivers/net/bnxt/tf_core/tf_tbl.c
@@ -156,7 +156,7 @@ tf_em_alloc_pg_tbl(struct tf_em_page_tbl *tp,
 		if (tfp_calloc(&parms) != 0)
 			goto cleanup;
 
-		tp->pg_pa_tbl[i] = (uint64_t)(uintptr_t)parms.mem_pa;
+		tp->pg_pa_tbl[i] = (uintptr_t)parms.mem_pa;
 		tp->pg_va_tbl[i] = parms.mem_va;
 
 		memset(tp->pg_va_tbl[i], 0, pg_size);
@@ -727,13 +727,13 @@ tf_em_validate_num_entries(struct tf_tbl_scope_cb *tbl_scope_cb,
 	tbl_scope_cb->em_ctx_info[TF_DIR_RX].em_tables[KEY1_TABLE].entry_size =
 		parms->rx_max_key_sz_in_bits / 8;
 
-	tbl_scope_cb->em_ctx_info[TF_DIR_RX].em_tables[RECORD_TABLE].num_entries =
-		parms->rx_num_flows_in_k * TF_KILOBYTE;
-	tbl_scope_cb->em_ctx_info[TF_DIR_RX].em_tables[RECORD_TABLE].entry_size =
-		parms->rx_max_action_entry_sz_in_bits / 8;
+	tbl_scope_cb->em_ctx_info[TF_DIR_RX].em_tables[RECORD_TABLE].num_entries
+		= parms->rx_num_flows_in_k * TF_KILOBYTE;
+	tbl_scope_cb->em_ctx_info[TF_DIR_RX].em_tables[RECORD_TABLE].entry_size
+		= parms->rx_max_action_entry_sz_in_bits / 8;
 
-	tbl_scope_cb->em_ctx_info[TF_DIR_RX].em_tables[EFC_TABLE].num_entries =
-		0;
+	tbl_scope_cb->em_ctx_info[TF_DIR_RX].em_tables[EFC_TABLE].num_entries
+		= 0;
 
 	/* Tx */
 	tbl_scope_cb->em_ctx_info[TF_DIR_TX].em_tables[KEY0_TABLE].num_entries =
@@ -746,13 +746,13 @@ tf_em_validate_num_entries(struct tf_tbl_scope_cb *tbl_scope_cb,
 	tbl_scope_cb->em_ctx_info[TF_DIR_TX].em_tables[KEY1_TABLE].entry_size =
 		parms->tx_max_key_sz_in_bits / 8;
 
-	tbl_scope_cb->em_ctx_info[TF_DIR_TX].em_tables[RECORD_TABLE].num_entries =
-		parms->tx_num_flows_in_k * TF_KILOBYTE;
-	tbl_scope_cb->em_ctx_info[TF_DIR_TX].em_tables[RECORD_TABLE].entry_size =
-		parms->tx_max_action_entry_sz_in_bits / 8;
+	tbl_scope_cb->em_ctx_info[TF_DIR_TX].em_tables[RECORD_TABLE].num_entries
+		= parms->tx_num_flows_in_k * TF_KILOBYTE;
+	tbl_scope_cb->em_ctx_info[TF_DIR_TX].em_tables[RECORD_TABLE].entry_size
+		= parms->tx_max_action_entry_sz_in_bits / 8;
 
-	tbl_scope_cb->em_ctx_info[TF_DIR_TX].em_tables[EFC_TABLE].num_entries =
-		0;
+	tbl_scope_cb->em_ctx_info[TF_DIR_TX].em_tables[EFC_TABLE].num_entries
+		= 0;
 
 	return 0;
 }
@@ -792,7 +792,8 @@ tf_set_tbl_entry_internal(struct tf *tfp,
 	index = parms->idx;
 
 	if (parms->type != TF_TBL_TYPE_FULL_ACT_RECORD &&
-	    parms->type != TF_TBL_TYPE_ACT_SP_SMAC_IPV4) {
+	    parms->type != TF_TBL_TYPE_ACT_SP_SMAC_IPV4 &&
+	    parms->type != TF_TBL_TYPE_ACT_STATS_64) {
 		PMD_DRV_LOG(ERR,
 			    "dir:%d, Type not supported, type:%d\n",
 			    parms->dir,
@@ -1179,7 +1180,8 @@ tf_alloc_tbl_entry_pool_internal(struct tf *tfp,
 	    parms->type != TF_TBL_TYPE_ACT_SP_SMAC_IPV4 &&
 	    parms->type != TF_TBL_TYPE_ACT_ENCAP_8B &&
 	    parms->type != TF_TBL_TYPE_ACT_ENCAP_16B &&
-	    parms->type != TF_TBL_TYPE_ACT_ENCAP_64B) {
+	    parms->type != TF_TBL_TYPE_ACT_ENCAP_64B &&
+	    parms->type != TF_TBL_TYPE_ACT_STATS_64) {
 		PMD_DRV_LOG(ERR,
 			    "dir:%d, Type not supported, type:%d\n",
 			    parms->dir,
@@ -1330,7 +1332,8 @@ tf_free_tbl_entry_pool_internal(struct tf *tfp,
 	    parms->type != TF_TBL_TYPE_ACT_SP_SMAC_IPV4 &&
 	    parms->type != TF_TBL_TYPE_ACT_ENCAP_8B &&
 	    parms->type != TF_TBL_TYPE_ACT_ENCAP_16B &&
-	    parms->type != TF_TBL_TYPE_ACT_ENCAP_64B) {
+	    parms->type != TF_TBL_TYPE_ACT_ENCAP_64B &&
+	    parms->type != TF_TBL_TYPE_ACT_STATS_64) {
 		PMD_DRV_LOG(ERR,
 			    "dir:%d, Type not supported, type:%d\n",
 			    parms->dir,
@@ -1801,3 +1804,91 @@ tf_free_tbl_entry(struct tf *tfp,
 			    rc);
 	return rc;
 }
+
+
+static void
+tf_dump_link_page_table(struct tf_em_page_tbl *tp,
+			struct tf_em_page_tbl *tp_next)
+{
+	uint64_t *pg_va;
+	uint32_t i;
+	uint32_t j;
+	uint32_t k = 0;
+
+	printf("pg_count:%d pg_size:0x%x\n",
+	       tp->pg_count,
+	       tp->pg_size);
+	for (i = 0; i < tp->pg_count; i++) {
+		pg_va = tp->pg_va_tbl[i];
+		printf("\t%p\n", (void *)pg_va);
+		for (j = 0; j < MAX_PAGE_PTRS(tp->pg_size); j++) {
+			printf("\t\t%p\n", (void *)(uintptr_t)pg_va[j]);
+			if (((pg_va[j] & 0x7) ==
+			     tfp_cpu_to_le_64(PTU_PTE_LAST |
+					      PTU_PTE_VALID)))
+				return;
+
+			if (!(pg_va[j] & tfp_cpu_to_le_64(PTU_PTE_VALID))) {
+				printf("** Invalid entry **\n");
+				return;
+			}
+
+			if (++k >= tp_next->pg_count) {
+				printf("** Shouldn't get here **\n");
+				return;
+			}
+		}
+	}
+}
+
+void tf_dump_dma(struct tf *tfp, uint32_t tbl_scope_id);
+
+void tf_dump_dma(struct tf *tfp, uint32_t tbl_scope_id)
+{
+	struct tf_session      *session;
+	struct tf_tbl_scope_cb *tbl_scope_cb;
+	struct tf_em_page_tbl *tp;
+	struct tf_em_page_tbl *tp_next;
+	struct tf_em_table *tbl;
+	int i;
+	int j;
+	int dir;
+
+	printf("called %s\n", __func__);
+
+	/* find session struct */
+	session = (struct tf_session *)tfp->session->core_data;
+
+	/* find control block for table scope */
+	tbl_scope_cb = tbl_scope_cb_find(session,
+					 tbl_scope_id);
+	if (tbl_scope_cb == NULL)
+		PMD_DRV_LOG(ERR, "No table scope\n");
+
+	for (dir = 0; dir < TF_DIR_MAX; dir++) {
+		printf("Direction %s:\n", (dir == TF_DIR_RX ? "Rx" : "Tx"));
+
+		for (j = KEY0_TABLE; j < MAX_TABLE; j++) {
+			tbl = &tbl_scope_cb->em_ctx_info[dir].em_tables[j];
+			printf
+	("Table: j:%d type:%d num_entries:%d entry_size:0x%x num_lvl:%d ",
+			       j,
+			       tbl->type,
+			       tbl->num_entries,
+			       tbl->entry_size,
+			       tbl->num_lvl);
+			if (tbl->pg_tbl[0].pg_va_tbl &&
+			    tbl->pg_tbl[0].pg_pa_tbl)
+				printf("%p %p\n",
+			       tbl->pg_tbl[0].pg_va_tbl[0],
+			       (void *)(uintptr_t)tbl->pg_tbl[0].pg_pa_tbl[0]);
+			for (i = 0; i < tbl->num_lvl - 1; i++) {
+				printf("Level:%d\n", i);
+				tp = &tbl->pg_tbl[i];
+				tp_next = &tbl->pg_tbl[i + 1];
+				tf_dump_link_page_table(tp, tp_next);
+			}
+			printf("\n");
+		}
+	}
+}
diff --git a/drivers/net/bnxt/tf_core/tf_tbl.h b/drivers/net/bnxt/tf_core/tf_tbl.h
index bdc6288..6cda487 100644
--- a/drivers/net/bnxt/tf_core/tf_tbl.h
+++ b/drivers/net/bnxt/tf_core/tf_tbl.h
@@ -76,38 +76,51 @@ struct tf_tbl_scope_cb {
 	uint32_t                  *ext_act_pool_mem[TF_DIR_MAX];
 };
 
-/** Hardware Page sizes supported for EEM: 4K, 8K, 64K, 256K, 1M, 2M, 4M, 1G.
- * Round-down other page sizes to the lower hardware page size supported.
+/**
+ * Hardware Page sizes supported for EEM:
+ *   4K, 8K, 64K, 256K, 1M, 2M, 4M, 1G.
+ *
+ * Round-down other page sizes to the lower hardware page
+ * size supported.
  */
-#define BNXT_PAGE_SHIFT 22 /** 2M */
+#define TF_EM_PAGE_SIZE_4K 12
+#define TF_EM_PAGE_SIZE_8K 13
+#define TF_EM_PAGE_SIZE_64K 16
+#define TF_EM_PAGE_SIZE_256K 18
+#define TF_EM_PAGE_SIZE_1M 20
+#define TF_EM_PAGE_SIZE_2M 21
+#define TF_EM_PAGE_SIZE_4M 22
+#define TF_EM_PAGE_SIZE_1G 30
+
+/* Set page size */
+#define PAGE_SIZE TF_EM_PAGE_SIZE_2M
 
-#if (BNXT_PAGE_SHIFT < 12)				/** < 4K >> 4K */
-#define TF_EM_PAGE_SHIFT 12
+#if (PAGE_SIZE == TF_EM_PAGE_SIZE_4K)	/** 4K */
+#define TF_EM_PAGE_SHIFT TF_EM_PAGE_SIZE_4K
 #define TF_EM_PAGE_SIZE_ENUM HWRM_TF_CTXT_MEM_RGTR_INPUT_PAGE_SIZE_4K
-#elif (BNXT_PAGE_SHIFT <= 13)			/** 4K, 8K */
-#define TF_EM_PAGE_SHIFT 13
+#elif (PAGE_SIZE == TF_EM_PAGE_SIZE_8K)	/** 8K */
+#define TF_EM_PAGE_SHIFT TF_EM_PAGE_SIZE_8K
 #define TF_EM_PAGE_SIZE_ENUM HWRM_TF_CTXT_MEM_RGTR_INPUT_PAGE_SIZE_8K
-#elif (BNXT_PAGE_SHIFT < 16)				/** 16K, 32K >> 8K */
-#define TF_EM_PAGE_SHIFT 15
-#define TF_EM_PAGE_SIZE_ENUM HWRM_TF_CTXT_MEM_RGTR_INPUT_PAGE_SIZE_32K
-#elif (BNXT_PAGE_SHIFT <= 17)			/** 64K, 128K >> 64K */
-#define TF_EM_PAGE_SHIFT 16
+#elif (PAGE_SIZE == TF_EM_PAGE_SIZE_64K)	/** 64K */
+#define TF_EM_PAGE_SHIFT TF_EM_PAGE_SIZE_64K
 #define TF_EM_PAGE_SIZE_ENUM HWRM_TF_CTXT_MEM_RGTR_INPUT_PAGE_SIZE_64K
-#elif (BNXT_PAGE_SHIFT <= 19)			/** 256K, 512K >> 256K */
-#define TF_EM_PAGE_SHIFT 18
+#elif (PAGE_SIZE == TF_EM_PAGE_SIZE_256K)	/** 256K */
+#define TF_EM_PAGE_SHIFT TF_EM_PAGE_SIZE_256K
 #define TF_EM_PAGE_SIZE_ENUM HWRM_TF_CTXT_MEM_RGTR_INPUT_PAGE_SIZE_256K
-#elif (BNXT_PAGE_SHIFT <= 21)			/** 1M */
-#define TF_EM_PAGE_SHIFT 20
+#elif (PAGE_SIZE == TF_EM_PAGE_SIZE_1M)	/** 1M */
+#define TF_EM_PAGE_SHIFT TF_EM_PAGE_SIZE_1M
 #define TF_EM_PAGE_SIZE_ENUM HWRM_TF_CTXT_MEM_RGTR_INPUT_PAGE_SIZE_1M
-#elif (BNXT_PAGE_SHIFT <= 22)			/** 2M, 4M */
-#define TF_EM_PAGE_SHIFT 21
+#elif (PAGE_SIZE == TF_EM_PAGE_SIZE_2M)	/** 2M */
+#define TF_EM_PAGE_SHIFT TF_EM_PAGE_SIZE_2M
 #define TF_EM_PAGE_SIZE_ENUM HWRM_TF_CTXT_MEM_RGTR_INPUT_PAGE_SIZE_2M
-#elif (BNXT_PAGE_SHIFT <= 29)			/** 8M ... 512M >> 4M */
-#define TF_EM_PAGE_SHIFT 22
+#elif (PAGE_SIZE == TF_EM_PAGE_SIZE_4M)	/** 4M */
+#define TF_EM_PAGE_SHIFT TF_EM_PAGE_SIZE_4M
 #define TF_EM_PAGE_SIZE_ENUM HWRM_TF_CTXT_MEM_RGTR_INPUT_PAGE_SIZE_4M
-#else						/** >= 1G >> 1G */
-#define TF_EM_PAGE_SHIFT	30
+#elif (PAGE_SIZE == TF_EM_PAGE_SIZE_1G)	/** 1G */
+#define TF_EM_PAGE_SHIFT TF_EM_PAGE_SIZE_1G
 #define TF_EM_PAGE_SIZE_ENUM HWRM_TF_CTXT_MEM_RGTR_INPUT_PAGE_SIZE_1G
+#else
+#error "Invalid Page Size specified. Please use a TF_EM_PAGE_SIZE_n define"
 #endif
 
 #define TF_EM_PAGE_SIZE	(1 << TF_EM_PAGE_SHIFT)
diff --git a/drivers/net/bnxt/tf_core/tfp.h b/drivers/net/bnxt/tf_core/tfp.h
index 8d5e94e..fe49b63 100644
--- a/drivers/net/bnxt/tf_core/tfp.h
+++ b/drivers/net/bnxt/tf_core/tfp.h
@@ -3,14 +3,23 @@
  * All rights reserved.
  */
 
-/* This header file defines the Portability structures and APIs for
+/*
+ * This header file defines the Portability structures and APIs for
  * TruFlow.
  */
 
 #ifndef _TFP_H_
 #define _TFP_H_
 
+#include <rte_config.h>
 #include <rte_spinlock.h>
+#include <rte_log.h>
+#include <rte_byteorder.h>
+
+/**
+ * DPDK/Driver specific log level for the BNXT Eth driver.
+ */
+extern int bnxt_logtype_driver;
 
 /** Spinlock
  */
@@ -18,13 +27,21 @@ struct tfp_spinlock_parms {
 	rte_spinlock_t slock;
 };
 
+#define TFP_DRV_LOG_RAW(level, fmt, args...) \
+	rte_log(RTE_LOG_ ## level, bnxt_logtype_driver, "%s(): " fmt, \
+		__func__, ## args)
+
+#define TFP_DRV_LOG(level, fmt, args...) \
+	TFP_DRV_LOG_RAW(level, fmt, ## args)
+
 /**
  * @file
  *
  * TrueFlow Portability API Header File
  */
 
-/** send message parameter definition
+/**
+ * send message parameter definition
  */
 struct tfp_send_msg_parms {
 	/**
@@ -62,7 +79,8 @@ struct tfp_send_msg_parms {
 	uint32_t *resp_data;
 };
 
-/** calloc parameter definition
+/**
+ * calloc parameter definition
  */
 struct tfp_calloc_parms {
 	/**
@@ -96,43 +114,15 @@ struct tfp_calloc_parms {
  * @ref tfp_send_msg_tunneled
  *
  * @ref tfp_calloc
- * @ref tfp_free
  * @ref tfp_memcpy
+ * @ref tfp_free
  *
  * @ref tfp_spinlock_init
  * @ref tfp_spinlock_lock
  * @ref tfp_spinlock_unlock
  *
- * @ref tfp_cpu_to_le_16
- * @ref tfp_le_to_cpu_16
- * @ref tfp_cpu_to_le_32
- * @ref tfp_le_to_cpu_32
- * @ref tfp_cpu_to_le_64
- * @ref tfp_le_to_cpu_64
- * @ref tfp_cpu_to_be_16
- * @ref tfp_be_to_cpu_16
- * @ref tfp_cpu_to_be_32
- * @ref tfp_be_to_cpu_32
- * @ref tfp_cpu_to_be_64
- * @ref tfp_be_to_cpu_64
  */
 
-#define tfp_cpu_to_le_16(val) rte_cpu_to_le_16(val)
-#define tfp_le_to_cpu_16(val) rte_le_to_cpu_16(val)
-#define tfp_cpu_to_le_32(val) rte_cpu_to_le_32(val)
-#define tfp_le_to_cpu_32(val) rte_le_to_cpu_32(val)
-#define tfp_cpu_to_le_64(val) rte_cpu_to_le_64(val)
-#define tfp_le_to_cpu_64(val) rte_le_to_cpu_64(val)
-#define tfp_cpu_to_be_16(val) rte_cpu_to_be_16(val)
-#define tfp_be_to_cpu_16(val) rte_be_to_cpu_16(val)
-#define tfp_cpu_to_be_32(val) rte_cpu_to_be_32(val)
-#define tfp_be_to_cpu_32(val) rte_be_to_cpu_32(val)
-#define tfp_cpu_to_be_64(val) rte_cpu_to_be_64(val)
-#define tfp_be_to_cpu_64(val) rte_be_to_cpu_64(val)
-#define tfp_bswap_16(val) rte_bswap16(val)
-#define tfp_bswap_32(val) rte_bswap32(val)
-#define tfp_bswap_64(val) rte_bswap64(val)
-
 /**
  * Provides communication capability from the TrueFlow API layer to
  * the TrueFlow firmware. The portability layer internally provides
@@ -162,10 +152,25 @@ int tfp_send_msg_direct(struct tf *tfp,
  *   -1             - Global error like not supported
  *   -EINVAL        - Parameter Error
  */
-int tfp_send_msg_tunneled(struct tf                 *tfp,
+int tfp_send_msg_tunneled(struct tf *tfp,
 			  struct tfp_send_msg_parms *parms);
 
 /**
+ * Sends OEM command message to Chimp
+ *
+ * [in] session, pointer to session handle
+ * [in] max_flows, max number of flows requested
+ *
+ * Returns:
+ *   0              - Success
+ *   -1             - Global error like not supported
+ *   -EINVAL        - Parameter Error
+ */
+int
+tfp_msg_hwrm_oem_cmd(struct tf *tfp,
+		     uint32_t max_flows);
+
+/**
  * Allocates zero'ed memory from the heap.
  *
  * NOTE: Also performs virt2phy address conversion by default thus is
@@ -179,10 +184,58 @@ int tfp_send_msg_tunneled(struct tf                 *tfp,
  *   -EINVAL        - Parameter error
  */
 int tfp_calloc(struct tfp_calloc_parms *parms);
-
-void tfp_free(void *addr);
 void tfp_memcpy(void *dest, void *src, size_t n);
+void tfp_free(void *addr);
+
 void tfp_spinlock_init(struct tfp_spinlock_parms *slock);
 void tfp_spinlock_lock(struct tfp_spinlock_parms *slock);
 void tfp_spinlock_unlock(struct tfp_spinlock_parms *slock);
+
+/**
+ * Lookup of the FID in the platform specific structure.
+ *
+ * [in] session
+ *   Pointer to session handle
+ *
+ * [out] fw_fid
+ *   Pointer to the fw_fid
+ *
+ * Returns:
+ *   0       - Success
+ *   -EINVAL - Parameter error
+ */
+int tfp_get_fid(struct tf *tfp, uint16_t *fw_fid);
+
+
+/*
+ * @ref tfp_cpu_to_le_16
+ * @ref tfp_le_to_cpu_16
+ * @ref tfp_cpu_to_le_32
+ * @ref tfp_le_to_cpu_32
+ * @ref tfp_cpu_to_le_64
+ * @ref tfp_le_to_cpu_64
+ * @ref tfp_cpu_to_be_16
+ * @ref tfp_be_to_cpu_16
+ * @ref tfp_cpu_to_be_32
+ * @ref tfp_be_to_cpu_32
+ * @ref tfp_cpu_to_be_64
+ * @ref tfp_be_to_cpu_64
+ */
+
+#define tfp_cpu_to_le_16(val) rte_cpu_to_le_16(val)
+#define tfp_le_to_cpu_16(val) rte_le_to_cpu_16(val)
+#define tfp_cpu_to_le_32(val) rte_cpu_to_le_32(val)
+#define tfp_le_to_cpu_32(val) rte_le_to_cpu_32(val)
+#define tfp_cpu_to_le_64(val) rte_cpu_to_le_64(val)
+#define tfp_le_to_cpu_64(val) rte_le_to_cpu_64(val)
+#define tfp_cpu_to_be_16(val) rte_cpu_to_be_16(val)
+#define tfp_be_to_cpu_16(val) rte_be_to_cpu_16(val)
+#define tfp_cpu_to_be_32(val) rte_cpu_to_be_32(val)
+#define tfp_be_to_cpu_32(val) rte_be_to_cpu_32(val)
+#define tfp_cpu_to_be_64(val) rte_cpu_to_be_64(val)
+#define tfp_be_to_cpu_64(val) rte_be_to_cpu_64(val)
+#define tfp_bswap_16(val) rte_bswap16(val)
+#define tfp_bswap_32(val) rte_bswap32(val)
+#define tfp_bswap_64(val) rte_bswap64(val)
+
 #endif /* _TFP_H_ */
-- 
2.7.4


^ permalink raw reply	[flat|nested] 271+ messages in thread

* [dpdk-dev] [PATCH 10/50] net/bnxt: modify EM insert and delete to use HWRM direct
  2020-06-12 13:28 [dpdk-dev] [PATCH 00/50] add features for host-based flow management Somnath Kotur
                   ` (8 preceding siblings ...)
  2020-06-12 13:28 ` [dpdk-dev] [PATCH 09/50] net/bnxt: add support for Exact Match Somnath Kotur
@ 2020-06-12 13:28 ` Somnath Kotur
  2020-06-12 13:28 ` [dpdk-dev] [PATCH 11/50] net/bnxt: add multi device support Somnath Kotur
                   ` (40 subsequent siblings)
  50 siblings, 0 replies; 271+ messages in thread
From: Somnath Kotur @ 2020-06-12 13:28 UTC (permalink / raw)
  To: dev; +Cc: ferruh.yigit

From: Pete Spreadborough <peter.spreadborough@broadcom.com>

Modify Exact Match insert and delete to use the HWRM messages directly.
Remove tunneled EM insert and delete message types.

Signed-off-by: Pete Spreadborough <peter.spreadborough@broadcom.com>
Reviewed-by: Randy Schacher <stuart.schacher@broadcom.com>
Signed-off-by: Venkat Duvvuru <venkatkumar.duvvuru@broadcom.com>
---
 drivers/net/bnxt/tf_core/hwrm_tf.h | 70 +++-----------------------------------
 drivers/net/bnxt/tf_core/tf_msg.c  | 66 ++++++++++++++++++++---------------
 2 files changed, 43 insertions(+), 93 deletions(-)

diff --git a/drivers/net/bnxt/tf_core/hwrm_tf.h b/drivers/net/bnxt/tf_core/hwrm_tf.h
index 439950e..d342c69 100644
--- a/drivers/net/bnxt/tf_core/hwrm_tf.h
+++ b/drivers/net/bnxt/tf_core/hwrm_tf.h
@@ -1,5 +1,5 @@
 /* SPDX-License-Identifier: BSD-3-Clause
- * Copyright(c) 2019-2020 Broadcom
+ * Copyright(c) 2019 Broadcom
  * All rights reserved.
  */
 #ifndef _HWRM_TF_H_
@@ -23,8 +23,6 @@ typedef enum tf_subtype {
 	HWRM_TFT_SESSION_SRAM_RESC_FREE = 727,
 	HWRM_TFT_SESSION_SRAM_RESC_FLUSH = 728,
 	HWRM_TFT_TBL_SCOPE_CFG = 731,
-	HWRM_TFT_EM_RULE_INSERT = 739,
-	HWRM_TFT_EM_RULE_DELETE = 740,
 	HWRM_TFT_REG_GET = 821,
 	HWRM_TFT_REG_SET = 822,
 	HWRM_TFT_TBL_TYPE_SET = 823,
@@ -83,10 +81,6 @@ struct tf_session_sram_resc_flush_input;
 struct tf_tbl_type_set_input;
 struct tf_tbl_type_get_input;
 struct tf_tbl_type_get_output;
-struct tf_em_internal_insert_input;
-struct tf_em_internal_insert_output;
-struct tf_em_internal_delete_input;
-struct tf_em_internal_delete_output;
 /* Input params for session attach */
 typedef struct tf_session_attach_input {
 	/* Firmware session id returned when HWRM_TF_SESSION_OPEN is sent */
@@ -351,7 +345,7 @@ typedef struct tf_session_hw_resc_alloc_output {
 	uint16_t			 range_prof_start;
 	/* Number range profiles allocated */
 	uint16_t			 range_prof_stride;
-	/* Starting index of range entries allocated to the session */
+	/* Starting index of range enntries allocated to the session */
 	uint16_t			 range_entries_start;
 	/* Number of range entries allocated */
 	uint16_t			 range_entries_stride;
@@ -453,7 +447,7 @@ typedef struct tf_session_hw_resc_free_input {
 	uint16_t			 range_prof_start;
 	/* Number range profiles allocated */
 	uint16_t			 range_prof_stride;
-	/* Starting index of range entries allocated to the session */
+	/* Starting index of range enntries allocated to the session */
 	uint16_t			 range_entries_start;
 	/* Number of range entries allocated */
 	uint16_t			 range_entries_stride;
@@ -555,7 +549,7 @@ typedef struct tf_session_hw_resc_flush_input {
 	uint16_t			 range_prof_start;
 	/* Number range profiles allocated */
 	uint16_t			 range_prof_stride;
-	/* Starting index of range entries allocated to the session */
+	/* Starting index of range enntries allocated to the session */
 	uint16_t			 range_entries_start;
 	/* Number of range entries allocated */
 	uint16_t			 range_entries_stride;
@@ -922,60 +916,4 @@ typedef struct tf_tbl_type_get_output {
 	uint8_t			  data[TF_BULK_RECV];
 } tf_tbl_type_get_output_t, *ptf_tbl_type_get_output_t;
 
-/* Input params for EM internal rule insert */
-typedef struct tf_em_internal_insert_input {
-	/* Firmware Session Id */
-	uint32_t			 fw_session_id;
-	/* flags */
-	uint16_t			 flags;
-	/* When set to 0, indicates the get apply to RX */
-#define TF_EM_INTERNAL_INSERT_INPUT_FLAGS_DIR_RX	  (0x0)
-	/* When set to 1, indicates the get apply to TX */
-#define TF_EM_INTERNAL_INSERT_INPUT_FLAGS_DIR_TX	  (0x1)
-	/* strength */
-	uint16_t			 strength;
-	/* index to action */
-	uint32_t			 action_ptr;
-	/* index of em record */
-	uint32_t			 em_record_idx;
-	/* EM Key value */
-	uint64_t			 em_key[8];
-	/* number of bits in em_key */
-	uint16_t			 em_key_bitlen;
-} tf_em_internal_insert_input_t, *ptf_em_internal_insert_input_t;
-
-/* Output params for EM internal rule insert */
-typedef struct tf_em_internal_insert_output {
-	/* EM record pointer index */
-	uint16_t			 rptr_index;
-	/* EM record offset 0~3 */
-	uint8_t			  rptr_entry;
-	/* Number of word entries consumed by the key */
-	uint8_t			  num_of_entries;
-} tf_em_internal_insert_output_t, *ptf_em_internal_insert_output_t;
-
-/* Input params for EM INTERNAL rule delete */
-typedef struct tf_em_internal_delete_input {
-	/* Session Id */
-	uint32_t			 tf_session_id;
-	/* flags */
-	uint16_t			 flags;
-	/* When set to 0, indicates the get apply to RX */
-#define TF_EM_INTERNAL_DELETE_INPUT_FLAGS_DIR_RX	  (0x0)
-	/* When set to 1, indicates the get apply to TX */
-#define TF_EM_INTERNAL_DELETE_INPUT_FLAGS_DIR_TX	  (0x1)
-	/* EM internal flow hanndle */
-	uint64_t			 flow_handle;
-	/* EM Key value */
-	uint64_t			 em_key[8];
-	/* number of bits in em_key */
-	uint16_t			 em_key_bitlen;
-} tf_em_internal_delete_input_t, *ptf_em_internal_delete_input_t;
-
-/* Input params for EM INTERNAL rule delete */
-typedef struct tf_em_internal_delete_output {
-	/* Original stack allocation index */
-	uint16_t			 em_index;
-} tf_em_internal_delete_output_t, *ptf_em_internal_delete_output_t;
-
 #endif /* _HWRM_TF_H_ */
diff --git a/drivers/net/bnxt/tf_core/tf_msg.c b/drivers/net/bnxt/tf_core/tf_msg.c
index 554a849..c8f6b88 100644
--- a/drivers/net/bnxt/tf_core/tf_msg.c
+++ b/drivers/net/bnxt/tf_core/tf_msg.c
@@ -1023,32 +1023,38 @@ int tf_msg_insert_em_internal_entry(struct tf *tfp,
 				uint8_t *rptr_entry,
 				uint8_t *num_of_entries)
 {
-	int rc;
-	struct tfp_send_msg_parms parms = { 0 };
-	struct tf_em_internal_insert_input req = { 0 };
-	struct tf_em_internal_insert_output resp = { 0 };
+	int                         rc;
+	struct tfp_send_msg_parms        parms = { 0 };
+	struct hwrm_tf_em_insert_input   req = { 0 };
+	struct hwrm_tf_em_insert_output  resp = { 0 };
 	struct tf_session *tfs = (struct tf_session *)(tfp->session->core_data);
 	struct tf_em_64b_entry *em_result =
 		(struct tf_em_64b_entry *)em_parms->em_record;
+	uint32_t flags;
 
 	req.fw_session_id =
 		tfp_cpu_to_le_32(tfs->session_id.internal.fw_session_id);
 	memcpy(req.em_key, em_parms->key, ((em_parms->key_sz_in_bits + 7) / 8));
-	req.flags = tfp_cpu_to_le_16(em_parms->dir);
+
+	flags = (em_parms->dir == TF_DIR_TX ?
+		 HWRM_TF_EM_INSERT_INPUT_FLAGS_DIR_TX :
+		 HWRM_TF_EM_INSERT_INPUT_FLAGS_DIR_RX);
+	req.flags = tfp_cpu_to_le_16(flags);
 	req.strength = (em_result->hdr.word1 & TF_LKUP_RECORD_STRENGTH_MASK) >>
 		TF_LKUP_RECORD_STRENGTH_SHIFT;
 	req.em_key_bitlen = em_parms->key_sz_in_bits;
 	req.action_ptr = em_result->hdr.pointer;
 	req.em_record_idx = *rptr_index;
 
-	MSG_PREP(parms,
-		 TF_KONG_MB,
-		 HWRM_TF,
-		 HWRM_TFT_EM_RULE_INSERT,
-		 req,
-		 resp);
+	parms.tf_type = HWRM_TF_EM_INSERT;
+	parms.req_data = (uint32_t *)&req;
+	parms.req_size = sizeof(req);
+	parms.resp_data = (uint32_t *)&resp;
+	parms.resp_size = sizeof(resp);
+	parms.mailbox = TF_KONG_MB;
 
-	rc = tfp_send_msg_tunneled(tfp, &parms);
+	rc = tfp_send_msg_direct(tfp,
+				 &parms);
 	if (rc)
 		return rc;
 
@@ -1056,7 +1062,7 @@ int tf_msg_insert_em_internal_entry(struct tf *tfp,
 	*rptr_index = resp.rptr_index;
 	*num_of_entries = resp.num_of_entries;
 
-	return tfp_le_to_cpu_32(parms.tf_resp_code);
+	return 0;
 }
 
 /**
@@ -1065,32 +1071,38 @@ int tf_msg_insert_em_internal_entry(struct tf *tfp,
 int tf_msg_delete_em_entry(struct tf *tfp,
 			   struct tf_delete_em_entry_parms *em_parms)
 {
-	int rc;
-	struct tfp_send_msg_parms parms = { 0 };
-	struct tf_em_internal_delete_input req = { 0 };
-	struct tf_em_internal_delete_output resp = { 0 };
+	int                             rc;
+	struct tfp_send_msg_parms       parms = { 0 };
+	struct hwrm_tf_em_delete_input  req = { 0 };
+	struct hwrm_tf_em_delete_output resp = { 0 };
+	uint32_t flags;
 	struct tf_session *tfs =
 		(struct tf_session *)(tfp->session->core_data);
 
-	req.tf_session_id =
+	req.fw_session_id =
 		tfp_cpu_to_le_32(tfs->session_id.internal.fw_session_id);
-	req.flags = tfp_cpu_to_le_16(em_parms->dir);
+
+	flags = (em_parms->dir == TF_DIR_TX ?
+		 HWRM_TF_EM_DELETE_INPUT_FLAGS_DIR_TX :
+		 HWRM_TF_EM_DELETE_INPUT_FLAGS_DIR_RX);
+	req.flags = tfp_cpu_to_le_16(flags);
 	req.flow_handle = tfp_cpu_to_le_64(em_parms->flow_handle);
 
-	MSG_PREP(parms,
-		 TF_KONG_MB,
-		 HWRM_TF,
-		 HWRM_TFT_EM_RULE_DELETE,
-		 req,
-		resp);
+	parms.tf_type = HWRM_TF_EM_DELETE;
+	parms.req_data = (uint32_t *)&req;
+	parms.req_size = sizeof(req);
+	parms.resp_data = (uint32_t *)&resp;
+	parms.resp_size = sizeof(resp);
+	parms.mailbox = TF_KONG_MB;
 
-	rc = tfp_send_msg_tunneled(tfp, &parms);
+	rc = tfp_send_msg_direct(tfp,
+				 &parms);
 	if (rc)
 		return rc;
 
 	em_parms->index = tfp_le_to_cpu_16(resp.em_index);
 
-	return tfp_le_to_cpu_32(parms.tf_resp_code);
+	return 0;
 }
 
 /**
-- 
2.7.4


^ permalink raw reply	[flat|nested] 271+ messages in thread

* [dpdk-dev] [PATCH 11/50] net/bnxt: add multi device support
  2020-06-12 13:28 [dpdk-dev] [PATCH 00/50] add features for host-based flow management Somnath Kotur
                   ` (9 preceding siblings ...)
  2020-06-12 13:28 ` [dpdk-dev] [PATCH 10/50] net/bnxt: modify EM insert and delete to use HWRM direct Somnath Kotur
@ 2020-06-12 13:28 ` Somnath Kotur
  2020-06-12 13:28 ` [dpdk-dev] [PATCH 12/50] net/bnxt: support bulk table get and mirror Somnath Kotur
                   ` (39 subsequent siblings)
  50 siblings, 0 replies; 271+ messages in thread
From: Somnath Kotur @ 2020-06-12 13:28 UTC (permalink / raw)
  To: dev; +Cc: ferruh.yigit

From: Michael Wildt <michael.wildt@broadcom.com>

Introduce new modules for Device, Resource Manager, Identifier,
Table Types, and TCAM.

Signed-off-by: Michael Wildt <michael.wildt@broadcom.com>
Reviewed-by: Randy Schacher <stuart.schacher@broadcom.com>
Signed-off-by: Venkat Duvvuru <venkatkumar.duvvuru@broadcom.com>
---
 drivers/net/bnxt/meson.build                  |   8 +
 drivers/net/bnxt/tf_core/Makefile             |   9 +
 drivers/net/bnxt/tf_core/cfa_resource_types.h | 266 +++++++++++++++++++
 drivers/net/bnxt/tf_core/tf_core.c            |   2 +
 drivers/net/bnxt/tf_core/tf_core.h            |  56 ++--
 drivers/net/bnxt/tf_core/tf_device.c          |  50 ++++
 drivers/net/bnxt/tf_core/tf_device.h          | 331 +++++++++++++++++++++++
 drivers/net/bnxt/tf_core/tf_device_p4.c       |  24 ++
 drivers/net/bnxt/tf_core/tf_device_p4.h       |  64 +++++
 drivers/net/bnxt/tf_core/tf_identifier.c      |  47 ++++
 drivers/net/bnxt/tf_core/tf_identifier.h      | 140 ++++++++++
 drivers/net/bnxt/tf_core/tf_rm.c              |  54 +---
 drivers/net/bnxt/tf_core/tf_rm.h              |  18 --
 drivers/net/bnxt/tf_core/tf_rm_new.c          | 102 +++++++
 drivers/net/bnxt/tf_core/tf_rm_new.h          | 368 ++++++++++++++++++++++++++
 drivers/net/bnxt/tf_core/tf_session.c         |  31 +++
 drivers/net/bnxt/tf_core/tf_session.h         |  54 ++++
 drivers/net/bnxt/tf_core/tf_shadow_tbl.c      |  63 +++++
 drivers/net/bnxt/tf_core/tf_shadow_tbl.h      | 240 +++++++++++++++++
 drivers/net/bnxt/tf_core/tf_shadow_tcam.c     |  63 +++++
 drivers/net/bnxt/tf_core/tf_shadow_tcam.h     | 239 +++++++++++++++++
 drivers/net/bnxt/tf_core/tf_tbl.c             |   1 +
 drivers/net/bnxt/tf_core/tf_tbl_type.c        |  78 ++++++
 drivers/net/bnxt/tf_core/tf_tbl_type.h        | 309 +++++++++++++++++++++
 drivers/net/bnxt/tf_core/tf_tcam.c            |  78 ++++++
 drivers/net/bnxt/tf_core/tf_tcam.h            | 314 ++++++++++++++++++++++
 drivers/net/bnxt/tf_core/tf_util.c            | 145 ++++++++++
 drivers/net/bnxt/tf_core/tf_util.h            |  41 +++
 28 files changed, 3101 insertions(+), 94 deletions(-)
 create mode 100644 drivers/net/bnxt/tf_core/cfa_resource_types.h
 create mode 100644 drivers/net/bnxt/tf_core/tf_device.c
 create mode 100644 drivers/net/bnxt/tf_core/tf_device.h
 create mode 100644 drivers/net/bnxt/tf_core/tf_device_p4.c
 create mode 100644 drivers/net/bnxt/tf_core/tf_device_p4.h
 create mode 100644 drivers/net/bnxt/tf_core/tf_identifier.c
 create mode 100644 drivers/net/bnxt/tf_core/tf_identifier.h
 create mode 100644 drivers/net/bnxt/tf_core/tf_rm_new.c
 create mode 100644 drivers/net/bnxt/tf_core/tf_rm_new.h
 create mode 100644 drivers/net/bnxt/tf_core/tf_session.c
 create mode 100644 drivers/net/bnxt/tf_core/tf_shadow_tbl.c
 create mode 100644 drivers/net/bnxt/tf_core/tf_shadow_tbl.h
 create mode 100644 drivers/net/bnxt/tf_core/tf_shadow_tcam.c
 create mode 100644 drivers/net/bnxt/tf_core/tf_shadow_tcam.h
 create mode 100644 drivers/net/bnxt/tf_core/tf_tbl_type.c
 create mode 100644 drivers/net/bnxt/tf_core/tf_tbl_type.h
 create mode 100644 drivers/net/bnxt/tf_core/tf_tcam.c
 create mode 100644 drivers/net/bnxt/tf_core/tf_tcam.h
 create mode 100644 drivers/net/bnxt/tf_core/tf_util.c
 create mode 100644 drivers/net/bnxt/tf_core/tf_util.h

diff --git a/drivers/net/bnxt/meson.build b/drivers/net/bnxt/meson.build
index 5c7859c..a50cb26 100644
--- a/drivers/net/bnxt/meson.build
+++ b/drivers/net/bnxt/meson.build
@@ -32,6 +32,14 @@ sources = files('bnxt_cpr.c',
 	'tf_core/tf_rm.c',
 	'tf_core/tf_tbl.c',
 	'tf_core/tfp.c',
+	'tf_core/tf_device_p4.c',
+	'tf_core/tf_identifier.c',
+	'tf_core/tf_shadow_tbl.c',
+	'tf_core/tf_shadow_tcam.c',
+	'tf_core/tf_tbl_type.c',
+	'tf_core/tf_tcam.c',
+	'tf_core/tf_util.c',
+	'tf_core/tf_rm_new.c',
 
 	'tf_ulp/bnxt_ulp.c',
 	'tf_ulp/ulp_mark_mgr.c',
diff --git a/drivers/net/bnxt/tf_core/Makefile b/drivers/net/bnxt/tf_core/Makefile
index 379da30..71df75b 100644
--- a/drivers/net/bnxt/tf_core/Makefile
+++ b/drivers/net/bnxt/tf_core/Makefile
@@ -14,4 +14,13 @@ SRCS-$(CONFIG_RTE_LIBRTE_BNXT_PMD) += tf_core/tfp.c
 SRCS-$(CONFIG_RTE_LIBRTE_BNXT_PMD) += tf_core/tf_msg.c
 SRCS-$(CONFIG_RTE_LIBRTE_BNXT_PMD) += tf_core/tf_em.c
 SRCS-$(CONFIG_RTE_LIBRTE_BNXT_PMD) += tf_core/tf_tbl.c
+SRCS-$(CONFIG_RTE_LIBRTE_BNXT_PMD) += tf_core/tf_device.c
+SRCS-$(CONFIG_RTE_LIBRTE_BNXT_PMD) += tf_core/tf_device_p4.c
+SRCS-$(CONFIG_RTE_LIBRTE_BNXT_PMD) += tf_core/tf_identifier.c
+SRCS-$(CONFIG_RTE_LIBRTE_BNXT_PMD) += tf_core/tf_shadow_tbl.c
+SRCS-$(CONFIG_RTE_LIBRTE_BNXT_PMD) += tf_core/tf_shadow_tcam.c
+SRCS-$(CONFIG_RTE_LIBRTE_BNXT_PMD) += tf_core/tf_tbl_type.c
+SRCS-$(CONFIG_RTE_LIBRTE_BNXT_PMD) += tf_core/tf_tcam.c
+SRCS-$(CONFIG_RTE_LIBRTE_BNXT_PMD) += tf_core/tf_util.c
+SRCS-$(CONFIG_RTE_LIBRTE_BNXT_PMD) += tf_core/tf_rm_new.c
 
diff --git a/drivers/net/bnxt/tf_core/cfa_resource_types.h b/drivers/net/bnxt/tf_core/cfa_resource_types.h
new file mode 100644
index 0000000..c0c1e75
--- /dev/null
+++ b/drivers/net/bnxt/tf_core/cfa_resource_types.h
@@ -0,0 +1,266 @@
+/*
+ * Copyright(c) 2001-2020, Broadcom. All rights reserved. The
+ * term Broadcom refers to Broadcom Inc. and/or its subsidiaries.
+ * Proprietary and Confidential Information.
+ *
+ * This source file is the property of Broadcom Corporation, and
+ * may not be copied or distributed in any isomorphic form without
+ * the prior written consent of Broadcom Corporation.
+ *
+ * DO NOT MODIFY!!! This file is automatically generated.
+ */
+
+#ifndef _CFA_RESOURCE_TYPES_H_
+#define _CFA_RESOURCE_TYPES_H_
+
+/* L2 Context TCAM */
+#define CFA_RESOURCE_TYPE_P59_L2_CTXT_TCAM    0x0UL
+/* Profile Func */
+#define CFA_RESOURCE_TYPE_P59_PROF_FUNC       0x1UL
+/* Profile TCAM */
+#define CFA_RESOURCE_TYPE_P59_PROF_TCAM       0x2UL
+/* Exact Match Profile Id */
+#define CFA_RESOURCE_TYPE_P59_EM_PROF_ID      0x3UL
+/* Wildcard TCAM Profile Id */
+#define CFA_RESOURCE_TYPE_P59_WC_TCAM_PROF_ID 0x4UL
+/* Wildcard TCAM */
+#define CFA_RESOURCE_TYPE_P59_WC_TCAM         0x5UL
+/* Meter Profile */
+#define CFA_RESOURCE_TYPE_P59_METER_PROF      0x6UL
+/* Meter */
+#define CFA_RESOURCE_TYPE_P59_METER           0x7UL
+/* Meter */
+#define CFA_RESOURCE_TYPE_P59_MIRROR          0x8UL
+/* Source Properties TCAM */
+#define CFA_RESOURCE_TYPE_P59_SP_TCAM         0x9UL
+/* Exact Match Flexible Key Builder */
+#define CFA_RESOURCE_TYPE_P59_EM_FKB          0xaUL
+/* Wildcard Flexible Key Builder */
+#define CFA_RESOURCE_TYPE_P59_WC_FKB          0xbUL
+/* Table Scope */
+#define CFA_RESOURCE_TYPE_P59_TBL_SCOPE       0xcUL
+/* L2 Func */
+#define CFA_RESOURCE_TYPE_P59_L2_FUNC         0xdUL
+/* EPOCH */
+#define CFA_RESOURCE_TYPE_P59_EPOCH           0xeUL
+/* Metadata */
+#define CFA_RESOURCE_TYPE_P59_METADATA        0xfUL
+/* Connection Tracking Rule TCAM */
+#define CFA_RESOURCE_TYPE_P59_CT_RULE_TCAM    0x10UL
+/* Range Profile */
+#define CFA_RESOURCE_TYPE_P59_RANGE_PROF      0x11UL
+/* Range */
+#define CFA_RESOURCE_TYPE_P59_RANGE           0x12UL
+/* Link Aggrigation */
+#define CFA_RESOURCE_TYPE_P59_LAG             0x13UL
+/* VEB TCAM */
+#define CFA_RESOURCE_TYPE_P59_VEB_TCAM        0x14UL
+#define CFA_RESOURCE_TYPE_P59_LAST           CFA_RESOURCE_TYPE_P59_VEB_TCAM
+
+
+/* SRAM Multicast Group */
+#define CFA_RESOURCE_TYPE_P58_SRAM_MCG             0x0UL
+/* SRAM Encap 8 byte record */
+#define CFA_RESOURCE_TYPE_P58_SRAM_ENCAP_8B        0x1UL
+/* SRAM Encap 16 byte record */
+#define CFA_RESOURCE_TYPE_P58_SRAM_ENCAP_16B       0x2UL
+/* SRAM Encap 64 byte record */
+#define CFA_RESOURCE_TYPE_P58_SRAM_ENCAP_64B       0x3UL
+/* SRAM Source Property MAC */
+#define CFA_RESOURCE_TYPE_P58_SRAM_SP_MAC          0x4UL
+/* SRAM Source Property MAC and IPv4 */
+#define CFA_RESOURCE_TYPE_P58_SRAM_SP_MAC_IPV4     0x5UL
+/* SRAM Source Property MAC and IPv6 */
+#define CFA_RESOURCE_TYPE_P58_SRAM_SP_MAC_IPV6     0x6UL
+/* SRAM Network Address Translation Source Port */
+#define CFA_RESOURCE_TYPE_P58_SRAM_NAT_SPORT       0x7UL
+/* SRAM Network Address Translation Destination Port */
+#define CFA_RESOURCE_TYPE_P58_SRAM_NAT_DPORT       0x8UL
+/* SRAM Network Address Translation Source IPv4 address */
+#define CFA_RESOURCE_TYPE_P58_SRAM_NAT_S_IPV4      0x9UL
+/* SRAM Network Address Translation Destination IPv4 address */
+#define CFA_RESOURCE_TYPE_P58_SRAM_NAT_D_IPV4      0xaUL
+/* SRAM Network Address Translation Source IPv4 address */
+#define CFA_RESOURCE_TYPE_P58_SRAM_NAT_S_IPV6      0xbUL
+/* SRAM Network Address Translation Destination IPv4 address */
+#define CFA_RESOURCE_TYPE_P58_SRAM_NAT_D_IPV6      0xcUL
+/* Meter */
+#define CFA_RESOURCE_TYPE_P58_SRAM_METER           0xdUL
+/* Flow State */
+#define CFA_RESOURCE_TYPE_P58_SRAM_FLOW_STATE      0xeUL
+/* Full Action Records */
+#define CFA_RESOURCE_TYPE_P58_SRAM_FULL_ACTION     0xfUL
+/* Action Record Format 0 */
+#define CFA_RESOURCE_TYPE_P58_SRAM_FORMAT_0_ACTION 0x10UL
+/* Action Record Format 2 */
+#define CFA_RESOURCE_TYPE_P58_SRAM_FORMAT_2_ACTION 0x11UL
+/* Action Record Format 3 */
+#define CFA_RESOURCE_TYPE_P58_SRAM_FORMAT_3_ACTION 0x12UL
+/* Action Record Format 4 */
+#define CFA_RESOURCE_TYPE_P58_SRAM_FORMAT_4_ACTION 0x13UL
+/* L2 Context TCAM */
+#define CFA_RESOURCE_TYPE_P58_L2_CTXT_TCAM         0x14UL
+/* Profile Func */
+#define CFA_RESOURCE_TYPE_P58_PROF_FUNC            0x15UL
+/* Profile TCAM */
+#define CFA_RESOURCE_TYPE_P58_PROF_TCAM            0x16UL
+/* Exact Match Profile Id */
+#define CFA_RESOURCE_TYPE_P58_EM_PROF_ID           0x17UL
+/* Wildcard Profile Id */
+#define CFA_RESOURCE_TYPE_P58_WC_TCAM_PROF_ID      0x18UL
+/* Wildcard TCAM */
+#define CFA_RESOURCE_TYPE_P58_WC_TCAM              0x19UL
+/* Meter profile */
+#define CFA_RESOURCE_TYPE_P58_METER_PROF           0x1aUL
+/* Meter */
+#define CFA_RESOURCE_TYPE_P58_METER                0x1bUL
+/* Meter */
+#define CFA_RESOURCE_TYPE_P58_MIRROR               0x1cUL
+/* Source Property TCAM */
+#define CFA_RESOURCE_TYPE_P58_SP_TCAM              0x1dUL
+/* Exact Match Flexible Key Builder */
+#define CFA_RESOURCE_TYPE_P58_EM_FKB               0x1eUL
+/* Wildcard Flexible Key Builder */
+#define CFA_RESOURCE_TYPE_P58_WC_FKB               0x1fUL
+/* VEB TCAM */
+#define CFA_RESOURCE_TYPE_P58_VEB_TCAM             0x20UL
+#define CFA_RESOURCE_TYPE_P58_LAST                CFA_RESOURCE_TYPE_P58_VEB_TCAM
+
+
+/* SRAM Multicast Group */
+#define CFA_RESOURCE_TYPE_P45_SRAM_MCG             0x0UL
+/* SRAM Encap 8 byte record */
+#define CFA_RESOURCE_TYPE_P45_SRAM_ENCAP_8B        0x1UL
+/* SRAM Encap 16 byte record */
+#define CFA_RESOURCE_TYPE_P45_SRAM_ENCAP_16B       0x2UL
+/* SRAM Encap 64 byte record */
+#define CFA_RESOURCE_TYPE_P45_SRAM_ENCAP_64B       0x3UL
+/* SRAM Source Property MAC */
+#define CFA_RESOURCE_TYPE_P45_SRAM_SP_MAC          0x4UL
+/* SRAM Source Property MAC and IPv4 */
+#define CFA_RESOURCE_TYPE_P45_SRAM_SP_MAC_IPV4     0x5UL
+/* SRAM Source Property MAC and IPv6 */
+#define CFA_RESOURCE_TYPE_P45_SRAM_SP_MAC_IPV6     0x6UL
+/* SRAM 64B Counters */
+#define CFA_RESOURCE_TYPE_P45_SRAM_COUNTER_64B     0x7UL
+/* SRAM Network Address Translation Source Port */
+#define CFA_RESOURCE_TYPE_P45_SRAM_NAT_SPORT       0x8UL
+/* SRAM Network Address Translation Destination Port */
+#define CFA_RESOURCE_TYPE_P45_SRAM_NAT_DPORT       0x9UL
+/* SRAM Network Address Translation Source IPv4 address */
+#define CFA_RESOURCE_TYPE_P45_SRAM_NAT_S_IPV4      0xaUL
+/* SRAM Network Address Translation Destination IPv4 address */
+#define CFA_RESOURCE_TYPE_P45_SRAM_NAT_D_IPV4      0xbUL
+/* SRAM Network Address Translation Source IPv6 address */
+#define CFA_RESOURCE_TYPE_P45_SRAM_NAT_S_IPV6      0xcUL
+/* SRAM Network Address Translation Destination IPv6 address */
+#define CFA_RESOURCE_TYPE_P45_SRAM_NAT_D_IPV6      0xdUL
+/* Meter */
+#define CFA_RESOURCE_TYPE_P45_SRAM_METER           0xeUL
+/* Flow State */
+#define CFA_RESOURCE_TYPE_P45_SRAM_FLOW_STATE      0xfUL
+/* Full Action Records */
+#define CFA_RESOURCE_TYPE_P45_SRAM_FULL_ACTION     0x10UL
+/* Action Record Format 0 */
+#define CFA_RESOURCE_TYPE_P45_SRAM_FORMAT_0_ACTION 0x11UL
+/* Action Record Format 2 */
+#define CFA_RESOURCE_TYPE_P45_SRAM_FORMAT_2_ACTION 0x12UL
+/* Action Record Format 3 */
+#define CFA_RESOURCE_TYPE_P45_SRAM_FORMAT_3_ACTION 0x13UL
+/* Action Record Format 4 */
+#define CFA_RESOURCE_TYPE_P45_SRAM_FORMAT_4_ACTION 0x14UL
+/* L2 Context TCAM */
+#define CFA_RESOURCE_TYPE_P45_L2_CTXT_TCAM         0x15UL
+/* Profile Func */
+#define CFA_RESOURCE_TYPE_P45_PROF_FUNC            0x16UL
+/* Profile TCAM */
+#define CFA_RESOURCE_TYPE_P45_PROF_TCAM            0x17UL
+/* Exact Match Profile Id */
+#define CFA_RESOURCE_TYPE_P45_EM_PROF_ID           0x18UL
+/* Exact Match Record */
+#define CFA_RESOURCE_TYPE_P45_EM_REC               0x19UL
+/* Wildcard Profile Id */
+#define CFA_RESOURCE_TYPE_P45_WC_TCAM_PROF_ID      0x1aUL
+/* Wildcard TCAM */
+#define CFA_RESOURCE_TYPE_P45_WC_TCAM              0x1bUL
+/* Meter profile */
+#define CFA_RESOURCE_TYPE_P45_METER_PROF           0x1cUL
+/* Meter */
+#define CFA_RESOURCE_TYPE_P45_METER                0x1dUL
+/* Meter */
+#define CFA_RESOURCE_TYPE_P45_MIRROR               0x1eUL
+/* Source Property TCAM */
+#define CFA_RESOURCE_TYPE_P45_SP_TCAM              0x1fUL
+/* VEB TCAM */
+#define CFA_RESOURCE_TYPE_P45_VEB_TCAM             0x20UL
+#define CFA_RESOURCE_TYPE_P45_LAST                CFA_RESOURCE_TYPE_P45_VEB_TCAM
+
+
+/* SRAM Multicast Group */
+#define CFA_RESOURCE_TYPE_P4_SRAM_MCG             0x0UL
+/* SRAM Encap 8 byte record */
+#define CFA_RESOURCE_TYPE_P4_SRAM_ENCAP_8B        0x1UL
+/* SRAM Encap 16 byte record */
+#define CFA_RESOURCE_TYPE_P4_SRAM_ENCAP_16B       0x2UL
+/* SRAM Encap 64 byte record */
+#define CFA_RESOURCE_TYPE_P4_SRAM_ENCAP_64B       0x3UL
+/* SRAM Source Property MAC */
+#define CFA_RESOURCE_TYPE_P4_SRAM_SP_MAC          0x4UL
+/* SRAM Source Property MAC and IPv4 */
+#define CFA_RESOURCE_TYPE_P4_SRAM_SP_MAC_IPV4     0x5UL
+/* SRAM Source Property MAC and IPv6 */
+#define CFA_RESOURCE_TYPE_P4_SRAM_SP_MAC_IPV6     0x6UL
+/* SRAM 64B Counters */
+#define CFA_RESOURCE_TYPE_P4_SRAM_COUNTER_64B     0x7UL
+/* SRAM Network Address Translation Source Port */
+#define CFA_RESOURCE_TYPE_P4_SRAM_NAT_SPORT       0x8UL
+/* SRAM Network Address Translation Destination Port */
+#define CFA_RESOURCE_TYPE_P4_SRAM_NAT_DPORT       0x9UL
+/* SRAM Network Address Translation Source IPv4 address */
+#define CFA_RESOURCE_TYPE_P4_SRAM_NAT_S_IPV4      0xaUL
+/* SRAM Network Address Translation Destination IPv4 address */
+#define CFA_RESOURCE_TYPE_P4_SRAM_NAT_D_IPV4      0xbUL
+/* SRAM Network Address Translation Source IPv6 address */
+#define CFA_RESOURCE_TYPE_P4_SRAM_NAT_S_IPV6      0xcUL
+/* SRAM Network Address Translation Destination IPv6 address */
+#define CFA_RESOURCE_TYPE_P4_SRAM_NAT_D_IPV6      0xdUL
+/* Meter */
+#define CFA_RESOURCE_TYPE_P4_SRAM_METER           0xeUL
+/* Flow State */
+#define CFA_RESOURCE_TYPE_P4_SRAM_FLOW_STATE      0xfUL
+/* Full Action Records */
+#define CFA_RESOURCE_TYPE_P4_SRAM_FULL_ACTION     0x10UL
+/* Action Record Format 0 */
+#define CFA_RESOURCE_TYPE_P4_SRAM_FORMAT_0_ACTION 0x11UL
+/* Action Record Format 2 */
+#define CFA_RESOURCE_TYPE_P4_SRAM_FORMAT_2_ACTION 0x12UL
+/* Action Record Format 3 */
+#define CFA_RESOURCE_TYPE_P4_SRAM_FORMAT_3_ACTION 0x13UL
+/* Action Record Format 4 */
+#define CFA_RESOURCE_TYPE_P4_SRAM_FORMAT_4_ACTION 0x14UL
+/* L2 Context TCAM */
+#define CFA_RESOURCE_TYPE_P4_L2_CTXT_TCAM         0x15UL
+/* Profile Func */
+#define CFA_RESOURCE_TYPE_P4_PROF_FUNC            0x16UL
+/* Profile TCAM */
+#define CFA_RESOURCE_TYPE_P4_PROF_TCAM            0x17UL
+/* Exact Match Profile Id */
+#define CFA_RESOURCE_TYPE_P4_EM_PROF_ID           0x18UL
+/* Exact Match Record */
+#define CFA_RESOURCE_TYPE_P4_EM_REC               0x19UL
+/* Wildcard Profile Id */
+#define CFA_RESOURCE_TYPE_P4_WC_TCAM_PROF_ID      0x1aUL
+/* Wildcard TCAM */
+#define CFA_RESOURCE_TYPE_P4_WC_TCAM              0x1bUL
+/* Meter profile */
+#define CFA_RESOURCE_TYPE_P4_METER_PROF           0x1cUL
+/* Meter */
+#define CFA_RESOURCE_TYPE_P4_METER                0x1dUL
+/* Meter */
+#define CFA_RESOURCE_TYPE_P4_MIRROR               0x1eUL
+/* Source Property TCAM */
+#define CFA_RESOURCE_TYPE_P4_SP_TCAM              0x1fUL
+#define CFA_RESOURCE_TYPE_P4_LAST                CFA_RESOURCE_TYPE_P4_SP_TCAM
+
+
+#endif /* _CFA_RESOURCE_TYPES_H_ */
diff --git a/drivers/net/bnxt/tf_core/tf_core.c b/drivers/net/bnxt/tf_core/tf_core.c
index ba54df6..58924b1 100644
--- a/drivers/net/bnxt/tf_core/tf_core.c
+++ b/drivers/net/bnxt/tf_core/tf_core.c
@@ -6,6 +6,7 @@
 #include <stdio.h>
 
 #include "tf_core.h"
+#include "tf_util.h"
 #include "tf_session.h"
 #include "tf_tbl.h"
 #include "tf_em.h"
@@ -229,6 +230,7 @@ tf_open_session(struct tf                    *tfp,
 
 	/* Initialize Session */
 	session->device_type = parms->device_type;
+	session->dev = NULL;
 	tf_rm_init(tfp);
 
 	/* Construct the Session ID */
diff --git a/drivers/net/bnxt/tf_core/tf_core.h b/drivers/net/bnxt/tf_core/tf_core.h
index 81ff760..becc50c 100644
--- a/drivers/net/bnxt/tf_core/tf_core.h
+++ b/drivers/net/bnxt/tf_core/tf_core.h
@@ -371,6 +371,35 @@ struct tf {
 	struct tf_session_info *session;
 };
 
+/**
+ * tf_session_resources parameter definition.
+ */
+struct tf_session_resources {
+	/** [in] Requested Identifier Resources
+	 *
+	 * The number of identifier resources requested for the session.
+	 * The index used is tf_identifier_type.
+	 */
+	uint16_t identifer_cnt[TF_DIR_MAX][TF_IDENT_TYPE_MAX];
+	/** [in] Requested Index Table resource counts
+	 *
+	 * The number of index table resources requested for the session.
+	 * The index used is tf_tbl_type.
+	 */
+	uint16_t tbl_cnt[TF_TBL_TYPE_MAX][TF_DIR_MAX];
+	/** [in] Requested TCAM Table resource counts
+	 *
+	 * The number of TCAM table resources requested for the session.
+	 * The index used is tf_tcam_tbl_type.
+	 */
+	uint16_t tcam_tbl_cnt[TF_TCAM_TBL_TYPE_MAX][TF_DIR_MAX];
+	/** [in] Requested EM resource counts
+	 *
+	 * The number of internal EM table resources requested for the session
+	 * The index used is tf_em_tbl_type.
+	 */
+	uint16_t em_tbl_cnt[TF_EM_TBL_TYPE_MAX][TF_DIR_MAX];
+};
 
 /**
  * tf_open_session parameters definition.
@@ -414,33 +443,14 @@ struct tf_open_session_parms {
 	union tf_session_id session_id;
 	/** [in] device type
 	 *
-	 * Device type is passed, one of Wh+, Brd2, Brd3, Brd4
+	 * Device type is passed, one of Wh+, SR, Thor, SR2
 	 */
 	enum tf_device_type device_type;
-	/** [in] Requested Identifier Resources
-	 *
-	 * The number of identifier resources requested for the session.
-	 * The index used is tf_identifier_type.
-	 */
-	uint16_t identifer_cnt[TF_IDENT_TYPE_MAX];
-	/** [in] Requested Index Table resource counts
-	 *
-	 * The number of index table resources requested for the session.
-	 * The index used is tf_tbl_type.
-	 */
-	uint16_t tbl_cnt[TF_TBL_TYPE_MAX];
-	/** [in] Requested TCAM Table resource counts
-	 *
-	 * The number of TCAM table resources requested for the session.
-	 * The index used is tf_tcam_tbl_type.
-	 */
-	uint16_t tcam_tbl_cnt[TF_TCAM_TBL_TYPE_MAX];
-	/** [in] Requested EM resource counts
+	/** [in] resources
 	 *
-	 * The number of internal EM table resources requested for the session
-	 * The index used is tf_em_tbl_type.
+	 * Resource allocation
 	 */
-	uint16_t em_tbl_cnt[TF_EM_TBL_TYPE_MAX];
+	struct tf_session_resources resources;
 };
 
 /**
diff --git a/drivers/net/bnxt/tf_core/tf_device.c b/drivers/net/bnxt/tf_core/tf_device.c
new file mode 100644
index 0000000..3b36831
--- /dev/null
+++ b/drivers/net/bnxt/tf_core/tf_device.c
@@ -0,0 +1,50 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2019-2020 Broadcom
+ * All rights reserved.
+ */
+
+#include "tf_device.h"
+#include "tf_device_p4.h"
+#include "tfp.h"
+#include "bnxt.h"
+
+struct tf;
+
+/**
+ * Device specific bind function
+ */
+static int
+dev_bind_p4(struct tf *tfp __rte_unused,
+	    struct tf_session_resources *resources __rte_unused,
+	    struct tf_dev_info *dev_info)
+{
+	/* Initialize the modules */
+
+	dev_info->ops = &tf_dev_ops_p4;
+	return 0;
+}
+
+int
+dev_bind(struct tf *tfp __rte_unused,
+	 enum tf_device_type type,
+	 struct tf_session_resources *resources,
+	 struct tf_dev_info *dev_info)
+{
+	switch (type) {
+	case TF_DEVICE_TYPE_WH:
+		return dev_bind_p4(tfp,
+				   resources,
+				   dev_info);
+	default:
+		TFP_DRV_LOG(ERR,
+			    "Device type not supported\n");
+		return -ENOTSUP;
+	}
+}
+
+int
+dev_unbind(struct tf *tfp __rte_unused,
+	   struct tf_dev_info *dev_handle __rte_unused)
+{
+	return 0;
+}
diff --git a/drivers/net/bnxt/tf_core/tf_device.h b/drivers/net/bnxt/tf_core/tf_device.h
new file mode 100644
index 0000000..8b63ff1
--- /dev/null
+++ b/drivers/net/bnxt/tf_core/tf_device.h
@@ -0,0 +1,331 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2019-2020 Broadcom
+ * All rights reserved.
+ */
+
+#ifndef _TF_DEVICE_H_
+#define _TF_DEVICE_H_
+
+#include "tf_core.h"
+#include "tf_identifier.h"
+#include "tf_tbl_type.h"
+#include "tf_tcam.h"
+
+struct tf;
+struct tf_session;
+
+/**
+ * The Device module provides a general device template. A supported
+ * device type should implement one or more of the listed function
+ * pointers according to its capabilities.
+ *
+ * If a device function pointer is NULL the device capability is not
+ * supported.
+ */
+
+/**
+ * TF device information
+ */
+struct tf_dev_info {
+	const struct tf_dev_ops *ops;
+};
+
+/**
+ * @page device Device
+ *
+ * @ref tf_dev_bind
+ *
+ * @ref tf_dev_unbind
+ */
+
+/**
+ * Device bind handles the initialization of the specified device
+ * type.
+ *
+ * [in] tfp
+ *   Pointer to TF handle
+ *
+ * [in] type
+ *   Device type
+ *
+ * [in] resources
+ *   Pointer to resource allocation information
+ *
+ * [out] dev_handle
+ *   Device handle
+ *
+ * Returns
+ *   - (0) if successful.
+ *   - (-EINVAL) on failure.
+ */
+int dev_bind(struct tf *tfp,
+	     enum tf_device_type type,
+	     struct tf_session_resources *resources,
+	     struct tf_dev_info *dev_handle);
+
+/**
+ * Device release handles cleanup of the device specific information.
+ *
+ * [in] tfp
+ *   Pointer to TF handle
+ *
+ * [in] dev_handle
+ *   Device handle
+ */
+int dev_unbind(struct tf *tfp,
+	       struct tf_dev_info *dev_handle);
+
+/**
+ * Truflow device specific function hooks structure
+ *
+ * The following device hooks can be defined; unless noted otherwise,
+ * they are optional and can be filled with a null pointer. The
+ * purpose of these hooks is to support Truflow device operations for
+ * different device variants.
+ */
+struct tf_dev_ops {
+	/**
+	 * Allocation of an identifier element.
+	 *
+	 * This API allocates the specified identifier element from a
+	 * device specific identifier DB. The allocated element is
+	 * returned.
+	 *
+	 * [in] tfp
+	 *   Pointer to TF handle
+	 *
+	 * [in] parms
+	 *   Pointer to identifier allocation parameters
+	 *
+	 * Returns
+	 *   - (0) if successful.
+	 *   - (-EINVAL) on failure.
+	 */
+	int (*tf_dev_alloc_ident)(struct tf *tfp,
+				  struct tf_ident_alloc_parms *parms);
+
+	/**
+	 * Free of an identifier element.
+	 *
+	 * This API free's a previous allocated identifier element from a
+	 * device specific identifier DB.
+	 *
+	 * [in] tfp
+	 *   Pointer to TF handle
+	 *
+	 * [in] parms
+	 *   Pointer to identifier free parameters
+	 *
+	 * Returns
+	 *   - (0) if successful.
+	 *   - (-EINVAL) on failure.
+	 */
+	int (*tf_dev_free_ident)(struct tf *tfp,
+				 struct tf_ident_free_parms *parms);
+
+	/**
+	 * Allocation of a table type element.
+	 *
+	 * This API allocates the specified table type element from a
+	 * device specific table type DB. The allocated element is
+	 * returned.
+	 *
+	 * [in] tfp
+	 *   Pointer to TF handle
+	 *
+	 * [in] parms
+	 *   Pointer to table type allocation parameters
+	 *
+	 * Returns
+	 *   - (0) if successful.
+	 *   - (-EINVAL) on failure.
+	 */
+	int (*tf_dev_alloc_tbl_type)(struct tf *tfp,
+				     struct tf_tbl_type_alloc_parms *parms);
+
+	/**
+	 * Free of a table type element.
+	 *
+	 * This API free's a previous allocated table type element from a
+	 * device specific table type DB.
+	 *
+	 * [in] tfp
+	 *   Pointer to TF handle
+	 *
+	 * [in] parms
+	 *   Pointer to table type free parameters
+	 *
+	 * Returns
+	 *   - (0) if successful.
+	 *   - (-EINVAL) on failure.
+	 */
+	int (*tf_dev_free_tbl_type)(struct tf *tfp,
+				    struct tf_tbl_type_free_parms *parms);
+
+	/**
+	 * Searches for the specified table type element in a shadow DB.
+	 *
+	 * This API searches for the specified table type element in a
+	 * device specific shadow DB. If the element is found the
+	 * reference count for the element is updated. If the element
+	 * is not found a new element is allocated from the table type
+	 * DB and then inserted into the shadow DB.
+	 *
+	 * [in] tfp
+	 *   Pointer to TF handle
+	 *
+	 * [in] parms
+	 *   Pointer to table type allocation and search parameters
+	 *
+	 * Returns
+	 *   - (0) if successful.
+	 *   - (-EINVAL) on failure.
+	 */
+	int (*tf_dev_alloc_search_tbl_type)
+			(struct tf *tfp,
+			struct tf_tbl_type_alloc_search_parms *parms);
+
+	/**
+	 * Sets the specified table type element.
+	 *
+	 * This API sets the specified element data by invoking the
+	 * firmware.
+	 *
+	 * [in] tfp
+	 *   Pointer to TF handle
+	 *
+	 * [in] parms
+	 *   Pointer to table type set parameters
+	 *
+	 * Returns
+	 *   - (0) if successful.
+	 *   - (-EINVAL) on failure.
+	 */
+	int (*tf_dev_set_tbl_type)(struct tf *tfp,
+				   struct tf_tbl_type_set_parms *parms);
+
+	/**
+	 * Retrieves the specified table type element.
+	 *
+	 * This API retrieves the specified element data by invoking the
+	 * firmware.
+	 *
+	 * [in] tfp
+	 *   Pointer to TF handle
+	 *
+	 * [in] parms
+	 *   Pointer to table type get parameters
+	 *
+	 * Returns
+	 *   - (0) if successful.
+	 *   - (-EINVAL) on failure.
+	 */
+	int (*tf_dev_get_tbl_type)(struct tf *tfp,
+				   struct tf_tbl_type_get_parms *parms);
+
+	/**
+	 * Allocation of a tcam element.
+	 *
+	 * This API allocates the specified tcam element from a device
+	 * specific tcam DB. The allocated element is returned.
+	 *
+	 * [in] tfp
+	 *   Pointer to TF handle
+	 *
+	 * [in] parms
+	 *   Pointer to tcam allocation parameters
+	 *
+	 * Returns
+	 *   - (0) if successful.
+	 *   - (-EINVAL) on failure.
+	 */
+	int (*tf_dev_alloc_tcam)(struct tf *tfp,
+				 struct tf_tcam_alloc_parms *parms);
+
+	/**
+	 * Free of a tcam element.
+	 *
+	 * This API free's a previous allocated tcam element from a
+	 * device specific tcam DB.
+	 *
+	 * [in] tfp
+	 *   Pointer to TF handle
+	 *
+	 * [in] parms
+	 *   Pointer to tcam free parameters
+	 *
+	 * Returns
+	 *   - (0) if successful.
+	 *   - (-EINVAL) on failure.
+	 */
+	int (*tf_dev_free_tcam)(struct tf *tfp,
+				struct tf_tcam_free_parms *parms);
+
+	/**
+	 * Searches for the specified tcam element in a shadow DB.
+	 *
+	 * This API searches for the specified tcam element in a
+	 * device specific shadow DB. If the element is found the
+	 * reference count for the element is updated. If the element
+	 * is not found a new element is allocated from the tcam DB
+	 * and then inserted into the shadow DB.
+	 *
+	 * [in] tfp
+	 *   Pointer to TF handle
+	 *
+	 * [in] parms
+	 *   Pointer to tcam allocation and search parameters
+	 *
+	 * Returns
+	 *   - (0) if successful.
+	 *   - (-EINVAL) on failure.
+	 */
+	int (*tf_dev_alloc_search_tcam)
+			(struct tf *tfp,
+			struct tf_tcam_alloc_search_parms *parms);
+
+	/**
+	 * Sets the specified tcam element.
+	 *
+	 * This API sets the specified element data by invoking the
+	 * firmware.
+	 *
+	 * [in] tfp
+	 *   Pointer to TF handle
+	 *
+	 * [in] parms
+	 *   Pointer to tcam set parameters
+	 *
+	 * Returns
+	 *   - (0) if successful.
+	 *   - (-EINVAL) on failure.
+	 */
+	int (*tf_dev_set_tcam)(struct tf *tfp,
+			       struct tf_tcam_set_parms *parms);
+
+	/**
+	 * Retrieves the specified tcam element.
+	 *
+	 * This API retrieves the specified element data by invoking the
+	 * firmware.
+	 *
+	 * [in] tfp
+	 *   Pointer to TF handle
+	 *
+	 * [in] parms
+	 *   Pointer to tcam get parameters
+	 *
+	 * Returns
+	 *   - (0) if successful.
+	 *   - (-EINVAL) on failure.
+	 */
+	int (*tf_dev_get_tcam)(struct tf *tfp,
+			       struct tf_tcam_get_parms *parms);
+};
+
+/**
+ * Supported device operation structures
+ */
+extern const struct tf_dev_ops tf_dev_ops_p4;
+
+#endif /* _TF_DEVICE_H_ */
diff --git a/drivers/net/bnxt/tf_core/tf_device_p4.c b/drivers/net/bnxt/tf_core/tf_device_p4.c
new file mode 100644
index 0000000..c3c4d1e
--- /dev/null
+++ b/drivers/net/bnxt/tf_core/tf_device_p4.c
@@ -0,0 +1,24 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2019-2020 Broadcom
+ * All rights reserved.
+ */
+
+#include "tf_device.h"
+#include "tf_identifier.h"
+#include "tf_tbl_type.h"
+#include "tf_tcam.h"
+
+const struct tf_dev_ops tf_dev_ops_p4 = {
+	.tf_dev_alloc_ident = tf_ident_alloc,
+	.tf_dev_free_ident = tf_ident_free,
+	.tf_dev_alloc_tbl_type = tf_tbl_type_alloc,
+	.tf_dev_free_tbl_type = tf_tbl_type_free,
+	.tf_dev_alloc_search_tbl_type = tf_tbl_type_alloc_search,
+	.tf_dev_set_tbl_type = tf_tbl_type_set,
+	.tf_dev_get_tbl_type = tf_tbl_type_get,
+	.tf_dev_alloc_tcam = tf_tcam_alloc,
+	.tf_dev_free_tcam = tf_tcam_free,
+	.tf_dev_alloc_search_tcam = tf_tcam_alloc_search,
+	.tf_dev_set_tcam = tf_tcam_set,
+	.tf_dev_get_tcam = tf_tcam_get,
+};
diff --git a/drivers/net/bnxt/tf_core/tf_device_p4.h b/drivers/net/bnxt/tf_core/tf_device_p4.h
new file mode 100644
index 0000000..84d90e3
--- /dev/null
+++ b/drivers/net/bnxt/tf_core/tf_device_p4.h
@@ -0,0 +1,64 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2019-2020 Broadcom
+ * All rights reserved.
+ */
+
+#ifndef _TF_DEVICE_P4_H_
+#define _TF_DEVICE_P4_H_
+
+#include <cfa_resource_types.h>
+
+#include "tf_core.h"
+#include "tf_rm_new.h"
+
+struct tf_rm_element_cfg tf_ident_p4[TF_IDENT_TYPE_MAX] = {
+	{ TF_RM_ELEM_CFG_PRIVATE, 0 /* CFA_RESOURCE_TYPE_P4_INVALID */ },
+	{ TF_RM_ELEM_CFG_HCAPI, CFA_RESOURCE_TYPE_P4_PROF_FUNC },
+	{ TF_RM_ELEM_CFG_HCAPI, CFA_RESOURCE_TYPE_P4_WC_TCAM_PROF_ID },
+	{ TF_RM_ELEM_CFG_HCAPI, CFA_RESOURCE_TYPE_P4_EM_PROF_ID },
+	{ TF_RM_ELEM_CFG_NULL, 0    /* CFA_RESOURCE_TYPE_P4_L2_FUNC */ }
+};
+
+struct tf_rm_element_cfg tf_tcam_p4[TF_TCAM_TBL_TYPE_MAX] = {
+	{ TF_RM_ELEM_CFG_HCAPI, CFA_RESOURCE_TYPE_P4_L2_CTXT_TCAM },
+	{ TF_RM_ELEM_CFG_HCAPI, CFA_RESOURCE_TYPE_P4_PROF_TCAM },
+	{ TF_RM_ELEM_CFG_HCAPI, CFA_RESOURCE_TYPE_P4_WC_TCAM },
+	{ TF_RM_ELEM_CFG_HCAPI, CFA_RESOURCE_TYPE_P4_SP_TCAM },
+	{ TF_RM_ELEM_CFG_NULL, 0 /* CFA_RESOURCE_TYPE_P4_CT_RULE_TCAM */ },
+	{ TF_RM_ELEM_CFG_NULL, 0  /* CFA_RESOURCE_TYPE_P4_VEB_TCAM */ }
+};
+
+struct tf_rm_element_cfg tf_tbl_p4[TF_TBL_TYPE_MAX] = {
+	{ TF_RM_ELEM_CFG_HCAPI, CFA_RESOURCE_TYPE_P4_SRAM_FULL_ACTION },
+	{ TF_RM_ELEM_CFG_HCAPI, CFA_RESOURCE_TYPE_P4_SRAM_MCG },
+	{ TF_RM_ELEM_CFG_HCAPI, CFA_RESOURCE_TYPE_P4_SRAM_ENCAP_8B },
+	{ TF_RM_ELEM_CFG_HCAPI, CFA_RESOURCE_TYPE_P4_SRAM_ENCAP_16B },
+	{ TF_RM_ELEM_CFG_NULL, 0, /* CFA_RESOURCE_TYPE_P4_SRAM_ENCAP_32B */ },
+	{ TF_RM_ELEM_CFG_HCAPI, CFA_RESOURCE_TYPE_P4_SRAM_ENCAP_64B },
+	{ TF_RM_ELEM_CFG_HCAPI, CFA_RESOURCE_TYPE_P4_SRAM_SP_MAC },
+	{ TF_RM_ELEM_CFG_NULL, 0 /* CFA_RESOURCE_TYPE_P4_SRAM_SP_SMAC_IPV4 */ },
+	{ TF_RM_ELEM_CFG_NULL, 0 /* CFA_RESOURCE_TYPE_P4_SRAM_SP_SMAC_IPV6 */ },
+	{ TF_RM_ELEM_CFG_HCAPI, CFA_RESOURCE_TYPE_P4_SRAM_COUNTER_64B },
+	{ TF_RM_ELEM_CFG_HCAPI, CFA_RESOURCE_TYPE_P4_SRAM_NAT_SPORT },
+	{ TF_RM_ELEM_CFG_HCAPI, CFA_RESOURCE_TYPE_P4_SRAM_NAT_DPORT },
+	{ TF_RM_ELEM_CFG_HCAPI, CFA_RESOURCE_TYPE_P4_SRAM_NAT_S_IPV4 },
+	{ TF_RM_ELEM_CFG_HCAPI, CFA_RESOURCE_TYPE_P4_SRAM_NAT_D_IPV4 },
+	{ TF_RM_ELEM_CFG_HCAPI, CFA_RESOURCE_TYPE_P4_SRAM_NAT_S_IPV6 },
+	{ TF_RM_ELEM_CFG_HCAPI, CFA_RESOURCE_TYPE_P4_SRAM_NAT_D_IPV6 },
+	{ TF_RM_ELEM_CFG_HCAPI, CFA_RESOURCE_TYPE_P4_METER_PROF },
+	{ TF_RM_ELEM_CFG_HCAPI, CFA_RESOURCE_TYPE_P4_METER },
+	{ TF_RM_ELEM_CFG_HCAPI, CFA_RESOURCE_TYPE_P4_MIRROR },
+	{ TF_RM_ELEM_CFG_NULL, /* CFA_RESOURCE_TYPE_P4_UPAR */ },
+	{ TF_RM_ELEM_CFG_NULL, /* CFA_RESOURCE_TYPE_P4_EPOC */ },
+	{ TF_RM_ELEM_CFG_NULL, /* CFA_RESOURCE_TYPE_P4_METADATA */ },
+	{ TF_RM_ELEM_CFG_NULL, /* CFA_RESOURCE_TYPE_P4_CT_STATE */ },
+	{ TF_RM_ELEM_CFG_NULL, /* CFA_RESOURCE_TYPE_P4_RANGE_PROF */ },
+	{ TF_RM_ELEM_CFG_NULL, /* CFA_RESOURCE_TYPE_P4_RANGE_ENTRY */ },
+	{ TF_RM_ELEM_CFG_NULL, /* CFA_RESOURCE_TYPE_P4_LAG */ },
+	{ TF_RM_ELEM_CFG_NULL, /* CFA_RESOURCE_TYPE_P4_VNIC_SVIF */ },
+	{ TF_RM_ELEM_CFG_NULL, /* CFA_RESOURCE_TYPE_P4_EM_FBK */ },
+	{ TF_RM_ELEM_CFG_NULL, /* CFA_RESOURCE_TYPE_P4_WC_FKB */ },
+	{ TF_RM_ELEM_CFG_NULL, /* CFA_RESOURCE_TYPE_P4_EXT */ }
+};
+
+#endif /* _TF_DEVICE_P4_H_ */
diff --git a/drivers/net/bnxt/tf_core/tf_identifier.c b/drivers/net/bnxt/tf_core/tf_identifier.c
new file mode 100644
index 0000000..726d0b4
--- /dev/null
+++ b/drivers/net/bnxt/tf_core/tf_identifier.c
@@ -0,0 +1,47 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2019-2020 Broadcom
+ * All rights reserved.
+ */
+
+#include <rte_common.h>
+
+#include "tf_identifier.h"
+
+struct tf;
+
+/**
+ * Identifier DBs.
+ */
+/* static void *ident_db[TF_DIR_MAX]; */
+
+/**
+ * Init flag, set on bind and cleared on unbind
+ */
+/* static uint8_t init; */
+
+int
+tf_ident_bind(struct tf *tfp __rte_unused,
+	      struct tf_ident_cfg *parms __rte_unused)
+{
+	return 0;
+}
+
+int
+tf_ident_unbind(struct tf *tfp __rte_unused)
+{
+	return 0;
+}
+
+int
+tf_ident_alloc(struct tf *tfp __rte_unused,
+	       struct tf_ident_alloc_parms *parms __rte_unused)
+{
+	return 0;
+}
+
+int
+tf_ident_free(struct tf *tfp __rte_unused,
+	      struct tf_ident_free_parms *parms __rte_unused)
+{
+	return 0;
+}
diff --git a/drivers/net/bnxt/tf_core/tf_identifier.h b/drivers/net/bnxt/tf_core/tf_identifier.h
new file mode 100644
index 0000000..b77c91b
--- /dev/null
+++ b/drivers/net/bnxt/tf_core/tf_identifier.h
@@ -0,0 +1,140 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2019-2020 Broadcom
+ * All rights reserved.
+ */
+
+#ifndef _TF_IDENTIFIER_H_
+#define _TF_IDENTIFIER_H_
+
+#include "tf_core.h"
+
+/**
+ * The Identifier module provides processing of Identifiers.
+ */
+
+struct tf_ident_cfg {
+	/**
+	 * Number of identifier types in each of the configuration
+	 * arrays
+	 */
+	uint16_t num_elements;
+
+	/**
+	 * TCAM configuration array
+	 */
+	struct tf_rm_element_cfg *ident_cfg[TF_DIR_MAX];
+};
+
+/**
+ * Identifier allcoation parameter definition
+ */
+struct tf_ident_alloc_parms {
+	/**
+	 * [in] receive or transmit direction
+	 */
+	enum tf_dir dir;
+	/**
+	 * [in] Identifier type
+	 */
+	enum tf_identifier_type ident_type;
+	/**
+	 * [out] Identifier allocated
+	 */
+	uint16_t id;
+};
+
+/**
+ * Identifier free parameter definition
+ */
+struct tf_ident_free_parms {
+	/**
+	 * [in]	 receive or transmit direction
+	 */
+	enum tf_dir dir;
+	/**
+	 * [in] Identifier type
+	 */
+	enum tf_identifier_type ident_type;
+	/**
+	 * [in] ID to free
+	 */
+	uint16_t id;
+};
+
+/**
+ * @page ident Identity Management
+ *
+ * @ref tf_ident_bind
+ *
+ * @ref tf_ident_unbind
+ *
+ * @ref tf_ident_alloc
+ *
+ * @ref tf_ident_free
+ */
+
+/**
+ * Initializes the Identifier module with the requested DBs. Must be
+ * invoked as the first thing before any of the access functions.
+ *
+ * [in] tfp
+ *   Pointer to TF handle, used for HCAPI communication
+ *
+ * [in] parms
+ *   Pointer to parameters
+ *
+ * Returns
+ *   - (0) if successful.
+ *   - (-EINVAL) on failure.
+ */
+int tf_ident_bind(struct tf *tfp,
+		  struct tf_ident_cfg *parms);
+
+/**
+ * Cleans up the private DBs and releases all the data.
+ *
+ * [in] tfp
+ *   Pointer to TF handle, used for HCAPI communication
+ *
+ * [in] parms
+ *   Pointer to parameters
+ *
+ * Returns
+ *   - (0) if successful.
+ *   - (-EINVAL) on failure.
+ */
+int tf_ident_unbind(struct tf *tfp);
+
+/**
+ * Allocates a single identifier type.
+ *
+ * [in] tfp
+ *   Pointer to TF handle, used for HCAPI communication
+ *
+ * [in] parms
+ *   Pointer to parameters
+ *
+ * Returns
+ *   - (0) if successful.
+ *   - (-EINVAL) on failure.
+ */
+int tf_ident_alloc(struct tf *tfp,
+		   struct tf_ident_alloc_parms *parms);
+
+/**
+ * Free's a single identifier type.
+ *
+ * [in] tfp
+ *   Pointer to TF handle, used for HCAPI communication
+ *
+ * [in] parms
+ *   Pointer to parameters
+ *
+ * Returns
+ *   - (0) if successful.
+ *   - (-EINVAL) on failure.
+ */
+int tf_ident_free(struct tf *tfp,
+		  struct tf_ident_free_parms *parms);
+
+#endif /* _TF_IDENTIFIER_H_ */
diff --git a/drivers/net/bnxt/tf_core/tf_rm.c b/drivers/net/bnxt/tf_core/tf_rm.c
index 38b1e71..2264704 100644
--- a/drivers/net/bnxt/tf_core/tf_rm.c
+++ b/drivers/net/bnxt/tf_core/tf_rm.c
@@ -9,6 +9,7 @@
 
 #include "tf_rm.h"
 #include "tf_core.h"
+#include "tf_util.h"
 #include "tf_session.h"
 #include "tf_resources.h"
 #include "tf_msg.h"
@@ -77,59 +78,6 @@
 	} while (0)
 
 const char
-*tf_dir_2_str(enum tf_dir dir)
-{
-	switch (dir) {
-	case TF_DIR_RX:
-		return "RX";
-	case TF_DIR_TX:
-		return "TX";
-	default:
-		return "Invalid direction";
-	}
-}
-
-const char
-*tf_ident_2_str(enum tf_identifier_type id_type)
-{
-	switch (id_type) {
-	case TF_IDENT_TYPE_L2_CTXT:
-		return "l2_ctxt_remap";
-	case TF_IDENT_TYPE_PROF_FUNC:
-		return "prof_func";
-	case TF_IDENT_TYPE_WC_PROF:
-		return "wc_prof";
-	case TF_IDENT_TYPE_EM_PROF:
-		return "em_prof";
-	case TF_IDENT_TYPE_L2_FUNC:
-		return "l2_func";
-	default:
-		return "Invalid identifier";
-	}
-}
-
-const char
-*tf_tcam_tbl_2_str(enum tf_tcam_tbl_type tcam_type)
-{
-	switch (tcam_type) {
-	case TF_TCAM_TBL_TYPE_L2_CTXT_TCAM:
-		return "l2_ctxt_tcam";
-	case TF_TCAM_TBL_TYPE_PROF_TCAM:
-		return "prof_tcam";
-	case TF_TCAM_TBL_TYPE_WC_TCAM:
-		return "wc_tcam";
-	case TF_TCAM_TBL_TYPE_VEB_TCAM:
-		return "veb_tcam";
-	case TF_TCAM_TBL_TYPE_SP_TCAM:
-		return "sp_tcam";
-	case TF_TCAM_TBL_TYPE_CT_RULE_TCAM:
-		return "ct_rule_tcam";
-	default:
-		return "Invalid tcam table type";
-	}
-}
-
-const char
 *tf_hcapi_hw_2_str(enum tf_resource_type_hw hw_type)
 {
 	switch (hw_type) {
diff --git a/drivers/net/bnxt/tf_core/tf_rm.h b/drivers/net/bnxt/tf_core/tf_rm.h
index e69d443..1a09f13 100644
--- a/drivers/net/bnxt/tf_core/tf_rm.h
+++ b/drivers/net/bnxt/tf_core/tf_rm.h
@@ -125,24 +125,6 @@ struct tf_rm_db {
 };
 
 /**
- * Helper function converting direction to text string
- */
-const char
-*tf_dir_2_str(enum tf_dir dir);
-
-/**
- * Helper function converting identifier to text string
- */
-const char
-*tf_ident_2_str(enum tf_identifier_type id_type);
-
-/**
- * Helper function converting tcam type to text string
- */
-const char
-*tf_tcam_tbl_2_str(enum tf_tcam_tbl_type tcam_type);
-
-/**
  * Helper function used to convert HW HCAPI resource type to a string.
  */
 const char
diff --git a/drivers/net/bnxt/tf_core/tf_rm_new.c b/drivers/net/bnxt/tf_core/tf_rm_new.c
new file mode 100644
index 0000000..51bb9ba
--- /dev/null
+++ b/drivers/net/bnxt/tf_core/tf_rm_new.c
@@ -0,0 +1,102 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2019-2020 Broadcom
+ * All rights reserved.
+ */
+
+#include <rte_common.h>
+
+#include "tf_rm_new.h"
+
+/**
+ * Resource query single entry. Used when accessing HCAPI RM on the
+ * firmware.
+ */
+struct tf_rm_query_entry {
+	/** Minimum guaranteed number of elements */
+	uint16_t min;
+	/** Maximum non-guaranteed number of elements */
+	uint16_t max;
+};
+
+/**
+ * Generic RM Element data type that an RM DB is build upon.
+ */
+struct tf_rm_element {
+	/**
+	 * RM Element configuration type. If Private then the
+	 * hcapi_type can be ignored. If Null then the element is not
+	 * valid for the device.
+	 */
+	enum tf_rm_elem_cfg_type type;
+
+	/**
+	 * HCAPI RM Type for the element.
+	 */
+	uint16_t hcapi_type;
+
+	/**
+	 * HCAPI RM allocated range information for the element.
+	 */
+	struct tf_rm_alloc_info alloc;
+
+	/**
+	 * Bit allocator pool for the element. Pool size is controlled
+	 * by the struct tf_session_resources at time of session creation.
+	 * Null indicates that the element is not used for the device.
+	 */
+	struct bitalloc *pool;
+};
+
+/**
+ * TF RM DB definition
+ */
+struct tf_rm_db {
+	/**
+	 * The DB consists of an array of elements
+	 */
+	struct tf_rm_element *db;
+};
+
+int
+tf_rm_create_db(struct tf *tfp __rte_unused,
+		struct tf_rm_create_db_parms *parms __rte_unused)
+{
+	return 0;
+}
+
+int
+tf_rm_free_db(struct tf *tfp __rte_unused,
+	      struct tf_rm_free_db_parms *parms __rte_unused)
+{
+	return 0;
+}
+
+int
+tf_rm_allocate(struct tf_rm_allocate_parms *parms __rte_unused)
+{
+	return 0;
+}
+
+int
+tf_rm_free(struct tf_rm_free_parms *parms __rte_unused)
+{
+	return 0;
+}
+
+int
+tf_rm_is_allocated(struct tf_rm_is_allocated_parms *parms __rte_unused)
+{
+	return 0;
+}
+
+int
+tf_rm_get_info(struct tf_rm_get_alloc_info_parms *parms __rte_unused)
+{
+	return 0;
+}
+
+int
+tf_rm_get_hcapi_type(struct tf_rm_get_hcapi_parms *parms __rte_unused)
+{
+	return 0;
+}
diff --git a/drivers/net/bnxt/tf_core/tf_rm_new.h b/drivers/net/bnxt/tf_core/tf_rm_new.h
new file mode 100644
index 0000000..72dba09
--- /dev/null
+++ b/drivers/net/bnxt/tf_core/tf_rm_new.h
@@ -0,0 +1,368 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2019-2020 Broadcom
+ * All rights reserved.
+ */
+
+#ifndef TF_RM_H_
+#define TF_RM_H_
+
+#include "tf_core.h"
+#include "bitalloc.h"
+
+struct tf;
+
+/**
+ * The Resource Manager (RM) module provides basic DB handling for
+ * internal resources. These resources exists within the actual device
+ * and are controlled by the HCAPI Resource Manager running on the
+ * firmware.
+ *
+ * The RM DBs are all intended to be indexed using TF types there for
+ * a lookup requires no additional conversion. The DB configuration
+ * specifies the TF Type to HCAPI Type mapping and it becomes the
+ * responsibility of the DB initialization to handle this static
+ * mapping.
+ *
+ * Accessor functions are providing access to the DB, thus hiding the
+ * implementation.
+ *
+ * The RM DB will work on its initial allocated sizes so the
+ * capability of dynamically growing a particular resource is not
+ * possible. If this capability later becomes a requirement then the
+ * MAX pool size of the Chip œneeds to be added to the tf_rm_elem_info
+ * structure and several new APIs would need to be added to allow for
+ * growth of a single TF resource type.
+ */
+
+/**
+ * Resource reservation single entry result. Used when accessing HCAPI
+ * RM on the firmware.
+ */
+struct tf_rm_entry {
+	/** Starting index of the allocated resource */
+	uint16_t start;
+	/** Number of allocated elements */
+	uint16_t stride;
+};
+
+/**
+ * RM Element configuration enumeration. Used by the Device to
+ * indicate how the RM elements the DB consists off, are to be
+ * configured at time of DB creation. The TF may present types to the
+ * ULP layer that is not controlled by HCAPI within the Firmware.
+ */
+enum tf_rm_elem_cfg_type {
+	TF_RM_ELEM_CFG_NULL,    /**< No configuration */
+	TF_RM_ELEM_CFG_HCAPI,   /**< HCAPI 'controlled' */
+	TF_RM_ELEM_CFG_PRIVATE, /**< Private thus not HCAPI 'controlled' */
+	TF_RM_TYPE_MAX
+};
+
+/**
+ * RM Element configuration structure, used by the Device to configure
+ * how an individual TF type is configured in regard to the HCAPI RM
+ * of same type.
+ */
+struct tf_rm_element_cfg {
+	/**
+	 * RM Element config controls how the DB for that element is
+	 * processed.
+	 */
+	enum tf_rm_elem_cfg_type cfg;
+
+	/* If a HCAPI to TF type conversion is required then TF type
+	 * can be added here.
+	 */
+
+	/**
+	 * HCAPI RM Type for the element. Used for TF to HCAPI type
+	 * conversion.
+	 */
+	uint16_t hcapi_type;
+};
+
+/**
+ * Allocation information for a single element.
+ */
+struct tf_rm_alloc_info {
+	/**
+	 * HCAPI RM allocated range information.
+	 *
+	 * NOTE:
+	 * In case of dynamic allocation support this would have
+	 * to be changed to linked list of tf_rm_entry instead.
+	 */
+	struct tf_rm_entry entry;
+};
+
+/**
+ * Create RM DB parameters
+ */
+struct tf_rm_create_db_parms {
+	/**
+	 * [in] Receive or transmit direction
+	 */
+	enum tf_dir dir;
+	/**
+	 * [in] Number of elements in the parameter structure
+	 */
+	uint16_t num_elements;
+	/**
+	 * [in] Parameter structure
+	 */
+	struct tf_rm_element_cfg *parms;
+	/**
+	 * [out] RM DB Handle
+	 */
+	void *tf_rm_db;
+};
+
+/**
+ * Free RM DB parameters
+ */
+struct tf_rm_free_db_parms {
+	/**
+	 * [in] Receive or transmit direction
+	 */
+	enum tf_dir dir;
+	/**
+	 * [in] RM DB Handle
+	 */
+	void *tf_rm_db;
+};
+
+/**
+ * Allocate RM parameters for a single element
+ */
+struct tf_rm_allocate_parms {
+	/**
+	 * [in] RM DB Handle
+	 */
+	void *tf_rm_db;
+	/**
+	 * [in] DB Index, indicates which DB entry to perform the
+	 * action on.
+	 */
+	uint16_t db_index;
+	/**
+	 * [in] Pointer to the allocated index in normalized
+	 * form. Normalized means the index has been adjusted,
+	 * i.e. Full Action Record offsets.
+	 */
+	uint32_t *index;
+};
+
+/**
+ * Free RM parameters for a single element
+ */
+struct tf_rm_free_parms {
+	/**
+	 * [in] RM DB Handle
+	 */
+	void *tf_rm_db;
+	/**
+	 * [in] DB Index, indicates which DB entry to perform the
+	 * action on.
+	 */
+	uint16_t db_index;
+	/**
+	 * [in] Index to free
+	 */
+	uint32_t index;
+};
+
+/**
+ * Is Allocated parameters for a single element
+ */
+struct tf_rm_is_allocated_parms {
+	/**
+	 * [in] RM DB Handle
+	 */
+	void *tf_rm_db;
+	/**
+	 * [in] DB Index, indicates which DB entry to perform the
+	 * action on.
+	 */
+	uint16_t db_index;
+	/**
+	 * [in] Index to free
+	 */
+	uint32_t index;
+	/**
+	 * [in] Pointer to flag that indicates the state of the query
+	 */
+	uint8_t *allocated;
+};
+
+/**
+ * Get Allocation information for a single element
+ */
+struct tf_rm_get_alloc_info_parms {
+	/**
+	 * [in] RM DB Handle
+	 */
+	void *tf_rm_db;
+	/**
+	 * [in] DB Index, indicates which DB entry to perform the
+	 * action on.
+	 */
+	uint16_t db_index;
+	/**
+	 * [out] Pointer to the requested allocation information for
+	 * the specified db_index
+	 */
+	struct tf_rm_alloc_info *info;
+};
+
+/**
+ * Get HCAPI type parameters for a single element
+ */
+struct tf_rm_get_hcapi_parms {
+	/**
+	 * [in] RM DB Handle
+	 */
+	void *tf_rm_db;
+	/**
+	 * [in] DB Index, indicates which DB entry to perform the
+	 * action on.
+	 */
+	uint16_t db_index;
+	/**
+	 * [out] Pointer to the hcapi type for the specified db_index
+	 */
+	uint16_t *hcapi_type;
+};
+
+/**
+ * @page rm Resource Manager
+ *
+ * @ref tf_rm_create_db
+ *
+ * @ref tf_rm_free_db
+ *
+ * @ref tf_rm_allocate
+ *
+ * @ref tf_rm_free
+ *
+ * @ref tf_rm_is_allocated
+ *
+ * @ref tf_rm_get_info
+ *
+ * @ref tf_rm_get_hcapi_type
+ */
+
+/**
+ * Creates and fills a Resource Manager (RM) DB with requested
+ * elements. The DB is indexed per the parms structure.
+ *
+ * [in] tfp
+ *   Pointer to TF handle, used for HCAPI communication
+ *
+ * [in] parms
+ *   Pointer to create parameters
+ *
+ * Returns
+ *   - (0) if successful.
+ *   - (-EINVAL) on failure.
+ */
+/*
+ * NOTE:
+ * - Fail on parameter check
+ * - Fail on DB creation, i.e. alloc amount is not possible or validation fails
+ * - Fail on DB creation if DB already exist
+ *
+ * - Allocs local DB
+ * - Does hcapi qcaps
+ * - Does hcapi reservation
+ * - Populates the pool with allocated elements
+ * - Returns handle to the created DB
+ */
+int tf_rm_create_db(struct tf *tfp,
+		    struct tf_rm_create_db_parms *parms);
+
+/**
+ * Closes the Resource Manager (RM) DB and frees all allocated
+ * resources per the associated database.
+ *
+ * [in] tfp
+ *   Pointer to TF handle, used for HCAPI communication
+ *
+ * [in] parms
+ *   Pointer to free parameters
+ *
+ * Returns
+ *   - (0) if successful.
+ *   - (-EINVAL) on failure.
+ */
+int tf_rm_free_db(struct tf *tfp,
+		  struct tf_rm_free_db_parms *parms);
+
+/**
+ * Allocates a single element for the type specified, within the DB.
+ *
+ * [in] parms
+ *   Pointer to allocate parameters
+ *
+ * Returns
+ *   - (0) if successful.
+ *   - (-EINVAL) on failure.
+ */
+int tf_rm_allocate(struct tf_rm_allocate_parms *parms);
+
+/**
+ * Free's a single element for the type specified, within the DB.
+ *
+ * [in] parms
+ *   Pointer to free parameters
+ *
+ * Returns
+ *   - (0) if successful.
+ *   - (-EpINVAL) on failure.
+ */
+int tf_rm_free(struct tf_rm_free_parms *parms);
+
+/**
+ * Performs an allocation verification check on a specified element.
+ *
+ * [in] parms
+ *   Pointer to is allocated parameters
+ *
+ * Returns
+ *   - (0) if successful.
+ *   - (-EINVAL) on failure.
+ */
+/*
+ * NOTE:
+ *  - If pool is set to Chip MAX, then the query index must be checked
+ *    against the allocated range and query index must be allocated as well.
+ *  - If pool is allocated size only, then check if query index is allocated.
+ */
+int tf_rm_is_allocated(struct tf_rm_is_allocated_parms *parms);
+
+/**
+ * Retrieves an elements allocation information from the Resource
+ * Manager (RM) DB.
+ *
+ * [in] parms
+ *   Pointer to get info parameters
+ *
+ * Returns
+ *   - (0) if successful.
+ *   - (-EINVAL) on failure.
+ */
+int tf_rm_get_info(struct tf_rm_get_alloc_info_parms *parms);
+
+/**
+ * Performs a lookup in the Resource Manager DB and retrives the
+ * requested HCAPI type.
+ *
+ * [in] parms
+ *   Pointer to get hcapi parameters
+ *
+ * Returns
+ *   - (0) if successful.
+ *   - (-EINVAL) on failure.
+ */
+int tf_rm_get_hcapi_type(struct tf_rm_get_hcapi_parms *parms);
+
+#endif /* TF_RM_H_ */
diff --git a/drivers/net/bnxt/tf_core/tf_session.c b/drivers/net/bnxt/tf_core/tf_session.c
new file mode 100644
index 0000000..c749945
--- /dev/null
+++ b/drivers/net/bnxt/tf_core/tf_session.c
@@ -0,0 +1,31 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2019-2020 Broadcom
+ * All rights reserved.
+ */
+
+int
+tf_session_get_session(struct tf *tfp,
+		       struct tf_session *tfs)
+{
+	if (tfp->session == NULL || tfp->session->core_data == NULL) {
+		TFP_DRV_LOG(ERR, "Session not created\n");
+		return -EINVAL;
+	}
+
+	tfs = (struct tf_session *)(tfp->session->core_data);
+
+	return 0;
+}
+
+int
+tf_session_get_device(struct tf_session *tfs,
+		      struct tf_device *tfd)
+{
+	if (tfs->dev == NULL) {
+		TFP_DRV_LOG(ERR, "Device not created\n");
+		return -EINVAL;
+	}
+	tfd = tfs->dev;
+
+	return 0;
+}
diff --git a/drivers/net/bnxt/tf_core/tf_session.h b/drivers/net/bnxt/tf_core/tf_session.h
index c9f4f8f..b1cc7a4 100644
--- a/drivers/net/bnxt/tf_core/tf_session.h
+++ b/drivers/net/bnxt/tf_core/tf_session.h
@@ -11,10 +11,21 @@
 
 #include "bitalloc.h"
 #include "tf_core.h"
+#include "tf_device.h"
 #include "tf_rm.h"
 #include "tf_tbl.h"
 #include "stack.h"
 
+/**
+ * The Session module provides session control support. A session is
+ * to the ULP layer known as a session_info instance. The session
+ * private data is the actual session.
+ *
+ * Session manages:
+ *   - The device and all the resources related to the device.
+ *   - Any session sharing between ULP applications
+ */
+
 /** Session defines
  */
 #define TF_SESSIONS_MAX	          1          /** max # sessions */
@@ -90,6 +101,9 @@ struct tf_session {
 	 */
 	uint8_t ref_count;
 
+	/** Device */
+	struct tf_dev_info *dev;
+
 	/** Session HW and SRAM resources */
 	struct tf_rm_db resc;
 
@@ -309,4 +323,44 @@ struct tf_session {
 	struct stack em_pool[TF_DIR_MAX];
 };
 
+/**
+ * @page session Session Management
+ *
+ * @ref tf_session_get_session
+ *
+ * @ref tf_session_get_device
+ */
+
+/**
+ * Looks up the private session information from the TF session info.
+ *
+ * [in] tfp
+ *   Pointer to TF handle
+ *
+ * [out] tfs
+ *   Pointer to the session
+ *
+ * Returns
+ *   - (0) if successful.
+ *   - (-EINVAL) on failure.
+ */
+int tf_session_get_session(struct tf *tfp,
+			   struct tf_session *tfs);
+
+/**
+ * Looks up the device information from the TF Session.
+ *
+ * [in] tfp
+ *   Pointer to TF handle
+ *
+ * [out] tfd
+ *   Pointer to the device
+ *
+ * Returns
+ *   - (0) if successful.
+ *   - (-EINVAL) on failure.
+ */
+int tf_session_get_device(struct tf_session *tfs,
+			  struct tf_dev_info *tfd);
+
 #endif /* _TF_SESSION_H_ */
diff --git a/drivers/net/bnxt/tf_core/tf_shadow_tbl.c b/drivers/net/bnxt/tf_core/tf_shadow_tbl.c
new file mode 100644
index 0000000..8f2b6de
--- /dev/null
+++ b/drivers/net/bnxt/tf_core/tf_shadow_tbl.c
@@ -0,0 +1,63 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2019-2020 Broadcom
+ * All rights reserved.
+ */
+
+#include <rte_common.h>
+
+#include "tf_shadow_tbl.h"
+
+/**
+ * Shadow table DB element
+ */
+struct tf_shadow_tbl_element {
+	/**
+	 * Hash table
+	 */
+	void *hash;
+
+	/**
+	 * Reference count, array of number of table type entries
+	 */
+	uint16_t *ref_count;
+};
+
+/**
+ * Shadow table DB definition
+ */
+struct tf_shadow_tbl_db {
+	/**
+	 * The DB consists of an array of elements
+	 */
+	struct tf_shadow_tbl_element *db;
+};
+
+int
+tf_shadow_tbl_create_db(struct tf_shadow_tbl_create_db_parms *parms __rte_unused)
+{
+	return 0;
+}
+
+int
+tf_shadow_tbl_free_db(struct tf_shadow_tbl_free_db_parms *parms __rte_unused)
+{
+	return 0;
+}
+
+int
+tf_shadow_tbl_search(struct tf_shadow_tbl_search_parms *parms __rte_unused)
+{
+	return 0;
+}
+
+int
+tf_shadow_tbl_insert(struct tf_shadow_tbl_insert_parms *parms __rte_unused)
+{
+	return 0;
+}
+
+int
+tf_shadow_tbl_remove(struct tf_shadow_tbl_remove_parms *parms __rte_unused)
+{
+	return 0;
+}
diff --git a/drivers/net/bnxt/tf_core/tf_shadow_tbl.h b/drivers/net/bnxt/tf_core/tf_shadow_tbl.h
new file mode 100644
index 0000000..dfd336e
--- /dev/null
+++ b/drivers/net/bnxt/tf_core/tf_shadow_tbl.h
@@ -0,0 +1,240 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2019-2020 Broadcom
+ * All rights reserved.
+ */
+
+#ifndef _TF_SHADOW_TBL_H_
+#define _TF_SHADOW_TBL_H_
+
+#include "tf_core.h"
+
+struct tf;
+
+/**
+ * The Shadow Table module provides shadow DB handling for table based
+ * TF types. A shadow DB provides the capability that allows for reuse
+ * of TF resources.
+ *
+ * A Shadow table DB is intended to be used by the Table Type module
+ * only.
+ */
+
+/**
+ * Shadow DB configuration information for a single table type.
+ *
+ * During Device initialization the HCAPI device specifics are learned
+ * and as well as the RM DB creation. From that those initial steps
+ * this structure can be populated.
+ *
+ * NOTE:
+ * If used in an array of table types then such array must be ordered
+ * by the TF type is represents.
+ */
+struct tf_shadow_tbl_cfg_parms {
+	/**
+	 * TF Table type
+	 */
+	enum tf_tbl_type type;
+
+	/**
+	 * Number of entries the Shadow DB needs to hold
+	 */
+	int num_entries;
+
+	/**
+	 * Element width for this table type
+	 */
+	int element_width;
+};
+
+/**
+ * Shadow table DB creation parameters
+ */
+struct tf_shadow_tbl_create_db_parms {
+	/**
+	 * [in] Configuration information for the shadow db
+	 */
+	struct tf_shadow_tbl_cfg_parms *cfg;
+	/**
+	 * [in] Number of elements in the parms structure
+	 */
+	uint16_t num_elements;
+	/**
+	 * [out] Shadow table DB handle
+	 */
+	void *tf_shadow_tbl_db;
+};
+
+/**
+ * Shadow table DB free parameters
+ */
+struct tf_shadow_tbl_free_db_parms {
+	/**
+	 * Shadow table DB handle
+	 */
+	void *tf_shadow_tbl_db;
+};
+
+/**
+ * Shadow table search parameters
+ */
+struct tf_shadow_tbl_search_parms {
+	/**
+	 * [in] Shadow table DB handle
+	 */
+	void *tf_shadow_tbl_db;
+	/**
+	 * [in] Table type
+	 */
+	enum tf_tbl_type type;
+	/**
+	 * [in] Pointer to entry blob value in remap table to match
+	 */
+	uint8_t *entry;
+	/**
+	 * [in] Size of the entry blob passed in bytes
+	 */
+	uint16_t entry_sz;
+	/**
+	 * [out] Index of the found element returned if hit
+	 */
+	uint16_t *index;
+	/**
+	 * [out] Reference count incremented if hit
+	 */
+	uint16_t *ref_cnt;
+};
+
+/**
+ * Shadow table insert parameters
+ */
+struct tf_shadow_tbl_insert_parms {
+	/**
+	 * [in] Shadow table DB handle
+	 */
+	void *tf_shadow_tbl_db;
+	/**
+	 * [in] Tbl type
+	 */
+	enum tf_tbl_type type;
+	/**
+	 * [in] Pointer to entry blob value in remap table to match
+	 */
+	uint8_t *entry;
+	/**
+	 * [in] Size of the entry blob passed in bytes
+	 */
+	uint16_t entry_sz;
+	/**
+	 * [in] Entry to update
+	 */
+	uint16_t index;
+	/**
+	 * [out] Reference count after insert
+	 */
+	uint16_t *ref_cnt;
+};
+
+/**
+ * Shadow table remove parameters
+ */
+struct tf_shadow_tbl_remove_parms {
+	/**
+	 * [in] Shadow table DB handle
+	 */
+	void *tf_shadow_tbl_db;
+	/**
+	 * [in] Tbl type
+	 */
+	enum tf_tbl_type type;
+	/**
+	 * [in] Entry to update
+	 */
+	uint16_t index;
+	/**
+	 * [out] Reference count after removal
+	 */
+	uint16_t *ref_cnt;
+};
+
+/**
+ * @page shadow_tbl Shadow table DB
+ *
+ * @ref tf_shadow_tbl_create_db
+ *
+ * @ref tf_shadow_tbl_free_db
+ *
+ * @reg tf_shadow_tbl_search
+ *
+ * @reg tf_shadow_tbl_insert
+ *
+ * @reg tf_shadow_tbl_remove
+ */
+
+/**
+ * Creates and fills a Shadow table DB. The DB is indexed per the
+ * parms structure.
+ *
+ * [in] parms
+ *   Pointer to create db parameters
+ *
+ * Returns
+ *   - (0) if successful.
+ *   - (-EINVAL) on failure.
+ */
+int tf_shadow_tbl_create_db(struct tf_shadow_tbl_create_db_parms *parms);
+
+/**
+ * Closes the Shadow table DB and frees all allocated
+ * resources per the associated database.
+ *
+ * [in] parms
+ *   Pointer to the free DB parameters
+ *
+ * Returns
+ *   - (0) if successful.
+ *   - (-EINVAL) on failure.
+ */
+int tf_shadow_tbl_free_db(struct tf_shadow_tbl_free_db_parms *parms);
+
+/**
+ * Search Shadow table db for matching result
+ *
+ * [in] parms
+ *   Pointer to the search parameters
+ *
+ * Returns
+ *   - (0) if successful, element was found.
+ *   - (-EINVAL) on failure.
+ */
+int tf_shadow_tbl_search(struct tf_shadow_tbl_search_parms *parms);
+
+/**
+ * Inserts an element into the Shadow table DB. Will fail if the
+ * elements ref_count is different from 0. Ref_count after insert will
+ * be incremented.
+ *
+ * [in] parms
+ *   Pointer to insert parameters
+ *
+ * Returns
+ *   - (0) if successful.
+ *   - (-EINVAL) on failure.
+ */
+int tf_shadow_tbl_insert(struct tf_shadow_tbl_insert_parms *parms);
+
+/**
+ * Removes an element from the Shadow table DB. Will fail if the
+ * elements ref_count is 0. Ref_count after removal will be
+ * decremented.
+ *
+ * [in] parms
+ *   Pointer to remove parameter
+ *
+ * Returns
+ *   - (0) if successful.
+ *   - (-EINVAL) on failure.
+ */
+int tf_shadow_tbl_remove(struct tf_shadow_tbl_remove_parms *parms);
+
+#endif /* _TF_SHADOW_TBL_H_ */
diff --git a/drivers/net/bnxt/tf_core/tf_shadow_tcam.c b/drivers/net/bnxt/tf_core/tf_shadow_tcam.c
new file mode 100644
index 0000000..c61b833
--- /dev/null
+++ b/drivers/net/bnxt/tf_core/tf_shadow_tcam.c
@@ -0,0 +1,63 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2019-2020 Broadcom
+ * All rights reserved.
+ */
+
+#include <rte_common.h>
+
+#include "tf_shadow_tcam.h"
+
+/**
+ * Shadow tcam DB element
+ */
+struct tf_shadow_tcam_element {
+	/**
+	 * Hash table
+	 */
+	void *hash;
+
+	/**
+	 * Reference count, array of number of tcam entries
+	 */
+	uint16_t *ref_count;
+};
+
+/**
+ * Shadow tcam DB definition
+ */
+struct tf_shadow_tcam_db {
+	/**
+	 * The DB consists of an array of elements
+	 */
+	struct tf_shadow_tcam_element *db;
+};
+
+int
+tf_shadow_tcam_create_db(struct tf_shadow_tcam_create_db_parms *parms __rte_unused)
+{
+	return 0;
+}
+
+int
+tf_shadow_tcam_free_db(struct tf_shadow_tcam_free_db_parms *parms __rte_unused)
+{
+	return 0;
+}
+
+int
+tf_shadow_tcam_search(struct tf_shadow_tcam_search_parms *parms __rte_unused)
+{
+	return 0;
+}
+
+int
+tf_shadow_tcam_insert(struct tf_shadow_tcam_insert_parms *parms __rte_unused)
+{
+	return 0;
+}
+
+int
+tf_shadow_tcam_remove(struct tf_shadow_tcam_remove_parms *parms __rte_unused)
+{
+	return 0;
+}
diff --git a/drivers/net/bnxt/tf_core/tf_shadow_tcam.h b/drivers/net/bnxt/tf_core/tf_shadow_tcam.h
new file mode 100644
index 0000000..e2c4e06
--- /dev/null
+++ b/drivers/net/bnxt/tf_core/tf_shadow_tcam.h
@@ -0,0 +1,239 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2019-2020 Broadcom
+ * All rights reserved.
+ */
+
+#ifndef _TF_SHADOW_TCAM_H_
+#define _TF_SHADOW_TCAM_H_
+
+#include "tf_core.h"
+
+struct tf;
+
+/**
+ * The Shadow tcam module provides shadow DB handling for tcam based
+ * TF types. A shadow DB provides the capability that allows for reuse
+ * of TF resources.
+ *
+ * A Shadow tcam DB is intended to be used by the Tcam module only.
+ */
+
+/**
+ * Shadow DB configuration information for a single tcam type.
+ *
+ * During Device initialization the HCAPI device specifics are learned
+ * and as well as the RM DB creation. From that those initial steps
+ * this structure can be populated.
+ *
+ * NOTE:
+ * If used in an array of tcam types then such array must be ordered
+ * by the TF type is represents.
+ */
+struct tf_shadow_tcam_cfg_parms {
+	/**
+	 * TF tcam type
+	 */
+	enum tf_tcam_tbl_type type;
+
+	/**
+	 * Number of entries the Shadow DB needs to hold
+	 */
+	int num_entries;
+
+	/**
+	 * Element width for this table type
+	 */
+	int element_width;
+};
+
+/**
+ * Shadow tcam DB creation parameters
+ */
+struct tf_shadow_tcam_create_db_parms {
+	/**
+	 * [in] Configuration information for the shadow db
+	 */
+	struct tf_shadow_tcam_cfg_parms *cfg;
+	/**
+	 * [in] Number of elements in the parms structure
+	 */
+	uint16_t num_elements;
+	/**
+	 * [out] Shadow tcam DB handle
+	 */
+	void *tf_shadow_tcam_db;
+};
+
+/**
+ * Shadow tcam DB free parameters
+ */
+struct tf_shadow_tcam_free_db_parms {
+	/**
+	 * Shadow tcam DB handle
+	 */
+	void *tf_shadow_tcam_db;
+};
+
+/**
+ * Shadow tcam search parameters
+ */
+struct tf_shadow_tcam_search_parms {
+	/**
+	 * [in] Shadow tcam DB handle
+	 */
+	void *tf_shadow_tcam_db;
+	/**
+	 * [in] TCAM tbl type
+	 */
+	enum tf_tcam_tbl_type type;
+	/**
+	 * [in] Pointer to entry blob value in remap table to match
+	 */
+	uint8_t *entry;
+	/**
+	 * [in] Size of the entry blob passed in bytes
+	 */
+	uint16_t entry_sz;
+	/**
+	 * [out] Index of the found element returned if hit
+	 */
+	uint16_t *index;
+	/**
+	 * [out] Reference count incremented if hit
+	 */
+	uint16_t *ref_cnt;
+};
+
+/**
+ * Shadow tcam insert parameters
+ */
+struct tf_shadow_tcam_insert_parms {
+	/**
+	 * [in] Shadow tcam DB handle
+	 */
+	void *tf_shadow_tcam_db;
+	/**
+	 * [in] TCAM tbl type
+	 */
+	enum tf_tcam_tbl_type type;
+	/**
+	 * [in] Pointer to entry blob value in remap table to match
+	 */
+	uint8_t *entry;
+	/**
+	 * [in] Size of the entry blob passed in bytes
+	 */
+	uint16_t entry_sz;
+	/**
+	 * [in] Entry to update
+	 */
+	uint16_t index;
+	/**
+	 * [out] Reference count after insert
+	 */
+	uint16_t *ref_cnt;
+};
+
+/**
+ * Shadow tcam remove parameters
+ */
+struct tf_shadow_tcam_remove_parms {
+	/**
+	 * [in] Shadow tcam DB handle
+	 */
+	void *tf_shadow_tcam_db;
+	/**
+	 * [in] TCAM tbl type
+	 */
+	enum tf_tcam_tbl_type type;
+	/**
+	 * [in] Entry to update
+	 */
+	uint16_t index;
+	/**
+	 * [out] Reference count after removal
+	 */
+	uint16_t *ref_cnt;
+};
+
+/**
+ * @page shadow_tcam Shadow tcam DB
+ *
+ * @ref tf_shadow_tcam_create_db
+ *
+ * @ref tf_shadow_tcam_free_db
+ *
+ * @reg tf_shadow_tcam_search
+ *
+ * @reg tf_shadow_tcam_insert
+ *
+ * @reg tf_shadow_tcam_remove
+ */
+
+/**
+ * Creates and fills a Shadow tcam DB. The DB is indexed per the
+ * parms structure.
+ *
+ * [in] parms
+ *   Pointer to create db parameters
+ *
+ * Returns
+ *   - (0) if successful.
+ *   - (-EINVAL) on failure.
+ */
+int tf_shadow_tcam_create_db(struct tf_shadow_tcam_create_db_parms *parms);
+
+/**
+ * Closes the Shadow tcam DB and frees all allocated
+ * resources per the associated database.
+ *
+ * [in] parms
+ *   Pointer to the free DB parameters
+ *
+ * Returns
+ *   - (0) if successful.
+ *   - (-EINVAL) on failure.
+ */
+int tf_shadow_tcam_free_db(struct tf_shadow_tcam_free_db_parms *parms);
+
+/**
+ * Search Shadow tcam db for matching result
+ *
+ * [in] parms
+ *   Pointer to the search parameters
+ *
+ * Returns
+ *   - (0) if successful, element was found.
+ *   - (-EINVAL) on failure.
+ */
+int tf_shadow_tcam_search(struct tf_shadow_tcam_search_parms *parms);
+
+/**
+ * Inserts an element into the Shadow tcam DB. Will fail if the
+ * elements ref_count is different from 0. Ref_count after insert will
+ * be incremented.
+ *
+ * [in] parms
+ *   Pointer to insert parameters
+ *
+ * Returns
+ *   - (0) if successful.
+ *   - (-EINVAL) on failure.
+ */
+int tf_shadow_tcam_insert(struct tf_shadow_tcam_insert_parms *parms);
+
+/**
+ * Removes an element from the Shadow tcam DB. Will fail if the
+ * elements ref_count is 0. Ref_count after removal will be
+ * decremented.
+ *
+ * [in] parms
+ *   Pointer to remove parameter
+ *
+ * Returns
+ *   - (0) if successful.
+ *   - (-EINVAL) on failure.
+ */
+int tf_shadow_tcam_remove(struct tf_shadow_tcam_remove_parms *parms);
+
+#endif /* _TF_SHADOW_TCAM_H_ */
diff --git a/drivers/net/bnxt/tf_core/tf_tbl.c b/drivers/net/bnxt/tf_core/tf_tbl.c
index b9c71d4..c403d81 100644
--- a/drivers/net/bnxt/tf_core/tf_tbl.c
+++ b/drivers/net/bnxt/tf_core/tf_tbl.c
@@ -15,6 +15,7 @@
 #include "hsi_struct_def_dpdk.h"
 
 #include "tf_core.h"
+#include "tf_util.h"
 #include "tf_em.h"
 #include "tf_msg.h"
 #include "tfp.h"
diff --git a/drivers/net/bnxt/tf_core/tf_tbl_type.c b/drivers/net/bnxt/tf_core/tf_tbl_type.c
new file mode 100644
index 0000000..a57a5dd
--- /dev/null
+++ b/drivers/net/bnxt/tf_core/tf_tbl_type.c
@@ -0,0 +1,78 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2019-2020 Broadcom
+ * All rights reserved.
+ */
+
+#include <rte_common.h>
+
+#include "tf_tbl_type.h"
+
+struct tf;
+
+/**
+ * Table Type DBs.
+ */
+/* static void *tbl_db[TF_DIR_MAX]; */
+
+/**
+ * Table Type Shadow DBs
+ */
+/* static void *shadow_tbl_db[TF_DIR_MAX]; */
+
+/**
+ * Init flag, set on bind and cleared on unbind
+ */
+/* static uint8_t init; */
+
+/**
+ * Shadow init flag, set on bind and cleared on unbind
+ */
+/* static uint8_t shadow_init; */
+
+int
+tf_tbl_type_bind(struct tf *tfp __rte_unused,
+		 struct tf_tbl_type_cfg_parms *parms __rte_unused)
+{
+	return 0;
+}
+
+int
+tf_tbl_type_unbind(struct tf *tfp __rte_unused)
+{
+	return 0;
+}
+
+int
+tf_tbl_type_alloc(struct tf *tfp __rte_unused,
+		  struct tf_tbl_type_alloc_parms *parms __rte_unused)
+{
+	return 0;
+}
+
+int
+tf_tbl_type_free(struct tf *tfp __rte_unused,
+		 struct tf_tbl_type_free_parms *parms __rte_unused)
+{
+	return 0;
+}
+
+int
+tf_tbl_type_alloc_search(struct tf *tfp __rte_unused,
+			 struct tf_tbl_type_alloc_search_parms *parms __rte_unused)
+{
+	return 0;
+}
+
+int
+tf_tbl_type_set(struct tf *tfp __rte_unused,
+		struct tf_tbl_type_set_parms *parms __rte_unused)
+{
+	return 0;
+}
+
+int
+tf_tbl_type_get(struct tf *tfp __rte_unused,
+		struct tf_tbl_type_get_parms *parms __rte_unused)
+{
+	return 0;
+}
diff --git a/drivers/net/bnxt/tf_core/tf_tbl_type.h b/drivers/net/bnxt/tf_core/tf_tbl_type.h
new file mode 100644
index 0000000..c880b36
--- /dev/null
+++ b/drivers/net/bnxt/tf_core/tf_tbl_type.h
@@ -0,0 +1,309 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2019-2020 Broadcom
+ * All rights reserved.
+ */
+
+#ifndef TF_TBL_TYPE_H_
+#define TF_TBL_TYPE_H_
+
+#include "tf_core.h"
+
+struct tf;
+
+/**
+ * The Table Type module provides processing of Internal TF table types.
+ */
+
+/**
+ * Table Type configuration parameters
+ */
+struct tf_tbl_type_cfg_parms {
+	/**
+	 * Number of table types in each of the configuration arrays
+	 */
+	uint16_t num_elements;
+
+	/**
+	 * Table Type element configuration array
+	 */
+	struct tf_rm_element_cfg *tbl_cfg[TF_DIR_MAX];
+
+	/**
+	 * Shadow table type configuration array
+	 */
+	struct tf_shadow_tbl_type_cfg *tbl_shadow_cfg[TF_DIR_MAX];
+};
+
+/**
+ * Table Type allocation parameters
+ */
+struct tf_tbl_type_alloc_parms {
+	/**
+	 * [in] Receive or transmit direction
+	 */
+	enum tf_dir dir;
+	/**
+	 * [in] Type of the allocation
+	 */
+	enum tf_tbl_type type;
+	/**
+	 * [out] Idx of allocated entry or found entry (if search_enable)
+	 */
+	uint32_t idx;
+};
+
+/**
+ * Table Type free parameters
+ */
+struct tf_tbl_type_free_parms {
+	/**
+	 * [in] Receive or transmit direction
+	 */
+	enum tf_dir dir;
+	/**
+	 * [in] Type of the allocation type
+	 */
+	enum tf_tbl_type type;
+	/**
+	 * [in] Index to free
+	 */
+	uint32_t idx;
+	/**
+	 * [out] Reference count after free, only valid if session has been
+	 * created with shadow_copy.
+	 */
+	uint16_t ref_cnt;
+};
+
+struct tf_tbl_type_alloc_search_parms {
+	/**
+	 * [in] Receive or transmit direction
+	 */
+	enum tf_dir dir;
+	/**
+	 * [in] Type of the allocation
+	 */
+	enum tf_tbl_type type;
+	/**
+	 * [in] Table scope identifier (ignored unless TF_TBL_TYPE_EXT)
+	 */
+	uint32_t tbl_scope_id;
+	/**
+	 * [in] Enable search for matching entry. If the table type is
+	 * internal the shadow copy will be searched before
+	 * alloc. Session must be configured with shadow copy enabled.
+	 */
+	uint8_t search_enable;
+	/**
+	 * [in] Result data to search for (if search_enable)
+	 */
+	uint8_t *result;
+	/**
+	 * [in] Result data size in bytes (if search_enable)
+	 */
+	uint16_t result_sz_in_bytes;
+	/**
+	 * [out] If search_enable, set if matching entry found
+	 */
+	uint8_t hit;
+	/**
+	 * [out] Current ref count after allocation (if search_enable)
+	 */
+	uint16_t ref_cnt;
+	/**
+	 * [out] Idx of allocated entry or found entry (if search_enable)
+	 */
+	uint32_t idx;
+};
+
+/**
+ * Table Type set parameters
+ */
+struct tf_tbl_type_set_parms {
+	/**
+	 * [in] Receive or transmit direction
+	 */
+	enum tf_dir dir;
+	/**
+	 * [in] Type of object to set
+	 */
+	enum tf_tbl_type type;
+	/**
+	 * [in] Entry data
+	 */
+	uint8_t *data;
+	/**
+	 * [in] Entry size
+	 */
+	uint16_t data_sz_in_bytes;
+	/**
+	 * [in] Entry index to write to
+	 */
+	uint32_t idx;
+};
+
+/**
+ * Table Type get parameters
+ */
+struct tf_tbl_type_get_parms {
+	/**
+	 * [in] Receive or transmit direction
+	 */
+	enum tf_dir dir;
+	/**
+	 * [in] Type of object to get
+	 */
+	enum tf_tbl_type type;
+	/**
+	 * [out] Entry data
+	 */
+	uint8_t *data;
+	/**
+	 * [out] Entry size
+	 */
+	uint16_t data_sz_in_bytes;
+	/**
+	 * [in] Entry index to read
+	 */
+	uint32_t idx;
+};
+
+/**
+ * @page tbl_type Table Type
+ *
+ * @ref tf_tbl_type_bind
+ *
+ * @ref tf_tbl_type_unbind
+ *
+ * @ref tf_tbl_type_alloc
+ *
+ * @ref tf_tbl_type_free
+ *
+ * @ref tf_tbl_type_alloc_search
+ *
+ * @ref tf_tbl_type_set
+ *
+ * @ref tf_tbl_type_get
+ */
+
+/**
+ * Initializes the Table Type module with the requested DBs. Must be
+ * invoked as the first thing before any of the access functions.
+ *
+ * [in] tfp
+ *   Pointer to TF handle, used for HCAPI communication
+ *
+ * [in] parms
+ *   Pointer to parameters
+ *
+ * Returns
+ *   - (0) if successful.
+ *   - (-EINVAL) on failure.
+ */
+int tf_tbl_type_bind(struct tf *tfp,
+		     struct tf_tbl_type_cfg_parms *parms);
+
+/**
+ * Cleans up the private DBs and releases all the data.
+ *
+ * [in] tfp
+ *   Pointer to TF handle, used for HCAPI communication
+ *
+ * [in] parms
+ *   Pointer to parameters
+ *
+ * Returns
+ *   - (0) if successful.
+ *   - (-EINVAL) on failure.
+ */
+int tf_tbl_type_unbind(struct tf *tfp);
+
+/**
+ * Allocates the requested table type from the internal RM DB.
+ *
+ * [in] tfp
+ *   Pointer to TF handle, used for HCAPI communication
+ *
+ * [in] parms
+ *   Pointer to parameters
+ *
+ * Returns
+ *   - (0) if successful.
+ *   - (-EINVAL) on failure.
+ */
+int tf_tbl_type_alloc(struct tf *tfp,
+		      struct tf_tbl_type_alloc_parms *parms);
+
+/**
+ * Free's the requested table type and returns it to the DB. If shadow
+ * DB is enabled its searched first and if found the element refcount
+ * is decremented. If refcount goes to 0 then its returned to the
+ * table type DB.
+ *
+ * [in] tfp
+ *   Pointer to TF handle, used for HCAPI communication
+ *
+ * [in] parms
+ *   Pointer to parameters
+ *
+ * Returns
+ *   - (0) if successful.
+ *   - (-EINVAL) on failure.
+ */
+int tf_tbl_type_free(struct tf *tfp,
+		     struct tf_tbl_type_free_parms *parms);
+
+/**
+ * Supported if Shadow DB is configured. Searches the Shadow DB for
+ * any matching element. If found the refcount in the shadow DB is
+ * updated accordingly. If not found a new element is allocated and
+ * installed into the shadow DB.
+ *
+ * [in] tfp
+ *   Pointer to TF handle, used for HCAPI communication
+ *
+ * [in] parms
+ *   Pointer to parameters
+ *
+ * Returns
+ *   - (0) if successful.
+ *   - (-EINVAL) on failure.
+ */
+int tf_tbl_type_alloc_search(struct tf *tfp,
+			     struct tf_tbl_type_alloc_search_parms *parms);
+
+/**
+ * Configures the requested element by sending a firmware request which
+ * then installs it into the device internal structures.
+ *
+ * [in] tfp
+ *   Pointer to TF handle, used for HCAPI communication
+ *
+ * [in] parms
+ *   Pointer to parameters
+ *
+ * Returns
+ *   - (0) if successful.
+ *   - (-EINVAL) on failure.
+ */
+int tf_tbl_type_set(struct tf *tfp,
+		    struct tf_tbl_type_set_parms *parms);
+
+/**
+ * Retrieves the requested element by sending a firmware request to get
+ * the element.
+ *
+ * [in] tfp
+ *   Pointer to TF handle, used for HCAPI communication
+ *
+ * [in] parms
+ *   Pointer to parameters
+ *
+ * Returns
+ *   - (0) if successful.
+ *   - (-EINVAL) on failure.
+ */
+int tf_tbl_type_get(struct tf *tfp,
+		    struct tf_tbl_type_get_parms *parms);
+
+#endif /* TF_TBL_TYPE_H */
diff --git a/drivers/net/bnxt/tf_core/tf_tcam.c b/drivers/net/bnxt/tf_core/tf_tcam.c
new file mode 100644
index 0000000..3ad99dd
--- /dev/null
+++ b/drivers/net/bnxt/tf_core/tf_tcam.c
@@ -0,0 +1,78 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2019-2020 Broadcom
+ * All rights reserved.
+ */
+
+#include <rte_common.h>
+
+#include "tf_tcam.h"
+
+struct tf;
+
+/**
+ * TCAM DBs.
+ */
+/* static void *tcam_db[TF_DIR_MAX]; */
+
+/**
+ * TCAM Shadow DBs
+ */
+/* static void *shadow_tcam_db[TF_DIR_MAX]; */
+
+/**
+ * Init flag, set on bind and cleared on unbind
+ */
+/* static uint8_t init; */
+
+/**
+ * Shadow init flag, set on bind and cleared on unbind
+ */
+/* static uint8_t shadow_init; */
+
+int
+tf_tcam_bind(struct tf *tfp __rte_unused,
+	     struct tf_tcam_cfg_parms *parms __rte_unused)
+{
+	return 0;
+}
+
+int
+tf_tcam_unbind(struct tf *tfp __rte_unused)
+{
+	return 0;
+}
+
+int
+tf_tcam_alloc(struct tf *tfp __rte_unused,
+	      struct tf_tcam_alloc_parms *parms __rte_unused)
+{
+	return 0;
+}
+
+int
+tf_tcam_free(struct tf *tfp __rte_unused,
+	     struct tf_tcam_free_parms *parms __rte_unused)
+{
+	return 0;
+}
+
+int
+tf_tcam_alloc_search(struct tf *tfp __rte_unused,
+		     struct tf_tcam_alloc_search_parms *parms __rte_unused)
+{
+	return 0;
+}
+
+int
+tf_tcam_set(struct tf *tfp __rte_unused,
+	    struct tf_tcam_set_parms *parms __rte_unused)
+{
+	return 0;
+}
+
+int
+tf_tcam_get(struct tf *tfp __rte_unused,
+	    struct tf_tcam_get_parms *parms __rte_unused)
+{
+	return 0;
+}
diff --git a/drivers/net/bnxt/tf_core/tf_tcam.h b/drivers/net/bnxt/tf_core/tf_tcam.h
new file mode 100644
index 0000000..1420c9e
--- /dev/null
+++ b/drivers/net/bnxt/tf_core/tf_tcam.h
@@ -0,0 +1,314 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2019-2020 Broadcom
+ * All rights reserved.
+ */
+
+#ifndef _TF_TCAM_H_
+#define _TF_TCAM_H_
+
+#include "tf_core.h"
+
+/**
+ * The TCAM module provides processing of Internal TCAM types.
+ */
+
+/**
+ * TCAM configuration parameters
+ */
+struct tf_tcam_cfg_parms {
+	/**
+	 * Number of tcam types in each of the configuration arrays
+	 */
+	uint16_t num_elements;
+
+	/**
+	 * TCAM configuration array
+	 */
+	struct tf_rm_element_cfg *tcam_cfg[TF_DIR_MAX];
+
+	/**
+	 * Shadow table type configuration array
+	 */
+	struct tf_shadow_tcam_cfg *tcam_shadow_cfg[TF_DIR_MAX];
+};
+
+/**
+ * TCAM allocation parameters
+ */
+struct tf_tcam_alloc_parms {
+	/**
+	 * [in] Receive or transmit direction
+	 */
+	enum tf_dir dir;
+	/**
+	 * [in] Type of the allocation
+	 */
+	enum tf_tcam_tbl_type type;
+	/**
+	 * [out] Idx of allocated entry or found entry (if search_enable)
+	 */
+	uint32_t idx;
+};
+
+/**
+ * TCAM free parameters
+ */
+struct tf_tcam_free_parms {
+	/**
+	 * [in] Receive or transmit direction
+	 */
+	enum tf_dir dir;
+	/**
+	 * [in] Type of the allocation type
+	 */
+	enum tf_tcam_tbl_type type;
+	/**
+	 * [in] Index to free
+	 */
+	uint32_t idx;
+	/**
+	 * [out] Reference count after free, only valid if session has been
+	 * created with shadow_copy.
+	 */
+	uint16_t ref_cnt;
+};
+
+/**
+ * TCAM allocate search parameters
+ */
+struct tf_tcam_alloc_search_parms {
+	/**
+	 * [in] receive or transmit direction
+	 */
+	enum tf_dir dir;
+	/**
+	 * [in] TCAM table type
+	 */
+	enum tf_tcam_tbl_type tcam_tbl_type;
+	/**
+	 * [in] Enable search for matching entry
+	 */
+	uint8_t search_enable;
+	/**
+	 * [in] Key data to match on (if search)
+	 */
+	uint8_t *key;
+	/**
+	 * [in] key size in bits (if search)
+	 */
+	uint16_t key_sz_in_bits;
+	/**
+	 * [in] Mask data to match on (if search)
+	 */
+	uint8_t *mask;
+	/**
+	 * [in] Priority of entry requested (definition TBD)
+	 */
+	uint32_t priority;
+	/**
+	 * [out] If search, set if matching entry found
+	 */
+	uint8_t hit;
+	/**
+	 * [out] Current refcnt after allocation
+	 */
+	uint16_t ref_cnt;
+	/**
+	 * [out] Idx allocated
+	 *
+	 */
+	uint16_t idx;
+};
+
+/**
+ * TCAM set parameters
+ */
+struct tf_tcam_set_parms {
+	/**
+	 * [in] Receive or transmit direction
+	 */
+	enum tf_dir dir;
+	/**
+	 * [in] Type of object to set
+	 */
+	enum tf_tcam_tbl_type type;
+	/**
+	 * [in] Entry data
+	 */
+	uint8_t *data;
+	/**
+	 * [in] Entry size
+	 */
+	uint16_t data_sz_in_bytes;
+	/**
+	 * [in] Entry index to write to
+	 */
+	uint32_t idx;
+};
+
+/**
+ * TCAM get parameters
+ */
+struct tf_tcam_get_parms {
+	/**
+	 * [in] Receive or transmit direction
+	 */
+	enum tf_dir dir;
+	/**
+	 * [in] Type of object to get
+	 */
+	enum tf_tcam_tbl_type type;
+	/**
+	 * [out] Entry data
+	 */
+	uint8_t *data;
+	/**
+	 * [out] Entry size
+	 */
+	uint16_t data_sz_in_bytes;
+	/**
+	 * [in] Entry index to read
+	 */
+	uint32_t idx;
+};
+
+/**
+ * @page tcam TCAM
+ *
+ * @ref tf_tcam_bind
+ *
+ * @ref tf_tcam_unbind
+ *
+ * @ref tf_tcam_alloc
+ *
+ * @ref tf_tcam_free
+ *
+ * @ref tf_tcam_alloc_search
+ *
+ * @ref tf_tcam_set
+ *
+ * @ref tf_tcam_get
+ *
+ */
+
+/**
+ * Initializes the TCAM module with the requested DBs. Must be
+ * invoked as the first thing before any of the access functions.
+ *
+ * [in] tfp
+ *   Pointer to TF handle, used for HCAPI communication
+ *
+ * [in] parms
+ *   Pointer to parameters
+ *
+ * Returns
+ *   - (0) if successful.
+ *   - (-EINVAL) on failure.
+ */
+int tf_tcam_bind(struct tf *tfp,
+		 struct tf_tcam_cfg_parms *parms);
+
+/**
+ * Cleans up the private DBs and releases all the data.
+ *
+ * [in] tfp
+ *   Pointer to TF handle, used for HCAPI communication
+ *
+ * [in] parms
+ *   Pointer to parameters
+ *
+ * Returns
+ *   - (0) if successful.
+ *   - (-EINVAL) on failure.
+ */
+int tf_tcam_unbind(struct tf *tfp);
+
+/**
+ * Allocates the requested tcam type from the internal RM DB.
+ *
+ * [in] tfp
+ *   Pointer to TF handle, used for HCAPI communication
+ *
+ * [in] parms
+ *   Pointer to parameters
+ *
+ * Returns
+ *   - (0) if successful.
+ *   - (-EINVAL) on failure.
+ */
+int tf_tcam_alloc(struct tf *tfp,
+		  struct tf_tcam_alloc_parms *parms);
+
+/**
+ * Free's the requested table type and returns it to the DB. If shadow
+ * DB is enabled its searched first and if found the element refcount
+ * is decremented. If refcount goes to 0 then its returned to the
+ * table type DB.
+ *
+ * [in] tfp
+ *   Pointer to TF handle, used for HCAPI communication
+ *
+ * [in] parms
+ *   Pointer to parameters
+ *
+ * Returns
+ *   - (0) if successful.
+ *   - (-EINVAL) on failure.
+ */
+int tf_tcam_free(struct tf *tfp,
+		 struct tf_tcam_free_parms *parms);
+
+/**
+ * Supported if Shadow DB is configured. Searches the Shadow DB for
+ * any matching element. If found the refcount in the shadow DB is
+ * updated accordingly. If not found a new element is allocated and
+ * installed into the shadow DB.
+ *
+ * [in] tfp
+ *   Pointer to TF handle, used for HCAPI communication
+ *
+ * [in] parms
+ *   Pointer to parameters
+ *
+ * Returns
+ *   - (0) if successful.
+ *   - (-EINVAL) on failure.
+ */
+int tf_tcam_alloc_search(struct tf *tfp,
+			 struct tf_tcam_alloc_search_parms *parms);
+
+/**
+ * Configures the requested element by sending a firmware request which
+ * then installs it into the device internal structures.
+ *
+ * [in] tfp
+ *   Pointer to TF handle, used for HCAPI communication
+ *
+ * [in] parms
+ *   Pointer to parameters
+ *
+ * Returns
+ *   - (0) if successful.
+ *   - (-EINVAL) on failure.
+ */
+int tf_tcam_set(struct tf *tfp,
+		struct tf_tcam_set_parms *parms);
+
+/**
+ * Retrieves the requested element by sending a firmware request to get
+ * the element.
+ *
+ * [in] tfp
+ *   Pointer to TF handle, used for HCAPI communication
+ *
+ * [in] parms
+ *   Pointer to parameters
+ *
+ * Returns
+ *   - (0) if successful.
+ *   - (-EINVAL) on failure.
+ */
+int tf_tcam_get(struct tf *tfp,
+		struct tf_tcam_get_parms *parms);
+
+#endif /* _TF_TCAM_H */
diff --git a/drivers/net/bnxt/tf_core/tf_util.c b/drivers/net/bnxt/tf_core/tf_util.c
new file mode 100644
index 0000000..a901054
--- /dev/null
+++ b/drivers/net/bnxt/tf_core/tf_util.c
@@ -0,0 +1,145 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2014-2019 Broadcom
+ * All rights reserved.
+ */
+
+#include <string.h>
+
+#include "tf_util.h"
+
+const char
+*tf_dir_2_str(enum tf_dir dir)
+{
+	switch (dir) {
+	case TF_DIR_RX:
+		return "RX";
+	case TF_DIR_TX:
+		return "TX";
+	default:
+		return "Invalid direction";
+	}
+}
+
+const char
+*tf_ident_2_str(enum tf_identifier_type id_type)
+{
+	switch (id_type) {
+	case TF_IDENT_TYPE_L2_CTXT:
+		return "l2_ctxt_remap";
+	case TF_IDENT_TYPE_PROF_FUNC:
+		return "prof_func";
+	case TF_IDENT_TYPE_WC_PROF:
+		return "wc_prof";
+	case TF_IDENT_TYPE_EM_PROF:
+		return "em_prof";
+	case TF_IDENT_TYPE_L2_FUNC:
+		return "l2_func";
+	default:
+		return "Invalid identifier";
+	}
+}
+
+const char
+*tf_tcam_tbl_2_str(enum tf_tcam_tbl_type tcam_type)
+{
+	switch (tcam_type) {
+	case TF_TCAM_TBL_TYPE_L2_CTXT_TCAM:
+		return "l2_ctxt_tcam";
+	case TF_TCAM_TBL_TYPE_PROF_TCAM:
+		return "prof_tcam";
+	case TF_TCAM_TBL_TYPE_WC_TCAM:
+		return "wc_tcam";
+	case TF_TCAM_TBL_TYPE_VEB_TCAM:
+		return "veb_tcam";
+	case TF_TCAM_TBL_TYPE_SP_TCAM:
+		return "sp_tcam";
+	case TF_TCAM_TBL_TYPE_CT_RULE_TCAM:
+		return "ct_rule_tcam";
+	default:
+		return "Invalid tcam table type";
+	}
+}
+
+const char
+*tf_tbl_type_2_str(enum tf_tbl_type tbl_type)
+{
+	switch (tbl_type) {
+	case TF_TBL_TYPE_FULL_ACT_RECORD:
+		return "Full Action record";
+	case TF_TBL_TYPE_MCAST_GROUPS:
+		return "Multicast Groups";
+	case TF_TBL_TYPE_ACT_ENCAP_8B:
+		return "Encap 8B";
+	case TF_TBL_TYPE_ACT_ENCAP_16B:
+		return "Encap 16B";
+	case TF_TBL_TYPE_ACT_ENCAP_32B:
+		return "Encap 32B";
+	case TF_TBL_TYPE_ACT_ENCAP_64B:
+		return "Encap 64B";
+	case TF_TBL_TYPE_ACT_SP_SMAC:
+		return "Source Properties SMAC";
+	case TF_TBL_TYPE_ACT_SP_SMAC_IPV4:
+		return "Source Properties SMAC IPv4";
+	case TF_TBL_TYPE_ACT_SP_SMAC_IPV6:
+		return "Source Properties SMAC IPv6";
+	case TF_TBL_TYPE_ACT_STATS_64:
+		return "Stats 64B";
+	case TF_TBL_TYPE_ACT_MODIFY_SPORT:
+		return "NAT Source Port";
+	case TF_TBL_TYPE_ACT_MODIFY_DPORT:
+		return "NAT Destination Port";
+	case TF_TBL_TYPE_ACT_MODIFY_IPV4_SRC:
+		return "NAT IPv4 Source";
+	case TF_TBL_TYPE_ACT_MODIFY_IPV4_DEST:
+		return "NAT IPv4 Destination";
+	case TF_TBL_TYPE_ACT_MODIFY_IPV6_SRC:
+		return "NAT IPv6 Source";
+	case TF_TBL_TYPE_ACT_MODIFY_IPV6_DEST:
+		return "NAT IPv6 Destination";
+	case TF_TBL_TYPE_METER_PROF:
+		return "Meter Profile";
+	case TF_TBL_TYPE_METER_INST:
+		return "Meter";
+	case TF_TBL_TYPE_MIRROR_CONFIG:
+		return "Mirror";
+	case TF_TBL_TYPE_UPAR:
+		return "UPAR";
+	case TF_TBL_TYPE_EPOCH0:
+		return "EPOCH0";
+	case TF_TBL_TYPE_EPOCH1:
+		return "EPOCH1";
+	case TF_TBL_TYPE_METADATA:
+		return "Metadata";
+	case TF_TBL_TYPE_CT_STATE:
+		return "Connection State";
+	case TF_TBL_TYPE_RANGE_PROF:
+		return "Range Profile";
+	case TF_TBL_TYPE_RANGE_ENTRY:
+		return "Range";
+	case TF_TBL_TYPE_LAG:
+		return "Link Aggregation";
+	case TF_TBL_TYPE_VNIC_SVIF:
+		return "VNIC SVIF";
+	case TF_TBL_TYPE_EM_FKB:
+		return "EM Flexible Key Builder";
+	case TF_TBL_TYPE_WC_FKB:
+		return "WC Flexible Key Builder";
+	case TF_TBL_TYPE_EXT:
+		return "External";
+	default:
+		return "Invalid tbl type";
+	}
+}
+
+const char
+*tf_em_tbl_type_2_str(enum tf_em_tbl_type em_type)
+{
+	switch (em_type) {
+	case TF_EM_TBL_TYPE_EM_RECORD:
+		return "EM Record";
+	case TF_EM_TBL_TYPE_TBL_SCOPE:
+		return "Table Scope";
+	default:
+		return "Invalid EM type";
+	}
+}
diff --git a/drivers/net/bnxt/tf_core/tf_util.h b/drivers/net/bnxt/tf_core/tf_util.h
new file mode 100644
index 0000000..4099629
--- /dev/null
+++ b/drivers/net/bnxt/tf_core/tf_util.h
@@ -0,0 +1,41 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2014-2019 Broadcom
+ * All rights reserved.
+ */
+
+#ifndef _TF_UTIL_H_
+#define _TF_UTIL_H_
+
+#include "tf_core.h"
+
+/**
+ * Helper function converting direction to text string
+ */
+const char
+*tf_dir_2_str(enum tf_dir dir);
+
+/**
+ * Helper function converting identifier to text string
+ */
+const char
+*tf_ident_2_str(enum tf_identifier_type id_type);
+
+/**
+ * Helper function converting tcam type to text string
+ */
+const char
+*tf_tcam_tbl_2_str(enum tf_tcam_tbl_type tcam_type);
+
+/**
+ * Helper function converting tbl type to text string
+ */
+const char
+*tf_tbl_type_2_str(enum tf_tbl_type tbl_type);
+
+/**
+ * Helper function converting em tbl type to text string
+ */
+const char
+*tf_em_tbl_type_2_str(enum tf_em_tbl_type em_type);
+
+#endif /* _TF_UTIL_H_ */
-- 
2.7.4


^ permalink raw reply	[flat|nested] 271+ messages in thread

* [dpdk-dev] [PATCH 12/50] net/bnxt: support bulk table get and mirror
  2020-06-12 13:28 [dpdk-dev] [PATCH 00/50] add features for host-based flow management Somnath Kotur
                   ` (10 preceding siblings ...)
  2020-06-12 13:28 ` [dpdk-dev] [PATCH 11/50] net/bnxt: add multi device support Somnath Kotur
@ 2020-06-12 13:28 ` Somnath Kotur
  2020-06-12 13:28 ` [dpdk-dev] [PATCH 13/50] net/bnxt: update multi device design support Somnath Kotur
                   ` (38 subsequent siblings)
  50 siblings, 0 replies; 271+ messages in thread
From: Somnath Kotur @ 2020-06-12 13:28 UTC (permalink / raw)
  To: dev; +Cc: ferruh.yigit

From: Shahaji Bhosle <sbhosle@broadcom.com>

- Add new bulk table type get using FW
  to DMA the data back to host.
- Add flag to allow records to be cleared if possible
- Set mirror using tf_alloc_tbl_entry

Signed-off-by: Shahaji Bhosle <sbhosle@broadcom.com>
Reviewed-by: Ajit Kumar Khaparde <ajit.khaparde@broadcom.com>
Signed-off-by: Venkat Duvvuru <venkatkumar.duvvuru@broadcom.com>
---
 drivers/net/bnxt/tf_core/hwrm_tf.h      |  37 +++++++++++-
 drivers/net/bnxt/tf_core/tf_common.h    |  54 +++++++++++++++++
 drivers/net/bnxt/tf_core/tf_core.c      |   2 +
 drivers/net/bnxt/tf_core/tf_core.h      |  55 ++++++++++++++++-
 drivers/net/bnxt/tf_core/tf_msg.c       |  70 +++++++++++++++++-----
 drivers/net/bnxt/tf_core/tf_msg.h       |  15 +++++
 drivers/net/bnxt/tf_core/tf_resources.h |   5 +-
 drivers/net/bnxt/tf_core/tf_tbl.c       | 103 ++++++++++++++++++++++++++++++++
 8 files changed, 319 insertions(+), 22 deletions(-)
 create mode 100644 drivers/net/bnxt/tf_core/tf_common.h

diff --git a/drivers/net/bnxt/tf_core/hwrm_tf.h b/drivers/net/bnxt/tf_core/hwrm_tf.h
index d342c69..c04d103 100644
--- a/drivers/net/bnxt/tf_core/hwrm_tf.h
+++ b/drivers/net/bnxt/tf_core/hwrm_tf.h
@@ -1,5 +1,5 @@
 /* SPDX-License-Identifier: BSD-3-Clause
- * Copyright(c) 2019 Broadcom
+ * Copyright(c) 2019-2020 Broadcom
  * All rights reserved.
  */
 #ifndef _HWRM_TF_H_
@@ -27,7 +27,8 @@ typedef enum tf_subtype {
 	HWRM_TFT_REG_SET = 822,
 	HWRM_TFT_TBL_TYPE_SET = 823,
 	HWRM_TFT_TBL_TYPE_GET = 824,
-	TF_SUBTYPE_LAST = HWRM_TFT_TBL_TYPE_GET,
+	HWRM_TFT_TBL_TYPE_GET_BULK = 825,
+	TF_SUBTYPE_LAST = HWRM_TFT_TBL_TYPE_GET_BULK,
 } tf_subtype_t;
 
 /* Request and Response compile time checking */
@@ -81,6 +82,8 @@ struct tf_session_sram_resc_flush_input;
 struct tf_tbl_type_set_input;
 struct tf_tbl_type_get_input;
 struct tf_tbl_type_get_output;
+struct tf_tbl_type_get_bulk_input;
+struct tf_tbl_type_get_bulk_output;
 /* Input params for session attach */
 typedef struct tf_session_attach_input {
 	/* Firmware session id returned when HWRM_TF_SESSION_OPEN is sent */
@@ -902,6 +905,8 @@ typedef struct tf_tbl_type_get_input {
 #define TF_TBL_TYPE_GET_INPUT_FLAGS_DIR_RX			(0x0)
 	/* When set to 1, indicates the get apply to TX */
 #define TF_TBL_TYPE_GET_INPUT_FLAGS_DIR_TX			(0x1)
+	/* When set to 1, indicates the clear entry on read */
+#define TF_TBL_TYPE_GET_INPUT_FLAGS_CLEAR_ON_READ	  (0x2)
 	/* Type of the object to set */
 	uint32_t			 type;
 	/* Index to get */
@@ -916,4 +921,32 @@ typedef struct tf_tbl_type_get_output {
 	uint8_t			  data[TF_BULK_RECV];
 } tf_tbl_type_get_output_t, *ptf_tbl_type_get_output_t;
 
+/* Input params for table type get */
+typedef struct tf_tbl_type_get_bulk_input {
+	/* Session Id */
+	uint32_t			 fw_session_id;
+	/* flags */
+	uint16_t			 flags;
+	/* When set to 0, indicates the get apply to RX */
+#define TF_TBL_TYPE_GET_BULK_INPUT_FLAGS_DIR_RX	   (0x0)
+	/* When set to 1, indicates the get apply to TX */
+#define TF_TBL_TYPE_GET_BULK_INPUT_FLAGS_DIR_TX	   (0x1)
+	/* When set to 1, indicates the clear entry on read */
+#define TF_TBL_TYPE_GET_BULK_INPUT_FLAGS_CLEAR_ON_READ	  (0x2)
+	/* Type of the object to set */
+	uint32_t			 type;
+	/* Starting index to get from */
+	uint32_t			 start_index;
+	/* Number of entries to get */
+	uint32_t			 num_entries;
+	/* Host memory where data will be stored */
+	uint64_t			 host_addr;
+} tf_tbl_type_get_bulk_input_t, *ptf_tbl_type_get_bulk_input_t;
+
+/* Output params for table type get */
+typedef struct tf_tbl_type_get_bulk_output {
+	/* Size of the total data read in bytes */
+	uint16_t			 size;
+} tf_tbl_type_get_bulk_output_t, *ptf_tbl_type_get_bulk_output_t;
+
 #endif /* _HWRM_TF_H_ */
diff --git a/drivers/net/bnxt/tf_core/tf_common.h b/drivers/net/bnxt/tf_core/tf_common.h
new file mode 100644
index 0000000..2aa4b86
--- /dev/null
+++ b/drivers/net/bnxt/tf_core/tf_common.h
@@ -0,0 +1,54 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2020 Broadcom
+ * All rights reserved.
+ */
+
+#ifndef _TF_COMMON_H_
+#define _TF_COMMON_H_
+
+/* Helper to check the parms */
+#define TF_CHECK_PARMS_SESSION(tfp, parms) do {	\
+		if ((parms) == NULL || (tfp) == NULL) { \
+			TFP_DRV_LOG(ERR, "Invalid Argument(s)\n"); \
+			return -EINVAL; \
+		} \
+		if ((tfp)->session == NULL || \
+		    (tfp)->session->core_data == NULL) { \
+			TFP_DRV_LOG(ERR, "%s: session error\n", \
+				    tf_dir_2_str((parms)->dir)); \
+			return -EINVAL; \
+		} \
+	} while (0)
+
+#define TF_CHECK_PARMS_SESSION_NO_DIR(tfp, parms) do {	\
+		if ((parms) == NULL || (tfp) == NULL) { \
+			TFP_DRV_LOG(ERR, "Invalid Argument(s)\n"); \
+			return -EINVAL; \
+		} \
+		if ((tfp)->session == NULL || \
+		    (tfp)->session->core_data == NULL) { \
+			TFP_DRV_LOG(ERR, "Session error\n"); \
+			return -EINVAL; \
+		} \
+	} while (0)
+
+#define TF_CHECK_PARMS(tfp, parms) do {	\
+		if ((parms) == NULL || (tfp) == NULL) { \
+			TFP_DRV_LOG(ERR, "Invalid Argument(s)\n"); \
+			return -EINVAL; \
+		} \
+	} while (0)
+
+#define TF_CHECK_TFP_SESSION(tfp) do { \
+		if ((tfp) == NULL) { \
+			TFP_DRV_LOG(ERR, "Invalid Argument(s)\n"); \
+			return -EINVAL; \
+		} \
+		if ((tfp)->session == NULL || \
+		    (tfp)->session->core_data == NULL) { \
+			TFP_DRV_LOG(ERR, "Session error\n"); \
+			return -EINVAL; \
+		} \
+	} while (0)
+
+#endif /* _TF_COMMON_H_ */
diff --git a/drivers/net/bnxt/tf_core/tf_core.c b/drivers/net/bnxt/tf_core/tf_core.c
index 58924b1..0098690 100644
--- a/drivers/net/bnxt/tf_core/tf_core.c
+++ b/drivers/net/bnxt/tf_core/tf_core.c
@@ -16,6 +16,8 @@
 #include "bitalloc.h"
 #include "bnxt.h"
 #include "rand.h"
+#include "tf_common.h"
+#include "hwrm_tf.h"
 
 static inline uint32_t SWAP_WORDS32(uint32_t val32)
 {
diff --git a/drivers/net/bnxt/tf_core/tf_core.h b/drivers/net/bnxt/tf_core/tf_core.h
index becc50c..96a1a79 100644
--- a/drivers/net/bnxt/tf_core/tf_core.h
+++ b/drivers/net/bnxt/tf_core/tf_core.h
@@ -1165,7 +1165,7 @@ struct tf_get_tbl_entry_parms {
 	 */
 	uint8_t *data;
 	/**
-	 * [out] Entry size
+	 * [in] Entry size
 	 */
 	uint16_t data_sz_in_bytes;
 	/**
@@ -1189,6 +1189,59 @@ int tf_get_tbl_entry(struct tf *tfp,
 		     struct tf_get_tbl_entry_parms *parms);
 
 /**
+ * tf_get_bulk_tbl_entry parameter definition
+ */
+struct tf_get_bulk_tbl_entry_parms {
+	/**
+	 * [in] Receive or transmit direction
+	 */
+	enum tf_dir dir;
+	/**
+	 * [in] Type of object to get
+	 */
+	enum tf_tbl_type type;
+	/**
+	 * [in] Clear hardware entries on reads only
+	 * supported for TF_TBL_TYPE_ACT_STATS_64
+	 */
+	bool clear_on_read;
+	/**
+	 * [in] Starting index to read from
+	 */
+	uint32_t starting_idx;
+	/**
+	 * [in] Number of sequential entries
+	 */
+	uint16_t num_entries;
+	/**
+	 * [in] Size of the single entry
+	 */
+	uint16_t entry_sz_in_bytes;
+	/**
+	 * [out] Host physical address, where the data
+	 * will be copied to by the firmware.
+	 * Use tfp_calloc() API and mem_pa
+	 * variable of the tfp_calloc_parms
+	 * structure for the physical address.
+	 */
+	uint64_t physical_mem_addr;
+};
+
+/**
+ * Bulk get index table entry
+ *
+ * Used to retrieve a previous set index table entry.
+ *
+ * Reads and compares with the shadow table copy (if enabled) (only
+ * for internal objects).
+ *
+ * Returns success or failure code. Failure will be returned if the
+ * provided data buffer is too small for the data type requested.
+ */
+int tf_get_bulk_tbl_entry(struct tf *tfp,
+		     struct tf_get_bulk_tbl_entry_parms *parms);
+
+/**
  * @page exact_match Exact Match Table
  *
  * @ref tf_insert_em_entry
diff --git a/drivers/net/bnxt/tf_core/tf_msg.c b/drivers/net/bnxt/tf_core/tf_msg.c
index c8f6b88..c755c85 100644
--- a/drivers/net/bnxt/tf_core/tf_msg.c
+++ b/drivers/net/bnxt/tf_core/tf_msg.c
@@ -1216,12 +1216,8 @@ tf_msg_get_tbl_entry(struct tf *tfp,
 	return tfp_le_to_cpu_32(parms.tf_resp_code);
 }
 
-#define TF_BYTES_PER_SLICE(tfp) 12
-#define NUM_SLICES(tfp, bytes) \
-	(((bytes) + TF_BYTES_PER_SLICE(tfp) - 1) / TF_BYTES_PER_SLICE(tfp))
-
 static int
-tf_msg_get_dma_buf(struct tf_msg_dma_buf *buf, int size)
+tf_msg_alloc_dma_buf(struct tf_msg_dma_buf *buf, int size)
 {
 	struct tfp_calloc_parms alloc_parms;
 	int rc;
@@ -1229,15 +1225,10 @@ tf_msg_get_dma_buf(struct tf_msg_dma_buf *buf, int size)
 	/* Allocate session */
 	alloc_parms.nitems = 1;
 	alloc_parms.size = size;
-	alloc_parms.alignment = 0;
+	alloc_parms.alignment = 4096;
 	rc = tfp_calloc(&alloc_parms);
-	if (rc) {
-		/* Log error */
-		PMD_DRV_LOG(ERR,
-			    "Failed to allocate tcam dma entry, rc:%d\n",
-			    rc);
+	if (rc)
 		return -ENOMEM;
-	}
 
 	buf->pa_addr = (uintptr_t)alloc_parms.mem_pa;
 	buf->va_addr = alloc_parms.mem_va;
@@ -1246,6 +1237,52 @@ tf_msg_get_dma_buf(struct tf_msg_dma_buf *buf, int size)
 }
 
 int
+tf_msg_get_bulk_tbl_entry(struct tf *tfp,
+			  struct tf_get_bulk_tbl_entry_parms *params)
+{
+	int rc;
+	struct tfp_send_msg_parms parms = { 0 };
+	struct tf_tbl_type_get_bulk_input req = { 0 };
+	struct tf_tbl_type_get_bulk_output resp = { 0 };
+	struct tf_session *tfs = (struct tf_session *)(tfp->session->core_data);
+	int data_size = 0;
+
+	/* Populate the request */
+	req.fw_session_id =
+		tfp_cpu_to_le_32(tfs->session_id.internal.fw_session_id);
+	req.flags = tfp_cpu_to_le_16((params->dir) |
+		((params->clear_on_read) ?
+		 TF_TBL_TYPE_GET_BULK_INPUT_FLAGS_CLEAR_ON_READ : 0x0));
+	req.type = tfp_cpu_to_le_32(params->type);
+	req.start_index = tfp_cpu_to_le_32(params->starting_idx);
+	req.num_entries = tfp_cpu_to_le_32(params->num_entries);
+
+	data_size = (params->num_entries * params->entry_sz_in_bytes);
+	req.host_addr = tfp_cpu_to_le_64(params->physical_mem_addr);
+
+	MSG_PREP(parms,
+		 TF_KONG_MB,
+		 HWRM_TF,
+		 HWRM_TFT_TBL_TYPE_GET_BULK,
+		 req,
+		 resp);
+
+	rc = tfp_send_msg_tunneled(tfp, &parms);
+	if (rc)
+		return rc;
+
+	/* Verify that we got enough buffer to return the requested data */
+	if (resp.size < data_size)
+		return -EINVAL;
+
+	return tfp_le_to_cpu_32(parms.tf_resp_code);
+}
+
+#define TF_BYTES_PER_SLICE(tfp) 12
+#define NUM_SLICES(tfp, bytes) \
+	(((bytes) + TF_BYTES_PER_SLICE(tfp) - 1) / TF_BYTES_PER_SLICE(tfp))
+
+int
 tf_msg_tcam_entry_set(struct tf *tfp,
 		      struct tf_set_tcam_entry_parms *parms)
 {
@@ -1282,9 +1319,9 @@ tf_msg_tcam_entry_set(struct tf *tfp,
 	} else {
 		/* use dma buffer */
 		req.flags |= HWRM_TF_TCAM_SET_INPUT_FLAGS_DMA;
-		rc = tf_msg_get_dma_buf(&buf, data_size);
-		if (rc != 0)
-			return rc;
+		rc = tf_msg_alloc_dma_buf(&buf, data_size);
+		if (rc)
+			goto cleanup;
 		data = buf.va_addr;
 		memcpy(&req.dev_data[0], &buf.pa_addr, sizeof(buf.pa_addr));
 	}
@@ -1303,8 +1340,9 @@ tf_msg_tcam_entry_set(struct tf *tfp,
 	rc = tfp_send_msg_direct(tfp,
 				 &mparms);
 	if (rc)
-		return rc;
+		goto cleanup;
 
+cleanup:
 	if (buf.va_addr != NULL)
 		tfp_free(buf.va_addr);
 
diff --git a/drivers/net/bnxt/tf_core/tf_msg.h b/drivers/net/bnxt/tf_core/tf_msg.h
index 89f7370..8d050c4 100644
--- a/drivers/net/bnxt/tf_core/tf_msg.h
+++ b/drivers/net/bnxt/tf_core/tf_msg.h
@@ -267,4 +267,19 @@ int tf_msg_get_tbl_entry(struct tf *tfp,
 			 uint8_t *data,
 			 uint32_t index);
 
+/**
+ * Sends bulk get message of a Table Type element to the firmware.
+ *
+ * [in] tfp
+ *   Pointer to session handle
+ *
+ * [in] parms
+ *   Pointer to table get bulk parameters
+ *
+ * Returns:
+ *  0 on Success else internal Truflow error
+ */
+int tf_msg_get_bulk_tbl_entry(struct tf *tfp,
+			  struct tf_get_bulk_tbl_entry_parms *parms);
+
 #endif  /* _TF_MSG_H_ */
diff --git a/drivers/net/bnxt/tf_core/tf_resources.h b/drivers/net/bnxt/tf_core/tf_resources.h
index 05e131f..9b7f5a0 100644
--- a/drivers/net/bnxt/tf_core/tf_resources.h
+++ b/drivers/net/bnxt/tf_core/tf_resources.h
@@ -149,11 +149,10 @@
 #define TF_RSVD_METER_INST_END_IDX_TX             0
 
 /* Mirror */
-/* Not yet supported fully in the infra */
-#define TF_RSVD_MIRROR_RX                         0
+#define TF_RSVD_MIRROR_RX                         1
 #define TF_RSVD_MIRROR_BEGIN_IDX_RX               0
 #define TF_RSVD_MIRROR_END_IDX_RX                 0
-#define TF_RSVD_MIRROR_TX                         0
+#define TF_RSVD_MIRROR_TX                         1
 #define TF_RSVD_MIRROR_BEGIN_IDX_TX               0
 #define TF_RSVD_MIRROR_END_IDX_TX                 0
 
diff --git a/drivers/net/bnxt/tf_core/tf_tbl.c b/drivers/net/bnxt/tf_core/tf_tbl.c
index c403d81..0f2979e 100644
--- a/drivers/net/bnxt/tf_core/tf_tbl.c
+++ b/drivers/net/bnxt/tf_core/tf_tbl.c
@@ -23,6 +23,7 @@
 #include "bnxt.h"
 #include "tf_resources.h"
 #include "tf_rm.h"
+#include "tf_common.h"
 
 #define PTU_PTE_VALID          0x1UL
 #define PTU_PTE_LAST           0x2UL
@@ -794,6 +795,7 @@ tf_set_tbl_entry_internal(struct tf *tfp,
 
 	if (parms->type != TF_TBL_TYPE_FULL_ACT_RECORD &&
 	    parms->type != TF_TBL_TYPE_ACT_SP_SMAC_IPV4 &&
+	    parms->type != TF_TBL_TYPE_MIRROR_CONFIG &&
 	    parms->type != TF_TBL_TYPE_ACT_STATS_64) {
 		PMD_DRV_LOG(ERR,
 			    "dir:%d, Type not supported, type:%d\n",
@@ -915,6 +917,76 @@ tf_get_tbl_entry_internal(struct tf *tfp,
 	return rc;
 }
 
+/**
+ * Internal function to get a Table Entry. Supports all Table Types
+ * except the TF_TBL_TYPE_EXT as that is handled as a table scope.
+ *
+ * [in] tfp
+ *   Pointer to TruFlow handle
+ *
+ * [in] parms
+ *   Pointer to input parameters
+ *
+ * Returns:
+ *   0       - Success
+ *   -EINVAL - Parameter error
+ */
+static int
+tf_get_bulk_tbl_entry_internal(struct tf *tfp,
+			  struct tf_get_bulk_tbl_entry_parms *parms)
+{
+	int rc;
+	int id;
+	uint32_t index;
+	struct bitalloc *session_pool;
+	struct tf_session *tfs = (struct tf_session *)(tfp->session->core_data);
+
+	/* Lookup the pool using the table type of the element */
+	rc = tf_rm_lookup_tbl_type_pool(tfs,
+					parms->dir,
+					parms->type,
+					&session_pool);
+	/* Error logging handled by tf_rm_lookup_tbl_type_pool */
+	if (rc)
+		return rc;
+
+	index = parms->starting_idx;
+
+	/*
+	 * Adjust the returned index/offset as there is no guarantee
+	 * that the start is 0 at time of RM allocation
+	 */
+	tf_rm_convert_index(tfs,
+			    parms->dir,
+			    parms->type,
+			    TF_RM_CONVERT_RM_BASE,
+			    parms->starting_idx,
+			    &index);
+
+	/* Verify that the entry has been previously allocated */
+	id = ba_inuse(session_pool, index);
+	if (id != 1) {
+		TFP_DRV_LOG(ERR,
+		   "%s, Invalid or not allocated index, type:%d, starting_idx:%d\n",
+		   tf_dir_2_str(parms->dir),
+		   parms->type,
+		   index);
+		return -EINVAL;
+	}
+
+	/* Get the entry */
+	rc = tf_msg_get_bulk_tbl_entry(tfp, parms);
+	if (rc) {
+		TFP_DRV_LOG(ERR,
+			    "%s, Bulk get failed, type:%d, rc:%s\n",
+			    tf_dir_2_str(parms->dir),
+			    parms->type,
+			    strerror(-rc));
+	}
+
+	return rc;
+}
+
 #if (TF_SHADOW == 1)
 /**
  * Allocate Tbl entry from the Shadow DB. Shadow DB is searched for
@@ -1182,6 +1254,7 @@ tf_alloc_tbl_entry_pool_internal(struct tf *tfp,
 	    parms->type != TF_TBL_TYPE_ACT_ENCAP_8B &&
 	    parms->type != TF_TBL_TYPE_ACT_ENCAP_16B &&
 	    parms->type != TF_TBL_TYPE_ACT_ENCAP_64B &&
+	    parms->type != TF_TBL_TYPE_MIRROR_CONFIG &&
 	    parms->type != TF_TBL_TYPE_ACT_STATS_64) {
 		PMD_DRV_LOG(ERR,
 			    "dir:%d, Type not supported, type:%d\n",
@@ -1665,6 +1738,36 @@ tf_get_tbl_entry(struct tf *tfp,
 
 /* API defined in tf_core.h */
 int
+tf_get_bulk_tbl_entry(struct tf *tfp,
+		 struct tf_get_bulk_tbl_entry_parms *parms)
+{
+	int rc = 0;
+
+	TF_CHECK_PARMS_SESSION(tfp, parms);
+
+	if (parms->type == TF_TBL_TYPE_EXT) {
+		/* Not supported, yet */
+		TFP_DRV_LOG(ERR,
+			    "%s, External table type not supported\n",
+			    tf_dir_2_str(parms->dir));
+
+		rc = -EOPNOTSUPP;
+	} else {
+		/* Internal table type processing */
+		rc = tf_get_bulk_tbl_entry_internal(tfp, parms);
+		if (rc)
+			TFP_DRV_LOG(ERR,
+				    "%s, Bulk get failed, type:%d, rc:%s\n",
+				    tf_dir_2_str(parms->dir),
+				    parms->type,
+				    strerror(-rc));
+	}
+
+	return rc;
+}
+
+/* API defined in tf_core.h */
+int
 tf_alloc_tbl_scope(struct tf *tfp,
 		   struct tf_alloc_tbl_scope_parms *parms)
 {
-- 
2.7.4


^ permalink raw reply	[flat|nested] 271+ messages in thread

* [dpdk-dev] [PATCH 13/50] net/bnxt: update multi device design support
  2020-06-12 13:28 [dpdk-dev] [PATCH 00/50] add features for host-based flow management Somnath Kotur
                   ` (11 preceding siblings ...)
  2020-06-12 13:28 ` [dpdk-dev] [PATCH 12/50] net/bnxt: support bulk table get and mirror Somnath Kotur
@ 2020-06-12 13:28 ` Somnath Kotur
  2020-06-12 13:28 ` [dpdk-dev] [PATCH 14/50] net/bnxt: support two-level priority for TCAMs Somnath Kotur
                   ` (37 subsequent siblings)
  50 siblings, 0 replies; 271+ messages in thread
From: Somnath Kotur @ 2020-06-12 13:28 UTC (permalink / raw)
  To: dev; +Cc: ferruh.yigit

From: Michael Wildt <michael.wildt@broadcom.com>

- Implement the modules RM, Device (WH+), Identifier.
- Update Session module.
- Implement new HWRMs for RM direct messaging.
- Add new parameter check macro's and clean up the header includes for
  i.e. tfp such that bnxt.h is not directly included in the new modules.
- Add cfa_resource_types, required for RM design.

Signed-off-by: Michael Wildt <michael.wildt@broadcom.com>
Reviewed-by: Randy Schacher <stuart.schacher@broadcom.com>
Signed-off-by: Venkat Duvvuru <venkatkumar.duvvuru@broadcom.com>
---
 drivers/net/bnxt/meson.build                  |   2 +
 drivers/net/bnxt/tf_core/Makefile             |   1 +
 drivers/net/bnxt/tf_core/cfa_resource_types.h | 291 ++++++++---------
 drivers/net/bnxt/tf_core/tf_common.h          |  24 ++
 drivers/net/bnxt/tf_core/tf_core.c            | 286 ++++++++++++++++-
 drivers/net/bnxt/tf_core/tf_core.h            |  12 +-
 drivers/net/bnxt/tf_core/tf_device.c          | 150 ++++++++-
 drivers/net/bnxt/tf_core/tf_device.h          |  79 ++++-
 drivers/net/bnxt/tf_core/tf_device_p4.c       |  78 ++++-
 drivers/net/bnxt/tf_core/tf_device_p4.h       |  79 +++--
 drivers/net/bnxt/tf_core/tf_identifier.c      | 142 ++++++++-
 drivers/net/bnxt/tf_core/tf_identifier.h      |  25 +-
 drivers/net/bnxt/tf_core/tf_msg.c             | 268 ++++++++++++++--
 drivers/net/bnxt/tf_core/tf_msg.h             |  59 ++++
 drivers/net/bnxt/tf_core/tf_rm_new.c          | 434 ++++++++++++++++++++++++--
 drivers/net/bnxt/tf_core/tf_rm_new.h          |  72 +++--
 drivers/net/bnxt/tf_core/tf_session.c         | 280 ++++++++++++++++-
 drivers/net/bnxt/tf_core/tf_session.h         | 118 ++++++-
 drivers/net/bnxt/tf_core/tf_tbl.h             |   4 +
 drivers/net/bnxt/tf_core/tf_tbl_type.c        |  30 +-
 drivers/net/bnxt/tf_core/tf_tbl_type.h        |  95 +++---
 drivers/net/bnxt/tf_core/tf_tcam.h            |  14 +-
 22 files changed, 2144 insertions(+), 399 deletions(-)

diff --git a/drivers/net/bnxt/meson.build b/drivers/net/bnxt/meson.build
index a50cb26..1f7df9d 100644
--- a/drivers/net/bnxt/meson.build
+++ b/drivers/net/bnxt/meson.build
@@ -32,6 +32,8 @@ sources = files('bnxt_cpr.c',
 	'tf_core/tf_rm.c',
 	'tf_core/tf_tbl.c',
 	'tf_core/tfp.c',
+	'tf_core/tf_session.c',
+	'tf_core/tf_device.c',
 	'tf_core/tf_device_p4.c',
 	'tf_core/tf_identifier.c',
 	'tf_core/tf_shadow_tbl.c',
diff --git a/drivers/net/bnxt/tf_core/Makefile b/drivers/net/bnxt/tf_core/Makefile
index 71df75b..7191c7f 100644
--- a/drivers/net/bnxt/tf_core/Makefile
+++ b/drivers/net/bnxt/tf_core/Makefile
@@ -14,6 +14,7 @@ SRCS-$(CONFIG_RTE_LIBRTE_BNXT_PMD) += tf_core/tfp.c
 SRCS-$(CONFIG_RTE_LIBRTE_BNXT_PMD) += tf_core/tf_msg.c
 SRCS-$(CONFIG_RTE_LIBRTE_BNXT_PMD) += tf_core/tf_em.c
 SRCS-$(CONFIG_RTE_LIBRTE_BNXT_PMD) += tf_core/tf_tbl.c
+SRCS-$(CONFIG_RTE_LIBRTE_BNXT_PMD) += tf_core/tf_session.c
 SRCS-$(CONFIG_RTE_LIBRTE_BNXT_PMD) += tf_core/tf_device.c
 SRCS-$(CONFIG_RTE_LIBRTE_BNXT_PMD) += tf_core/tf_device_p4.c
 SRCS-$(CONFIG_RTE_LIBRTE_BNXT_PMD) += tf_core/tf_identifier.c
diff --git a/drivers/net/bnxt/tf_core/cfa_resource_types.h b/drivers/net/bnxt/tf_core/cfa_resource_types.h
index c0c1e75..11e8892 100644
--- a/drivers/net/bnxt/tf_core/cfa_resource_types.h
+++ b/drivers/net/bnxt/tf_core/cfa_resource_types.h
@@ -12,6 +12,11 @@
 
 #ifndef _CFA_RESOURCE_TYPES_H_
 #define _CFA_RESOURCE_TYPES_H_
+/*
+ * This is the constant used to define invalid CFA
+ * resource types across all devices.
+ */
+#define CFA_RESOURCE_TYPE_INVALID 65535
 
 /* L2 Context TCAM */
 #define CFA_RESOURCE_TYPE_P59_L2_CTXT_TCAM    0x0UL
@@ -58,209 +63,205 @@
 #define CFA_RESOURCE_TYPE_P59_LAST           CFA_RESOURCE_TYPE_P59_VEB_TCAM
 
 
-/* SRAM Multicast Group */
-#define CFA_RESOURCE_TYPE_P58_SRAM_MCG             0x0UL
-/* SRAM Encap 8 byte record */
-#define CFA_RESOURCE_TYPE_P58_SRAM_ENCAP_8B        0x1UL
-/* SRAM Encap 16 byte record */
-#define CFA_RESOURCE_TYPE_P58_SRAM_ENCAP_16B       0x2UL
-/* SRAM Encap 64 byte record */
-#define CFA_RESOURCE_TYPE_P58_SRAM_ENCAP_64B       0x3UL
-/* SRAM Source Property MAC */
-#define CFA_RESOURCE_TYPE_P58_SRAM_SP_MAC          0x4UL
-/* SRAM Source Property MAC and IPv4 */
-#define CFA_RESOURCE_TYPE_P58_SRAM_SP_MAC_IPV4     0x5UL
-/* SRAM Source Property MAC and IPv6 */
-#define CFA_RESOURCE_TYPE_P58_SRAM_SP_MAC_IPV6     0x6UL
-/* SRAM Network Address Translation Source Port */
-#define CFA_RESOURCE_TYPE_P58_SRAM_NAT_SPORT       0x7UL
-/* SRAM Network Address Translation Destination Port */
-#define CFA_RESOURCE_TYPE_P58_SRAM_NAT_DPORT       0x8UL
-/* SRAM Network Address Translation Source IPv4 address */
-#define CFA_RESOURCE_TYPE_P58_SRAM_NAT_S_IPV4      0x9UL
-/* SRAM Network Address Translation Destination IPv4 address */
-#define CFA_RESOURCE_TYPE_P58_SRAM_NAT_D_IPV4      0xaUL
-/* SRAM Network Address Translation Source IPv4 address */
-#define CFA_RESOURCE_TYPE_P58_SRAM_NAT_S_IPV6      0xbUL
-/* SRAM Network Address Translation Destination IPv4 address */
-#define CFA_RESOURCE_TYPE_P58_SRAM_NAT_D_IPV6      0xcUL
+/* Multicast Group */
+#define CFA_RESOURCE_TYPE_P58_MCG             0x0UL
+/* Encap 8 byte record */
+#define CFA_RESOURCE_TYPE_P58_ENCAP_8B        0x1UL
+/* Encap 16 byte record */
+#define CFA_RESOURCE_TYPE_P58_ENCAP_16B       0x2UL
+/* Encap 64 byte record */
+#define CFA_RESOURCE_TYPE_P58_ENCAP_64B       0x3UL
+/* Source Property MAC */
+#define CFA_RESOURCE_TYPE_P58_SP_MAC          0x4UL
+/* Source Property MAC and IPv4 */
+#define CFA_RESOURCE_TYPE_P58_SP_MAC_IPV4     0x5UL
+/* Source Property MAC and IPv6 */
+#define CFA_RESOURCE_TYPE_P58_SP_MAC_IPV6     0x6UL
+/* Network Address Translation Source Port */
+#define CFA_RESOURCE_TYPE_P58_NAT_SPORT       0x7UL
+/* Network Address Translation Destination Port */
+#define CFA_RESOURCE_TYPE_P58_NAT_DPORT       0x8UL
+/* Network Address Translation Source IPv4 address */
+#define CFA_RESOURCE_TYPE_P58_NAT_S_IPV4      0x9UL
+/* Network Address Translation Destination IPv4 address */
+#define CFA_RESOURCE_TYPE_P58_NAT_D_IPV4      0xaUL
+/* Network Address Translation Source IPv4 address */
+#define CFA_RESOURCE_TYPE_P58_NAT_S_IPV6      0xbUL
+/* Network Address Translation Destination IPv4 address */
+#define CFA_RESOURCE_TYPE_P58_NAT_D_IPV6      0xcUL
 /* Meter */
-#define CFA_RESOURCE_TYPE_P58_SRAM_METER           0xdUL
+#define CFA_RESOURCE_TYPE_P58_METER           0xdUL
 /* Flow State */
-#define CFA_RESOURCE_TYPE_P58_SRAM_FLOW_STATE      0xeUL
+#define CFA_RESOURCE_TYPE_P58_FLOW_STATE      0xeUL
 /* Full Action Records */
-#define CFA_RESOURCE_TYPE_P58_SRAM_FULL_ACTION     0xfUL
+#define CFA_RESOURCE_TYPE_P58_FULL_ACTION     0xfUL
 /* Action Record Format 0 */
-#define CFA_RESOURCE_TYPE_P58_SRAM_FORMAT_0_ACTION 0x10UL
+#define CFA_RESOURCE_TYPE_P58_FORMAT_0_ACTION 0x10UL
 /* Action Record Format 2 */
-#define CFA_RESOURCE_TYPE_P58_SRAM_FORMAT_2_ACTION 0x11UL
+#define CFA_RESOURCE_TYPE_P58_FORMAT_2_ACTION 0x11UL
 /* Action Record Format 3 */
-#define CFA_RESOURCE_TYPE_P58_SRAM_FORMAT_3_ACTION 0x12UL
+#define CFA_RESOURCE_TYPE_P58_FORMAT_3_ACTION 0x12UL
 /* Action Record Format 4 */
-#define CFA_RESOURCE_TYPE_P58_SRAM_FORMAT_4_ACTION 0x13UL
+#define CFA_RESOURCE_TYPE_P58_FORMAT_4_ACTION 0x13UL
 /* L2 Context TCAM */
-#define CFA_RESOURCE_TYPE_P58_L2_CTXT_TCAM         0x14UL
+#define CFA_RESOURCE_TYPE_P58_L2_CTXT_TCAM    0x14UL
 /* Profile Func */
-#define CFA_RESOURCE_TYPE_P58_PROF_FUNC            0x15UL
+#define CFA_RESOURCE_TYPE_P58_PROF_FUNC       0x15UL
 /* Profile TCAM */
-#define CFA_RESOURCE_TYPE_P58_PROF_TCAM            0x16UL
+#define CFA_RESOURCE_TYPE_P58_PROF_TCAM       0x16UL
 /* Exact Match Profile Id */
-#define CFA_RESOURCE_TYPE_P58_EM_PROF_ID           0x17UL
+#define CFA_RESOURCE_TYPE_P58_EM_PROF_ID      0x17UL
 /* Wildcard Profile Id */
-#define CFA_RESOURCE_TYPE_P58_WC_TCAM_PROF_ID      0x18UL
+#define CFA_RESOURCE_TYPE_P58_WC_TCAM_PROF_ID 0x18UL
+/* Exact Match Record */
+#define CFA_RESOURCE_TYPE_P58_EM_REC          0x19UL
 /* Wildcard TCAM */
-#define CFA_RESOURCE_TYPE_P58_WC_TCAM              0x19UL
+#define CFA_RESOURCE_TYPE_P58_WC_TCAM         0x1aUL
 /* Meter profile */
-#define CFA_RESOURCE_TYPE_P58_METER_PROF           0x1aUL
+#define CFA_RESOURCE_TYPE_P58_METER_PROF      0x1bUL
 /* Meter */
-#define CFA_RESOURCE_TYPE_P58_METER                0x1bUL
-/* Meter */
-#define CFA_RESOURCE_TYPE_P58_MIRROR               0x1cUL
+#define CFA_RESOURCE_TYPE_P58_MIRROR          0x1cUL
 /* Source Property TCAM */
-#define CFA_RESOURCE_TYPE_P58_SP_TCAM              0x1dUL
+#define CFA_RESOURCE_TYPE_P58_SP_TCAM         0x1dUL
 /* Exact Match Flexible Key Builder */
-#define CFA_RESOURCE_TYPE_P58_EM_FKB               0x1eUL
+#define CFA_RESOURCE_TYPE_P58_EM_FKB          0x1eUL
 /* Wildcard Flexible Key Builder */
-#define CFA_RESOURCE_TYPE_P58_WC_FKB               0x1fUL
+#define CFA_RESOURCE_TYPE_P58_WC_FKB          0x1fUL
 /* VEB TCAM */
-#define CFA_RESOURCE_TYPE_P58_VEB_TCAM             0x20UL
-#define CFA_RESOURCE_TYPE_P58_LAST                CFA_RESOURCE_TYPE_P58_VEB_TCAM
+#define CFA_RESOURCE_TYPE_P58_VEB_TCAM        0x20UL
+#define CFA_RESOURCE_TYPE_P58_LAST           CFA_RESOURCE_TYPE_P58_VEB_TCAM
 
 
-/* SRAM Multicast Group */
-#define CFA_RESOURCE_TYPE_P45_SRAM_MCG             0x0UL
-/* SRAM Encap 8 byte record */
-#define CFA_RESOURCE_TYPE_P45_SRAM_ENCAP_8B        0x1UL
-/* SRAM Encap 16 byte record */
-#define CFA_RESOURCE_TYPE_P45_SRAM_ENCAP_16B       0x2UL
-/* SRAM Encap 64 byte record */
-#define CFA_RESOURCE_TYPE_P45_SRAM_ENCAP_64B       0x3UL
-/* SRAM Source Property MAC */
-#define CFA_RESOURCE_TYPE_P45_SRAM_SP_MAC          0x4UL
-/* SRAM Source Property MAC and IPv4 */
-#define CFA_RESOURCE_TYPE_P45_SRAM_SP_MAC_IPV4     0x5UL
-/* SRAM Source Property MAC and IPv6 */
-#define CFA_RESOURCE_TYPE_P45_SRAM_SP_MAC_IPV6     0x6UL
-/* SRAM 64B Counters */
-#define CFA_RESOURCE_TYPE_P45_SRAM_COUNTER_64B     0x7UL
-/* SRAM Network Address Translation Source Port */
-#define CFA_RESOURCE_TYPE_P45_SRAM_NAT_SPORT       0x8UL
-/* SRAM Network Address Translation Destination Port */
-#define CFA_RESOURCE_TYPE_P45_SRAM_NAT_DPORT       0x9UL
-/* SRAM Network Address Translation Source IPv4 address */
-#define CFA_RESOURCE_TYPE_P45_SRAM_NAT_S_IPV4      0xaUL
-/* SRAM Network Address Translation Destination IPv4 address */
-#define CFA_RESOURCE_TYPE_P45_SRAM_NAT_D_IPV4      0xbUL
-/* SRAM Network Address Translation Source IPv6 address */
-#define CFA_RESOURCE_TYPE_P45_SRAM_NAT_S_IPV6      0xcUL
-/* SRAM Network Address Translation Destination IPv6 address */
-#define CFA_RESOURCE_TYPE_P45_SRAM_NAT_D_IPV6      0xdUL
+/* Multicast Group */
+#define CFA_RESOURCE_TYPE_P45_MCG             0x0UL
+/* Encap 8 byte record */
+#define CFA_RESOURCE_TYPE_P45_ENCAP_8B        0x1UL
+/* Encap 16 byte record */
+#define CFA_RESOURCE_TYPE_P45_ENCAP_16B       0x2UL
+/* Encap 64 byte record */
+#define CFA_RESOURCE_TYPE_P45_ENCAP_64B       0x3UL
+/* Source Property MAC */
+#define CFA_RESOURCE_TYPE_P45_SP_MAC          0x4UL
+/* Source Property MAC and IPv4 */
+#define CFA_RESOURCE_TYPE_P45_SP_MAC_IPV4     0x5UL
+/* Source Property MAC and IPv6 */
+#define CFA_RESOURCE_TYPE_P45_SP_MAC_IPV6     0x6UL
+/* 64B Counters */
+#define CFA_RESOURCE_TYPE_P45_COUNTER_64B     0x7UL
+/* Network Address Translation Source Port */
+#define CFA_RESOURCE_TYPE_P45_NAT_SPORT       0x8UL
+/* Network Address Translation Destination Port */
+#define CFA_RESOURCE_TYPE_P45_NAT_DPORT       0x9UL
+/* Network Address Translation Source IPv4 address */
+#define CFA_RESOURCE_TYPE_P45_NAT_S_IPV4      0xaUL
+/* Network Address Translation Destination IPv4 address */
+#define CFA_RESOURCE_TYPE_P45_NAT_D_IPV4      0xbUL
+/* Network Address Translation Source IPv6 address */
+#define CFA_RESOURCE_TYPE_P45_NAT_S_IPV6      0xcUL
+/* Network Address Translation Destination IPv6 address */
+#define CFA_RESOURCE_TYPE_P45_NAT_D_IPV6      0xdUL
 /* Meter */
-#define CFA_RESOURCE_TYPE_P45_SRAM_METER           0xeUL
+#define CFA_RESOURCE_TYPE_P45_METER           0xeUL
 /* Flow State */
-#define CFA_RESOURCE_TYPE_P45_SRAM_FLOW_STATE      0xfUL
+#define CFA_RESOURCE_TYPE_P45_FLOW_STATE      0xfUL
 /* Full Action Records */
-#define CFA_RESOURCE_TYPE_P45_SRAM_FULL_ACTION     0x10UL
+#define CFA_RESOURCE_TYPE_P45_FULL_ACTION     0x10UL
 /* Action Record Format 0 */
-#define CFA_RESOURCE_TYPE_P45_SRAM_FORMAT_0_ACTION 0x11UL
+#define CFA_RESOURCE_TYPE_P45_FORMAT_0_ACTION 0x11UL
 /* Action Record Format 2 */
-#define CFA_RESOURCE_TYPE_P45_SRAM_FORMAT_2_ACTION 0x12UL
+#define CFA_RESOURCE_TYPE_P45_FORMAT_2_ACTION 0x12UL
 /* Action Record Format 3 */
-#define CFA_RESOURCE_TYPE_P45_SRAM_FORMAT_3_ACTION 0x13UL
+#define CFA_RESOURCE_TYPE_P45_FORMAT_3_ACTION 0x13UL
 /* Action Record Format 4 */
-#define CFA_RESOURCE_TYPE_P45_SRAM_FORMAT_4_ACTION 0x14UL
+#define CFA_RESOURCE_TYPE_P45_FORMAT_4_ACTION 0x14UL
 /* L2 Context TCAM */
-#define CFA_RESOURCE_TYPE_P45_L2_CTXT_TCAM         0x15UL
+#define CFA_RESOURCE_TYPE_P45_L2_CTXT_TCAM    0x15UL
 /* Profile Func */
-#define CFA_RESOURCE_TYPE_P45_PROF_FUNC            0x16UL
+#define CFA_RESOURCE_TYPE_P45_PROF_FUNC       0x16UL
 /* Profile TCAM */
-#define CFA_RESOURCE_TYPE_P45_PROF_TCAM            0x17UL
+#define CFA_RESOURCE_TYPE_P45_PROF_TCAM       0x17UL
 /* Exact Match Profile Id */
-#define CFA_RESOURCE_TYPE_P45_EM_PROF_ID           0x18UL
+#define CFA_RESOURCE_TYPE_P45_EM_PROF_ID      0x18UL
 /* Exact Match Record */
-#define CFA_RESOURCE_TYPE_P45_EM_REC               0x19UL
+#define CFA_RESOURCE_TYPE_P45_EM_REC          0x19UL
 /* Wildcard Profile Id */
-#define CFA_RESOURCE_TYPE_P45_WC_TCAM_PROF_ID      0x1aUL
+#define CFA_RESOURCE_TYPE_P45_WC_TCAM_PROF_ID 0x1aUL
 /* Wildcard TCAM */
-#define CFA_RESOURCE_TYPE_P45_WC_TCAM              0x1bUL
+#define CFA_RESOURCE_TYPE_P45_WC_TCAM         0x1bUL
 /* Meter profile */
-#define CFA_RESOURCE_TYPE_P45_METER_PROF           0x1cUL
-/* Meter */
-#define CFA_RESOURCE_TYPE_P45_METER                0x1dUL
+#define CFA_RESOURCE_TYPE_P45_METER_PROF      0x1cUL
 /* Meter */
-#define CFA_RESOURCE_TYPE_P45_MIRROR               0x1eUL
+#define CFA_RESOURCE_TYPE_P45_MIRROR          0x1dUL
 /* Source Property TCAM */
-#define CFA_RESOURCE_TYPE_P45_SP_TCAM              0x1fUL
+#define CFA_RESOURCE_TYPE_P45_SP_TCAM         0x1eUL
 /* VEB TCAM */
-#define CFA_RESOURCE_TYPE_P45_VEB_TCAM             0x20UL
-#define CFA_RESOURCE_TYPE_P45_LAST                CFA_RESOURCE_TYPE_P45_VEB_TCAM
+#define CFA_RESOURCE_TYPE_P45_VEB_TCAM        0x1fUL
+#define CFA_RESOURCE_TYPE_P45_LAST           CFA_RESOURCE_TYPE_P45_VEB_TCAM
 
 
-/* SRAM Multicast Group */
-#define CFA_RESOURCE_TYPE_P4_SRAM_MCG             0x0UL
-/* SRAM Encap 8 byte record */
-#define CFA_RESOURCE_TYPE_P4_SRAM_ENCAP_8B        0x1UL
-/* SRAM Encap 16 byte record */
-#define CFA_RESOURCE_TYPE_P4_SRAM_ENCAP_16B       0x2UL
-/* SRAM Encap 64 byte record */
-#define CFA_RESOURCE_TYPE_P4_SRAM_ENCAP_64B       0x3UL
-/* SRAM Source Property MAC */
-#define CFA_RESOURCE_TYPE_P4_SRAM_SP_MAC          0x4UL
-/* SRAM Source Property MAC and IPv4 */
-#define CFA_RESOURCE_TYPE_P4_SRAM_SP_MAC_IPV4     0x5UL
-/* SRAM Source Property MAC and IPv6 */
-#define CFA_RESOURCE_TYPE_P4_SRAM_SP_MAC_IPV6     0x6UL
-/* SRAM 64B Counters */
-#define CFA_RESOURCE_TYPE_P4_SRAM_COUNTER_64B     0x7UL
-/* SRAM Network Address Translation Source Port */
-#define CFA_RESOURCE_TYPE_P4_SRAM_NAT_SPORT       0x8UL
-/* SRAM Network Address Translation Destination Port */
-#define CFA_RESOURCE_TYPE_P4_SRAM_NAT_DPORT       0x9UL
-/* SRAM Network Address Translation Source IPv4 address */
-#define CFA_RESOURCE_TYPE_P4_SRAM_NAT_S_IPV4      0xaUL
-/* SRAM Network Address Translation Destination IPv4 address */
-#define CFA_RESOURCE_TYPE_P4_SRAM_NAT_D_IPV4      0xbUL
-/* SRAM Network Address Translation Source IPv6 address */
-#define CFA_RESOURCE_TYPE_P4_SRAM_NAT_S_IPV6      0xcUL
-/* SRAM Network Address Translation Destination IPv6 address */
-#define CFA_RESOURCE_TYPE_P4_SRAM_NAT_D_IPV6      0xdUL
+/* Multicast Group */
+#define CFA_RESOURCE_TYPE_P4_MCG             0x0UL
+/* Encap 8 byte record */
+#define CFA_RESOURCE_TYPE_P4_ENCAP_8B        0x1UL
+/* Encap 16 byte record */
+#define CFA_RESOURCE_TYPE_P4_ENCAP_16B       0x2UL
+/* Encap 64 byte record */
+#define CFA_RESOURCE_TYPE_P4_ENCAP_64B       0x3UL
+/* Source Property MAC */
+#define CFA_RESOURCE_TYPE_P4_SP_MAC          0x4UL
+/* Source Property MAC and IPv4 */
+#define CFA_RESOURCE_TYPE_P4_SP_MAC_IPV4     0x5UL
+/* Source Property MAC and IPv6 */
+#define CFA_RESOURCE_TYPE_P4_SP_MAC_IPV6     0x6UL
+/* 64B Counters */
+#define CFA_RESOURCE_TYPE_P4_COUNTER_64B     0x7UL
+/* Network Address Translation Source Port */
+#define CFA_RESOURCE_TYPE_P4_NAT_SPORT       0x8UL
+/* Network Address Translation Destination Port */
+#define CFA_RESOURCE_TYPE_P4_NAT_DPORT       0x9UL
+/* Network Address Translation Source IPv4 address */
+#define CFA_RESOURCE_TYPE_P4_NAT_S_IPV4      0xaUL
+/* Network Address Translation Destination IPv4 address */
+#define CFA_RESOURCE_TYPE_P4_NAT_D_IPV4      0xbUL
+/* Network Address Translation Source IPv6 address */
+#define CFA_RESOURCE_TYPE_P4_NAT_S_IPV6      0xcUL
+/* Network Address Translation Destination IPv6 address */
+#define CFA_RESOURCE_TYPE_P4_NAT_D_IPV6      0xdUL
 /* Meter */
-#define CFA_RESOURCE_TYPE_P4_SRAM_METER           0xeUL
+#define CFA_RESOURCE_TYPE_P4_METER           0xeUL
 /* Flow State */
-#define CFA_RESOURCE_TYPE_P4_SRAM_FLOW_STATE      0xfUL
+#define CFA_RESOURCE_TYPE_P4_FLOW_STATE      0xfUL
 /* Full Action Records */
-#define CFA_RESOURCE_TYPE_P4_SRAM_FULL_ACTION     0x10UL
+#define CFA_RESOURCE_TYPE_P4_FULL_ACTION     0x10UL
 /* Action Record Format 0 */
-#define CFA_RESOURCE_TYPE_P4_SRAM_FORMAT_0_ACTION 0x11UL
+#define CFA_RESOURCE_TYPE_P4_FORMAT_0_ACTION 0x11UL
 /* Action Record Format 2 */
-#define CFA_RESOURCE_TYPE_P4_SRAM_FORMAT_2_ACTION 0x12UL
+#define CFA_RESOURCE_TYPE_P4_FORMAT_2_ACTION 0x12UL
 /* Action Record Format 3 */
-#define CFA_RESOURCE_TYPE_P4_SRAM_FORMAT_3_ACTION 0x13UL
+#define CFA_RESOURCE_TYPE_P4_FORMAT_3_ACTION 0x13UL
 /* Action Record Format 4 */
-#define CFA_RESOURCE_TYPE_P4_SRAM_FORMAT_4_ACTION 0x14UL
+#define CFA_RESOURCE_TYPE_P4_FORMAT_4_ACTION 0x14UL
 /* L2 Context TCAM */
-#define CFA_RESOURCE_TYPE_P4_L2_CTXT_TCAM         0x15UL
+#define CFA_RESOURCE_TYPE_P4_L2_CTXT_TCAM    0x15UL
 /* Profile Func */
-#define CFA_RESOURCE_TYPE_P4_PROF_FUNC            0x16UL
+#define CFA_RESOURCE_TYPE_P4_PROF_FUNC       0x16UL
 /* Profile TCAM */
-#define CFA_RESOURCE_TYPE_P4_PROF_TCAM            0x17UL
+#define CFA_RESOURCE_TYPE_P4_PROF_TCAM       0x17UL
 /* Exact Match Profile Id */
-#define CFA_RESOURCE_TYPE_P4_EM_PROF_ID           0x18UL
+#define CFA_RESOURCE_TYPE_P4_EM_PROF_ID      0x18UL
 /* Exact Match Record */
-#define CFA_RESOURCE_TYPE_P4_EM_REC               0x19UL
+#define CFA_RESOURCE_TYPE_P4_EM_REC          0x19UL
 /* Wildcard Profile Id */
-#define CFA_RESOURCE_TYPE_P4_WC_TCAM_PROF_ID      0x1aUL
+#define CFA_RESOURCE_TYPE_P4_WC_TCAM_PROF_ID 0x1aUL
 /* Wildcard TCAM */
-#define CFA_RESOURCE_TYPE_P4_WC_TCAM              0x1bUL
+#define CFA_RESOURCE_TYPE_P4_WC_TCAM         0x1bUL
 /* Meter profile */
-#define CFA_RESOURCE_TYPE_P4_METER_PROF           0x1cUL
-/* Meter */
-#define CFA_RESOURCE_TYPE_P4_METER                0x1dUL
+#define CFA_RESOURCE_TYPE_P4_METER_PROF      0x1cUL
 /* Meter */
-#define CFA_RESOURCE_TYPE_P4_MIRROR               0x1eUL
+#define CFA_RESOURCE_TYPE_P4_MIRROR          0x1dUL
 /* Source Property TCAM */
-#define CFA_RESOURCE_TYPE_P4_SP_TCAM              0x1fUL
-#define CFA_RESOURCE_TYPE_P4_LAST                CFA_RESOURCE_TYPE_P4_SP_TCAM
+#define CFA_RESOURCE_TYPE_P4_SP_TCAM         0x1eUL
+#define CFA_RESOURCE_TYPE_P4_LAST           CFA_RESOURCE_TYPE_P4_SP_TCAM
 
 
 #endif /* _CFA_RESOURCE_TYPES_H_ */
diff --git a/drivers/net/bnxt/tf_core/tf_common.h b/drivers/net/bnxt/tf_core/tf_common.h
index 2aa4b86..ec3bca8 100644
--- a/drivers/net/bnxt/tf_core/tf_common.h
+++ b/drivers/net/bnxt/tf_core/tf_common.h
@@ -51,4 +51,28 @@
 		} \
 	} while (0)
 
+
+#define TF_CHECK_PARMS1(parms) do {					\
+		if ((parms) == NULL) {					\
+			TFP_DRV_LOG(ERR, "Invalid Argument(s)\n");	\
+			return -EINVAL;					\
+		}							\
+	} while (0)
+
+#define TF_CHECK_PARMS2(parms1, parms2) do {				\
+		if ((parms1) == NULL || (parms2) == NULL) {		\
+			TFP_DRV_LOG(ERR, "Invalid Argument(s)\n");	\
+			return -EINVAL;					\
+		}							\
+	} while (0)
+
+#define TF_CHECK_PARMS3(parms1, parms2, parms3) do {			\
+		if ((parms1) == NULL ||					\
+		    (parms2) == NULL ||					\
+		    (parms3) == NULL) {					\
+			TFP_DRV_LOG(ERR, "Invalid Argument(s)\n");	\
+			return -EINVAL;					\
+		}							\
+	} while (0)
+
 #endif /* _TF_COMMON_H_ */
diff --git a/drivers/net/bnxt/tf_core/tf_core.c b/drivers/net/bnxt/tf_core/tf_core.c
index 0098690..28a6bbd 100644
--- a/drivers/net/bnxt/tf_core/tf_core.c
+++ b/drivers/net/bnxt/tf_core/tf_core.c
@@ -85,7 +85,7 @@ tf_create_em_pool(struct tf_session *session,
 
 	/* Create empty stack
 	 */
-	rc = stack_init(num_entries, parms.mem_va, pool);
+	rc = stack_init(num_entries, (uint32_t *)parms.mem_va, pool);
 
 	if (rc != 0) {
 		TFP_DRV_LOG(ERR, "EM pool stack init failure %s\n",
@@ -231,7 +231,6 @@ tf_open_session(struct tf                    *tfp,
 		   TF_SESSION_NAME_MAX);
 
 	/* Initialize Session */
-	session->device_type = parms->device_type;
 	session->dev = NULL;
 	tf_rm_init(tfp);
 
@@ -276,7 +275,9 @@ tf_open_session(struct tf                    *tfp,
 
 	/* Initialize EM pool */
 	for (dir = 0; dir < TF_DIR_MAX; dir++) {
-		rc = tf_create_em_pool(session, dir, TF_SESSION_EM_POOL_SIZE);
+		rc = tf_create_em_pool(session,
+				       (enum tf_dir)dir,
+				       TF_SESSION_EM_POOL_SIZE);
 		if (rc) {
 			TFP_DRV_LOG(ERR,
 				    "EM Pool initialization failed\n");
@@ -314,6 +315,64 @@ tf_open_session(struct tf                    *tfp,
 }
 
 int
+tf_open_session_new(struct tf *tfp,
+		    struct tf_open_session_parms *parms)
+{
+	int rc;
+	unsigned int domain, bus, slot, device;
+	struct tf_session_open_session_parms oparms;
+
+	TF_CHECK_PARMS(tfp, parms);
+
+	/* Filter out any non-supported device types on the Core
+	 * side. It is assumed that the Firmware will be supported if
+	 * firmware open session succeeds.
+	 */
+	if (parms->device_type != TF_DEVICE_TYPE_WH) {
+		TFP_DRV_LOG(ERR,
+			    "Unsupported device type %d\n",
+			    parms->device_type);
+		return -ENOTSUP;
+	}
+
+	/* Verify control channel and build the beginning of session_id */
+	rc = sscanf(parms->ctrl_chan_name,
+		    "%x:%x:%x.%d",
+		    &domain,
+		    &bus,
+		    &slot,
+		    &device);
+	if (rc != 4) {
+		TFP_DRV_LOG(ERR,
+			    "Failed to scan device ctrl_chan_name\n");
+		return -EINVAL;
+	}
+
+	parms->session_id.internal.domain = domain;
+	parms->session_id.internal.bus = bus;
+	parms->session_id.internal.device = device;
+	oparms.open_cfg = parms;
+
+	rc = tf_session_open_session(tfp, &oparms);
+	/* Logging handled by tf_session_open_session */
+	if (rc)
+		return rc;
+
+	TFP_DRV_LOG(INFO,
+		    "Session created, session_id:%d\n",
+		    parms->session_id.id);
+
+	TFP_DRV_LOG(INFO,
+		    "domain:%d, bus:%d, device:%d, fw_session_id:%d\n",
+		    parms->session_id.internal.domain,
+		    parms->session_id.internal.bus,
+		    parms->session_id.internal.device,
+		    parms->session_id.internal.fw_session_id);
+
+	return 0;
+}
+
+int
 tf_attach_session(struct tf *tfp __rte_unused,
 		  struct tf_attach_session_parms *parms __rte_unused)
 {
@@ -342,6 +401,69 @@ tf_attach_session(struct tf *tfp __rte_unused,
 }
 
 int
+tf_attach_session_new(struct tf *tfp,
+		      struct tf_attach_session_parms *parms)
+{
+	int rc;
+	unsigned int domain, bus, slot, device;
+	struct tf_session_attach_session_parms aparms;
+
+	TF_CHECK_PARMS2(tfp, parms);
+
+	/* Verify control channel */
+	rc = sscanf(parms->ctrl_chan_name,
+		    "%x:%x:%x.%d",
+		    &domain,
+		    &bus,
+		    &slot,
+		    &device);
+	if (rc != 4) {
+		TFP_DRV_LOG(ERR,
+			    "Failed to scan device ctrl_chan_name\n");
+		return -EINVAL;
+	}
+
+	/* Verify 'attach' channel */
+	rc = sscanf(parms->attach_chan_name,
+		    "%x:%x:%x.%d",
+		    &domain,
+		    &bus,
+		    &slot,
+		    &device);
+	if (rc != 4) {
+		TFP_DRV_LOG(ERR,
+			    "Failed to scan device attach_chan_name\n");
+		return -EINVAL;
+	}
+
+	/* Prepare return value of session_id, using ctrl_chan_name
+	 * device values as it becomes the session id.
+	 */
+	parms->session_id.internal.domain = domain;
+	parms->session_id.internal.bus = bus;
+	parms->session_id.internal.device = device;
+	aparms.attach_cfg = parms;
+	rc = tf_session_attach_session(tfp,
+				       &aparms);
+	/* Logging handled by dev_bind */
+	if (rc)
+		return rc;
+
+	TFP_DRV_LOG(INFO,
+		    "Attached to session, session_id:%d\n",
+		    parms->session_id.id);
+
+	TFP_DRV_LOG(INFO,
+		    "domain:%d, bus:%d, device:%d, fw_session_id:%d\n",
+		    parms->session_id.internal.domain,
+		    parms->session_id.internal.bus,
+		    parms->session_id.internal.device,
+		    parms->session_id.internal.fw_session_id);
+
+	return rc;
+}
+
+int
 tf_close_session(struct tf *tfp)
 {
 	int rc;
@@ -380,7 +502,7 @@ tf_close_session(struct tf *tfp)
 	if (tfs->ref_count == 0) {
 		/* Free EM pool */
 		for (dir = 0; dir < TF_DIR_MAX; dir++)
-			tf_free_em_pool(tfs, dir);
+			tf_free_em_pool(tfs, (enum tf_dir)dir);
 
 		tfp_free(tfp->session->core_data);
 		tfp_free(tfp->session);
@@ -401,6 +523,39 @@ tf_close_session(struct tf *tfp)
 	return rc_close;
 }
 
+int
+tf_close_session_new(struct tf *tfp)
+{
+	int rc;
+	struct tf_session_close_session_parms cparms = { 0 };
+	union tf_session_id session_id = { 0 };
+	uint8_t ref_count;
+
+	TF_CHECK_PARMS1(tfp);
+
+	cparms.ref_count = &ref_count;
+	cparms.session_id = &session_id;
+	rc = tf_session_close_session(tfp,
+				      &cparms);
+	/* Logging handled by tf_session_close_session */
+	if (rc)
+		return rc;
+
+	TFP_DRV_LOG(INFO,
+		    "Closed session, session_id:%d, ref_count:%d\n",
+		    cparms.session_id->id,
+		    *cparms.ref_count);
+
+	TFP_DRV_LOG(INFO,
+		    "domain:%d, bus:%d, device:%d, fw_session_id:%d\n",
+		    cparms.session_id->internal.domain,
+		    cparms.session_id->internal.bus,
+		    cparms.session_id->internal.device,
+		    cparms.session_id->internal.fw_session_id);
+
+	return rc;
+}
+
 /** insert EM hash entry API
  *
  *    returns:
@@ -539,10 +694,67 @@ int tf_alloc_identifier(struct tf *tfp,
 	return 0;
 }
 
-/** free identifier resource
- *
- * Returns success or failure code.
- */
+int
+tf_alloc_identifier_new(struct tf *tfp,
+			struct tf_alloc_identifier_parms *parms)
+{
+	int rc;
+	struct tf_session *tfs;
+	struct tf_dev_info *dev;
+	struct tf_ident_alloc_parms aparms;
+	uint16_t id;
+
+	TF_CHECK_PARMS2(tfp, parms);
+
+	/* Can't do static initialization due to UT enum check */
+	memset(&aparms, 0, sizeof(struct tf_ident_alloc_parms));
+
+	/* Retrieve the session information */
+	rc = tf_session_get_session(tfp, &tfs);
+	if (rc) {
+		TFP_DRV_LOG(ERR,
+			    "%s: Failed to lookup session, rc:%s\n",
+			    tf_dir_2_str(parms->dir),
+			    strerror(-rc));
+		return rc;
+	}
+
+	/* Retrieve the device information */
+	rc = tf_session_get_device(tfs, &dev);
+	if (rc) {
+		TFP_DRV_LOG(ERR,
+			    "%s: Failed to lookup device, rc:%s\n",
+			    tf_dir_2_str(parms->dir),
+			    strerror(-rc));
+		return rc;
+	}
+
+	if (dev->ops->tf_dev_alloc_ident == NULL) {
+		rc = -EOPNOTSUPP;
+		TFP_DRV_LOG(ERR,
+			    "%s: Operation not supported, rc:%s\n",
+			    tf_dir_2_str(parms->dir),
+			    strerror(-rc));
+		return -EOPNOTSUPP;
+	}
+
+	aparms.dir = parms->dir;
+	aparms.ident_type = parms->ident_type;
+	aparms.id = &id;
+	rc = dev->ops->tf_dev_alloc_ident(tfp, &aparms);
+	if (rc) {
+		TFP_DRV_LOG(ERR,
+			    "%s: Identifier allocation failed, rc:%s\n",
+			    tf_dir_2_str(parms->dir),
+			    strerror(-rc));
+		return rc;
+	}
+
+	parms->id = id;
+
+	return 0;
+}
+
 int tf_free_identifier(struct tf *tfp,
 		       struct tf_free_identifier_parms *parms)
 {
@@ -619,6 +831,64 @@ int tf_free_identifier(struct tf *tfp,
 }
 
 int
+tf_free_identifier_new(struct tf *tfp,
+		       struct tf_free_identifier_parms *parms)
+{
+	int rc;
+	struct tf_session *tfs;
+	struct tf_dev_info *dev;
+	struct tf_ident_free_parms fparms;
+
+	TF_CHECK_PARMS2(tfp, parms);
+
+	/* Can't do static initialization due to UT enum check */
+	memset(&fparms, 0, sizeof(struct tf_ident_free_parms));
+
+	/* Retrieve the session information */
+	rc = tf_session_get_session(tfp, &tfs);
+	if (rc) {
+		TFP_DRV_LOG(ERR,
+			    "%s: Failed to lookup session, rc:%s\n",
+			    tf_dir_2_str(parms->dir),
+			    strerror(-rc));
+		return rc;
+	}
+
+	/* Retrieve the device information */
+	rc = tf_session_get_device(tfs, &dev);
+	if (rc) {
+		TFP_DRV_LOG(ERR,
+			    "%s: Failed to lookup device, rc:%s\n",
+			    tf_dir_2_str(parms->dir),
+			    strerror(-rc));
+		return rc;
+	}
+
+	if (dev->ops->tf_dev_free_ident == NULL) {
+		rc = -EOPNOTSUPP;
+		TFP_DRV_LOG(ERR,
+			    "%s: Operation not supported, rc:%s\n",
+			    tf_dir_2_str(parms->dir),
+			    strerror(-rc));
+		return -EOPNOTSUPP;
+	}
+
+	fparms.dir = parms->dir;
+	fparms.ident_type = parms->ident_type;
+	fparms.id = parms->id;
+	rc = dev->ops->tf_dev_free_ident(tfp, &fparms);
+	if (rc) {
+		TFP_DRV_LOG(ERR,
+			    "%s: Identifier allocation failed, rc:%s\n",
+			    tf_dir_2_str(parms->dir),
+			    strerror(-rc));
+		return rc;
+	}
+
+	return 0;
+}
+
+int
 tf_alloc_tcam_entry(struct tf *tfp,
 		    struct tf_alloc_tcam_entry_parms *parms)
 {
diff --git a/drivers/net/bnxt/tf_core/tf_core.h b/drivers/net/bnxt/tf_core/tf_core.h
index 96a1a79..74ed24e 100644
--- a/drivers/net/bnxt/tf_core/tf_core.h
+++ b/drivers/net/bnxt/tf_core/tf_core.h
@@ -380,7 +380,7 @@ struct tf_session_resources {
 	 * The number of identifier resources requested for the session.
 	 * The index used is tf_identifier_type.
 	 */
-	uint16_t identifer_cnt[TF_DIR_MAX][TF_IDENT_TYPE_MAX];
+	uint16_t identifier_cnt[TF_IDENT_TYPE_MAX][TF_DIR_MAX];
 	/** [in] Requested Index Table resource counts
 	 *
 	 * The number of index table resources requested for the session.
@@ -480,6 +480,9 @@ struct tf_open_session_parms {
 int tf_open_session(struct tf *tfp,
 		    struct tf_open_session_parms *parms);
 
+int tf_open_session_new(struct tf *tfp,
+			struct tf_open_session_parms *parms);
+
 struct tf_attach_session_parms {
 	/** [in] ctrl_chan_name
 	 *
@@ -542,6 +545,8 @@ struct tf_attach_session_parms {
  */
 int tf_attach_session(struct tf *tfp,
 		      struct tf_attach_session_parms *parms);
+int tf_attach_session_new(struct tf *tfp,
+			  struct tf_attach_session_parms *parms);
 
 /**
  * Closes an existing session. Cleans up all hardware and firmware
@@ -551,6 +556,7 @@ int tf_attach_session(struct tf *tfp,
  * Returns success or failure code.
  */
 int tf_close_session(struct tf *tfp);
+int tf_close_session_new(struct tf *tfp);
 
 /**
  * @page  ident Identity Management
@@ -602,6 +608,8 @@ struct tf_free_identifier_parms {
  */
 int tf_alloc_identifier(struct tf *tfp,
 			struct tf_alloc_identifier_parms *parms);
+int tf_alloc_identifier_new(struct tf *tfp,
+			    struct tf_alloc_identifier_parms *parms);
 
 /** free identifier resource
  *
@@ -613,6 +621,8 @@ int tf_alloc_identifier(struct tf *tfp,
  */
 int tf_free_identifier(struct tf *tfp,
 		       struct tf_free_identifier_parms *parms);
+int tf_free_identifier_new(struct tf *tfp,
+			   struct tf_free_identifier_parms *parms);
 
 /**
  * @page dram_table DRAM Table Scope Interface
diff --git a/drivers/net/bnxt/tf_core/tf_device.c b/drivers/net/bnxt/tf_core/tf_device.c
index 3b36831..4c46cad 100644
--- a/drivers/net/bnxt/tf_core/tf_device.c
+++ b/drivers/net/bnxt/tf_core/tf_device.c
@@ -6,45 +6,169 @@
 #include "tf_device.h"
 #include "tf_device_p4.h"
 #include "tfp.h"
-#include "bnxt.h"
 
 struct tf;
 
+/* Forward declarations */
+static int dev_unbind_p4(struct tf *tfp);
+
 /**
- * Device specific bind function
+ * Device specific bind function, WH+
+ *
+ * [in] tfp
+ *   Pointer to TF handle
+ *
+ * [in] shadow_copy
+ *   Flag controlling shadow copy DB creation
+ *
+ * [in] resources
+ *   Pointer to resource allocation information
+ *
+ * [out] dev_handle
+ *   Device handle
+ *
+ * Returns
+ *   - (0) if successful.
+ *   - (-EINVAL) on parameter or internal failure.
  */
 static int
-dev_bind_p4(struct tf *tfp __rte_unused,
-	    struct tf_session_resources *resources __rte_unused,
-	    struct tf_dev_info *dev_info)
+dev_bind_p4(struct tf *tfp,
+	    bool shadow_copy,
+	    struct tf_session_resources *resources,
+	    struct tf_dev_info *dev_handle)
 {
+	int rc;
+	int frc;
+	struct tf_ident_cfg_parms ident_cfg;
+	struct tf_tbl_cfg_parms tbl_cfg;
+	struct tf_tcam_cfg_parms tcam_cfg;
+
 	/* Initialize the modules */
 
-	dev_info->ops = &tf_dev_ops_p4;
+	ident_cfg.num_elements = TF_IDENT_TYPE_MAX;
+	ident_cfg.cfg = tf_ident_p4;
+	ident_cfg.shadow_copy = shadow_copy;
+	ident_cfg.resources = resources;
+	rc = tf_ident_bind(tfp, &ident_cfg);
+	if (rc) {
+		TFP_DRV_LOG(ERR,
+			    "Identifier initialization failure\n");
+		goto fail;
+	}
+
+	tbl_cfg.num_elements = TF_TBL_TYPE_MAX;
+	tbl_cfg.cfg = tf_tbl_p4;
+	tbl_cfg.shadow_copy = shadow_copy;
+	tbl_cfg.resources = resources;
+	rc = tf_tbl_bind(tfp, &tbl_cfg);
+	if (rc) {
+		TFP_DRV_LOG(ERR,
+			    "Table initialization failure\n");
+		goto fail;
+	}
+
+	tcam_cfg.num_elements = TF_TCAM_TBL_TYPE_MAX;
+	tcam_cfg.cfg = tf_tcam_p4;
+	tcam_cfg.shadow_copy = shadow_copy;
+	tcam_cfg.resources = resources;
+	rc = tf_tcam_bind(tfp, &tcam_cfg);
+	if (rc) {
+		TFP_DRV_LOG(ERR,
+			    "TCAM initialization failure\n");
+		goto fail;
+	}
+
+	dev_handle->type = TF_DEVICE_TYPE_WH;
+	dev_handle->ops = &tf_dev_ops_p4;
+
 	return 0;
+
+ fail:
+	/* Cleanup of already created modules */
+	frc = dev_unbind_p4(tfp);
+	if (frc)
+		return frc;
+
+	return rc;
+}
+
+/**
+ * Device specific unbind function, WH+
+ *
+ * [in] tfp
+ *   Pointer to TF handle
+ *
+ * Returns
+ *   - (0) if successful.
+ *   - (-EINVAL) on failure.
+ */
+static int
+dev_unbind_p4(struct tf *tfp)
+{
+	int rc = 0;
+	bool fail = false;
+
+	/* Unbind all the support modules. As this is only done on
+	 * close we only report errors as everything has to be cleaned
+	 * up regardless.
+	 */
+	rc = tf_ident_unbind(tfp);
+	if (rc) {
+		TFP_DRV_LOG(ERR,
+			    "Device unbind failed, Identifier\n");
+		fail = true;
+	}
+
+	rc = tf_tbl_unbind(tfp);
+	if (rc) {
+		TFP_DRV_LOG(ERR,
+			    "Device unbind failed, Table Type\n");
+		fail = true;
+	}
+
+	rc = tf_tcam_unbind(tfp);
+	if (rc) {
+		TFP_DRV_LOG(ERR,
+			    "Device unbind failed, TCAM\n");
+		fail = true;
+	}
+
+	if (fail)
+		return -1;
+
+	return rc;
 }
 
 int
 dev_bind(struct tf *tfp __rte_unused,
 	 enum tf_device_type type,
+	 bool shadow_copy,
 	 struct tf_session_resources *resources,
-	 struct tf_dev_info *dev_info)
+	 struct tf_dev_info *dev_handle)
 {
 	switch (type) {
 	case TF_DEVICE_TYPE_WH:
 		return dev_bind_p4(tfp,
+				   shadow_copy,
 				   resources,
-				   dev_info);
+				   dev_handle);
 	default:
 		TFP_DRV_LOG(ERR,
-			    "Device type not supported\n");
-		return -ENOTSUP;
+			    "No such device\n");
+		return -ENODEV;
 	}
 }
 
 int
-dev_unbind(struct tf *tfp __rte_unused,
-	   struct tf_dev_info *dev_handle __rte_unused)
+dev_unbind(struct tf *tfp,
+	   struct tf_dev_info *dev_handle)
 {
-	return 0;
+	switch (dev_handle->type) {
+	case TF_DEVICE_TYPE_WH:
+		return dev_unbind_p4(tfp);
+	default:
+		TFP_DRV_LOG(ERR,
+			    "No such device\n");
+		return -ENODEV;
+	}
 }
diff --git a/drivers/net/bnxt/tf_core/tf_device.h b/drivers/net/bnxt/tf_core/tf_device.h
index 8b63ff1..6aeb6fe 100644
--- a/drivers/net/bnxt/tf_core/tf_device.h
+++ b/drivers/net/bnxt/tf_core/tf_device.h
@@ -27,6 +27,7 @@ struct tf_session;
  * TF device information
  */
 struct tf_dev_info {
+	enum tf_device_type type;
 	const struct tf_dev_ops *ops;
 };
 
@@ -56,10 +57,12 @@ struct tf_dev_info {
  *
  * Returns
  *   - (0) if successful.
- *   - (-EINVAL) on failure.
+ *   - (-EINVAL) parameter failure.
+ *   - (-ENODEV) no such device supported.
  */
 int dev_bind(struct tf *tfp,
 	     enum tf_device_type type,
+	     bool shadow_copy,
 	     struct tf_session_resources *resources,
 	     struct tf_dev_info *dev_handle);
 
@@ -71,6 +74,11 @@ int dev_bind(struct tf *tfp,
  *
  * [in] dev_handle
  *   Device handle
+ *
+ * Returns
+ *   - (0) if successful.
+ *   - (-EINVAL) parameter failure.
+ *   - (-ENODEV) no such device supported.
  */
 int dev_unbind(struct tf *tfp,
 	       struct tf_dev_info *dev_handle);
@@ -85,6 +93,44 @@ int dev_unbind(struct tf *tfp,
  */
 struct tf_dev_ops {
 	/**
+	 * Retrives the MAX number of resource types that the device
+	 * supports.
+	 *
+	 * [in] tfp
+	 *   Pointer to TF handle
+	 *
+	 * [out] max_types
+	 *   Pointer to MAX number of types the device supports
+	 *
+	 * Returns
+	 *   - (0) if successful.
+	 *   - (-EINVAL) on failure.
+	 */
+	int (*tf_dev_get_max_types)(struct tf *tfp,
+				    uint16_t *max_types);
+
+	/**
+	 * Retrieves the WC TCAM slice information that the device
+	 * supports.
+	 *
+	 * [in] tfp
+	 *   Pointer to TF handle
+	 *
+	 * [out] slice_size
+	 *   Pointer to slice size the device supports
+	 *
+	 * [out] num_slices_per_row
+	 *   Pointer to number of slices per row the device supports
+	 *
+	 * Returns
+	 *   - (0) if successful.
+	 *   - (-EINVAL) on failure.
+	 */
+	int (*tf_dev_get_wc_tcam_slices)(struct tf *tfp,
+					 uint16_t *slice_size,
+					 uint16_t *num_slices_per_row);
+
+	/**
 	 * Allocation of an identifier element.
 	 *
 	 * This API allocates the specified identifier element from a
@@ -134,14 +180,14 @@ struct tf_dev_ops {
 	 *   Pointer to TF handle
 	 *
 	 * [in] parms
-	 *   Pointer to table type allocation parameters
+	 *   Pointer to table allocation parameters
 	 *
 	 * Returns
 	 *   - (0) if successful.
 	 *   - (-EINVAL) on failure.
 	 */
-	int (*tf_dev_alloc_tbl_type)(struct tf *tfp,
-				     struct tf_tbl_type_alloc_parms *parms);
+	int (*tf_dev_alloc_tbl)(struct tf *tfp,
+				struct tf_tbl_alloc_parms *parms);
 
 	/**
 	 * Free of a table type element.
@@ -153,14 +199,14 @@ struct tf_dev_ops {
 	 *   Pointer to TF handle
 	 *
 	 * [in] parms
-	 *   Pointer to table type free parameters
+	 *   Pointer to table free parameters
 	 *
 	 * Returns
 	 *   - (0) if successful.
 	 *   - (-EINVAL) on failure.
 	 */
-	int (*tf_dev_free_tbl_type)(struct tf *tfp,
-				    struct tf_tbl_type_free_parms *parms);
+	int (*tf_dev_free_tbl)(struct tf *tfp,
+			       struct tf_tbl_free_parms *parms);
 
 	/**
 	 * Searches for the specified table type element in a shadow DB.
@@ -175,15 +221,14 @@ struct tf_dev_ops {
 	 *   Pointer to TF handle
 	 *
 	 * [in] parms
-	 *   Pointer to table type allocation and search parameters
+	 *   Pointer to table allocation and search parameters
 	 *
 	 * Returns
 	 *   - (0) if successful.
 	 *   - (-EINVAL) on failure.
 	 */
-	int (*tf_dev_alloc_search_tbl_type)
-			(struct tf *tfp,
-			struct tf_tbl_type_alloc_search_parms *parms);
+	int (*tf_dev_alloc_search_tbl)(struct tf *tfp,
+				       struct tf_tbl_alloc_search_parms *parms);
 
 	/**
 	 * Sets the specified table type element.
@@ -195,14 +240,14 @@ struct tf_dev_ops {
 	 *   Pointer to TF handle
 	 *
 	 * [in] parms
-	 *   Pointer to table type set parameters
+	 *   Pointer to table set parameters
 	 *
 	 * Returns
 	 *   - (0) if successful.
 	 *   - (-EINVAL) on failure.
 	 */
-	int (*tf_dev_set_tbl_type)(struct tf *tfp,
-				   struct tf_tbl_type_set_parms *parms);
+	int (*tf_dev_set_tbl)(struct tf *tfp,
+			      struct tf_tbl_set_parms *parms);
 
 	/**
 	 * Retrieves the specified table type element.
@@ -214,14 +259,14 @@ struct tf_dev_ops {
 	 *   Pointer to TF handle
 	 *
 	 * [in] parms
-	 *   Pointer to table type get parameters
+	 *   Pointer to table get parameters
 	 *
 	 * Returns
 	 *   - (0) if successful.
 	 *   - (-EINVAL) on failure.
 	 */
-	int (*tf_dev_get_tbl_type)(struct tf *tfp,
-				   struct tf_tbl_type_get_parms *parms);
+	int (*tf_dev_get_tbl)(struct tf *tfp,
+			       struct tf_tbl_get_parms *parms);
 
 	/**
 	 * Allocation of a tcam element.
diff --git a/drivers/net/bnxt/tf_core/tf_device_p4.c b/drivers/net/bnxt/tf_core/tf_device_p4.c
index c3c4d1e..c235976 100644
--- a/drivers/net/bnxt/tf_core/tf_device_p4.c
+++ b/drivers/net/bnxt/tf_core/tf_device_p4.c
@@ -3,19 +3,87 @@
  * All rights reserved.
  */
 
+#include <rte_common.h>
+#include <cfa_resource_types.h>
+
 #include "tf_device.h"
 #include "tf_identifier.h"
 #include "tf_tbl_type.h"
 #include "tf_tcam.h"
 
+/**
+ * Device specific function that retrieves the MAX number of HCAPI
+ * types the device supports.
+ *
+ * [in] tfp
+ *   Pointer to TF handle
+ *
+ * [out] max_types
+ *   Pointer to the MAX number of HCAPI types supported
+ *
+ * Returns
+ *   - (0) if successful.
+ *   - (-EINVAL) on failure.
+ */
+static int
+tf_dev_p4_get_max_types(struct tf *tfp __rte_unused,
+			uint16_t *max_types)
+{
+	if (max_types == NULL)
+		return -EINVAL;
+
+	*max_types = CFA_RESOURCE_TYPE_P4_LAST + 1;
+
+	return 0;
+}
+
+/**
+ * Device specific function that retrieves the WC TCAM slices the
+ * device supports.
+ *
+ * [in] tfp
+ *   Pointer to TF handle
+ *
+ * [out] slice_size
+ *   Pointer to the WC TCAM slice size
+ *
+ * [out] num_slices_per_row
+ *   Pointer to the WC TCAM row slice configuration
+ *
+ * Returns
+ *   - (0) if successful.
+ *   - (-EINVAL) on failure.
+ */
+static int
+tf_dev_p4_get_wc_tcam_slices(struct tf *tfp __rte_unused,
+			     uint16_t *slice_size,
+			     uint16_t *num_slices_per_row)
+{
+#define CFA_P4_WC_TCAM_SLICE_SIZE       12
+#define CFA_P4_WC_TCAM_SLICES_PER_ROW    2
+
+	if (slice_size == NULL || num_slices_per_row == NULL)
+		return -EINVAL;
+
+	*slice_size = CFA_P4_WC_TCAM_SLICE_SIZE;
+	*num_slices_per_row = CFA_P4_WC_TCAM_SLICES_PER_ROW;
+
+	return 0;
+}
+
+/**
+ * Truflow P4 device specific functions
+ */
 const struct tf_dev_ops tf_dev_ops_p4 = {
+	.tf_dev_get_max_types = tf_dev_p4_get_max_types,
+	.tf_dev_get_wc_tcam_slices = tf_dev_p4_get_wc_tcam_slices,
 	.tf_dev_alloc_ident = tf_ident_alloc,
 	.tf_dev_free_ident = tf_ident_free,
-	.tf_dev_alloc_tbl_type = tf_tbl_type_alloc,
-	.tf_dev_free_tbl_type = tf_tbl_type_free,
-	.tf_dev_alloc_search_tbl_type = tf_tbl_type_alloc_search,
-	.tf_dev_set_tbl_type = tf_tbl_type_set,
-	.tf_dev_get_tbl_type = tf_tbl_type_get,
+	.tf_dev_alloc_tbl = tf_tbl_alloc,
+	.tf_dev_free_tbl = tf_tbl_free,
+	.tf_dev_alloc_search_tbl = tf_tbl_alloc_search,
+	.tf_dev_set_tbl = tf_tbl_set,
+	.tf_dev_get_tbl = tf_tbl_get,
 	.tf_dev_alloc_tcam = tf_tcam_alloc,
 	.tf_dev_free_tcam = tf_tcam_free,
 	.tf_dev_alloc_search_tcam = tf_tcam_alloc_search,
diff --git a/drivers/net/bnxt/tf_core/tf_device_p4.h b/drivers/net/bnxt/tf_core/tf_device_p4.h
index 84d90e3..5cd02b2 100644
--- a/drivers/net/bnxt/tf_core/tf_device_p4.h
+++ b/drivers/net/bnxt/tf_core/tf_device_p4.h
@@ -12,11 +12,12 @@
 #include "tf_rm_new.h"
 
 struct tf_rm_element_cfg tf_ident_p4[TF_IDENT_TYPE_MAX] = {
-	{ TF_RM_ELEM_CFG_PRIVATE, 0 /* CFA_RESOURCE_TYPE_P4_INVALID */ },
+	{ TF_RM_ELEM_CFG_PRIVATE, CFA_RESOURCE_TYPE_INVALID },
 	{ TF_RM_ELEM_CFG_HCAPI, CFA_RESOURCE_TYPE_P4_PROF_FUNC },
 	{ TF_RM_ELEM_CFG_HCAPI, CFA_RESOURCE_TYPE_P4_WC_TCAM_PROF_ID },
 	{ TF_RM_ELEM_CFG_HCAPI, CFA_RESOURCE_TYPE_P4_EM_PROF_ID },
-	{ TF_RM_ELEM_CFG_NULL, 0    /* CFA_RESOURCE_TYPE_P4_L2_FUNC */ }
+	/* CFA_RESOURCE_TYPE_P4_L2_FUNC */
+	{ TF_RM_ELEM_CFG_NULL, CFA_RESOURCE_TYPE_INVALID }
 };
 
 struct tf_rm_element_cfg tf_tcam_p4[TF_TCAM_TBL_TYPE_MAX] = {
@@ -24,41 +25,57 @@ struct tf_rm_element_cfg tf_tcam_p4[TF_TCAM_TBL_TYPE_MAX] = {
 	{ TF_RM_ELEM_CFG_HCAPI, CFA_RESOURCE_TYPE_P4_PROF_TCAM },
 	{ TF_RM_ELEM_CFG_HCAPI, CFA_RESOURCE_TYPE_P4_WC_TCAM },
 	{ TF_RM_ELEM_CFG_HCAPI, CFA_RESOURCE_TYPE_P4_SP_TCAM },
-	{ TF_RM_ELEM_CFG_NULL, 0 /* CFA_RESOURCE_TYPE_P4_CT_RULE_TCAM */ },
-	{ TF_RM_ELEM_CFG_NULL, 0  /* CFA_RESOURCE_TYPE_P4_VEB_TCAM */ }
+	/* CFA_RESOURCE_TYPE_P4_CT_RULE_TCAM */
+	{ TF_RM_ELEM_CFG_NULL, CFA_RESOURCE_TYPE_INVALID },
+	/* CFA_RESOURCE_TYPE_P4_VEB_TCAM */
+	{ TF_RM_ELEM_CFG_NULL, CFA_RESOURCE_TYPE_INVALID }
 };
 
 struct tf_rm_element_cfg tf_tbl_p4[TF_TBL_TYPE_MAX] = {
-	{ TF_RM_ELEM_CFG_HCAPI, CFA_RESOURCE_TYPE_P4_SRAM_FULL_ACTION },
-	{ TF_RM_ELEM_CFG_HCAPI, CFA_RESOURCE_TYPE_P4_SRAM_MCG },
-	{ TF_RM_ELEM_CFG_HCAPI, CFA_RESOURCE_TYPE_P4_SRAM_ENCAP_8B },
-	{ TF_RM_ELEM_CFG_HCAPI, CFA_RESOURCE_TYPE_P4_SRAM_ENCAP_16B },
-	{ TF_RM_ELEM_CFG_NULL, 0, /* CFA_RESOURCE_TYPE_P4_SRAM_ENCAP_32B */ },
-	{ TF_RM_ELEM_CFG_HCAPI, CFA_RESOURCE_TYPE_P4_SRAM_ENCAP_64B },
-	{ TF_RM_ELEM_CFG_HCAPI, CFA_RESOURCE_TYPE_P4_SRAM_SP_MAC },
-	{ TF_RM_ELEM_CFG_NULL, 0 /* CFA_RESOURCE_TYPE_P4_SRAM_SP_SMAC_IPV4 */ },
-	{ TF_RM_ELEM_CFG_NULL, 0 /* CFA_RESOURCE_TYPE_P4_SRAM_SP_SMAC_IPV6 */ },
-	{ TF_RM_ELEM_CFG_HCAPI, CFA_RESOURCE_TYPE_P4_SRAM_COUNTER_64B },
-	{ TF_RM_ELEM_CFG_HCAPI, CFA_RESOURCE_TYPE_P4_SRAM_NAT_SPORT },
-	{ TF_RM_ELEM_CFG_HCAPI, CFA_RESOURCE_TYPE_P4_SRAM_NAT_DPORT },
-	{ TF_RM_ELEM_CFG_HCAPI, CFA_RESOURCE_TYPE_P4_SRAM_NAT_S_IPV4 },
-	{ TF_RM_ELEM_CFG_HCAPI, CFA_RESOURCE_TYPE_P4_SRAM_NAT_D_IPV4 },
-	{ TF_RM_ELEM_CFG_HCAPI, CFA_RESOURCE_TYPE_P4_SRAM_NAT_S_IPV6 },
-	{ TF_RM_ELEM_CFG_HCAPI, CFA_RESOURCE_TYPE_P4_SRAM_NAT_D_IPV6 },
+	{ TF_RM_ELEM_CFG_HCAPI, CFA_RESOURCE_TYPE_P4_FULL_ACTION },
+	{ TF_RM_ELEM_CFG_HCAPI, CFA_RESOURCE_TYPE_P4_MCG },
+	{ TF_RM_ELEM_CFG_HCAPI, CFA_RESOURCE_TYPE_P4_ENCAP_8B },
+	{ TF_RM_ELEM_CFG_HCAPI, CFA_RESOURCE_TYPE_P4_ENCAP_16B },
+	/* CFA_RESOURCE_TYPE_P4_SRAM_ENCAP_32B */
+	{ TF_RM_ELEM_CFG_NULL, CFA_RESOURCE_TYPE_INVALID },
+	{ TF_RM_ELEM_CFG_HCAPI, CFA_RESOURCE_TYPE_P4_ENCAP_64B },
+	{ TF_RM_ELEM_CFG_HCAPI, CFA_RESOURCE_TYPE_P4_SP_MAC },
+	/* CFA_RESOURCE_TYPE_P4_SRAM_SP_SMAC_IPV4 */
+	{ TF_RM_ELEM_CFG_NULL, CFA_RESOURCE_TYPE_INVALID },
+	/* CFA_RESOURCE_TYPE_P4_SRAM_SP_SMAC_IPV6 */
+	{ TF_RM_ELEM_CFG_NULL, CFA_RESOURCE_TYPE_INVALID },
+	{ TF_RM_ELEM_CFG_HCAPI, CFA_RESOURCE_TYPE_P4_COUNTER_64B },
+	{ TF_RM_ELEM_CFG_HCAPI, CFA_RESOURCE_TYPE_P4_NAT_SPORT },
+	{ TF_RM_ELEM_CFG_HCAPI, CFA_RESOURCE_TYPE_P4_NAT_DPORT },
+	{ TF_RM_ELEM_CFG_HCAPI, CFA_RESOURCE_TYPE_P4_NAT_S_IPV4 },
+	{ TF_RM_ELEM_CFG_HCAPI, CFA_RESOURCE_TYPE_P4_NAT_D_IPV4 },
+	{ TF_RM_ELEM_CFG_HCAPI, CFA_RESOURCE_TYPE_P4_NAT_S_IPV6 },
+	{ TF_RM_ELEM_CFG_HCAPI, CFA_RESOURCE_TYPE_P4_NAT_D_IPV6 },
 	{ TF_RM_ELEM_CFG_HCAPI, CFA_RESOURCE_TYPE_P4_METER_PROF },
 	{ TF_RM_ELEM_CFG_HCAPI, CFA_RESOURCE_TYPE_P4_METER },
 	{ TF_RM_ELEM_CFG_HCAPI, CFA_RESOURCE_TYPE_P4_MIRROR },
-	{ TF_RM_ELEM_CFG_NULL, /* CFA_RESOURCE_TYPE_P4_UPAR */ },
-	{ TF_RM_ELEM_CFG_NULL, /* CFA_RESOURCE_TYPE_P4_EPOC */ },
-	{ TF_RM_ELEM_CFG_NULL, /* CFA_RESOURCE_TYPE_P4_METADATA */ },
-	{ TF_RM_ELEM_CFG_NULL, /* CFA_RESOURCE_TYPE_P4_CT_STATE */ },
-	{ TF_RM_ELEM_CFG_NULL, /* CFA_RESOURCE_TYPE_P4_RANGE_PROF */ },
-	{ TF_RM_ELEM_CFG_NULL, /* CFA_RESOURCE_TYPE_P4_RANGE_ENTRY */ },
-	{ TF_RM_ELEM_CFG_NULL, /* CFA_RESOURCE_TYPE_P4_LAG */ },
-	{ TF_RM_ELEM_CFG_NULL, /* CFA_RESOURCE_TYPE_P4_VNIC_SVIF */ },
-	{ TF_RM_ELEM_CFG_NULL, /* CFA_RESOURCE_TYPE_P4_EM_FBK */ },
-	{ TF_RM_ELEM_CFG_NULL, /* CFA_RESOURCE_TYPE_P4_WC_FKB */ },
-	{ TF_RM_ELEM_CFG_NULL, /* CFA_RESOURCE_TYPE_P4_EXT */ }
+	/* CFA_RESOURCE_TYPE_P4_UPAR */
+	{ TF_RM_ELEM_CFG_NULL, CFA_RESOURCE_TYPE_INVALID },
+	/* CFA_RESOURCE_TYPE_P4_EPOC */
+	{ TF_RM_ELEM_CFG_NULL, CFA_RESOURCE_TYPE_INVALID },
+	/* CFA_RESOURCE_TYPE_P4_METADATA */
+	{ TF_RM_ELEM_CFG_NULL, CFA_RESOURCE_TYPE_INVALID },
+	/* CFA_RESOURCE_TYPE_P4_CT_STATE */
+	{ TF_RM_ELEM_CFG_NULL, CFA_RESOURCE_TYPE_INVALID },
+	/* CFA_RESOURCE_TYPE_P4_RANGE_PROF */
+	{ TF_RM_ELEM_CFG_NULL, CFA_RESOURCE_TYPE_INVALID },
+	/* CFA_RESOURCE_TYPE_P4_RANGE_ENTRY */
+	{ TF_RM_ELEM_CFG_NULL, CFA_RESOURCE_TYPE_INVALID },
+	/* CFA_RESOURCE_TYPE_P4_LAG */
+	{ TF_RM_ELEM_CFG_NULL, CFA_RESOURCE_TYPE_INVALID },
+	/* CFA_RESOURCE_TYPE_P4_VNIC_SVIF */
+	{ TF_RM_ELEM_CFG_NULL, CFA_RESOURCE_TYPE_INVALID },
+	/* CFA_RESOURCE_TYPE_P4_EM_FBK */
+	{ TF_RM_ELEM_CFG_NULL, CFA_RESOURCE_TYPE_INVALID },
+	/* CFA_RESOURCE_TYPE_P4_WC_FKB */
+	{ TF_RM_ELEM_CFG_NULL, CFA_RESOURCE_TYPE_INVALID },
+	/* CFA_RESOURCE_TYPE_P4_EXT */
+	{ TF_RM_ELEM_CFG_NULL, CFA_RESOURCE_TYPE_INVALID }
 };
 
 #endif /* _TF_DEVICE_P4_H_ */
diff --git a/drivers/net/bnxt/tf_core/tf_identifier.c b/drivers/net/bnxt/tf_core/tf_identifier.c
index 726d0b4..e89f976 100644
--- a/drivers/net/bnxt/tf_core/tf_identifier.c
+++ b/drivers/net/bnxt/tf_core/tf_identifier.c
@@ -6,42 +6,172 @@
 #include <rte_common.h>
 
 #include "tf_identifier.h"
+#include "tf_common.h"
+#include "tf_rm_new.h"
+#include "tf_util.h"
+#include "tfp.h"
 
 struct tf;
 
 /**
  * Identifier DBs.
  */
-/* static void *ident_db[TF_DIR_MAX]; */
+static void *ident_db[TF_DIR_MAX];
 
 /**
  * Init flag, set on bind and cleared on unbind
  */
-/* static uint8_t init; */
+static uint8_t init;
 
 int
-tf_ident_bind(struct tf *tfp __rte_unused,
-	      struct tf_ident_cfg *parms __rte_unused)
+tf_ident_bind(struct tf *tfp,
+	      struct tf_ident_cfg_parms *parms)
 {
+	int rc;
+	int i;
+	struct tf_rm_create_db_parms db_cfg = { 0 };
+
+	TF_CHECK_PARMS2(tfp, parms);
+
+	if (init) {
+		TFP_DRV_LOG(ERR,
+			    "Identifier already initialized\n");
+		return -EINVAL;
+	}
+
+	db_cfg.num_elements = parms->num_elements;
+
+	for (i = 0; i < TF_DIR_MAX; i++) {
+		db_cfg.dir = i;
+		db_cfg.num_elements = parms->num_elements;
+		db_cfg.cfg = parms->cfg;
+		db_cfg.alloc_num = parms->resources->identifier_cnt[i];
+		db_cfg.rm_db = ident_db[i];
+		rc = tf_rm_create_db(tfp, &db_cfg);
+		if (rc) {
+			TFP_DRV_LOG(ERR,
+				    "%s: Identifier DB creation failed\n",
+				    tf_dir_2_str(i));
+			return rc;
+		}
+	}
+
+	init = 1;
+
 	return 0;
 }
 
 int
 tf_ident_unbind(struct tf *tfp __rte_unused)
 {
+	int rc;
+	int i;
+	struct tf_rm_free_db_parms fparms = { 0 };
+
+	TF_CHECK_PARMS1(tfp);
+
+	/* Bail if nothing has been initialized done silent as to
+	 * allow for creation cleanup.
+	 */
+	if (!init)
+		return -EINVAL;
+
+	for (i = 0; i < TF_DIR_MAX; i++) {
+		fparms.dir = i;
+		fparms.rm_db = ident_db[i];
+		rc = tf_rm_free_db(tfp, &fparms);
+		if (rc)
+			return rc;
+
+		ident_db[i] = NULL;
+	}
+
+	init = 0;
+
 	return 0;
 }
 
 int
 tf_ident_alloc(struct tf *tfp __rte_unused,
-	       struct tf_ident_alloc_parms *parms __rte_unused)
+	       struct tf_ident_alloc_parms *parms)
 {
+	int rc;
+	struct tf_rm_allocate_parms aparms = { 0 };
+
+	TF_CHECK_PARMS2(tfp, parms);
+
+	if (!init) {
+		TFP_DRV_LOG(ERR,
+			    "%s: No Identifier DBs created\n",
+			    tf_dir_2_str(parms->dir));
+		return -EINVAL;
+	}
+
+	/* Allocate requested element */
+	aparms.rm_db = ident_db[parms->dir];
+	aparms.db_index = parms->ident_type;
+	aparms.index = (uint32_t *)&parms->id;
+	rc = tf_rm_allocate(&aparms);
+	if (rc) {
+		TFP_DRV_LOG(ERR,
+			    "%s: Failed allocate, type:%d\n",
+			    tf_dir_2_str(parms->dir),
+			    parms->ident_type);
+		return rc;
+	}
+
 	return 0;
 }
 
 int
 tf_ident_free(struct tf *tfp __rte_unused,
-	      struct tf_ident_free_parms *parms __rte_unused)
+	      struct tf_ident_free_parms *parms)
 {
+	int rc;
+	struct tf_rm_is_allocated_parms aparms = { 0 };
+	struct tf_rm_free_parms fparms = { 0 };
+	int allocated = 0;
+
+	TF_CHECK_PARMS2(tfp, parms);
+
+	if (!init) {
+		TFP_DRV_LOG(ERR,
+			    "%s: No Identifier DBs created\n",
+			    tf_dir_2_str(parms->dir));
+		return -EINVAL;
+	}
+
+	/* Check if element is in use */
+	aparms.rm_db = ident_db[parms->dir];
+	aparms.db_index = parms->ident_type;
+	aparms.index = parms->id;
+	aparms.allocated = &allocated;
+	rc = tf_rm_is_allocated(&aparms);
+	if (rc)
+		return rc;
+
+	if (!allocated) {
+		TFP_DRV_LOG(ERR,
+			    "%s: Entry already free, type:%d, index:%d\n",
+			    tf_dir_2_str(parms->dir),
+			    parms->ident_type,
+			    parms->id);
+		return rc;
+	}
+
+	/* Free requested element */
+	fparms.rm_db = ident_db[parms->dir];
+	fparms.db_index = parms->ident_type;
+	fparms.index = parms->id;
+	rc = tf_rm_free(&fparms);
+	if (rc) {
+		TFP_DRV_LOG(ERR,
+			    "%s: Free failed, type:%d, index:%d\n",
+			    tf_dir_2_str(parms->dir),
+			    parms->ident_type,
+			    parms->id);
+		return rc;
+	}
+
 	return 0;
 }
diff --git a/drivers/net/bnxt/tf_core/tf_identifier.h b/drivers/net/bnxt/tf_core/tf_identifier.h
index b77c91b..1c5319b 100644
--- a/drivers/net/bnxt/tf_core/tf_identifier.h
+++ b/drivers/net/bnxt/tf_core/tf_identifier.h
@@ -12,21 +12,28 @@
  * The Identifier module provides processing of Identifiers.
  */
 
-struct tf_ident_cfg {
+struct tf_ident_cfg_parms {
 	/**
-	 * Number of identifier types in each of the configuration
-	 * arrays
+	 * [in] Number of identifier types in each of the
+	 * configuration arrays
 	 */
 	uint16_t num_elements;
-
 	/**
-	 * TCAM configuration array
+	 * [in] Identifier configuration array
+	 */
+	struct tf_rm_element_cfg *cfg;
+	/**
+	 * [in] Boolean controlling the request shadow copy.
 	 */
-	struct tf_rm_element_cfg *ident_cfg[TF_DIR_MAX];
+	bool shadow_copy;
+	/**
+	 * [in] Session resource allocations
+	 */
+	struct tf_session_resources *resources;
 };
 
 /**
- * Identifier allcoation parameter definition
+ * Identifier allocation parameter definition
  */
 struct tf_ident_alloc_parms {
 	/**
@@ -40,7 +47,7 @@ struct tf_ident_alloc_parms {
 	/**
 	 * [out] Identifier allocated
 	 */
-	uint16_t id;
+	uint16_t *id;
 };
 
 /**
@@ -88,7 +95,7 @@ struct tf_ident_free_parms {
  *   - (-EINVAL) on failure.
  */
 int tf_ident_bind(struct tf *tfp,
-		  struct tf_ident_cfg *parms);
+		  struct tf_ident_cfg_parms *parms);
 
 /**
  * Cleans up the private DBs and releases all the data.
diff --git a/drivers/net/bnxt/tf_core/tf_msg.c b/drivers/net/bnxt/tf_core/tf_msg.c
index c755c85..e08a96f 100644
--- a/drivers/net/bnxt/tf_core/tf_msg.c
+++ b/drivers/net/bnxt/tf_core/tf_msg.c
@@ -6,15 +6,13 @@
 #include <inttypes.h>
 #include <stdbool.h>
 #include <stdlib.h>
-
-#include "bnxt.h"
-#include "tf_core.h"
-#include "tf_session.h"
-#include "tfp.h"
+#include <string.h>
 
 #include "tf_msg_common.h"
 #include "tf_msg.h"
-#include "hsi_struct_def_dpdk.h"
+#include "tf_util.h"
+#include "tf_session.h"
+#include "tfp.h"
 #include "hwrm_tf.h"
 #include "tf_em.h"
 
@@ -141,6 +139,51 @@ tf_tcam_tbl_2_hwrm(enum tf_tcam_tbl_type tcam_type,
 }
 
 /**
+ * Allocates a DMA buffer that can be used for message transfer.
+ *
+ * [in] buf
+ *   Pointer to DMA buffer structure
+ *
+ * [in] size
+ *   Requested size of the buffer in bytes
+ *
+ * Returns:
+ *    0      - Success
+ *   -ENOMEM - Unable to allocate buffer, no memory
+ */
+static int
+tf_msg_alloc_dma_buf(struct tf_msg_dma_buf *buf, int size)
+{
+	struct tfp_calloc_parms alloc_parms;
+	int rc;
+
+	/* Allocate session */
+	alloc_parms.nitems = 1;
+	alloc_parms.size = size;
+	alloc_parms.alignment = 4096;
+	rc = tfp_calloc(&alloc_parms);
+	if (rc)
+		return -ENOMEM;
+
+	buf->pa_addr = (uintptr_t)alloc_parms.mem_pa;
+	buf->va_addr = alloc_parms.mem_va;
+
+	return 0;
+}
+
+/**
+ * Free's a previous allocated DMA buffer.
+ *
+ * [in] buf
+ *   Pointer to DMA buffer structure
+ */
+static void
+tf_msg_free_dma_buf(struct tf_msg_dma_buf *buf)
+{
+	tfp_free(buf->va_addr);
+}
+
+/**
  * Sends session open request to TF Firmware
  */
 int
@@ -154,7 +197,7 @@ tf_msg_session_open(struct tf *tfp,
 	struct tfp_send_msg_parms parms = { 0 };
 
 	/* Populate the request */
-	memcpy(&req.session_name, ctrl_chan_name, TF_SESSION_NAME_MAX);
+	tfp_memcpy(&req.session_name, ctrl_chan_name, TF_SESSION_NAME_MAX);
 
 	parms.tf_type = HWRM_TF_SESSION_OPEN;
 	parms.req_data = (uint32_t *)&req;
@@ -870,6 +913,180 @@ tf_msg_session_sram_resc_flush(struct tf *tfp,
 	return tfp_le_to_cpu_32(parms.tf_resp_code);
 }
 
+int
+tf_msg_session_resc_qcaps(struct tf *tfp,
+			  enum tf_dir dir,
+			  uint16_t size,
+			  struct tf_rm_resc_req_entry *query,
+			  enum tf_rm_resc_resv_strategy *resv_strategy)
+{
+	int rc;
+	int i;
+	struct tfp_send_msg_parms parms = { 0 };
+	struct hwrm_tf_session_resc_qcaps_input req = { 0 };
+	struct hwrm_tf_session_resc_qcaps_output resp = { 0 };
+	uint8_t fw_session_id;
+	struct tf_msg_dma_buf qcaps_buf = { 0 };
+	struct tf_rm_resc_req_entry *data;
+	int dma_size;
+
+	if (size == 0 || query == NULL || resv_strategy == NULL) {
+		TFP_DRV_LOG(ERR,
+			    "%s: Resource QCAPS parameter error, rc:%s\n",
+			    tf_dir_2_str(dir),
+			    strerror(-EINVAL));
+		return -EINVAL;
+	}
+
+	rc = tf_session_get_fw_session_id(tfp, &fw_session_id);
+	if (rc) {
+		TFP_DRV_LOG(ERR,
+			    "%s: Unable to lookup FW id, rc:%s\n",
+			    tf_dir_2_str(dir),
+			    strerror(-rc));
+		return rc;
+	}
+
+	/* Prepare DMA buffer */
+	dma_size = size * sizeof(struct tf_rm_resc_req_entry);
+	rc = tf_msg_alloc_dma_buf(&qcaps_buf, dma_size);
+	if (rc)
+		return rc;
+
+	/* Populate the request */
+	req.fw_session_id = tfp_cpu_to_le_32(fw_session_id);
+	req.flags = tfp_cpu_to_le_16(dir);
+	req.qcaps_size = size;
+	req.qcaps_addr = qcaps_buf.pa_addr;
+
+	parms.tf_type = HWRM_TF_SESSION_RESC_QCAPS;
+	parms.req_data = (uint32_t *)&req;
+	parms.req_size = sizeof(req);
+	parms.resp_data = (uint32_t *)&resp;
+	parms.resp_size = sizeof(resp);
+	parms.mailbox = TF_KONG_MB;
+
+	rc = tfp_send_msg_direct(tfp, &parms);
+	if (rc)
+		return rc;
+
+	/* Process the response
+	 * Should always get expected number of entries
+	 */
+	if (resp.size != size) {
+		TFP_DRV_LOG(ERR,
+			    "%s: QCAPS message error, rc:%s\n",
+			    tf_dir_2_str(dir),
+			    strerror(-EINVAL));
+		return -EINVAL;
+	}
+
+	/* Post process the response */
+	data = (struct tf_rm_resc_req_entry *)qcaps_buf.va_addr;
+	for (i = 0; i < size; i++) {
+		query[i].type = tfp_cpu_to_le_32(data[i].type);
+		query[i].min = tfp_le_to_cpu_16(data[i].min);
+		query[i].max = tfp_le_to_cpu_16(data[i].max);
+	}
+
+	*resv_strategy = resp.flags &
+	      HWRM_TF_SESSION_RESC_QCAPS_OUTPUT_FLAGS_SESS_RESV_STRATEGY_MASK;
+
+	tf_msg_free_dma_buf(&qcaps_buf);
+
+	return rc;
+}
+
+int
+tf_msg_session_resc_alloc(struct tf *tfp,
+			  enum tf_dir dir,
+			  uint16_t size,
+			  struct tf_rm_resc_req_entry *request,
+			  struct tf_rm_resc_entry *resv)
+{
+	int rc;
+	int i;
+	struct tfp_send_msg_parms parms = { 0 };
+	struct hwrm_tf_session_resc_alloc_input req = { 0 };
+	struct hwrm_tf_session_resc_alloc_output resp = { 0 };
+	uint8_t fw_session_id;
+	struct tf_msg_dma_buf req_buf = { 0 };
+	struct tf_msg_dma_buf resv_buf = { 0 };
+	struct tf_rm_resc_req_entry *req_data;
+	struct tf_rm_resc_entry *resv_data;
+	int dma_size;
+
+	rc = tf_session_get_fw_session_id(tfp, &fw_session_id);
+	if (rc) {
+		TFP_DRV_LOG(ERR,
+			    "%s: Unable to lookup FW id, rc:%s\n",
+			    tf_dir_2_str(dir),
+			    strerror(-rc));
+		return rc;
+	}
+
+	/* Prepare DMA buffers */
+	dma_size = size * sizeof(struct tf_rm_resc_req_entry);
+	rc = tf_msg_alloc_dma_buf(&req_buf, dma_size);
+	if (rc)
+		return rc;
+
+	dma_size = size * sizeof(struct tf_rm_resc_entry);
+	rc = tf_msg_alloc_dma_buf(&resv_buf, dma_size);
+	if (rc)
+		return rc;
+
+	/* Populate the request */
+	req.fw_session_id = tfp_cpu_to_le_32(fw_session_id);
+	req.flags = tfp_cpu_to_le_16(dir);
+	req.req_size = size;
+
+	req_data = (struct tf_rm_resc_req_entry *)req_buf.va_addr;
+	for (i = 0; i < size; i++) {
+		req_data[i].type = tfp_cpu_to_le_32(request[i].type);
+		req_data[i].min = tfp_cpu_to_le_16(request[i].min);
+		req_data[i].max = tfp_cpu_to_le_16(request[i].max);
+	}
+
+	req.req_addr = req_buf.pa_addr;
+	req.resp_addr = resv_buf.pa_addr;
+
+	parms.tf_type = HWRM_TF_SESSION_RESC_ALLOC;
+	parms.req_data = (uint32_t *)&req;
+	parms.req_size = sizeof(req);
+	parms.resp_data = (uint32_t *)&resp;
+	parms.resp_size = sizeof(resp);
+	parms.mailbox = TF_KONG_MB;
+
+	rc = tfp_send_msg_direct(tfp, &parms);
+	if (rc)
+		return rc;
+
+	/* Process the response
+	 * Should always get expected number of entries
+	 */
+	if (resp.size != size) {
+		TFP_DRV_LOG(ERR,
+			    "%s: Alloc message error, rc:%s\n",
+			    tf_dir_2_str(dir),
+			    strerror(-EINVAL));
+		return -EINVAL;
+	}
+
+	/* Post process the response */
+	resv_data = (struct tf_rm_resc_entry *)resv_buf.va_addr;
+	for (i = 0; i < size; i++) {
+		resv[i].type = tfp_cpu_to_le_32(resv_data[i].type);
+		resv[i].start = tfp_cpu_to_le_16(resv_data[i].start);
+		resv[i].stride = tfp_cpu_to_le_16(resv_data[i].stride);
+	}
+
+	tf_msg_free_dma_buf(&req_buf);
+	tf_msg_free_dma_buf(&resv_buf);
+
+	return rc;
+}
+
 /**
  * Sends EM mem register request to Firmware
  */
@@ -1034,7 +1251,9 @@ int tf_msg_insert_em_internal_entry(struct tf *tfp,
 
 	req.fw_session_id =
 		tfp_cpu_to_le_32(tfs->session_id.internal.fw_session_id);
-	memcpy(req.em_key, em_parms->key, ((em_parms->key_sz_in_bits + 7) / 8));
+	tfp_memcpy(req.em_key,
+		   em_parms->key,
+		   ((em_parms->key_sz_in_bits + 7) / 8));
 
 	flags = (em_parms->dir == TF_DIR_TX ?
 		 HWRM_TF_EM_INSERT_INPUT_FLAGS_DIR_TX :
@@ -1216,26 +1435,6 @@ tf_msg_get_tbl_entry(struct tf *tfp,
 	return tfp_le_to_cpu_32(parms.tf_resp_code);
 }
 
-static int
-tf_msg_alloc_dma_buf(struct tf_msg_dma_buf *buf, int size)
-{
-	struct tfp_calloc_parms alloc_parms;
-	int rc;
-
-	/* Allocate session */
-	alloc_parms.nitems = 1;
-	alloc_parms.size = size;
-	alloc_parms.alignment = 4096;
-	rc = tfp_calloc(&alloc_parms);
-	if (rc)
-		return -ENOMEM;
-
-	buf->pa_addr = (uintptr_t)alloc_parms.mem_pa;
-	buf->va_addr = alloc_parms.mem_va;
-
-	return 0;
-}
-
 int
 tf_msg_get_bulk_tbl_entry(struct tf *tfp,
 			  struct tf_get_bulk_tbl_entry_parms *params)
@@ -1323,12 +1522,14 @@ tf_msg_tcam_entry_set(struct tf *tfp,
 		if (rc)
 			goto cleanup;
 		data = buf.va_addr;
-		memcpy(&req.dev_data[0], &buf.pa_addr, sizeof(buf.pa_addr));
+		tfp_memcpy(&req.dev_data[0],
+			   &buf.pa_addr,
+			   sizeof(buf.pa_addr));
 	}
 
-	memcpy(&data[0], parms->key, key_bytes);
-	memcpy(&data[key_bytes], parms->mask, key_bytes);
-	memcpy(&data[req.result_offset], parms->result, result_bytes);
+	tfp_memcpy(&data[0], parms->key, key_bytes);
+	tfp_memcpy(&data[key_bytes], parms->mask, key_bytes);
+	tfp_memcpy(&data[req.result_offset], parms->result, result_bytes);
 
 	mparms.tf_type = HWRM_TF_TCAM_SET;
 	mparms.req_data = (uint32_t *)&req;
@@ -1343,8 +1544,7 @@ tf_msg_tcam_entry_set(struct tf *tfp,
 		goto cleanup;
 
 cleanup:
-	if (buf.va_addr != NULL)
-		tfp_free(buf.va_addr);
+	tf_msg_free_dma_buf(&buf);
 
 	return rc;
 }
diff --git a/drivers/net/bnxt/tf_core/tf_msg.h b/drivers/net/bnxt/tf_core/tf_msg.h
index 8d050c4..06f52ef 100644
--- a/drivers/net/bnxt/tf_core/tf_msg.h
+++ b/drivers/net/bnxt/tf_core/tf_msg.h
@@ -6,8 +6,12 @@
 #ifndef _TF_MSG_H_
 #define _TF_MSG_H_
 
+#include <rte_common.h>
+#include <hsi_struct_def_dpdk.h>
+
 #include "tf_tbl.h"
 #include "tf_rm.h"
+#include "tf_rm_new.h"
 
 struct tf;
 
@@ -122,6 +126,61 @@ int tf_msg_session_sram_resc_flush(struct tf *tfp,
 				   struct tf_rm_entry *sram_entry);
 
 /**
+ * Sends session HW resource query capability request to TF Firmware
+ *
+ * [in] tfp
+ *   Pointer to TF handle
+ *
+ * [in] dir
+ *   Receive or Transmit direction
+ *
+ * [in] size
+ *   Number of elements in the query. Should be set to the max
+ *   elements for the device type
+ *
+ * [out] query
+ *   Pointer to an array of query elements
+ *
+ * [out] resv_strategy
+ *   Pointer to the reservation strategy
+ *
+ * Returns:
+ *   0 on Success else internal Truflow error
+ */
+int tf_msg_session_resc_qcaps(struct tf *tfp,
+			      enum tf_dir dir,
+			      uint16_t size,
+			      struct tf_rm_resc_req_entry *query,
+			      enum tf_rm_resc_resv_strategy *resv_strategy);
+
+/**
+ * Sends session HW resource allocation request to TF Firmware
+ *
+ * [in] tfp
+ *   Pointer to TF handle
+ *
+ * [in] dir
+ *   Receive or Transmit direction
+ *
+ * [in] size
+ *   Number of elements in the req and resv arrays
+ *
+ * [in] req
+ *   Pointer to an array of request elements
+ *
+ * [in] resv
+ *   Pointer to an array of reserved elements
+ *
+ * Returns:
+ *   0 on Success else internal Truflow error
+ */
+int tf_msg_session_resc_alloc(struct tf *tfp,
+			      enum tf_dir dir,
+			      uint16_t size,
+			      struct tf_rm_resc_req_entry *request,
+			      struct tf_rm_resc_entry *resv);
+
+/**
  * Sends EM internal insert request to Firmware
  */
 int tf_msg_insert_em_internal_entry(struct tf *tfp,
diff --git a/drivers/net/bnxt/tf_core/tf_rm_new.c b/drivers/net/bnxt/tf_core/tf_rm_new.c
index 51bb9ba..7cadb23 100644
--- a/drivers/net/bnxt/tf_core/tf_rm_new.c
+++ b/drivers/net/bnxt/tf_core/tf_rm_new.c
@@ -3,20 +3,18 @@
  * All rights reserved.
  */
 
+#include <string.h>
+
 #include <rte_common.h>
 
-#include "tf_rm_new.h"
+#include <cfa_resource_types.h>
 
-/**
- * Resource query single entry. Used when accessing HCAPI RM on the
- * firmware.
- */
-struct tf_rm_query_entry {
-	/** Minimum guaranteed number of elements */
-	uint16_t min;
-	/** Maximum non-guaranteed number of elements */
-	uint16_t max;
-};
+#include "tf_rm_new.h"
+#include "tf_util.h"
+#include "tf_session.h"
+#include "tf_device.h"
+#include "tfp.h"
+#include "tf_msg.h"
 
 /**
  * Generic RM Element data type that an RM DB is build upon.
@@ -27,7 +25,7 @@ struct tf_rm_element {
 	 * hcapi_type can be ignored. If Null then the element is not
 	 * valid for the device.
 	 */
-	enum tf_rm_elem_cfg_type type;
+	enum tf_rm_elem_cfg_type cfg_type;
 
 	/**
 	 * HCAPI RM Type for the element.
@@ -50,53 +48,435 @@ struct tf_rm_element {
 /**
  * TF RM DB definition
  */
-struct tf_rm_db {
+struct tf_rm_new_db {
+	/**
+	 * Number of elements in the DB
+	 */
+	uint16_t num_entries;
+
+	/**
+	 * Direction this DB controls.
+	 */
+	enum tf_dir dir;
+
 	/**
 	 * The DB consists of an array of elements
 	 */
 	struct tf_rm_element *db;
 };
 
+
+/**
+ * Resource Manager Adjust of base index definitions.
+ */
+enum tf_rm_adjust_type {
+	TF_RM_ADJUST_ADD_BASE, /**< Adds base to the index */
+	TF_RM_ADJUST_RM_BASE   /**< Removes base from the index */
+};
+
+/**
+ * Adjust an index according to the allocation information.
+ *
+ * All resources are controlled in a 0 based pool. Some resources, by
+ * design, are not 0 based, i.e. Full Action Records (SRAM) thus they
+ * need to be adjusted before they are handed out.
+ *
+ * [in] db
+ *   Pointer to the db, used for the lookup
+ *
+ * [in] action
+ *   Adjust action
+ *
+ * [in] db_index
+ *   DB index for the element type
+ *
+ * [in] index
+ *   Index to convert
+ *
+ * [out] adj_index
+ *   Adjusted index
+ *
+ * Returns:
+ *     0          - Success
+ *   - EOPNOTSUPP - Operation not supported
+ */
+static int
+tf_rm_adjust_index(struct tf_rm_element *db,
+		   enum tf_rm_adjust_type action,
+		   uint32_t db_index,
+		   uint32_t index,
+		   uint32_t *adj_index)
+{
+	int rc = 0;
+	uint32_t base_index;
+
+	base_index = db[db_index].alloc.entry.start;
+
+	switch (action) {
+	case TF_RM_ADJUST_RM_BASE:
+		*adj_index = index - base_index;
+		break;
+	case TF_RM_ADJUST_ADD_BASE:
+		*adj_index = index + base_index;
+		break;
+	default:
+		return -EOPNOTSUPP;
+	}
+
+	return rc;
+}
+
 int
-tf_rm_create_db(struct tf *tfp __rte_unused,
-		struct tf_rm_create_db_parms *parms __rte_unused)
+tf_rm_create_db(struct tf *tfp,
+		struct tf_rm_create_db_parms *parms)
 {
+	int rc;
+	int i;
+	struct tf_session *tfs;
+	struct tf_dev_info *dev;
+	uint16_t max_types;
+	struct tfp_calloc_parms cparms;
+	struct tf_rm_resc_req_entry *query;
+	enum tf_rm_resc_resv_strategy resv_strategy;
+	struct tf_rm_resc_req_entry *req;
+	struct tf_rm_resc_entry *resv;
+	struct tf_rm_new_db *rm_db;
+	struct tf_rm_element *db;
+	uint32_t pool_size;
+
+	/* Retrieve the session information */
+	rc = tf_session_get_session(tfp, &tfs);
+	if (rc)
+		return rc;
+
+	/* Retrieve device information */
+	rc = tf_session_get_device(tfs, &dev);
+	if (rc)
+		return rc;
+
+	/* Need device max number of elements for the RM QCAPS */
+	rc = dev->ops->tf_dev_get_max_types(tfp, &max_types);
+	if (rc)
+		return rc;
+
+	cparms.nitems = max_types;
+	cparms.size = sizeof(struct tf_rm_resc_req_entry);
+	cparms.alignment = 0;
+	rc = tfp_calloc(&cparms);
+	if (rc)
+		return rc;
+
+	query = (struct tf_rm_resc_req_entry *)cparms.mem_va;
+
+	/* Get Firmware Capabilities */
+	rc = tf_msg_session_resc_qcaps(tfp,
+				       parms->dir,
+				       max_types,
+				       query,
+				       &resv_strategy);
+	if (rc)
+		return rc;
+
+	/* Process capabilities against db requirements */
+
+	/* Alloc request, alignment already set */
+	cparms.nitems = parms->num_elements;
+	cparms.size = sizeof(struct tf_rm_resc_req_entry);
+	rc = tfp_calloc(&cparms);
+	if (rc)
+		return rc;
+	req = (struct tf_rm_resc_req_entry *)cparms.mem_va;
+
+	/* Alloc reservation, alignment and nitems already set */
+	cparms.size = sizeof(struct tf_rm_resc_entry);
+	rc = tfp_calloc(&cparms);
+	if (rc)
+		return rc;
+	resv = (struct tf_rm_resc_entry *)cparms.mem_va;
+
+	/* Build the request */
+	for (i = 0; i < parms->num_elements; i++) {
+		/* Skip any non HCAPI cfg elements */
+		if (parms->cfg[i].cfg_type == TF_RM_ELEM_CFG_HCAPI) {
+			req[i].type = parms->cfg[i].hcapi_type;
+			/* Check that we can get the full amount allocated */
+			if (parms->alloc_num[i] <=
+			    query[parms->cfg[i].hcapi_type].max) {
+				req[i].min = parms->alloc_num[i];
+				req[i].max = parms->alloc_num[i];
+			} else {
+				TFP_DRV_LOG(ERR,
+					    "%s: Resource failure, type:%d\n",
+					    tf_dir_2_str(parms->dir),
+					    parms->cfg[i].hcapi_type);
+				TFP_DRV_LOG(ERR,
+					"req:%d, avail:%d\n",
+					parms->alloc_num[i],
+					query[parms->cfg[i].hcapi_type].max);
+				return -EINVAL;
+			}
+		} else {
+			/* Skip the element */
+			req[i].type = CFA_RESOURCE_TYPE_INVALID;
+		}
+	}
+
+	rc = tf_msg_session_resc_alloc(tfp,
+				       parms->dir,
+				       parms->num_elements,
+				       req,
+				       resv);
+	if (rc)
+		return rc;
+
+	/* Build the RM DB per the request */
+	cparms.nitems = 1;
+	cparms.size = sizeof(struct tf_rm_new_db);
+	rc = tfp_calloc(&cparms);
+	if (rc)
+		return rc;
+	rm_db = (void *)cparms.mem_va;
+
+	/* Build the DB within RM DB */
+	cparms.nitems = parms->num_elements;
+	cparms.size = sizeof(struct tf_rm_element);
+	rc = tfp_calloc(&cparms);
+	if (rc)
+		return rc;
+	rm_db->db = (struct tf_rm_element *)cparms.mem_va;
+
+	db = rm_db->db;
+	for (i = 0; i < parms->num_elements; i++) {
+		/* If allocation failed for a single entry the DB
+		 * creation is considered a failure.
+		 */
+		if (parms->alloc_num[i] != resv[i].stride) {
+			TFP_DRV_LOG(ERR,
+				    "%s: Alloc failed, type:%d\n",
+				    tf_dir_2_str(parms->dir),
+				    i);
+			TFP_DRV_LOG(ERR,
+				    "req:%d, alloc:%d\n",
+				    parms->alloc_num[i],
+				    resv[i].stride);
+			goto fail;
+		}
+
+		db[i].cfg_type = parms->cfg[i].cfg_type;
+		db[i].hcapi_type = parms->cfg[i].hcapi_type;
+		db[i].alloc.entry.start = resv[i].start;
+		db[i].alloc.entry.stride = resv[i].stride;
+
+		/* Create pool */
+		pool_size = (BITALLOC_SIZEOF(resv[i].stride) /
+			     sizeof(struct bitalloc));
+		/* Alloc request, alignment already set */
+		cparms.nitems = pool_size;
+		cparms.size = sizeof(struct bitalloc);
+		rc = tfp_calloc(&cparms);
+		if (rc)
+			return rc;
+		db[i].pool = (struct bitalloc *)cparms.mem_va;
+	}
+
+	rm_db->num_entries = i;
+	rm_db->dir = parms->dir;
+	parms->rm_db = (void *)rm_db;
+
+	tfp_free((void *)req);
+	tfp_free((void *)resv);
+
 	return 0;
+
+ fail:
+	tfp_free((void *)req);
+	tfp_free((void *)resv);
+	tfp_free((void *)db->pool);
+	tfp_free((void *)db);
+	tfp_free((void *)rm_db);
+	parms->rm_db = NULL;
+
+	return -EINVAL;
 }
 
 int
 tf_rm_free_db(struct tf *tfp __rte_unused,
-	      struct tf_rm_free_db_parms *parms __rte_unused)
+	      struct tf_rm_free_db_parms *parms)
 {
-	return 0;
+	int rc = 0;
+	int i;
+	struct tf_rm_new_db *rm_db;
+
+	/* Traverse the DB and clear each pool.
+	 * NOTE:
+	 *   Firmware is not cleared. It will be cleared on close only.
+	 */
+	rm_db = (struct tf_rm_new_db *)parms->rm_db;
+	for (i = 0; i < rm_db->num_entries; i++)
+		tfp_free((void *)rm_db->db->pool);
+
+	tfp_free((void *)parms->rm_db);
+
+	return rc;
 }
 
 int
-tf_rm_allocate(struct tf_rm_allocate_parms *parms __rte_unused)
+tf_rm_allocate(struct tf_rm_allocate_parms *parms)
 {
-	return 0;
+	int rc = 0;
+	int id;
+	struct tf_rm_new_db *rm_db;
+	enum tf_rm_elem_cfg_type cfg_type;
+
+	if (parms == NULL || parms->rm_db == NULL)
+		return -EINVAL;
+
+	rm_db = (struct tf_rm_new_db *)parms->rm_db;
+	cfg_type = rm_db->db[parms->db_index].cfg_type;
+
+	/* Bail out if not controlled by RM */
+	if (cfg_type != TF_RM_ELEM_CFG_HCAPI &&
+	    cfg_type != TF_RM_ELEM_CFG_PRIVATE)
+		return -ENOTSUP;
+
+	id = ba_alloc(rm_db->db[parms->db_index].pool);
+	if (id == BA_FAIL) {
+		TFP_DRV_LOG(ERR,
+			    "%s: Allocation failed, rc:%s\n",
+			    tf_dir_2_str(rm_db->dir),
+			    strerror(-rc));
+		return -ENOMEM;
+	}
+
+	/* Adjust for any non zero start value */
+	rc = tf_rm_adjust_index(rm_db->db,
+				TF_RM_ADJUST_ADD_BASE,
+				parms->db_index,
+				id,
+				parms->index);
+	if (rc) {
+		TFP_DRV_LOG(ERR,
+			    "%s: Alloc adjust of base index failed, rc:%s\n",
+			    tf_dir_2_str(rm_db->dir),
+			    strerror(-rc));
+		return -1;
+	}
+
+	return rc;
 }
 
 int
-tf_rm_free(struct tf_rm_free_parms *parms __rte_unused)
+tf_rm_free(struct tf_rm_free_parms *parms)
 {
-	return 0;
+	int rc = 0;
+	uint32_t adj_index;
+	struct tf_rm_new_db *rm_db;
+	enum tf_rm_elem_cfg_type cfg_type;
+
+	if (parms == NULL || parms->rm_db == NULL)
+		return -EINVAL;
+
+	rm_db = (struct tf_rm_new_db *)parms->rm_db;
+	cfg_type = rm_db->db[parms->db_index].cfg_type;
+
+	/* Bail out if not controlled by RM */
+	if (cfg_type != TF_RM_ELEM_CFG_HCAPI &&
+	    cfg_type != TF_RM_ELEM_CFG_PRIVATE)
+		return -ENOTSUP;
+
+	/* Adjust for any non zero start value */
+	rc = tf_rm_adjust_index(rm_db->db,
+				TF_RM_ADJUST_RM_BASE,
+				parms->db_index,
+				parms->index,
+				&adj_index);
+	if (rc)
+		return rc;
+
+	rc = ba_free(rm_db->db[parms->db_index].pool, adj_index);
+	/* No logging direction matters and that is not available here */
+	if (rc)
+		return rc;
+
+	return rc;
 }
 
 int
-tf_rm_is_allocated(struct tf_rm_is_allocated_parms *parms __rte_unused)
+tf_rm_is_allocated(struct tf_rm_is_allocated_parms *parms)
 {
-	return 0;
+	int rc = 0;
+	uint32_t adj_index;
+	struct tf_rm_new_db *rm_db;
+	enum tf_rm_elem_cfg_type cfg_type;
+
+	if (parms == NULL || parms->rm_db == NULL)
+		return -EINVAL;
+
+	rm_db = (struct tf_rm_new_db *)parms->rm_db;
+	cfg_type = rm_db->db[parms->db_index].cfg_type;
+
+	/* Bail out if not controlled by RM */
+	if (cfg_type != TF_RM_ELEM_CFG_HCAPI &&
+	    cfg_type != TF_RM_ELEM_CFG_PRIVATE)
+		return -ENOTSUP;
+
+	/* Adjust for any non zero start value */
+	rc = tf_rm_adjust_index(rm_db->db,
+				TF_RM_ADJUST_RM_BASE,
+				parms->db_index,
+				parms->index,
+				&adj_index);
+	if (rc)
+		return rc;
+
+	*parms->allocated = ba_inuse(rm_db->db[parms->db_index].pool,
+				     adj_index);
+
+	return rc;
 }
 
 int
-tf_rm_get_info(struct tf_rm_get_alloc_info_parms *parms __rte_unused)
+tf_rm_get_info(struct tf_rm_get_alloc_info_parms *parms)
 {
-	return 0;
+	int rc = 0;
+	struct tf_rm_new_db *rm_db;
+	enum tf_rm_elem_cfg_type cfg_type;
+
+	if (parms == NULL || parms->rm_db == NULL)
+		return -EINVAL;
+
+	rm_db = (struct tf_rm_new_db *)parms->rm_db;
+	cfg_type = rm_db->db[parms->db_index].cfg_type;
+
+	/* Bail out if not controlled by RM */
+	if (cfg_type != TF_RM_ELEM_CFG_HCAPI &&
+	    cfg_type != TF_RM_ELEM_CFG_PRIVATE)
+		return -ENOTSUP;
+
+	parms->info = &rm_db->db[parms->db_index].alloc;
+
+	return rc;
 }
 
 int
-tf_rm_get_hcapi_type(struct tf_rm_get_hcapi_parms *parms __rte_unused)
+tf_rm_get_hcapi_type(struct tf_rm_get_hcapi_parms *parms)
 {
-	return 0;
+	int rc = 0;
+	struct tf_rm_new_db *rm_db;
+	enum tf_rm_elem_cfg_type cfg_type;
+
+	if (parms == NULL || parms->rm_db == NULL)
+		return -EINVAL;
+
+	rm_db = (struct tf_rm_new_db *)parms->rm_db;
+	cfg_type = rm_db->db[parms->db_index].cfg_type;
+
+	/* Bail out if not controlled by RM */
+	if (cfg_type != TF_RM_ELEM_CFG_HCAPI &&
+	    cfg_type != TF_RM_ELEM_CFG_PRIVATE)
+		return -ENOTSUP;
+
+	*parms->hcapi_type = rm_db->db[parms->db_index].hcapi_type;
+
+	return rc;
 }
diff --git a/drivers/net/bnxt/tf_core/tf_rm_new.h b/drivers/net/bnxt/tf_core/tf_rm_new.h
index 72dba09..6d8234d 100644
--- a/drivers/net/bnxt/tf_core/tf_rm_new.h
+++ b/drivers/net/bnxt/tf_core/tf_rm_new.h
@@ -3,8 +3,8 @@
  * All rights reserved.
  */
 
-#ifndef TF_RM_H_
-#define TF_RM_H_
+#ifndef TF_RM_NEW_H_
+#define TF_RM_NEW_H_
 
 #include "tf_core.h"
 #include "bitalloc.h"
@@ -32,13 +32,16 @@ struct tf;
  * MAX pool size of the Chip œneeds to be added to the tf_rm_elem_info
  * structure and several new APIs would need to be added to allow for
  * growth of a single TF resource type.
+ *
+ * The access functions does not check for NULL pointers as it's a
+ * support module, not called directly.
  */
 
 /**
  * Resource reservation single entry result. Used when accessing HCAPI
  * RM on the firmware.
  */
-struct tf_rm_entry {
+struct tf_rm_new_entry {
 	/** Starting index of the allocated resource */
 	uint16_t start;
 	/** Number of allocated elements */
@@ -52,13 +55,33 @@ struct tf_rm_entry {
  * ULP layer that is not controlled by HCAPI within the Firmware.
  */
 enum tf_rm_elem_cfg_type {
-	TF_RM_ELEM_CFG_NULL,    /**< No configuration */
-	TF_RM_ELEM_CFG_HCAPI,   /**< HCAPI 'controlled' */
-	TF_RM_ELEM_CFG_PRIVATE, /**< Private thus not HCAPI 'controlled' */
+	/** No configuration */
+	TF_RM_ELEM_CFG_NULL,
+	/** HCAPI 'controlled' */
+	TF_RM_ELEM_CFG_HCAPI,
+	/** Private thus not HCAPI 'controlled' */
+	TF_RM_ELEM_CFG_PRIVATE,
+	/**
+	 * Shared element thus it belongs to a shared FW Session and
+	 * is not controlled by the Host.
+	 */
+	TF_RM_ELEM_CFG_SHARED,
 	TF_RM_TYPE_MAX
 };
 
 /**
+ * RM Reservation strategy enumeration. Type of strategy comes from
+ * the HCAPI RM QCAPS handshake.
+ */
+enum tf_rm_resc_resv_strategy {
+	TF_RM_RESC_RESV_STATIC_PARTITION,
+	TF_RM_RESC_RESV_STRATEGY_1,
+	TF_RM_RESC_RESV_STRATEGY_2,
+	TF_RM_RESC_RESV_STRATEGY_3,
+	TF_RM_RESC_RESV_MAX
+};
+
+/**
  * RM Element configuration structure, used by the Device to configure
  * how an individual TF type is configured in regard to the HCAPI RM
  * of same type.
@@ -68,7 +91,7 @@ struct tf_rm_element_cfg {
 	 * RM Element config controls how the DB for that element is
 	 * processed.
 	 */
-	enum tf_rm_elem_cfg_type cfg;
+	enum tf_rm_elem_cfg_type cfg_type;
 
 	/* If a HCAPI to TF type conversion is required then TF type
 	 * can be added here.
@@ -92,7 +115,7 @@ struct tf_rm_alloc_info {
 	 * In case of dynamic allocation support this would have
 	 * to be changed to linked list of tf_rm_entry instead.
 	 */
-	struct tf_rm_entry entry;
+	struct tf_rm_new_entry entry;
 };
 
 /**
@@ -104,17 +127,21 @@ struct tf_rm_create_db_parms {
 	 */
 	enum tf_dir dir;
 	/**
-	 * [in] Number of elements in the parameter structure
+	 * [in] Number of elements.
 	 */
 	uint16_t num_elements;
 	/**
-	 * [in] Parameter structure
+	 * [in] Parameter structure array. Array size is num_elements.
+	 */
+	struct tf_rm_element_cfg *cfg;
+	/**
+	 * Allocation number array. Array size is num_elements.
 	 */
-	struct tf_rm_element_cfg *parms;
+	uint16_t *alloc_num;
 	/**
 	 * [out] RM DB Handle
 	 */
-	void *tf_rm_db;
+	void *rm_db;
 };
 
 /**
@@ -128,7 +155,7 @@ struct tf_rm_free_db_parms {
 	/**
 	 * [in] RM DB Handle
 	 */
-	void *tf_rm_db;
+	void *rm_db;
 };
 
 /**
@@ -138,7 +165,7 @@ struct tf_rm_allocate_parms {
 	/**
 	 * [in] RM DB Handle
 	 */
-	void *tf_rm_db;
+	void *rm_db;
 	/**
 	 * [in] DB Index, indicates which DB entry to perform the
 	 * action on.
@@ -159,7 +186,7 @@ struct tf_rm_free_parms {
 	/**
 	 * [in] RM DB Handle
 	 */
-	void *tf_rm_db;
+	void *rm_db;
 	/**
 	 * [in] DB Index, indicates which DB entry to perform the
 	 * action on.
@@ -168,7 +195,7 @@ struct tf_rm_free_parms {
 	/**
 	 * [in] Index to free
 	 */
-	uint32_t index;
+	uint16_t index;
 };
 
 /**
@@ -178,7 +205,7 @@ struct tf_rm_is_allocated_parms {
 	/**
 	 * [in] RM DB Handle
 	 */
-	void *tf_rm_db;
+	void *rm_db;
 	/**
 	 * [in] DB Index, indicates which DB entry to perform the
 	 * action on.
@@ -191,7 +218,7 @@ struct tf_rm_is_allocated_parms {
 	/**
 	 * [in] Pointer to flag that indicates the state of the query
 	 */
-	uint8_t *allocated;
+	int *allocated;
 };
 
 /**
@@ -201,7 +228,7 @@ struct tf_rm_get_alloc_info_parms {
 	/**
 	 * [in] RM DB Handle
 	 */
-	void *tf_rm_db;
+	void *rm_db;
 	/**
 	 * [in] DB Index, indicates which DB entry to perform the
 	 * action on.
@@ -221,7 +248,7 @@ struct tf_rm_get_hcapi_parms {
 	/**
 	 * [in] RM DB Handle
 	 */
-	void *tf_rm_db;
+	void *rm_db;
 	/**
 	 * [in] DB Index, indicates which DB entry to perform the
 	 * action on.
@@ -306,6 +333,7 @@ int tf_rm_free_db(struct tf *tfp,
  * Returns
  *   - (0) if successful.
  *   - (-EINVAL) on failure.
+ *   - (-ENOMEM) if pool is empty
  */
 int tf_rm_allocate(struct tf_rm_allocate_parms *parms);
 
@@ -317,7 +345,7 @@ int tf_rm_allocate(struct tf_rm_allocate_parms *parms);
  *
  * Returns
  *   - (0) if successful.
- *   - (-EpINVAL) on failure.
+ *   - (-EINVAL) on failure.
  */
 int tf_rm_free(struct tf_rm_free_parms *parms);
 
@@ -365,4 +393,4 @@ int tf_rm_get_info(struct tf_rm_get_alloc_info_parms *parms);
  */
 int tf_rm_get_hcapi_type(struct tf_rm_get_hcapi_parms *parms);
 
-#endif /* TF_RM_H_ */
+#endif /* TF_RM_NEW_H_ */
diff --git a/drivers/net/bnxt/tf_core/tf_session.c b/drivers/net/bnxt/tf_core/tf_session.c
index c749945..2f769d8 100644
--- a/drivers/net/bnxt/tf_core/tf_session.c
+++ b/drivers/net/bnxt/tf_core/tf_session.c
@@ -3,29 +3,293 @@
  * All rights reserved.
  */
 
+#include <string.h>
+
+#include <rte_common.h>
+
+#include "tf_session.h"
+#include "tf_common.h"
+#include "tf_msg.h"
+#include "tfp.h"
+
+int
+tf_session_open_session(struct tf *tfp,
+			struct tf_session_open_session_parms *parms)
+{
+	int rc;
+	struct tf_session *session;
+	struct tfp_calloc_parms cparms;
+	uint8_t fw_session_id;
+	union tf_session_id *session_id;
+
+	TF_CHECK_PARMS2(tfp, parms);
+
+	/* Open FW session and get a new session_id */
+	rc = tf_msg_session_open(tfp,
+				 parms->open_cfg->ctrl_chan_name,
+				 &fw_session_id);
+	if (rc) {
+		/* Log error */
+		if (rc == -EEXIST)
+			TFP_DRV_LOG(ERR,
+				    "Session is already open, rc:%s\n",
+				    strerror(-rc));
+		else
+			TFP_DRV_LOG(ERR,
+				    "Open message send failed, rc:%s\n",
+				    strerror(-rc));
+
+		parms->open_cfg->session_id.id = TF_FW_SESSION_ID_INVALID;
+		return rc;
+	}
+
+	/* Allocate session */
+	cparms.nitems = 1;
+	cparms.size = sizeof(struct tf_session_info);
+	cparms.alignment = 0;
+	rc = tfp_calloc(&cparms);
+	if (rc) {
+		/* Log error */
+		TFP_DRV_LOG(ERR,
+			    "Failed to allocate session info, rc:%s\n",
+			    strerror(-rc));
+		goto cleanup;
+	}
+	tfp->session = (struct tf_session_info *)cparms.mem_va;
+
+	/* Allocate core data for the session */
+	cparms.nitems = 1;
+	cparms.size = sizeof(struct tf_session);
+	cparms.alignment = 0;
+	rc = tfp_calloc(&cparms);
+	if (rc) {
+		/* Log error */
+		TFP_DRV_LOG(ERR,
+			    "Failed to allocate session data, rc:%s\n",
+			    strerror(-rc));
+		goto cleanup;
+	}
+	tfp->session->core_data = cparms.mem_va;
+
+	/* Initialize Session and Device */
+	session = (struct tf_session *)tfp->session->core_data;
+	session->ver.major = 0;
+	session->ver.minor = 0;
+	session->ver.update = 0;
+
+	session_id = &parms->open_cfg->session_id;
+	session->session_id.internal.domain = session_id->internal.domain;
+	session->session_id.internal.bus = session_id->internal.bus;
+	session->session_id.internal.device = session_id->internal.device;
+	session->session_id.internal.fw_session_id = fw_session_id;
+	/* Return the allocated fw session id */
+	session_id->internal.fw_session_id = fw_session_id;
+
+	session->shadow_copy = parms->open_cfg->shadow_copy;
+
+	tfp_memcpy(session->ctrl_chan_name,
+		   parms->open_cfg->ctrl_chan_name,
+		   TF_SESSION_NAME_MAX);
+
+	rc = dev_bind(tfp,
+		      parms->open_cfg->device_type,
+		      session->shadow_copy,
+		      &parms->open_cfg->resources,
+		      session->dev);
+	/* Logging handled by dev_bind */
+	if (rc)
+		return rc;
+
+	/* Query for Session Config
+	 */
+	rc = tf_msg_session_qcfg(tfp);
+	if (rc) {
+		TFP_DRV_LOG(ERR,
+			    "Query config message send failed, rc:%s\n",
+			    strerror(-rc));
+		goto cleanup_close;
+	}
+
+	session->ref_count++;
+
+	return 0;
+
+ cleanup:
+	tfp_free(tfp->session->core_data);
+	tfp_free(tfp->session);
+	tfp->session = NULL;
+	return rc;
+
+ cleanup_close:
+	tf_close_session(tfp);
+	return -EINVAL;
+}
+
+int
+tf_session_attach_session(struct tf *tfp __rte_unused,
+			  struct tf_session_attach_session_parms *parms __rte_unused)
+{
+#if 0
+
+	/* A shared session is similar to single session. It consists
+	 * of two parts the tf_session_info element which remains
+	 * private to the caller and the session within this element
+	 * which is shared. The session it self holds the dynamic
+	 * data, i.e. the device and its sub modules.
+	 *
+	 * Firmware side is updated about any sharing as well.
+	 */
+
+	/* - Open the shared memory for the attach_chan_name
+	 * - Point to the shared session for this Device instance
+	 * - Check that session is valid
+	 * - Attach to the firmware so it can record there is more
+	 *   than one client of the session.
+	 */
+
+	if (tfp->session->session_id.id != TF_SESSION_ID_INVALID) {
+		rc = tf_msg_session_attach(tfp,
+					   parms->ctrl_chan_name,
+					   parms->session_id);
+	}
+#endif /* 0 */
+	int rc = -EOPNOTSUPP;
+
+	TF_CHECK_PARMS2(tfp, parms);
+
+	TFP_DRV_LOG(ERR,
+		    "Attach not yet supported, rc:%s\n",
+		    strerror(-rc));
+	return rc;
+}
+
+int
+tf_session_close_session(struct tf *tfp,
+			 struct tf_session_close_session_parms *parms)
+{
+	int rc;
+	struct tf_session *tfs;
+	struct tf_dev_info *tfd;
+
+	TF_CHECK_PARMS2(tfp, parms);
+
+	rc = tf_session_get_session(tfp, &tfs);
+	if (rc) {
+		TFP_DRV_LOG(ERR,
+			    "Session lookup failed, rc:%s\n",
+			    strerror(-rc));
+		return rc;
+	}
+
+	if (tfs->session_id.id == TF_SESSION_ID_INVALID) {
+		rc = -EINVAL;
+		TFP_DRV_LOG(ERR,
+			    "Invalid session id, unable to close, rc:%s\n",
+			    strerror(-rc));
+		return rc;
+	}
+
+	/* Record the session we're closing so the caller knows the
+	 * details.
+	 */
+	*parms->session_id = tfs->session_id;
+
+	rc = tf_session_get_device(tfs, &tfd);
+	if (rc) {
+		TFP_DRV_LOG(ERR,
+			    "Device lookup failed, rc:%s\n",
+			    strerror(-rc));
+		return rc;
+	}
+
+	/* In case we're attached only the session client gets closed */
+	rc = tf_msg_session_close(tfp);
+	if (rc) {
+		/* Log error */
+		TFP_DRV_LOG(ERR,
+			    "FW Session close failed, rc:%s\n",
+			    strerror(-rc));
+	}
+
+	tfs->ref_count--;
+
+	/* Final cleanup as we're last user of the session */
+	if (tfs->ref_count == 0) {
+		/* Unbind the device */
+		rc = dev_unbind(tfp, tfd);
+		if (rc) {
+			/* Log error */
+			TFP_DRV_LOG(ERR,
+				    "Device unbind failed, rc:%s\n",
+				    strerror(-rc));
+		}
+
+		tfp_free(tfp->session->core_data);
+		tfp_free(tfp->session);
+		tfp->session = NULL;
+	}
+
+	return 0;
+}
+
 int
 tf_session_get_session(struct tf *tfp,
-		       struct tf_session *tfs)
+		       struct tf_session **tfs)
 {
+	int rc;
+
 	if (tfp->session == NULL || tfp->session->core_data == NULL) {
-		TFP_DRV_LOG(ERR, "Session not created\n");
-		return -EINVAL;
+		rc = -EINVAL;
+		TFP_DRV_LOG(ERR,
+			    "Session not created, rc:%s\n",
+			    strerror(-rc));
+		return rc;
 	}
 
-	tfs = (struct tf_session *)(tfp->session->core_data);
+	*tfs = (struct tf_session *)(tfp->session->core_data);
 
 	return 0;
 }
 
 int
 tf_session_get_device(struct tf_session *tfs,
-		      struct tf_device *tfd)
+		      struct tf_dev_info **tfd)
 {
+	int rc;
+
 	if (tfs->dev == NULL) {
-		TFP_DRV_LOG(ERR, "Device not created\n");
-		return -EINVAL;
+		rc = -EINVAL;
+		TFP_DRV_LOG(ERR,
+			    "Device not created, rc:%s\n",
+			    strerror(-rc));
+		return rc;
 	}
-	tfd = tfs->dev;
+
+	*tfd = tfs->dev;
+
+	return 0;
+}
+
+int
+tf_session_get_fw_session_id(struct tf *tfp,
+			     uint8_t *fw_session_id)
+{
+	int rc;
+	struct tf_session *tfs;
+
+	if (tfp->session == NULL) {
+		rc = -EINVAL;
+		TFP_DRV_LOG(ERR,
+			    "Session not created, rc:%s\n",
+			    strerror(-rc));
+		return rc;
+	}
+
+	rc = tf_session_get_session(tfp, &tfs);
+	if (rc)
+		return rc;
+
+	*fw_session_id = tfs->session_id.internal.fw_session_id;
 
 	return 0;
 }
diff --git a/drivers/net/bnxt/tf_core/tf_session.h b/drivers/net/bnxt/tf_core/tf_session.h
index b1cc7a4..9279251 100644
--- a/drivers/net/bnxt/tf_core/tf_session.h
+++ b/drivers/net/bnxt/tf_core/tf_session.h
@@ -63,12 +63,7 @@ struct tf_session {
 	 */
 	struct tf_session_version ver;
 
-	/** Device type, provided by tf_open_session().
-	 */
-	enum tf_device_type device_type;
-
-	/** Session ID, allocated by FW on tf_open_session().
-	 */
+	/** Session ID, allocated by FW on tf_open_session() */
 	union tf_session_id session_id;
 
 	/**
@@ -101,7 +96,7 @@ struct tf_session {
 	 */
 	uint8_t ref_count;
 
-	/** Device */
+	/** Device handle */
 	struct tf_dev_info *dev;
 
 	/** Session HW and SRAM resources */
@@ -324,12 +319,96 @@ struct tf_session {
 };
 
 /**
+ * Session open parameter definition
+ */
+struct tf_session_open_session_parms {
+	/**
+	 * [in] Pointer to the TF open session configuration
+	 */
+	struct tf_open_session_parms *open_cfg;
+};
+
+/**
+ * Session attach parameter definition
+ */
+struct tf_session_attach_session_parms {
+	/**
+	 * [in] Pointer to the TF attach session configuration
+	 */
+	struct tf_attach_session_parms *attach_cfg;
+};
+
+/**
+ * Session close parameter definition
+ */
+struct tf_session_close_session_parms {
+	uint8_t *ref_count;
+	union tf_session_id *session_id;
+};
+
+/**
  * @page session Session Management
  *
+ * @ref tf_session_open_session
+ *
+ * @ref tf_session_attach_session
+ *
+ * @ref tf_session_close_session
+ *
  * @ref tf_session_get_session
  *
  * @ref tf_session_get_device
+ *
+ * @ref tf_session_get_fw_session_id
+ */
+
+/**
+ * Creates a host session with a corresponding firmware session.
+ *
+ * [in] tfp
+ *   Pointer to TF handle
+ *
+ * [in] parms
+ *   Pointer to the session open parameters
+ *
+ * Returns
+ *   - (0) if successful.
+ *   - (-EINVAL) on failure.
  */
+int tf_session_open_session(struct tf *tfp,
+			    struct tf_session_open_session_parms *parms);
+
+/**
+ * Attaches a previous created session.
+ *
+ * [in] tfp
+ *   Pointer to TF handle
+ *
+ * [in] parms
+ *   Pointer to the session attach parameters
+ *
+ * Returns
+ *   - (0) if successful.
+ *   - (-EINVAL) on failure.
+ */
+int tf_session_attach_session(struct tf *tfp,
+			      struct tf_session_attach_session_parms *parms);
+
+/**
+ * Closes a previous created session.
+ *
+ * [in] tfp
+ *   Pointer to TF handle
+ *
+ * [in/out] parms
+ *   Pointer to the session close parameters.
+ *
+ * Returns
+ *   - (0) if successful.
+ *   - (-EINVAL) on failure.
+ */
+int tf_session_close_session(struct tf *tfp,
+			     struct tf_session_close_session_parms *parms);
 
 /**
  * Looks up the private session information from the TF session info.
@@ -338,14 +417,14 @@ struct tf_session {
  *   Pointer to TF handle
  *
  * [out] tfs
- *   Pointer to the session
+ *   Pointer pointer to the session
  *
  * Returns
  *   - (0) if successful.
  *   - (-EINVAL) on failure.
  */
 int tf_session_get_session(struct tf *tfp,
-			   struct tf_session *tfs);
+			   struct tf_session **tfs);
 
 /**
  * Looks up the device information from the TF Session.
@@ -354,13 +433,30 @@ int tf_session_get_session(struct tf *tfp,
  *   Pointer to TF handle
  *
  * [out] tfd
- *   Pointer to the device
+ *   Pointer pointer to the device
  *
  * Returns
  *   - (0) if successful.
  *   - (-EINVAL) on failure.
  */
 int tf_session_get_device(struct tf_session *tfs,
-			  struct tf_dev_info *tfd);
+			  struct tf_dev_info **tfd);
+
+/**
+ * Looks up the FW session id of the firmware connection for the
+ * requested TF handle.
+ *
+ * [in] tfp
+ *   Pointer to TF handle
+ *
+ * [out] session_id
+ *   Pointer to the session_id
+ *
+ * Returns
+ *   - (0) if successful.
+ *   - (-EINVAL) on failure.
+ */
+int tf_session_get_fw_session_id(struct tf *tfp,
+				 uint8_t *fw_session_id);
 
 #endif /* _TF_SESSION_H_ */
diff --git a/drivers/net/bnxt/tf_core/tf_tbl.h b/drivers/net/bnxt/tf_core/tf_tbl.h
index 6cda487..b335a9c 100644
--- a/drivers/net/bnxt/tf_core/tf_tbl.h
+++ b/drivers/net/bnxt/tf_core/tf_tbl.h
@@ -7,8 +7,12 @@
 #define _TF_TBL_H_
 
 #include <stdint.h>
+
+#include "tf_core.h"
 #include "stack.h"
 
+struct tf_session;
+
 enum tf_pg_tbl_lvl {
 	PT_LVL_0,
 	PT_LVL_1,
diff --git a/drivers/net/bnxt/tf_core/tf_tbl_type.c b/drivers/net/bnxt/tf_core/tf_tbl_type.c
index a57a5dd..b79706f 100644
--- a/drivers/net/bnxt/tf_core/tf_tbl_type.c
+++ b/drivers/net/bnxt/tf_core/tf_tbl_type.c
@@ -10,12 +10,12 @@
 struct tf;
 
 /**
- * Table Type DBs.
+ * Table DBs.
  */
 /* static void *tbl_db[TF_DIR_MAX]; */
 
 /**
- * Table Type Shadow DBs
+ * Table Shadow DBs
  */
 /* static void *shadow_tbl_db[TF_DIR_MAX]; */
 
@@ -30,49 +30,49 @@ struct tf;
 /* static uint8_t shadow_init; */
 
 int
-tf_tbl_type_bind(struct tf *tfp __rte_unused,
-		 struct tf_tbl_type_cfg_parms *parms __rte_unused)
+tf_tbl_bind(struct tf *tfp __rte_unused,
+	    struct tf_tbl_cfg_parms *parms __rte_unused)
 {
 	return 0;
 }
 
 int
-tf_tbl_type_unbind(struct tf *tfp __rte_unused)
+tf_tbl_unbind(struct tf *tfp __rte_unused)
 {
 	return 0;
 }
 
 int
-tf_tbl_type_alloc(struct tf *tfp __rte_unused,
-		  struct tf_tbl_type_alloc_parms *parms __rte_unused)
+tf_tbl_alloc(struct tf *tfp __rte_unused,
+	     struct tf_tbl_alloc_parms *parms __rte_unused)
 {
 	return 0;
 }
 
 int
-tf_tbl_type_free(struct tf *tfp __rte_unused,
-		 struct tf_tbl_type_free_parms *parms __rte_unused)
+tf_tbl_free(struct tf *tfp __rte_unused,
+	    struct tf_tbl_free_parms *parms __rte_unused)
 {
 	return 0;
 }
 
 int
-tf_tbl_type_alloc_search(struct tf *tfp __rte_unused,
-			 struct tf_tbl_type_alloc_search_parms *parms __rte_unused)
+tf_tbl_alloc_search(struct tf *tfp __rte_unused,
+		    struct tf_tbl_alloc_search_parms *parms __rte_unused)
 {
 	return 0;
 }
 
 int
-tf_tbl_type_set(struct tf *tfp __rte_unused,
-		struct tf_tbl_type_set_parms *parms __rte_unused)
+tf_tbl_set(struct tf *tfp __rte_unused,
+	   struct tf_tbl_set_parms *parms __rte_unused)
 {
 	return 0;
 }
 
 int
-tf_tbl_type_get(struct tf *tfp __rte_unused,
-		struct tf_tbl_type_get_parms *parms __rte_unused)
+tf_tbl_get(struct tf *tfp __rte_unused,
+	   struct tf_tbl_get_parms *parms __rte_unused)
 {
 	return 0;
 }
diff --git a/drivers/net/bnxt/tf_core/tf_tbl_type.h b/drivers/net/bnxt/tf_core/tf_tbl_type.h
index c880b36..11f2aa3 100644
--- a/drivers/net/bnxt/tf_core/tf_tbl_type.h
+++ b/drivers/net/bnxt/tf_core/tf_tbl_type.h
@@ -11,33 +11,39 @@
 struct tf;
 
 /**
- * The Table Type module provides processing of Internal TF table types.
+ * The Table module provides processing of Internal TF table types.
  */
 
 /**
- * Table Type configuration parameters
+ * Table configuration parameters
  */
-struct tf_tbl_type_cfg_parms {
+struct tf_tbl_cfg_parms {
 	/**
 	 * Number of table types in each of the configuration arrays
 	 */
 	uint16_t num_elements;
-
 	/**
 	 * Table Type element configuration array
 	 */
-	struct tf_rm_element_cfg *tbl_cfg[TF_DIR_MAX];
-
+	struct tf_rm_element_cfg *cfg;
 	/**
 	 * Shadow table type configuration array
 	 */
-	struct tf_shadow_tbl_type_cfg *tbl_shadow_cfg[TF_DIR_MAX];
+	struct tf_shadow_tbl_cfg *shadow_cfg;
+	/**
+	 * Boolean controlling the request shadow copy.
+	 */
+	bool shadow_copy;
+	/**
+	 * Session resource allocations
+	 */
+	struct tf_session_resources *resources;
 };
 
 /**
- * Table Type allocation parameters
+ * Table allocation parameters
  */
-struct tf_tbl_type_alloc_parms {
+struct tf_tbl_alloc_parms {
 	/**
 	 * [in] Receive or transmit direction
 	 */
@@ -53,9 +59,9 @@ struct tf_tbl_type_alloc_parms {
 };
 
 /**
- * Table Type free parameters
+ * Table free parameters
  */
-struct tf_tbl_type_free_parms {
+struct tf_tbl_free_parms {
 	/**
 	 * [in] Receive or transmit direction
 	 */
@@ -75,7 +81,10 @@ struct tf_tbl_type_free_parms {
 	uint16_t ref_cnt;
 };
 
-struct tf_tbl_type_alloc_search_parms {
+/**
+ * Table allocate search parameters
+ */
+struct tf_tbl_alloc_search_parms {
 	/**
 	 * [in] Receive or transmit direction
 	 */
@@ -117,9 +126,9 @@ struct tf_tbl_type_alloc_search_parms {
 };
 
 /**
- * Table Type set parameters
+ * Table set parameters
  */
-struct tf_tbl_type_set_parms {
+struct tf_tbl_set_parms {
 	/**
 	 * [in] Receive or transmit direction
 	 */
@@ -143,9 +152,9 @@ struct tf_tbl_type_set_parms {
 };
 
 /**
- * Table Type get parameters
+ * Table get parameters
  */
-struct tf_tbl_type_get_parms {
+struct tf_tbl_get_parms {
 	/**
 	 * [in] Receive or transmit direction
 	 */
@@ -169,39 +178,39 @@ struct tf_tbl_type_get_parms {
 };
 
 /**
- * @page tbl_type Table Type
+ * @page tbl Table
  *
- * @ref tf_tbl_type_bind
+ * @ref tf_tbl_bind
  *
- * @ref tf_tbl_type_unbind
+ * @ref tf_tbl_unbind
  *
- * @ref tf_tbl_type_alloc
+ * @ref tf_tbl_alloc
  *
- * @ref tf_tbl_type_free
+ * @ref tf_tbl_free
  *
- * @ref tf_tbl_type_alloc_search
+ * @ref tf_tbl_alloc_search
  *
- * @ref tf_tbl_type_set
+ * @ref tf_tbl_set
  *
- * @ref tf_tbl_type_get
+ * @ref tf_tbl_get
  */
 
 /**
- * Initializes the Table Type module with the requested DBs. Must be
+ * Initializes the Table module with the requested DBs. Must be
  * invoked as the first thing before any of the access functions.
  *
  * [in] tfp
  *   Pointer to TF handle, used for HCAPI communication
  *
  * [in] parms
- *   Pointer to parameters
+ *   Pointer to Table configuration parameters
  *
  * Returns
  *   - (0) if successful.
  *   - (-EINVAL) on failure.
  */
-int tf_tbl_type_bind(struct tf *tfp,
-		     struct tf_tbl_type_cfg_parms *parms);
+int tf_tbl_bind(struct tf *tfp,
+		struct tf_tbl_cfg_parms *parms);
 
 /**
  * Cleans up the private DBs and releases all the data.
@@ -216,7 +225,7 @@ int tf_tbl_type_bind(struct tf *tfp,
  *   - (0) if successful.
  *   - (-EINVAL) on failure.
  */
-int tf_tbl_type_unbind(struct tf *tfp);
+int tf_tbl_unbind(struct tf *tfp);
 
 /**
  * Allocates the requested table type from the internal RM DB.
@@ -225,14 +234,14 @@ int tf_tbl_type_unbind(struct tf *tfp);
  *   Pointer to TF handle, used for HCAPI communication
  *
  * [in] parms
- *   Pointer to parameters
+ *   Pointer to Table allocation parameters
  *
  * Returns
  *   - (0) if successful.
  *   - (-EINVAL) on failure.
  */
-int tf_tbl_type_alloc(struct tf *tfp,
-		      struct tf_tbl_type_alloc_parms *parms);
+int tf_tbl_alloc(struct tf *tfp,
+		 struct tf_tbl_alloc_parms *parms);
 
 /**
  * Free's the requested table type and returns it to the DB. If shadow
@@ -244,14 +253,14 @@ int tf_tbl_type_alloc(struct tf *tfp,
  *   Pointer to TF handle, used for HCAPI communication
  *
  * [in] parms
- *   Pointer to parameters
+ *   Pointer to Table free parameters
  *
  * Returns
  *   - (0) if successful.
  *   - (-EINVAL) on failure.
  */
-int tf_tbl_type_free(struct tf *tfp,
-		     struct tf_tbl_type_free_parms *parms);
+int tf_tbl_free(struct tf *tfp,
+		struct tf_tbl_free_parms *parms);
 
 /**
  * Supported if Shadow DB is configured. Searches the Shadow DB for
@@ -269,8 +278,8 @@ int tf_tbl_type_free(struct tf *tfp,
  *   - (0) if successful.
  *   - (-EINVAL) on failure.
  */
-int tf_tbl_type_alloc_search(struct tf *tfp,
-			     struct tf_tbl_type_alloc_search_parms *parms);
+int tf_tbl_alloc_search(struct tf *tfp,
+			struct tf_tbl_alloc_search_parms *parms);
 
 /**
  * Configures the requested element by sending a firmware request which
@@ -280,14 +289,14 @@ int tf_tbl_type_alloc_search(struct tf *tfp,
  *   Pointer to TF handle, used for HCAPI communication
  *
  * [in] parms
- *   Pointer to parameters
+ *   Pointer to Table set parameters
  *
  * Returns
  *   - (0) if successful.
  *   - (-EINVAL) on failure.
  */
-int tf_tbl_type_set(struct tf *tfp,
-		    struct tf_tbl_type_set_parms *parms);
+int tf_tbl_set(struct tf *tfp,
+	       struct tf_tbl_set_parms *parms);
 
 /**
  * Retrieves the requested element by sending a firmware request to get
@@ -297,13 +306,13 @@ int tf_tbl_type_set(struct tf *tfp,
  *   Pointer to TF handle, used for HCAPI communication
  *
  * [in] parms
- *   Pointer to parameters
+ *   Pointer to Table get parameters
  *
  * Returns
  *   - (0) if successful.
  *   - (-EINVAL) on failure.
  */
-int tf_tbl_type_get(struct tf *tfp,
-		    struct tf_tbl_type_get_parms *parms);
+int tf_tbl_get(struct tf *tfp,
+	       struct tf_tbl_get_parms *parms);
 
 #endif /* TF_TBL_TYPE_H */
diff --git a/drivers/net/bnxt/tf_core/tf_tcam.h b/drivers/net/bnxt/tf_core/tf_tcam.h
index 1420c9e..68c25eb 100644
--- a/drivers/net/bnxt/tf_core/tf_tcam.h
+++ b/drivers/net/bnxt/tf_core/tf_tcam.h
@@ -20,16 +20,22 @@ struct tf_tcam_cfg_parms {
 	 * Number of tcam types in each of the configuration arrays
 	 */
 	uint16_t num_elements;
-
 	/**
 	 * TCAM configuration array
 	 */
-	struct tf_rm_element_cfg *tcam_cfg[TF_DIR_MAX];
-
+	struct tf_rm_element_cfg *cfg;
 	/**
 	 * Shadow table type configuration array
 	 */
-	struct tf_shadow_tcam_cfg *tcam_shadow_cfg[TF_DIR_MAX];
+	struct tf_shadow_tcam_cfg *shadow_cfg;
+	/**
+	 * Boolean controlling the request shadow copy.
+	 */
+	bool shadow_copy;
+	/**
+	 * Session resource allocations
+	 */
+	struct tf_session_resources *resources;
 };
 
 /**
-- 
2.7.4


^ permalink raw reply	[flat|nested] 271+ messages in thread

* [dpdk-dev] [PATCH 14/50] net/bnxt: support two-level priority for TCAMs
  2020-06-12 13:28 [dpdk-dev] [PATCH 00/50] add features for host-based flow management Somnath Kotur
                   ` (12 preceding siblings ...)
  2020-06-12 13:28 ` [dpdk-dev] [PATCH 13/50] net/bnxt: update multi device design support Somnath Kotur
@ 2020-06-12 13:28 ` Somnath Kotur
  2020-06-12 13:28 ` [dpdk-dev] [PATCH 15/50] net/bnxt: add HCAPI interface support Somnath Kotur
                   ` (36 subsequent siblings)
  50 siblings, 0 replies; 271+ messages in thread
From: Somnath Kotur @ 2020-06-12 13:28 UTC (permalink / raw)
  To: dev; +Cc: ferruh.yigit

From: Shahaji Bhosle <sbhosle@broadcom.com>

Allow TCAM indexes to be allocated from top or bottom.
If the priority is set to 0, allocate from the
lowest tcam indexes i.e. from top. Any other value,
allocate it from the highest tcam indexes i.e. from
bottom.

Signed-off-by: Shahaji Bhosle <sbhosle@broadcom.com>
Reviewed-by: Randy Schacher <stuart.schacher@broadcom.com>
Reviewed-by: Ajit Kumar Khaparde <ajit.khaparde@broadcom.com>
Signed-off-by: Venkat Duvvuru <venkatkumar.duvvuru@broadcom.com>
---
 drivers/net/bnxt/tf_core/tf_core.c | 36 +++++++++++++++++++++++++++++-------
 drivers/net/bnxt/tf_core/tf_core.h |  4 +++-
 drivers/net/bnxt/tf_core/tf_em.c   |  6 ++----
 drivers/net/bnxt/tf_core/tf_tbl.c  |  2 +-
 4 files changed, 35 insertions(+), 13 deletions(-)

diff --git a/drivers/net/bnxt/tf_core/tf_core.c b/drivers/net/bnxt/tf_core/tf_core.c
index 28a6bbd..7d9bca8 100644
--- a/drivers/net/bnxt/tf_core/tf_core.c
+++ b/drivers/net/bnxt/tf_core/tf_core.c
@@ -893,7 +893,7 @@ tf_alloc_tcam_entry(struct tf *tfp,
 		    struct tf_alloc_tcam_entry_parms *parms)
 {
 	int rc;
-	int index;
+	int index = 0;
 	struct tf_session *tfs;
 	struct bitalloc *session_pool;
 
@@ -916,12 +916,34 @@ tf_alloc_tcam_entry(struct tf *tfp,
 	if (rc)
 		return rc;
 
-	index = ba_alloc(session_pool);
-	if (index == BA_FAIL) {
-		PMD_DRV_LOG(ERR, "%s: %s: No resource available\n",
-			    tf_dir_2_str(parms->dir),
-			    tf_tcam_tbl_2_str(parms->tcam_tbl_type));
-		return -ENOMEM;
+	/*
+	 * priority  0: allocate from top of the tcam i.e. high
+	 * priority !0: allocate index from bottom i.e lowest
+	 */
+	if (parms->priority) {
+		for (index = session_pool->size - 1; index >= 0; index--) {
+			if (ba_inuse(session_pool,
+					  index) == BA_ENTRY_FREE) {
+				break;
+			}
+		}
+		if (ba_alloc_index(session_pool,
+				   index) == BA_FAIL) {
+			TFP_DRV_LOG(ERR,
+				    "%s: %s: ba_alloc index %d failed\n",
+				    tf_dir_2_str(parms->dir),
+				    tf_tcam_tbl_2_str(parms->tcam_tbl_type),
+				    index);
+			return -ENOMEM;
+		}
+	} else {
+		index = ba_alloc(session_pool);
+		if (index == BA_FAIL) {
+			TFP_DRV_LOG(ERR, "%s: %s: Out of resource\n",
+				    tf_dir_2_str(parms->dir),
+				    tf_tcam_tbl_2_str(parms->tcam_tbl_type));
+			return -ENOMEM;
+		}
 	}
 
 	parms->idx = index;
diff --git a/drivers/net/bnxt/tf_core/tf_core.h b/drivers/net/bnxt/tf_core/tf_core.h
index 74ed24e..f1ef00b 100644
--- a/drivers/net/bnxt/tf_core/tf_core.h
+++ b/drivers/net/bnxt/tf_core/tf_core.h
@@ -799,7 +799,9 @@ struct tf_alloc_tcam_entry_parms {
 	 */
 	uint8_t *mask;
 	/**
-	 * [in] Priority of entry requested (definition TBD)
+	 * [in] Priority of entry requested
+	 * 0: index from top i.e. highest priority first
+	 * !0: index from bottom i.e lowest priority first
 	 */
 	uint32_t priority;
 	/**
diff --git a/drivers/net/bnxt/tf_core/tf_em.c b/drivers/net/bnxt/tf_core/tf_em.c
index fd1797e..91cbc62 100644
--- a/drivers/net/bnxt/tf_core/tf_em.c
+++ b/drivers/net/bnxt/tf_core/tf_em.c
@@ -479,8 +479,7 @@ int tf_insert_em_internal_entry(struct tf *tfp,
 	rc = stack_pop(pool, &index);
 
 	if (rc != 0) {
-		PMD_DRV_LOG
-		   (ERR,
+		TFP_DRV_LOG(ERR,
 		   "dir:%d, EM entry index allocation failed\n",
 		   parms->dir);
 		return rc;
@@ -495,8 +494,7 @@ int tf_insert_em_internal_entry(struct tf *tfp,
 	if (rc != 0)
 		return -1;
 
-	PMD_DRV_LOG
-		   (ERR,
+	TFP_DRV_LOG(INFO,
 		   "Internal entry @ Index:%d rptr_index:0x%x rptr_entry:0x%x num_of_entries:%d\n",
 		   index * TF_SESSION_EM_ENTRY_SIZE,
 		   rptr_index,
diff --git a/drivers/net/bnxt/tf_core/tf_tbl.c b/drivers/net/bnxt/tf_core/tf_tbl.c
index 0f2979e..f9bfae7 100644
--- a/drivers/net/bnxt/tf_core/tf_tbl.c
+++ b/drivers/net/bnxt/tf_core/tf_tbl.c
@@ -1967,7 +1967,7 @@ void tf_dump_dma(struct tf *tfp, uint32_t tbl_scope_id)
 	tbl_scope_cb = tbl_scope_cb_find(session,
 					 tbl_scope_id);
 	if (tbl_scope_cb == NULL)
-		PMD_DRV_LOG(ERR, "No table scope\n");
+		TFP_DRV_LOG(ERR, "No table scope\n");
 
 	for (dir = 0; dir < TF_DIR_MAX; dir++) {
 		printf("Direction %s:\n", (dir == TF_DIR_RX ? "Rx" : "Tx"));
-- 
2.7.4


^ permalink raw reply	[flat|nested] 271+ messages in thread

* [dpdk-dev] [PATCH 15/50] net/bnxt: add HCAPI interface support
  2020-06-12 13:28 [dpdk-dev] [PATCH 00/50] add features for host-based flow management Somnath Kotur
                   ` (13 preceding siblings ...)
  2020-06-12 13:28 ` [dpdk-dev] [PATCH 14/50] net/bnxt: support two-level priority for TCAMs Somnath Kotur
@ 2020-06-12 13:28 ` Somnath Kotur
  2020-06-12 13:29 ` [dpdk-dev] [PATCH 16/50] net/bnxt: add core changes for EM and EEM lookups Somnath Kotur
                   ` (35 subsequent siblings)
  50 siblings, 0 replies; 271+ messages in thread
From: Somnath Kotur @ 2020-06-12 13:28 UTC (permalink / raw)
  To: dev; +Cc: ferruh.yigit

From: Pete Spreadborough <peter.spreadborough@broadcom.com>

Add new hardware shim APIs to support multiple
device generations

Signed-off-by: Pete Spreadborough <peter.spreadborough@broadcom.com>
Reviewed-by: Randy Schacher <stuart.schacher@broadcom.com>
Reviewed-by: Ajit Kumar Khaparde <ajit.khaparde@broadcom.com>
Signed-off-by: Venkat Duvvuru <venkatkumar.duvvuru@broadcom.com>
---
 drivers/net/bnxt/hcapi/Makefile           |   7 +
 drivers/net/bnxt/hcapi/hcapi_cfa.h        | 272 ++++++++++++
 drivers/net/bnxt/hcapi/hcapi_cfa_common.c |  92 ++++
 drivers/net/bnxt/hcapi/hcapi_cfa_defs.h   | 669 ++++++++++++++++++++++++++++++
 drivers/net/bnxt/hcapi/hcapi_cfa_p4.c     | 411 ++++++++++++++++++
 drivers/net/bnxt/hcapi/hcapi_cfa_p4.h     | 451 ++++++++++++++++++++
 drivers/net/bnxt/meson.build              |   3 +
 drivers/net/bnxt/tf_core/tf_em.c          |  22 +-
 drivers/net/bnxt/tf_core/tf_tbl.c         |  92 ++--
 drivers/net/bnxt/tf_core/tf_tbl.h         |  24 +-
 10 files changed, 1974 insertions(+), 69 deletions(-)
 create mode 100644 drivers/net/bnxt/hcapi/Makefile
 create mode 100644 drivers/net/bnxt/hcapi/hcapi_cfa.h
 create mode 100644 drivers/net/bnxt/hcapi/hcapi_cfa_common.c
 create mode 100644 drivers/net/bnxt/hcapi/hcapi_cfa_defs.h
 create mode 100644 drivers/net/bnxt/hcapi/hcapi_cfa_p4.c
 create mode 100644 drivers/net/bnxt/hcapi/hcapi_cfa_p4.h

diff --git a/drivers/net/bnxt/hcapi/Makefile b/drivers/net/bnxt/hcapi/Makefile
new file mode 100644
index 0000000..c4c91b6
--- /dev/null
+++ b/drivers/net/bnxt/hcapi/Makefile
@@ -0,0 +1,7 @@
+# SPDX-License-Identifier: BSD-3-Clause
+# Copyright(c) 2019-2020 Broadcom Limited.
+# All rights reserved.
+
+SRCS-$(CONFIG_RTE_LIBRTE_BNXT_PMD) += hcapi/hcapi_cfa_common.c
+SRCS-$(CONFIG_RTE_LIBRTE_BNXT_PMD) += hcapi/hcapi_cfa_p4.c
+
diff --git a/drivers/net/bnxt/hcapi/hcapi_cfa.h b/drivers/net/bnxt/hcapi/hcapi_cfa.h
new file mode 100644
index 0000000..a27c749
--- /dev/null
+++ b/drivers/net/bnxt/hcapi/hcapi_cfa.h
@@ -0,0 +1,272 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2019-2020 Broadcom
+ * All rights reserved.
+ */
+
+#ifndef _HCAPI_CFA_H_
+#define _HCAPI_CFA_H_
+
+#include <stdio.h>
+#include <string.h>
+#include <stdbool.h>
+#include <stdint.h>
+#include <stddef.h>
+
+#include "hcapi_cfa_defs.h"
+
+#define SUPPORT_CFA_HW_P4  1
+
+#if SUPPORT_CFA_HW_P4 && SUPPORT_CFA_HW_P58 && SUPPORT_CFA_HW_P59
+#define SUPPORT_CFA_HW_ALL  1
+#endif
+
+/**
+ * Index used for the sram_entries field
+ */
+enum hcapi_cfa_resc_type_sram {
+	HCAPI_CFA_RESC_TYPE_SRAM_FULL_ACTION,
+	HCAPI_CFA_RESC_TYPE_SRAM_MCG,
+	HCAPI_CFA_RESC_TYPE_SRAM_ENCAP_8B,
+	HCAPI_CFA_RESC_TYPE_SRAM_ENCAP_16B,
+	HCAPI_CFA_RESC_TYPE_SRAM_ENCAP_64B,
+	HCAPI_CFA_RESC_TYPE_SRAM_SP_SMAC,
+	HCAPI_CFA_RESC_TYPE_SRAM_SP_SMAC_IPV4,
+	HCAPI_CFA_RESC_TYPE_SRAM_SP_SMAC_IPV6,
+	HCAPI_CFA_RESC_TYPE_SRAM_COUNTER_64B,
+	HCAPI_CFA_RESC_TYPE_SRAM_NAT_SPORT,
+	HCAPI_CFA_RESC_TYPE_SRAM_NAT_DPORT,
+	HCAPI_CFA_RESC_TYPE_SRAM_NAT_S_IPV4,
+	HCAPI_CFA_RESC_TYPE_SRAM_NAT_D_IPV4,
+	HCAPI_CFA_RESC_TYPE_SRAM_MAX
+};
+
+/**
+ * Index used for the hw_entries field in struct cfa_rm_db
+ */
+enum hcapi_cfa_resc_type_hw {
+	/* common HW resources for all chip variants */
+	HCAPI_CFA_RESC_TYPE_HW_L2_CTXT_TCAM,
+	HCAPI_CFA_RESC_TYPE_HW_PROF_FUNC,
+	HCAPI_CFA_RESC_TYPE_HW_PROF_TCAM,
+	HCAPI_CFA_RESC_TYPE_HW_EM_PROF_ID,
+	HCAPI_CFA_RESC_TYPE_HW_EM_REC,
+	HCAPI_CFA_RESC_TYPE_HW_WC_TCAM_PROF_ID,
+	HCAPI_CFA_RESC_TYPE_HW_WC_TCAM,
+	HCAPI_CFA_RESC_TYPE_HW_METER_PROF,
+	HCAPI_CFA_RESC_TYPE_HW_METER_INST,
+	HCAPI_CFA_RESC_TYPE_HW_MIRROR,
+	HCAPI_CFA_RESC_TYPE_HW_UPAR,
+	/* Wh+/SR specific HW resources */
+	HCAPI_CFA_RESC_TYPE_HW_SP_TCAM,
+	/* Thor, SR2 common HW resources */
+	HCAPI_CFA_RESC_TYPE_HW_FKB,
+	/* SR specific HW resources */
+	HCAPI_CFA_RESC_TYPE_HW_TBL_SCOPE,
+	HCAPI_CFA_RESC_TYPE_HW_L2_FUNC,
+	HCAPI_CFA_RESC_TYPE_HW_EPOCH0,
+	HCAPI_CFA_RESC_TYPE_HW_EPOCH1,
+	HCAPI_CFA_RESC_TYPE_HW_METADATA,
+	HCAPI_CFA_RESC_TYPE_HW_CT_STATE,
+	HCAPI_CFA_RESC_TYPE_HW_RANGE_PROF,
+	HCAPI_CFA_RESC_TYPE_HW_RANGE_ENTRY,
+	HCAPI_CFA_RESC_TYPE_HW_LAG_ENTRY,
+	HCAPI_CFA_RESC_TYPE_HW_MAX
+};
+
+struct hcapi_cfa_key_result {
+	uint64_t bucket_mem_ptr;
+	uint8_t bucket_idx;
+};
+
+/* common CFA register access macros */
+#define CFA_REG(x)		OFFSETOF(cfa_reg_t, cfa_##x)
+
+#ifndef REG_WR
+#define REG_WR(_p, x, y)  (*((uint32_t volatile *)(x)) = (y))
+#endif
+#ifndef REG_RD
+#define REG_RD(_p, x)  (*((uint32_t volatile *)(x)))
+#endif
+#define CFA_REG_RD(_p, x)	REG_RD(0, (uint32_t)(_p)->base_addr + CFA_REG(x))
+#define CFA_REG_WR(_p, x, y)	REG_WR(0, (uint32_t)(_p)->base_addr + CFA_REG(x), y)
+
+
+/* Constants used by Resource Manager Registration*/
+#define RM_CLIENT_NAME_MAX_LEN          32
+
+/**
+ *  Resource Manager Data Structures used for resource requests
+ */
+struct hcapi_cfa_resc_req_entry {
+	uint16_t min;
+	uint16_t max;
+};
+
+struct hcapi_cfa_resc_req {
+	/* Wh+/SR specific onchip Action SRAM resources */
+	/* Validity of each sram type is indicated by the
+	 * corresponding sram type bit in the sram_resc_flags. When
+	 * set to 1, the CFA sram resource type is valid and amount of
+	 * resources for this type is reserved. Each sram resource
+	 * pool is identified by the starting index and number of
+	 * resources in the pool.
+	 */
+	uint32_t sram_resc_flags;
+	struct hcapi_cfa_resc_req_entry sram_resc[HCAPI_CFA_RESC_TYPE_SRAM_MAX];
+
+	/* Validity of each resource type is indicated by the
+	 * corresponding resource type bit in the hw_resc_flags. When
+	 * set to 1, the CFA resource type is valid and amount of
+	 * resource of this type is reserved. Each resource pool is
+	 * identified by the starting index and the number of
+	 * resources in the pool.
+	 */
+	uint32_t hw_resc_flags;
+	struct hcapi_cfa_resc_req_entry hw_resc[HCAPI_CFA_RESC_TYPE_HW_MAX];
+};
+
+struct hcapi_cfa_resc_req_db {
+	struct hcapi_cfa_resc_req rx;
+	struct hcapi_cfa_resc_req tx;
+};
+
+struct hcapi_cfa_resc_entry {
+	uint16_t start;
+	uint16_t stride;
+	uint16_t tag;
+};
+
+struct hcapi_cfa_resc {
+	/* Wh+/SR specific onchip Action SRAM resources */
+	/* Validity of each sram type is indicated by the
+	 * corresponding sram type bit in the sram_resc_flags. When
+	 * set to 1, the CFA sram resource type is valid and amount of
+	 * resources for this type is reserved. Each sram resource
+	 * pool is identified by the starting index and number of
+	 * resources in the pool.
+	 */
+	uint32_t sram_resc_flags;
+	struct hcapi_cfa_resc_entry sram_resc[HCAPI_CFA_RESC_TYPE_SRAM_MAX];
+
+	/* Validity of each resource type is indicated by the
+	 * corresponding resource type bit in the hw_resc_flags. When
+	 * set to 1, the CFA resource type is valid and amount of
+	 * resource of this type is reserved. Each resource pool is
+	 * identified by the startin index and the number of resources
+	 * in the pool.
+	 */
+	uint32_t hw_resc_flags;
+	struct hcapi_cfa_resc_entry hw_resc[HCAPI_CFA_RESC_TYPE_HW_MAX];
+};
+
+struct hcapi_cfa_resc_db {
+	struct hcapi_cfa_resc rx;
+	struct hcapi_cfa_resc tx;
+};
+
+/**
+ * This is the main data structure used by the CFA Resource
+ * Manager.  This data structure holds all the state and table
+ * management information.
+ */
+typedef struct hcapi_cfa_rm_data {
+    uint32_t dummy_data;
+} hcapi_cfa_rm_data_t;
+
+/* End RM support */
+
+struct hcapi_cfa_devops;
+
+struct hcapi_cfa_devinfo {
+			  uint8_t global_cfg_data[CFA_GLOBAL_CFG_DATA_SZ];
+			  struct hcapi_cfa_layout_tbl layouts;
+			  struct hcapi_cfa_devops *devops;
+};
+
+int hcapi_cfa_dev_bind(enum hcapi_cfa_ver hw_ver,
+			struct hcapi_cfa_devinfo *dev_info);
+
+int hcapi_cfa_key_compile_layout(
+				 struct hcapi_cfa_key_template *key_template,
+				 struct hcapi_cfa_key_layout *key_layout);
+uint64_t hcapi_cfa_key_hash(uint64_t *key_data, uint16_t bitlen);
+int hcapi_cfa_action_compile_layout(
+				    struct hcapi_cfa_action_template *act_template,
+				    struct hcapi_cfa_action_layout *act_layout);
+int hcapi_cfa_action_init_obj(
+			      uint64_t *act_obj,
+			      struct hcapi_cfa_action_layout *act_layout);
+int hcapi_cfa_action_compute_ptr(
+				 uint64_t *act_obj,
+				 struct hcapi_cfa_action_layout *act_layout,
+				 uint32_t base_ptr);
+
+int hcapi_cfa_action_hw_op(struct hcapi_cfa_hwop *op,
+			   uint8_t *act_tbl,
+			   struct hcapi_cfa_data *act_obj);
+int hcapi_cfa_dev_hw_op(struct hcapi_cfa_hwop *op, uint16_t tbl_id,
+			struct hcapi_cfa_data *obj_data);
+int hcapi_cfa_rm_register_client(hcapi_cfa_rm_data_t *data,
+				 const char *client_name,
+				 int *client_id);
+int hcapi_cfa_rm_unregister_client(hcapi_cfa_rm_data_t *data,
+				   int client_id);
+int hcapi_cfa_rm_query_resources(hcapi_cfa_rm_data_t *data,
+				 int client_id,
+				 uint16_t chnl_id,
+				 struct hcapi_cfa_resc_req_db *req_db);
+int hcapi_cfa_rm_query_resources_one(hcapi_cfa_rm_data_t *data,
+				     int clien_id,
+				     struct hcapi_cfa_resc_db *resc_db);
+int hcapi_cfa_rm_reserve_resources(hcapi_cfa_rm_data_t *data,
+				   int client_id,
+				   struct hcapi_cfa_resc_req_db *resc_req,
+				   struct hcapi_cfa_resc_db *resc_db);
+int hcapi_cfa_rm_release_resources(hcapi_cfa_rm_data_t *data,
+				   int client_id,
+				   struct hcapi_cfa_resc_req_db *resc_req,
+				   struct hcapi_cfa_resc_db *resc_db);
+int hcapi_cfa_rm_initialize(hcapi_cfa_rm_data_t *data);
+
+#if SUPPORT_CFA_HW_P4
+
+int hcapi_cfa_p4_dev_hw_op(struct hcapi_cfa_hwop *op, uint16_t tbl_id,
+			    struct hcapi_cfa_data *obj_data);
+int hcapi_cfa_p4_prof_l2ctxt_hwop(struct hcapi_cfa_hwop *op,
+				   struct hcapi_cfa_data *obj_data);
+int hcapi_cfa_p4_prof_l2ctxtrmp_hwop(struct hcapi_cfa_hwop *op,
+				      struct hcapi_cfa_data *obj_data);
+int hcapi_cfa_p4_prof_tcam_hwop(struct hcapi_cfa_hwop *op,
+				 struct hcapi_cfa_data *obj_data);
+int hcapi_cfa_p4_prof_tcamrmp_hwop(struct hcapi_cfa_hwop *op,
+				    struct hcapi_cfa_data *obj_data);
+int hcapi_cfa_p4_wc_tcam_hwop(struct hcapi_cfa_hwop *op,
+			       struct hcapi_cfa_data *obj_data);
+int hcapi_cfa_p4_wc_tcam_rec_hwop(struct hcapi_cfa_hwop *op,
+				   struct hcapi_cfa_data *obj_data);
+#endif /* SUPPORT_CFA_HW_P4 */
+/**
+ *  HCAPI CFA device HW operation function callback definition
+ *  This is standardized function callback hook to install different
+ *  CFA HW table programming function callback.
+ */
+
+struct hcapi_cfa_tbl_cb {
+	/**
+	 * This function callback provides the functionality to read/write
+	 * HW table entry from a HW table.
+	 *
+	 * @param[in] op
+	 *   A pointer to the Hardware operation parameter
+	 *
+	 * @param[in] obj_data
+	 *   A pointer to the HW data object for the hardware operation
+	 *
+	 * @return
+	 *   0 for SUCCESS, negative value for FAILURE
+	 */
+	int (*hwop_cb)(struct hcapi_cfa_hwop *op,
+		       struct hcapi_cfa_data *obj_data);
+};
+
+#endif  /* HCAPI_CFA_H_ */
diff --git a/drivers/net/bnxt/hcapi/hcapi_cfa_common.c b/drivers/net/bnxt/hcapi/hcapi_cfa_common.c
new file mode 100644
index 0000000..39afd4d
--- /dev/null
+++ b/drivers/net/bnxt/hcapi/hcapi_cfa_common.c
@@ -0,0 +1,92 @@
+/*
+ *   Copyright(c) 2019-2020 Broadcom Limited.
+ *   All rights reserved.
+ */
+
+#include "bitstring.h"
+#include "hcapi_cfa_defs.h"
+#include <errno.h>
+#include "assert.h"
+
+/* HCAPI CFA common PUT APIs */
+int hcapi_cfa_put_field(uint64_t *data_buf,
+			const struct hcapi_cfa_layout *layout,
+			uint16_t field_id, uint64_t val)
+{
+	assert(layout);
+
+	if (field_id > layout->array_sz)
+		/* Invalid field_id */
+		return -EINVAL;
+
+	if (layout->is_msb_order)
+		bs_put_msb(data_buf,
+			   layout->field_array[field_id].bitpos,
+			   layout->field_array[field_id].bitlen, val);
+	else
+		bs_put_lsb(data_buf,
+			   layout->field_array[field_id].bitpos,
+			   layout->field_array[field_id].bitlen, val);
+	return 0;
+}
+
+int hcapi_cfa_put_fields(uint64_t *obj_data,
+			 const struct hcapi_cfa_layout *layout,
+			 struct hcapi_cfa_data_obj *field_tbl,
+			 uint16_t field_tbl_sz)
+{
+	int i;
+	uint16_t bitpos;
+	uint8_t bitlen;
+	uint16_t field_id;
+
+	assert(layout);
+	assert(field_tbl);
+
+	if (layout->is_msb_order) {
+		for (i = 0; i < field_tbl_sz; i++) {
+			field_id = field_tbl[i].field_id;
+			if (field_id > layout->array_sz)
+				return -EINVAL;
+			bitpos = layout->field_array[field_id].bitpos;
+			bitlen = layout->field_array[field_id].bitlen;
+			bs_put_msb(obj_data, bitpos, bitlen,
+				   field_tbl[i].val);
+		}
+	} else {
+		for (i = 0; i < field_tbl_sz; i++) {
+			field_id = field_tbl[i].field_id;
+			if (field_id > layout->array_sz)
+				return -EINVAL;
+			bitpos = layout->field_array[field_id].bitpos;
+			bitlen = layout->field_array[field_id].bitlen;
+			bs_put_lsb(obj_data, bitpos, bitlen,
+				   field_tbl[i].val);
+		}
+	}
+	return 0;
+}
+
+/* HCAPI CFA common GET APIs */
+int hcapi_cfa_get_field(uint64_t *obj_data,
+			const struct hcapi_cfa_layout *layout,
+			uint16_t field_id,
+			uint64_t *val)
+{
+	assert(layout);
+	assert(val);
+
+	if (field_id > layout->array_sz)
+		/* Invalid field_id */
+		return -EINVAL;
+
+	if (layout->is_msb_order)
+		*val = bs_get_msb(obj_data,
+				  layout->field_array[field_id].bitpos,
+				  layout->field_array[field_id].bitlen);
+	else
+		*val = bs_get_lsb(obj_data,
+				  layout->field_array[field_id].bitpos,
+				  layout->field_array[field_id].bitlen);
+	return 0;
+}
diff --git a/drivers/net/bnxt/hcapi/hcapi_cfa_defs.h b/drivers/net/bnxt/hcapi/hcapi_cfa_defs.h
new file mode 100644
index 0000000..ca562d2
--- /dev/null
+++ b/drivers/net/bnxt/hcapi/hcapi_cfa_defs.h
@@ -0,0 +1,669 @@
+/*
+  *   Copyright(c) Broadcom Limited.
+  *   All rights reserved.
+  */
+
+/*!
+ *   \file
+ *   \brief Exported functions for CFA HW programming
+ */
+#ifndef _HCAPI_CFA_DEFS_H_
+#define _HCAPI_CFA_DEFS_H_
+
+#include <stdio.h>
+#include <string.h>
+#include <stdbool.h>
+#include <stdint.h>
+#include <stddef.h>
+
+#define SUPPORT_CFA_HW_ALL 0
+#define SUPPORT_CFA_HW_P4  1
+#define SUPPORT_CFA_HW_P58 0
+#define SUPPORT_CFA_HW_P59 0
+
+#define CFA_BITS_PER_BYTE (8)
+#define __CFA_ALIGN_MASK(x, mask) (((x) + (mask)) & ~(mask))
+#define CFA_ALIGN(x, a) __CFA_ALIGN_MASK(x, (a)-1)
+#define CFA_ALIGN_128(x) CFA_ALIGN(x, 128)
+#define CFA_ALIGN_32(x) CFA_ALIGN(x, 32)
+
+#define NUM_WORDS_ALIGN_32BIT(x)                                               \
+	(CFA_ALIGN_32(x) / (sizeof(uint32_t) * CFA_BITS_PER_BYTE))
+#define NUM_WORDS_ALIGN_128BIT(x)                                              \
+	(CFA_ALIGN_128(x) / (sizeof(uint32_t) * CFA_BITS_PER_BYTE))
+
+#define CFA_GLOBAL_CFG_DATA_SZ (100)
+
+#if SUPPORT_CFA_HW_P4 && SUPPORT_CFA_HW_P58 && SUPPORT_CFA_HW_P59
+#define SUPPORT_CFA_HW_ALL (1)
+#endif
+
+#include "hcapi_cfa_p4.h"
+#define CFA_PROF_L2CTXT_TCAM_MAX_FIELD_CNT CFA_P40_PROF_L2_CTXT_TCAM_MAX_FLD
+#define CFA_PROF_L2CTXT_REMAP_MAX_FIELD_CNT CFA_P40_PROF_L2_CTXT_RMP_DR_MAX_FLD
+#define CFA_PROF_MAX_KEY_CFG_SZ sizeof(struct cfa_p4_prof_key_cfg)
+#define CFA_KEY_MAX_FIELD_CNT 41
+#define CFA_ACT_MAX_TEMPLATE_SZ sizeof(struct cfa_p4_action_template)
+
+/**
+ * CFA HW version definition
+ */
+enum hcapi_cfa_ver {
+	HCAPI_CFA_P40 = 0, /**< CFA phase 4.0 */
+	HCAPI_CFA_P45 = 1, /**< CFA phase 4.5 */
+	HCAPI_CFA_P58 = 2, /**< CFA phase 5.8 */
+	HCAPI_CFA_P59 = 3, /**< CFA phase 5.9 */
+	HCAPI_CFA_PMAX = 4
+};
+
+/**
+ * CFA direction definition
+ */
+enum hcapi_cfa_dir {
+	HCAPI_CFA_DIR_RX = 0, /**< Receive */
+	HCAPI_CFA_DIR_TX = 1, /**< Transmit */
+	HCAPI_CFA_DIR_MAX = 2
+};
+
+/**
+ * CFA HW OPCODE definition
+ */
+enum hcapi_cfa_hwops {
+	HCAPI_CFA_HWOPS_PUT, /**< Write to HW operation */
+	HCAPI_CFA_HWOPS_GET, /**< Read from HW operation */
+	HCAPI_CFA_HWOPS_ADD, /**< For operations which require more then simple
+			      *   writes to HW, this operation is used.  The
+			      *   distinction with this operation when compared
+			      *   to the PUT ops is that this operation is used
+			      *   in conjunction with the HCAPI_CFA_HWOPS_DEL
+			      *   op to remove the operations issued by the
+			      *   ADD OP.
+			      */
+	HCAPI_CFA_HWOPS_DEL, /**< This issues operations to clear the hardware.
+			      *   This operation is used in conjunction
+			      *   with the HCAPI_CFA_HWOPS_ADD op and is the
+			      *   way to undo/clear the ADD op.
+			      */
+	HCAPI_CFA_HWOPS_MAX
+};
+
+/**
+ * CFA HW KEY CONTROL OPCODE definition
+ */
+enum hcapi_cfa_key_ctrlops {
+	HCAPI_CFA_KEY_CTRLOPS_INSERT, /**< insert control bits */
+	HCAPI_CFA_KEY_CTRLOPS_STRIP, /**< strip control bits */
+	HCAPI_CFA_KEY_CTRLOPS_MAX
+};
+
+/**
+ * CFA HW field structure definition
+ */
+struct hcapi_cfa_field {
+	/** [in] Starting bit position pf the HW field within a HW table
+	 *  entry.
+	 */
+	uint16_t bitpos;
+	/** [in] Number of bits for the HW field. */
+	uint8_t bitlen;
+};
+
+/**
+ * CFA HW table entry layout structure definition
+ */
+struct hcapi_cfa_layout {
+	/** [out] Bit order of layout */
+	bool is_msb_order;
+	/** [out] Size in bits of entry */
+	uint32_t total_sz_in_bits;
+	/** [out] data pointer of the HW layout fields array */
+	const struct hcapi_cfa_field *field_array;
+	/** [out] number of HW field entries in the HW layout field array */
+	uint32_t array_sz;
+};
+
+/**
+ * CFA HW data object definition
+ */
+struct hcapi_cfa_data_obj {
+	/** [in] HW field identifier. Used as an index to a HW table layout */
+	uint16_t field_id;
+	/** [in] Value of the HW field */
+	uint64_t val;
+};
+
+/**
+ * CFA HW definition
+ */
+struct hcapi_cfa_hw {
+	/** [in] HW table base address for the operation with optional device
+	 *  handle. For on-chip HW table operation, this is the either the TX
+	 *  or RX CFA HW base address. For off-chip table, this field is the
+	 *  base memory address of the off-chip table.
+	 */
+	uint64_t base_addr;
+	/** [in] Optional opaque device handle. It is generally used to access
+	 *  an GRC register space through PCIE BAR and passed to the BAR memory
+	 *  accessor routine.
+	 */
+	void *handle;
+};
+
+/**
+ * CFA HW operation definition
+ *
+ */
+struct hcapi_cfa_hwop {
+	/** [in] HW opcode */
+	enum hcapi_cfa_hwops opcode;
+	/** [in] CFA HW information used by accessor routines.
+	 */
+	struct hcapi_cfa_hw hw;
+};
+
+/**
+ * CFA HW data structure definition
+ */
+struct hcapi_cfa_data {
+	/** [in] physical offset to the HW table for the data to be
+	 *  written to.  If this is an array of registers, this is the
+	 *  index into the array of registers.  For writing keys, this
+	 *  is the byte offset into the memory wher the key should be
+	 *  written.
+	 */
+	union {
+		uint32_t index;
+		uint32_t byte_offset;
+	};
+	/** [in] HW data buffer pointer */
+	uint8_t *data;
+	/** [in] HW data mask buffer pointer */
+	uint8_t *data_mask;
+	/** [in] size of the HW data buffer in bytes */
+	uint16_t data_sz;
+};
+
+/*********************** Truflow start ***************************/
+enum hcapi_cfa_pg_tbl_lvl {
+	PT_LVL_0,
+	PT_LVL_1,
+	PT_LVL_2,
+	PT_LVL_MAX
+};
+
+enum hcapi_cfa_em_table_type {
+	KEY0_TABLE,
+	KEY1_TABLE,
+	RECORD_TABLE,
+	EFC_TABLE,
+	MAX_TABLE
+};
+
+struct hcapi_cfa_em_page_tbl {
+	uint32_t	pg_count;
+	uint32_t	pg_size;
+	void		**pg_va_tbl;
+	uint64_t	*pg_pa_tbl;
+};
+
+struct hcapi_cfa_em_table {
+	int				type;
+	uint32_t			num_entries;
+	uint16_t			ctx_id;
+	uint32_t			entry_size;
+	int				num_lvl;
+	uint32_t			page_cnt[PT_LVL_MAX];
+	uint64_t			num_data_pages;
+	void				*l0_addr;
+	uint64_t			l0_dma_addr;
+	struct hcapi_cfa_em_page_tbl    pg_tbl[PT_LVL_MAX];
+};
+
+struct hcapi_cfa_em_ctx_mem_info {
+	struct hcapi_cfa_em_table		em_tables[MAX_TABLE];
+};
+
+/*********************** Truflow end ****************************/
+
+/**
+ * CFA HW key table definition
+ *
+ * Applicable to EEM and off-chip EM table only.
+ */
+struct hcapi_cfa_key_tbl {
+	/** [in] For EEM, this is the KEY0 base mem pointer. For off-chip EM,
+	 *  this is the base mem pointer of the key table.
+	 */
+	uint8_t *base0;
+	/** [in] total size of the key table in bytes. For EEM, this size is
+	 *  same for both KEY0 and KEY1 table.
+	 */
+	uint32_t size;
+	/** [in] number of key buckets, applicable for newer chips */
+	uint32_t num_buckets;
+	/** [in] For EEM, this is KEY1 base mem pointer. Fo off-chip EM,
+	 *  this is the key record memory base pointer within the key table,
+	 *  applicable for newer chip
+	 */
+	uint8_t *base1;
+};
+
+/**
+ * CFA HW key buffer definition
+ */
+struct hcapi_cfa_key_obj {
+	/** [in] pointer to the key data buffer */
+	uint32_t *data;
+	/** [in] buffer len in bits */
+	uint32_t len;
+	/** [in] Pointer to the key layout */
+	struct hcapi_cfa_key_layout *layout;
+};
+
+/**
+ * CFA HW key data definition
+ */
+struct hcapi_cfa_key_data {
+	/** [in] For on-chip key table, it is the offset in unit of smallest
+	 *  key. For off-chip key table, it is the byte offset relative
+	 *  to the key record memory base.
+	 */
+	uint32_t offset;
+	/** [in] HW key data buffer pointer */
+	uint8_t *data;
+	/** [in] size of the key in bytes */
+	uint16_t size;
+};
+
+/**
+ * CFA HW key location definition
+ */
+struct hcapi_cfa_key_loc {
+	/** [out] on-chip EM bucket offset or off-chip EM bucket mem pointer */
+	uint64_t bucket_mem_ptr;
+	/** [out] index within the EM bucket */
+	uint8_t bucket_idx;
+};
+
+/**
+ * CFA HW layout table definition
+ */
+struct hcapi_cfa_layout_tbl {
+	/** [out] data pointer to an array of fix formatted layouts supported.
+	 *  The index to the array is the CFA HW table ID
+	 */
+	const struct hcapi_cfa_layout *tbl;
+	/** [out] number of fix formatted layouts in the layout array */
+	uint16_t num_layouts;
+};
+
+/**
+ * Key template consists of key fields that can be enabled/disabled
+ * individually.
+ */
+struct hcapi_cfa_key_template {
+	/** [in] key field enable field array, set 1 to the correspeonding
+	 *  field enable to make a field valid
+	 */
+	uint8_t field_en[CFA_KEY_MAX_FIELD_CNT];
+	/** [in] Identified if the key template is for TCAM. If false, the
+	 *  the key template is for EM. This field is mandantory for device that
+	 *  only support fix key formats.
+	 */
+	bool is_wc_tcam_key;
+};
+
+/**
+ * key layout consist of field array, key bitlen, key ID, and other meta data
+ * pertain to a key
+ */
+struct hcapi_cfa_key_layout {
+	/** [out] key layout data */
+	struct hcapi_cfa_layout *layout;
+	/** [out] actual key size in number of bits */
+	uint16_t bitlen;
+	/** [out] key identifier and this field is only valid for device
+	 *  that supports fix key formats
+	 */
+	uint16_t id;
+	/** [out] Indentified the key layout is WC TCAM key */
+	bool is_wc_tcam_key;
+	/** [out] total slices size, valid for WC TCAM key only. It can be
+	 *  used by the user to determine the total size of WC TCAM key slices
+	 *  in bytes. */
+	uint16_t slices_size;
+};
+
+/**
+ * key layout memory contents
+ */
+struct hcapi_cfa_key_layout_contents {
+	/** key layouts */
+	struct hcapi_cfa_key_layout key_layout;
+
+	/** layout */
+	struct hcapi_cfa_layout layout;
+
+	/** fields */
+	struct hcapi_cfa_field field_array[CFA_KEY_MAX_FIELD_CNT];
+};
+
+/**
+ * Action template consists of action fields that can be enabled/disabled
+ * individually.
+ */
+struct hcapi_cfa_action_template {
+	/** [in] CFA version for the action template */
+	enum hcapi_cfa_ver hw_ver;
+	/** [in] action field enable field array, set 1 to the correspeonding
+	 *  field enable to make a field valid
+	 */
+	uint8_t data[CFA_ACT_MAX_TEMPLATE_SZ];
+};
+
+/**
+ * action layout consist of field array, action wordlen and action format ID
+ */
+struct hcapi_cfa_action_layout {
+	/** [in] action identifier */
+	uint16_t id;
+	/** [out] action layout data */
+	struct hcapi_cfa_layout *layout;
+	/** [out] actual action record size in number of bits */
+	uint16_t wordlen;
+};
+
+/**
+ *  \defgroup CFA_HCAPI_PUT_API
+ *  HCAPI used for writing to the hardware
+ *  @{
+ */
+
+/**
+ * This API provides the functionality to program a specified value to a
+ * HW field based on the provided programming layout.
+ *
+ * @param[in,out] obj_data
+ *   A data pointer to a CFA HW key/mask data
+ *
+ * @param[in] layout
+ *   A pointer to CFA HW programming layout
+ *
+ * @param[in] field_id
+ *   ID of the HW field to be programmed
+ *
+ * @param[in] val
+ *   Value of the HW field to be programmed
+ *
+ * @return
+ *   0 for SUCCESS, negative value for FAILURE
+ */
+int hcapi_cfa_put_field(uint64_t *data_buf,
+			const struct hcapi_cfa_layout *layout,
+			uint16_t field_id, uint64_t val);
+
+/**
+ * This API provides the functionality to program an array of field values
+ * with corresponding field IDs to a number of profiler sub-block fields
+ * based on the fixed profiler sub-block hardware programming layout.
+ *
+ * @param[in, out] obj_data
+ *   A pointer to a CFA profiler key/mask object data
+ *
+ * @param[in] layout
+ *   A pointer to CFA HW programming layout
+ *
+ * @param[in] field_tbl
+ *   A pointer to an array that consists of the object field
+ *   ID/value pairs
+ *
+ * @param[in] field_tbl_sz
+ *   Number of entries in the table
+ *
+ * @return
+ *   0 for SUCCESS, negative value for FAILURE
+ */
+int hcapi_cfa_put_fields(uint64_t *obj_data,
+			 const struct hcapi_cfa_layout *layout,
+			 struct hcapi_cfa_data_obj *field_tbl,
+			 uint16_t field_tbl_sz);
+
+/**
+ * This API provides the functionality to write a value to a
+ * field within the bit position and bit length of a HW data
+ * object based on a provided programming layout.
+ *
+ * @param[in, out] act_obj
+ *   A pointer of the action object to be initialized
+ *
+ * @param[in] layout
+ *   A pointer of the programming layout
+ *
+ * @param field_id
+ *   [in] Identifier of the HW field
+ *
+ * @param[in] bitpos_adj
+ *   Bit position adjustment value
+ *
+ * @param[in] bitlen_adj
+ *   Bit length adjustment value
+ *
+ * @param[in] val
+ *   HW field value to be programmed
+ *
+ * @return
+ *   0 for SUCCESS, negative value for FAILURE
+ */
+int hcapi_cfa_put_field_rel(uint64_t *obj_data,
+			    const struct hcapi_cfa_layout *layout,
+			    uint16_t field_id, int16_t bitpos_adj,
+			    int16_t bitlen_adj, uint64_t val);
+
+/*@}*/
+
+/**
+ *  \defgroup CFA_HCAPI_GET_API
+ *  HCAPI used for writing to the hardware
+ *  @{
+ */
+
+/**
+ * This API provides the functionality to get the word length of
+ * a layout object.
+ *
+ * @param[in] layout
+ *   A pointer of the HW layout
+ *
+ * @return
+ *   Word length of the layout object
+ */
+uint16_t hcapi_cfa_get_wordlen(const struct hcapi_cfa_layout *layout);
+
+/**
+ * The API provides the functionality to get bit offset and bit
+ * length information of a field from a programming layout.
+ *
+ * @param[in] layout
+ *   A pointer of the action layout
+ *
+ * @param[out] slice
+ *   A pointer to the action offset info data structure
+ *
+ * @return
+ *   0 for SUCCESS, negative value for FAILURE
+ */
+int hcapi_cfa_get_slice(const struct hcapi_cfa_layout *layout,
+			uint16_t field_id, struct hcapi_cfa_field *slice);
+
+/**
+ * This API provides the functionality to read the value of a
+ * CFA HW field from CFA HW data object based on the hardware
+ * programming layout.
+ *
+ * @param[in] obj_data
+ *   A pointer to a CFA HW key/mask object data
+ *
+ * @param[in] layout
+ *   A pointer to CFA HW programming layout
+ *
+ * @param[in] field_id
+ *   ID of the HW field to be programmed
+ *
+ * @param[out] val
+ *   Value of the HW field
+ *
+ * @return
+ *   0 for SUCCESS, negative value for FAILURE
+ */
+int hcapi_cfa_get_field(uint64_t *obj_data,
+			const struct hcapi_cfa_layout *layout,
+			uint16_t field_id, uint64_t *val);
+
+/**
+ * This API provides the functionality to read a number of
+ * HW fields from a CFA HW data object based on the hardware
+ * programming layout.
+ *
+ * @param[in] obj_data
+ *   A pointer to a CFA profiler key/mask object data
+ *
+ * @param[in] layout
+ *   A pointer to CFA HW programming layout
+ *
+ * @param[in, out] field_tbl
+ *   A pointer to an array that consists of the object field
+ *   ID/value pairs
+ *
+ * @param[in] field_tbl_sz
+ *   Number of entries in the table
+ *
+ * @return
+ *   0 for SUCCESS, negative value for FAILURE
+ */
+int hcapi_cfa_get_fields(uint64_t *obj_data,
+			 const struct hcapi_cfa_layout *layout,
+			 struct hcapi_cfa_data_obj *field_tbl,
+			 uint16_t field_tbl_sz);
+
+/**
+ * Get a value to a specific location relative to a HW field
+ *
+ * This API provides the functionality to read HW field from
+ * a section of a HW data object identified by the bit position
+ * and bit length from a given programming layout in order to avoid
+ * reading the entire HW data object.
+ *
+ * @param[in] obj_data
+ *   A pointer of the data object to read from
+ *
+ * @param[in] layout
+ *   A pointer of the programming layout
+ *
+ * @param[in] field_id
+ *   Identifier of the HW field
+ *
+ * @param[in] bitpos_adj
+ *   Bit position adjustment value
+ *
+ * @param[in] bitlen_adj
+ *   Bit length adjustment value
+ *
+ * @param[out] val
+ *   Value of the HW field
+ *
+ * @return
+ *   0 for SUCCESS, negative value for FAILURE
+ */
+int hcapi_cfa_get_field_rel(uint64_t *obj_data,
+			    const struct hcapi_cfa_layout *layout,
+			    uint16_t field_id, int16_t bitpos_adj,
+			    int16_t bitlen_adj, uint64_t *val);
+
+/**
+ * This function is used to initialize a layout_contents structure
+ *
+ * The struct hcapi_cfa_key_layout is complex as there are three
+ * layers of abstraction.  Each of those layer need to be properly
+ * initialized.
+ *
+ * @param[in] layout_contents
+ *  A pointer of the layout contents to initialize
+ *
+ * @return
+ *   0 for SUCCESS, negative value for FAILURE
+ */
+int hcapi_cfa_init_key_layout_contents(
+	struct hcapi_cfa_key_layout_contents *layout_contents);
+
+/**
+ * This function is used to validate a key template
+ *
+ * The struct hcapi_cfa_key_template is complex as there are three
+ * layers of abstraction.  Each of those layer need to be properly
+ * validated.
+ *
+ * @param[in] key_template
+ *  A pointer of the key template contents to validate
+ *
+ * @return
+ *   0 for SUCCESS, negative value for FAILURE
+ */
+int hcapi_cfa_is_valid_key_template(struct hcapi_cfa_key_template *key_template);
+
+/**
+ * This function is used to validate a key layout
+ *
+ * The struct hcapi_cfa_key_layout is complex as there are three
+ * layers of abstraction.  Each of those layer need to be properly
+ * validated.
+ *
+ * @param[in] key_layout
+ *  A pointer of the key layout contents to validate
+ *
+ * @return
+ *   0 for SUCCESS, negative value for FAILURE
+ */
+int hcapi_cfa_is_valid_key_layout(struct hcapi_cfa_key_layout *key_layout);
+
+/**
+ * This function is used to hash E/EM keys
+ *
+ *
+ * @param[in] key_data
+ *  A pointer of the key
+ *
+ * @param[in] bitlen
+ *  Number of bits in the key
+ *
+ * @return
+ *   CRC32 and Lookup3 hashes of the input key
+ */
+uint64_t hcapi_cfa_key_hash(uint64_t *key_data,
+			    uint16_t bitlen);
+
+/**
+ * This function is used to execute an operation
+ *
+ *
+ * @param[in] op
+ *  Operation
+ *
+ * @param[in] key_tbl
+ *  Table
+ *
+ * @param[in] key_obj
+ *  Key data
+ *
+ * @param[in] key_key_loc
+ *
+ * @return
+ *   0 for SUCCESS, negative value for FAILURE
+ */
+int hcapi_cfa_key_hw_op(struct hcapi_cfa_hwop *op,
+			struct hcapi_cfa_key_tbl *key_tbl,
+			struct hcapi_cfa_key_data *key_obj,
+			struct hcapi_cfa_key_loc *key_loc);
+
+uint64_t hcapi_get_table_page(struct hcapi_cfa_em_table *mem,
+			      uint32_t offset);
+#endif /* HCAPI_CFA_DEFS_H_ */
diff --git a/drivers/net/bnxt/hcapi/hcapi_cfa_p4.c b/drivers/net/bnxt/hcapi/hcapi_cfa_p4.c
new file mode 100644
index 0000000..5b5cac8
--- /dev/null
+++ b/drivers/net/bnxt/hcapi/hcapi_cfa_p4.c
@@ -0,0 +1,411 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2019-2020 Broadcom
+ * All rights reserved.
+ */
+
+#include <stdint.h>
+#include <stdlib.h>
+#include <stdbool.h>
+#include <string.h>
+#include "lookup3.h"
+#include "rand.h"
+
+#include "hcapi_cfa_defs.h"
+
+#if 0
+uint64_t hcapi_cfa_key_hash(uint64_t *key_data,
+			    uint16_t bitlen);
+int hcapi_cfa_key_hw_op(struct hcapi_cfa_hwop *op,
+			struct hcapi_cfa_key_tbl *key_tbl,
+			struct hcapi_cfa_key_data *key_obj,
+			struct hcapi_cfa_key_loc *key_loc);
+#endif
+
+#define HCAPI_CFA_LKUP_SEED_MEM_SIZE 512
+#define TF_EM_PAGE_SIZE (1 << 21)
+uint32_t hcapi_cfa_lkup_lkup3_init_cfg;
+uint32_t hcapi_cfa_lkup_em_seed_mem[HCAPI_CFA_LKUP_SEED_MEM_SIZE];
+bool hcapi_cfa_lkup_init;
+
+static __inline uint32_t SWAP_WORDS32(uint32_t val32)
+{
+	return (((val32 & 0x0000ffff) << 16) |
+		((val32 & 0xffff0000) >> 16));
+}
+
+static void hcapi_cfa_seeds_init(void)
+{
+	int i;
+	uint32_t r;
+
+	if (hcapi_cfa_lkup_init == true)
+		return;
+
+	hcapi_cfa_lkup_init = true;
+
+	/* Initialize the lfsr */
+	rand_init();
+
+	/* RX and TX use the same seed values */
+	hcapi_cfa_lkup_lkup3_init_cfg = SWAP_WORDS32(rand32());
+
+	for (i = 0; i < HCAPI_CFA_LKUP_SEED_MEM_SIZE / 2; i++) {
+		r = SWAP_WORDS32(rand32());
+		hcapi_cfa_lkup_em_seed_mem[i * 2] = r;
+		r = SWAP_WORDS32(rand32());
+		hcapi_cfa_lkup_em_seed_mem[i * 2 + 1] = (r & 0x1);
+	}
+}
+
+/* CRC32i support for Key0 hash */
+#define ucrc32(ch, crc) (crc32tbl[((crc) ^ (ch)) & 0xff] ^ ((crc) >> 8))
+#define crc32(x, y) crc32i(~0, x, y)
+
+static const uint32_t crc32tbl[] = {	/* CRC polynomial 0xedb88320 */
+0x00000000, 0x77073096, 0xee0e612c, 0x990951ba,
+0x076dc419, 0x706af48f, 0xe963a535, 0x9e6495a3,
+0x0edb8832, 0x79dcb8a4, 0xe0d5e91e, 0x97d2d988,
+0x09b64c2b, 0x7eb17cbd, 0xe7b82d07, 0x90bf1d91,
+0x1db71064, 0x6ab020f2, 0xf3b97148, 0x84be41de,
+0x1adad47d, 0x6ddde4eb, 0xf4d4b551, 0x83d385c7,
+0x136c9856, 0x646ba8c0, 0xfd62f97a, 0x8a65c9ec,
+0x14015c4f, 0x63066cd9, 0xfa0f3d63, 0x8d080df5,
+0x3b6e20c8, 0x4c69105e, 0xd56041e4, 0xa2677172,
+0x3c03e4d1, 0x4b04d447, 0xd20d85fd, 0xa50ab56b,
+0x35b5a8fa, 0x42b2986c, 0xdbbbc9d6, 0xacbcf940,
+0x32d86ce3, 0x45df5c75, 0xdcd60dcf, 0xabd13d59,
+0x26d930ac, 0x51de003a, 0xc8d75180, 0xbfd06116,
+0x21b4f4b5, 0x56b3c423, 0xcfba9599, 0xb8bda50f,
+0x2802b89e, 0x5f058808, 0xc60cd9b2, 0xb10be924,
+0x2f6f7c87, 0x58684c11, 0xc1611dab, 0xb6662d3d,
+0x76dc4190, 0x01db7106, 0x98d220bc, 0xefd5102a,
+0x71b18589, 0x06b6b51f, 0x9fbfe4a5, 0xe8b8d433,
+0x7807c9a2, 0x0f00f934, 0x9609a88e, 0xe10e9818,
+0x7f6a0dbb, 0x086d3d2d, 0x91646c97, 0xe6635c01,
+0x6b6b51f4, 0x1c6c6162, 0x856530d8, 0xf262004e,
+0x6c0695ed, 0x1b01a57b, 0x8208f4c1, 0xf50fc457,
+0x65b0d9c6, 0x12b7e950, 0x8bbeb8ea, 0xfcb9887c,
+0x62dd1ddf, 0x15da2d49, 0x8cd37cf3, 0xfbd44c65,
+0x4db26158, 0x3ab551ce, 0xa3bc0074, 0xd4bb30e2,
+0x4adfa541, 0x3dd895d7, 0xa4d1c46d, 0xd3d6f4fb,
+0x4369e96a, 0x346ed9fc, 0xad678846, 0xda60b8d0,
+0x44042d73, 0x33031de5, 0xaa0a4c5f, 0xdd0d7cc9,
+0x5005713c, 0x270241aa, 0xbe0b1010, 0xc90c2086,
+0x5768b525, 0x206f85b3, 0xb966d409, 0xce61e49f,
+0x5edef90e, 0x29d9c998, 0xb0d09822, 0xc7d7a8b4,
+0x59b33d17, 0x2eb40d81, 0xb7bd5c3b, 0xc0ba6cad,
+0xedb88320, 0x9abfb3b6, 0x03b6e20c, 0x74b1d29a,
+0xead54739, 0x9dd277af, 0x04db2615, 0x73dc1683,
+0xe3630b12, 0x94643b84, 0x0d6d6a3e, 0x7a6a5aa8,
+0xe40ecf0b, 0x9309ff9d, 0x0a00ae27, 0x7d079eb1,
+0xf00f9344, 0x8708a3d2, 0x1e01f268, 0x6906c2fe,
+0xf762575d, 0x806567cb, 0x196c3671, 0x6e6b06e7,
+0xfed41b76, 0x89d32be0, 0x10da7a5a, 0x67dd4acc,
+0xf9b9df6f, 0x8ebeeff9, 0x17b7be43, 0x60b08ed5,
+0xd6d6a3e8, 0xa1d1937e, 0x38d8c2c4, 0x4fdff252,
+0xd1bb67f1, 0xa6bc5767, 0x3fb506dd, 0x48b2364b,
+0xd80d2bda, 0xaf0a1b4c, 0x36034af6, 0x41047a60,
+0xdf60efc3, 0xa867df55, 0x316e8eef, 0x4669be79,
+0xcb61b38c, 0xbc66831a, 0x256fd2a0, 0x5268e236,
+0xcc0c7795, 0xbb0b4703, 0x220216b9, 0x5505262f,
+0xc5ba3bbe, 0xb2bd0b28, 0x2bb45a92, 0x5cb36a04,
+0xc2d7ffa7, 0xb5d0cf31, 0x2cd99e8b, 0x5bdeae1d,
+0x9b64c2b0, 0xec63f226, 0x756aa39c, 0x026d930a,
+0x9c0906a9, 0xeb0e363f, 0x72076785, 0x05005713,
+0x95bf4a82, 0xe2b87a14, 0x7bb12bae, 0x0cb61b38,
+0x92d28e9b, 0xe5d5be0d, 0x7cdcefb7, 0x0bdbdf21,
+0x86d3d2d4, 0xf1d4e242, 0x68ddb3f8, 0x1fda836e,
+0x81be16cd, 0xf6b9265b, 0x6fb077e1, 0x18b74777,
+0x88085ae6, 0xff0f6a70, 0x66063bca, 0x11010b5c,
+0x8f659eff, 0xf862ae69, 0x616bffd3, 0x166ccf45,
+0xa00ae278, 0xd70dd2ee, 0x4e048354, 0x3903b3c2,
+0xa7672661, 0xd06016f7, 0x4969474d, 0x3e6e77db,
+0xaed16a4a, 0xd9d65adc, 0x40df0b66, 0x37d83bf0,
+0xa9bcae53, 0xdebb9ec5, 0x47b2cf7f, 0x30b5ffe9,
+0xbdbdf21c, 0xcabac28a, 0x53b39330, 0x24b4a3a6,
+0xbad03605, 0xcdd70693, 0x54de5729, 0x23d967bf,
+0xb3667a2e, 0xc4614ab8, 0x5d681b02, 0x2a6f2b94,
+0xb40bbe37, 0xc30c8ea1, 0x5a05df1b, 0x2d02ef8d
+};
+
+static uint32_t hcapi_cfa_crc32i(uint32_t crc, const uint8_t *buf, size_t len)
+{
+	int l;
+
+#ifdef TF_EEM_DEBUG
+	TFP_DRV_LOG(DEBUG, "CRC2:");
+#endif
+	for (l = (len - 1); l >= 0; l--) {
+		crc = ucrc32(buf[l], crc);
+#ifdef TF_EEM_DEBUG
+		TFP_DRV_LOG(DEBUG,
+			    "%02X %08X %08X\n",
+			    (buf[l] & 0xff),
+			    crc,
+			    ~crc);
+#endif
+	}
+
+#ifdef TF_EEM_DEBUG
+	TFP_DRV_LOG(DEBUG, "\n");
+#endif
+
+	return ~crc;
+}
+
+static uint32_t hcapi_cfa_crc32_hash(uint8_t *key)
+{
+	int i;
+	uint32_t index;
+	uint32_t val1, val2;
+	uint8_t temp[4];
+	uint8_t *kptr = key;
+
+	/* Do byte-wise XOR of the 52-byte HASH key first. */
+	index = *key;
+	kptr--;
+
+	for (i = CFA_P4_EEM_KEY_MAX_SIZE - 2; i >= 0; i--) {
+		index = index ^ *kptr;
+		kptr--;
+	}
+
+	/* Get seeds */
+	val1 = hcapi_cfa_lkup_em_seed_mem[index * 2];
+	val2 = hcapi_cfa_lkup_em_seed_mem[index * 2 + 1];
+
+	temp[3] = (uint8_t)(val1 >> 24);
+	temp[2] = (uint8_t)(val1 >> 16);
+	temp[1] = (uint8_t)(val1 >> 8);
+	temp[0] = (uint8_t)(val1 & 0xff);
+	val1 = 0;
+
+	/* Start with seed */
+	if (!(val2 & 0x1))
+		val1 = hcapi_cfa_crc32i(~val1, temp, 4);
+
+	val1 = hcapi_cfa_crc32i(~val1,
+		      (key - (CFA_P4_EEM_KEY_MAX_SIZE - 1)),
+		      CFA_P4_EEM_KEY_MAX_SIZE);
+
+	/* End with seed */
+	if (val2 & 0x1)
+		val1 = hcapi_cfa_crc32i(~val1, temp, 4);
+
+	return val1;
+}
+
+static uint32_t hcapi_cfa_lookup3_hash(uint8_t *in_key)
+{
+	uint32_t val1;
+
+	val1 = hashword(((const uint32_t *)(uintptr_t *)in_key) + 1,
+			 CFA_P4_EEM_KEY_MAX_SIZE / (sizeof(uint32_t)),
+			 hcapi_cfa_lkup_lkup3_init_cfg);
+
+	return val1;
+}
+
+
+uint64_t hcapi_get_table_page(struct hcapi_cfa_em_table *mem,
+			      uint32_t offset)
+{
+	int level = 0;
+	int page = offset / TF_EM_PAGE_SIZE;
+	uint64_t addr;
+
+	if (mem == NULL)
+		return 0;
+
+	/*
+	 * Use the level according to the num_level of page table
+	 */
+	level = mem->num_lvl - 1;
+
+	addr = (uint64_t)mem->pg_tbl[level].pg_va_tbl[page];
+
+#if 0
+	TFP_DRV_LOG(DEBUG, "dir:%d offset:0x%x level:%d page:%d addr:%p\n",
+		    dir, offset, level, page, addr);
+#endif
+
+	return addr;
+}
+
+/** Approximation of HCAPI hcapi_cfa_key_hash()
+ *
+ * Return:
+ *
+ */
+uint64_t hcapi_cfa_key_hash(uint64_t *key_data,
+			    uint16_t bitlen)
+{
+	uint32_t key0_hash;
+	uint32_t key1_hash;
+
+	/*
+	 * Init the seeds if needed
+	 */
+	if (hcapi_cfa_lkup_init == false)
+		hcapi_cfa_seeds_init();
+
+	key0_hash = hcapi_cfa_crc32_hash(((uint8_t *)key_data) +
+					      (bitlen / 8) - 1);
+
+	key1_hash = hcapi_cfa_lookup3_hash((uint8_t *)key_data);
+
+	return ((uint64_t)key0_hash) << 32 | (uint64_t)key1_hash;
+}
+
+static int hcapi_cfa_key_hw_op_put(struct hcapi_cfa_hwop *op,
+				   struct hcapi_cfa_key_data *key_obj)
+{
+	int rc = 0;
+
+	memcpy((uint8_t *)op->hw.base_addr +
+	       key_obj->offset,
+	       key_obj->data,
+	       key_obj->size);
+
+	return rc;
+}
+
+static int hcapi_cfa_key_hw_op_get(struct hcapi_cfa_hwop *op,
+				   struct hcapi_cfa_key_data *key_obj)
+{
+	int rc = 0;
+
+	memcpy(key_obj->data,
+	       (uint8_t *)op->hw.base_addr +
+	       key_obj->offset,
+	       key_obj->size);
+
+	return rc;
+}
+
+static int hcapi_cfa_key_hw_op_add(struct hcapi_cfa_hwop *op,
+				   struct hcapi_cfa_key_data *key_obj)
+{
+	int rc = 0;
+	struct cfa_p4_eem_64b_entry table_entry;
+
+	/*
+	 * Is entry free?
+	 */
+	memcpy(&table_entry,
+	       (uint8_t *)op->hw.base_addr +
+	       key_obj->offset,
+	       key_obj->size);
+
+	/*
+	 * If this is entry is valid then report failure
+	 */
+	if (table_entry.hdr.word1 & (1 << CFA_P4_EEM_ENTRY_VALID_SHIFT))
+		return -1;
+
+	memcpy((uint8_t *)op->hw.base_addr +
+	       key_obj->offset,
+	       key_obj->data,
+	       key_obj->size);
+
+	return rc;
+}
+
+static int hcapi_cfa_key_hw_op_del(struct hcapi_cfa_hwop *op,
+				   struct hcapi_cfa_key_data *key_obj)
+{
+	int rc = 0;
+	struct cfa_p4_eem_64b_entry table_entry;
+
+	/*
+	 * Read entry
+	 */
+	memcpy(&table_entry,
+	       (uint8_t *)op->hw.base_addr +
+	       key_obj->offset,
+	       key_obj->size);
+
+	/*
+	 * If this is not a valid entry then report failure.
+	 */
+	if (table_entry.hdr.word1 & (1 << CFA_P4_EEM_ENTRY_VALID_SHIFT)) {
+		/*
+		 * If a key has been provided then verify the key matches
+		 * before deleting the entry.
+		 */
+		if (key_obj->data != NULL) {
+			if (memcmp(&table_entry,
+				   key_obj->data,
+				   key_obj->size) != 0)
+				return -1;
+		}
+	} else {
+		return -1;
+	}
+
+
+	/*
+	 * Delete entry
+	 */
+	memset((uint8_t *)op->hw.base_addr +
+	       key_obj->offset,
+	       0,
+	       key_obj->size);
+
+	return rc;
+}
+
+
+/** Apporiximation of hcapi_cfa_key_hw_op()
+ *
+ *
+ */
+int hcapi_cfa_key_hw_op(struct hcapi_cfa_hwop *op,
+			struct hcapi_cfa_key_tbl *key_tbl,
+			struct hcapi_cfa_key_data *key_obj,
+			struct hcapi_cfa_key_loc *key_loc __attribute__((unused)))
+{
+	int rc = 0;
+
+	if (op == NULL ||
+	    key_tbl == NULL ||
+	    key_obj == NULL ||
+	    key_loc == NULL)
+		return -1;
+
+	op->hw.base_addr =
+		hcapi_get_table_page((struct hcapi_cfa_em_table *)key_tbl->base0,
+				     key_obj->offset);
+
+	if (op->hw.base_addr == 0)
+		return -1;
+
+	switch (op->opcode) {
+	case HCAPI_CFA_HWOPS_PUT: /**< Write to HW operation */
+		rc = hcapi_cfa_key_hw_op_put(op, key_obj);
+		break;
+	case HCAPI_CFA_HWOPS_GET: /**< Read from HW operation */
+		rc = hcapi_cfa_key_hw_op_get(op, key_obj);
+		break;
+	case HCAPI_CFA_HWOPS_ADD: /**< For operations which require more then simple
+			      *   writes to HW, this operation is used.  The
+			      *   distinction with this operation when compared
+			      *   to the PUT ops is that this operation is used
+			      *   in conjunction with the HCAPI_CFA_HWOPS_DEL
+			      *   op to remove the operations issued by the
+			      *   ADD OP.
+			      */
+
+		rc = hcapi_cfa_key_hw_op_add(op, key_obj);
+
+		break;
+	case HCAPI_CFA_HWOPS_DEL:
+		rc = hcapi_cfa_key_hw_op_del(op, key_obj);
+		break;
+	default:
+		rc = -1;
+		break;
+	}
+
+	return rc;
+}
diff --git a/drivers/net/bnxt/hcapi/hcapi_cfa_p4.h b/drivers/net/bnxt/hcapi/hcapi_cfa_p4.h
new file mode 100644
index 0000000..0c11876
--- /dev/null
+++ b/drivers/net/bnxt/hcapi/hcapi_cfa_p4.h
@@ -0,0 +1,451 @@
+/*
+  *   Copyright(c) Broadcom Limited.
+  *   All rights reserved.
+  */
+
+#ifndef _HCAPI_CFA_P4_H_
+#define _HCPAI_CFA_P4_H_
+
+#include "cfa_p40_hw.h"
+
+/** CFA phase 4 fix formatted table(layout) ID definition
+ *
+ */
+enum cfa_p4_tbl_id {
+	CFA_P4_TBL_L2CTXT_TCAM = 0,
+	CFA_P4_TBL_L2CTXT_REMAP,
+	CFA_P4_TBL_PROF_TCAM,
+	CFA_P4_TBL_PROF_TCAM_REMAP,
+	CFA_P4_TBL_WC_TCAM,
+	CFA_P4_TBL_WC_TCAM_REC,
+	CFA_P4_TBL_WC_TCAM_REMAP,
+	CFA_P4_TBL_VEB_TCAM,
+	CFA_P4_TBL_SP_TCAM,
+	CFA_P4_TBL_MAX
+};
+
+#define CFA_P4_PROF_MAX_KEYS 4
+enum cfa_p4_mac_sel_mode {
+	CFA_P4_MAC_SEL_MODE_FIRST = 0,
+	CFA_P4_MAC_SEL_MODE_LOWEST = 1,
+};
+
+struct cfa_p4_prof_key_cfg {
+	uint8_t mac_sel[CFA_P4_PROF_MAX_KEYS];
+#define CFA_P4_PROF_MAC_SEL_DMAC0 (1 << 0)
+#define CFA_P4_PROF_MAC_SEL_T_MAC0 (1 << 1)
+#define CFA_P4_PROF_MAC_SEL_OUTERMOST_MAC0 (1 << 2)
+#define CFA_P4_PROF_MAC_SEL_DMAC1 (1 << 3)
+#define CFA_P4_PROF_MAC_SEL_T_MAC1 (1 << 4)
+#define CFA_P4_PROF_MAC_OUTERMOST_MAC1 (1 << 5)
+	uint8_t pass_cnt;
+	enum cfa_p4_mac_sel_mode mode;
+};
+
+/**
+ * CFA action layout definition
+ */
+
+#define CFA_P4_ACTION_MAX_LAYOUT_SIZE 184
+
+/**
+ * Action object template structure
+ *
+ * Template structure presents data fields that are necessary to know
+ * at the beginning of Action Builder (AB) processing. Like before the
+ * AB compilation. One such example could be a template that is
+ * flexible in size (Encap Record) and the presence of these fields
+ * allows for determining the template size as well as where the
+ * fields are located in the record.
+ *
+ * The template may also present fields that are not made visible to
+ * the caller by way of the action fields.
+ *
+ * Template fields also allow for additional checking on user visible
+ * fields. One such example could be the encap pointer behavior on a
+ * CFA_P4_ACT_OBJ_TYPE_ACT or CFA_P4_ACT_OBJ_TYPE_ACT_SRAM.
+ */
+struct cfa_p4_action_template {
+	/** Action Object type
+	 *
+	 * Controls the type of the Action Template
+	 */
+	enum {
+		/** Select this type to build an Action Record Object
+		 */
+		CFA_P4_ACT_OBJ_TYPE_ACT,
+		/** Select this type to build an Action Statistics
+		 * Object
+		 */
+		CFA_P4_ACT_OBJ_TYPE_STAT,
+		/** Select this type to build a SRAM Action Record
+		 * Object.
+		 */
+		CFA_P4_ACT_OBJ_TYPE_ACT_SRAM,
+		/** Select this type to build a SRAM Action
+		 * Encapsulation Object.
+		 */
+		CFA_P4_ACT_OBJ_TYPE_ENCAP_SRAM,
+		/** Select this type to build a SRAM Action Modify
+		 * Object, with IPv4 capability.
+		 */
+		/* In case of Stingray the term Modify is used for the 'NAT
+		 * action'. Action builder is leveraged to fill in the NAT
+		 * object which then can be referenced by the action
+		 * record.
+		 */
+		CFA_P4_ACT_OBJ_TYPE_MODIFY_IPV4_SRAM,
+		/** Select this type to build a SRAM Action Source
+		 * Property Object.
+		 */
+		/* In case of Stingray this is not a 'pure' action record.
+		 * Action builder is leveraged to full in the Source Property
+		 * object which can then be referenced by the action
+		 * record.
+		 */
+		CFA_P4_ACT_OBJ_TYPE_SRC_PROP_SRAM,
+		/** Select this type to build a SRAM Action Statistics
+		 * Object
+		 */
+		CFA_P4_ACT_OBJ_TYPE_STAT_SRAM,
+	} obj_type;
+
+	/** Action Control
+	 *
+	 * Controls the internals of the Action Template
+	 *
+	 * act is valid when:
+	 * (obj_type == CFA_P4_ACT_OBJ_TYPE_ACT)
+	 */
+	/*
+	 * Stat and encap are always inline for EEM as table scope
+	 * allocation does not allow for separate Stats allocation,
+	 * but has the xx_inline flags as to be forward compatible
+	 * with Stingray 2, always treated as TRUE.
+	 */
+	struct {
+		/** Set to CFA_HCAPI_TRUE to enable statistics
+		 */
+		uint8_t stat_enable;
+		/** Set to CFA_HCAPI_TRUE to enable statistics to be inlined
+		 */
+		uint8_t stat_inline;
+
+		/** Set to CFA_HCAPI_TRUE to enable encapsulation
+		 */
+		uint8_t encap_enable;
+		/** Set to CFA_HCAPI_TRUE to enable encapsulation to be inlined
+		 */
+		uint8_t encap_inline;
+	} act;
+
+	/** Modify Setting
+	 *
+	 * Controls the type of the Modify Action the template is
+	 * describing
+	 *
+	 * modify is valid when:
+	 * (obj_type == CFA_P4_ACT_OBJ_TYPE_MODIFY_SRAM)
+	 */
+	enum {
+		/** Set to enable Modify of Source IPv4 Address
+		 */
+		CFA_P4_MR_REPLACE_SOURCE_IPV4 = 0,
+		/** Set to enable Modify of Destination IPv4 Address
+		 */
+		CFA_P4_MR_REPLACE_DEST_IPV4
+	} modify;
+
+	/** Encap Control
+	 * Controls the type of encapsulation the template is
+	 * describing
+	 *
+	 * encap is valid when:
+	 * ((obj_type == CFA_P4_ACT_OBJ_TYPE_ACT) &&
+	 *   act.encap_enable) ||
+	 * ((obj_type == CFA_P4_ACT_OBJ_TYPE_SRC_PROP_SRAM)
+	 */
+	struct {
+		/* Direction is required as Stingray Encap on RX is
+		 * limited to l2 and VTAG only.
+		 */
+		/** Receive or Transmit direction
+		 */
+		uint8_t direction;
+		/** Set to CFA_HCAPI_TRUE to enable L2 capability in the
+		 *  template
+		 */
+		uint8_t l2_enable;
+		/** vtag controls the Encap Vector - VTAG Encoding, 4 bits
+		 *
+		 * <ul>
+		 * <li> CFA_P4_ACT_ENCAP_VTAGS_PUSH_0, default, no VLAN
+		 *      Tags applied
+		 * <li> CFA_P4_ACT_ENCAP_VTAGS_PUSH_1, adds capability to
+		 *      set 1 VLAN Tag. Action Template compile adds
+		 *      the following field to the action object
+		 *      ::TF_ER_VLAN1
+		 * <li> CFA_P4_ACT_ENCAP_VTAGS_PUSH_2, adds capability to
+		 *      set 2 VLAN Tags. Action Template compile adds
+		 *      the following fields to the action object
+		 *      ::TF_ER_VLAN1 and ::TF_ER_VLAN2
+		 * </ul>
+		 */
+		enum { CFA_P4_ACT_ENCAP_VTAGS_PUSH_0 = 0,
+		       CFA_P4_ACT_ENCAP_VTAGS_PUSH_1,
+		       CFA_P4_ACT_ENCAP_VTAGS_PUSH_2 } vtag;
+
+		/*
+		 * The remaining fields are NOT supported when
+		 * direction is RX and ((obj_type ==
+		 * CFA_P4_ACT_OBJ_TYPE_ACT) && act.encap_enable).
+		 * ab_compile_layout will perform the checking and
+		 * skip remaining fields.
+		 */
+		/** L3 Encap controls the Encap Vector - L3 Encoding,
+		 *  3 bits. Defines the type of L3 Encapsulation the
+		 *  template is describing.
+		 * <ul>
+		 * <li> CFA_P4_ACT_ENCAP_L3_NONE, default, no L3
+		 *      Encapsulation processing.
+		 * <li> CFA_P4_ACT_ENCAP_L3_IPV4, enables L3 IPv4
+		 *      Encapsulation.
+		 * <li> CFA_P4_ACT_ENCAP_L3_IPV6, enables L3 IPv6
+		 *      Encapsulation.
+		 * <li> CFA_P4_ACT_ENCAP_L3_MPLS_8847, enables L3 MPLS
+		 *      8847 Encapsulation.
+		 * <li> CFA_P4_ACT_ENCAP_L3_MPLS_8848, enables L3 MPLS
+		 *      8848 Encapsulation.
+		 * </ul>
+		 */
+		enum {
+			/** Set to disable any L3 encapsulation
+			 * processing, default
+			 */
+			CFA_P4_ACT_ENCAP_L3_NONE = 0,
+			/** Set to enable L3 IPv4 encapsulation
+			 */
+			CFA_P4_ACT_ENCAP_L3_IPV4 = 4,
+			/** Set to enable L3 IPv6 encapsulation
+			 */
+			CFA_P4_ACT_ENCAP_L3_IPV6 = 5,
+			/** Set to enable L3 MPLS 8847 encapsulation
+			 */
+			CFA_P4_ACT_ENCAP_L3_MPLS_8847 = 6,
+			/** Set to enable L3 MPLS 8848 encapsulation
+			 */
+			CFA_P4_ACT_ENCAP_L3_MPLS_8848 = 7
+		} l3;
+
+#define CFA_P4_ACT_ENCAP_MAX_MPLS_LABELS 8
+		/** 1-8 labels, valid when
+		 * (l3 == CFA_P4_ACT_ENCAP_L3_MPLS_8847) ||
+		 * (l3 == CFA_P4_ACT_ENCAP_L3_MPLS_8848)
+		 *
+		 * MAX number of MPLS Labels 8.
+		 */
+		uint8_t l3_num_mpls_labels;
+
+		/** Set to CFA_HCAPI_TRUE to enable L4 capability in the
+		 * template.
+		 *
+		 * CFA_HCAPI_TRUE adds ::TF_EN_UDP_SRC_PORT and
+		 * ::TF_EN_UDP_DST_PORT to the template.
+		 */
+		uint8_t l4_enable;
+
+		/** Tunnel Encap controls the Encap Vector - Tunnel
+		 *  Encap, 3 bits. Defines the type of Tunnel
+		 *  encapsulation the template is describing
+		 * <ul>
+		 * <li> CFA_P4_ACT_ENCAP_TNL_NONE, default, no Tunnel
+		 *      Encapsulation processing.
+		 * <li> CFA_P4_ACT_ENCAP_TNL_GENERIC_FULL
+		 * <li> CFA_P4_ACT_ENCAP_TNL_VXLAN. NOTE: Expects
+		 *      l4_enable set to CFA_P4_TRUE;
+		 * <li> CFA_P4_ACT_ENCAP_TNL_NGE. NOTE: Expects l4_enable
+		 *      set to CFA_P4_TRUE;
+		 * <li> CFA_P4_ACT_ENCAP_TNL_NVGRE. NOTE: only valid if
+		 *      l4_enable set to CFA_HCAPI_FALSE.
+		 * <li> CFA_P4_ACT_ENCAP_TNL_GRE.NOTE: only valid if
+		 *      l4_enable set to CFA_HCAPI_FALSE.
+		 * <li> CFA_P4_ACT_ENCAP_TNL_GENERIC_AFTER_TL4
+		 * <li> CFA_P4_ACT_ENCAP_TNL_GENERIC_AFTER_TNL
+		 * </ul>
+		 */
+		enum {
+			/** Set to disable Tunnel header encapsulation
+			 * processing, default
+			 */
+			CFA_P4_ACT_ENCAP_TNL_NONE = 0,
+			/** Set to enable Tunnel Generic Full header
+			 * encapsulation
+			 */
+			CFA_P4_ACT_ENCAP_TNL_GENERIC_FULL,
+			/** Set to enable VXLAN header encapsulation
+			 */
+			CFA_P4_ACT_ENCAP_TNL_VXLAN,
+			/** Set to enable NGE (VXLAN2) header encapsulation
+			 */
+			CFA_P4_ACT_ENCAP_TNL_NGE,
+			/** Set to enable NVGRE header encapsulation
+			 */
+			CFA_P4_ACT_ENCAP_TNL_NVGRE,
+			/** Set to enable GRE header encapsulation
+			 */
+			CFA_P4_ACT_ENCAP_TNL_GRE,
+			/** Set to enable Generic header after Tunnel
+			 * L4 encapsulation
+			 */
+			CFA_P4_ACT_ENCAP_TNL_GENERIC_AFTER_TL4,
+			/** Set to enable Generic header after Tunnel
+			 * encapsulation
+			 */
+			CFA_P4_ACT_ENCAP_TNL_GENERIC_AFTER_TNL
+		} tnl;
+
+		/** Number of bytes of generic tunnel header,
+		 * valid when
+		 * (tnl == CFA_P4_ACT_ENCAP_TNL_GENERIC_FULL) ||
+		 * (tnl == CFA_P4_ACT_ENCAP_TNL_GENERIC_AFTER_TL4) ||
+		 * (tnl == CFA_P4_ACT_ENCAP_TNL_GENERIC_AFTER_TNL)
+		 */
+		uint8_t tnl_generic_size;
+		/** Number of 32b words of nge options,
+		 * valid when
+		 * (tnl == CFA_P4_ACT_ENCAP_TNL_NGE)
+		 */
+		uint8_t tnl_nge_op_len;
+		/* Currently not planned */
+		/* Custom Header */
+		/*	uint8_t custom_enable; */
+	} encap;
+};
+
+/**
+ * Enumeration of SRAM entry types, used for allocation of
+ * fixed SRAM entities. The memory model for CFA HCAPI
+ * determines if an SRAM entry type is supported.
+ */
+enum cfa_p4_action_sram_entry_type {
+	/* NOTE: Any additions to this enum must be reflected on FW
+	 * side as well.
+	 */
+
+	/** SRAM Action Record */
+	CFA_P4_ACTION_SRAM_ENTRY_TYPE_ACT,
+	/** SRAM Action Encap 8 Bytes */
+	CFA_P4_ACTION_SRAM_ENTRY_TYPE_ENCAP_8B,
+	/** SRAM Action Encap 16 Bytes */
+	CFA_P4_ACTION_SRAM_ENTRY_TYPE_ENCAP_16B,
+	/** SRAM Action Encap 64 Bytes */
+	CFA_P4_ACTION_SRAM_ENTRY_TYPE_ENCAP_64B,
+	/** SRAM Action Modify IPv4 Source */
+	CFA_P4_ACTION_SRAM_ENTRY_TYPE_MODIFY_IPV4_SRC,
+	/** SRAM Action Modify IPv4 Destination */
+	CFA_P4_ACTION_SRAM_ENTRY_TYPE_MODIFY_IPV4_DEST,
+	/** SRAM Action Source Properties SMAC */
+	CFA_P4_ACTION_SRAM_ENTRY_TYPE_SP_SMAC,
+	/** SRAM Action Source Properties SMAC IPv4 */
+	CFA_P4_ACTION_SRAM_ENTRY_TYPE_SP_SMAC_IPV4,
+	/** SRAM Action Source Properties SMAC IPv6 */
+	CFA_P4_ACTION_SRAM_ENTRY_TYPE_SP_SMAC_IPV6,
+	/** SRAM Action Statistics 64 Bits */
+	CFA_P4_ACTION_SRAM_ENTRY_TYPE_STATS_64,
+	CFA_P4_ACTION_SRAM_ENTRY_TYPE_MAX
+};
+
+/**
+ * SRAM Action Record structure holding either an action index or an
+ * action ptr.
+ */
+union cfa_p4_action_sram_act_record {
+	/** SRAM Action idx specifies the offset of the SRAM
+	 * element within its SRAM Entry Type block. This
+	 * index can be written into i.e. an L2 Context. Use
+	 * this type for all SRAM Action Record types except
+	 * SRAM Full Action records. Use act_ptr instead.
+	 */
+	uint16_t act_idx;
+	/** SRAM Full Action is special in that it needs an
+	 * action record pointer. This pointer can be written
+	 * into i.e. a Wildcard TCAM entry.
+	 */
+	uint32_t act_ptr;
+};
+
+/**
+ * cfa_p4_action_param parameter definition
+ */
+struct cfa_p4_action_param {
+	/**
+	 * [in] receive or transmit direction
+	 */
+	uint8_t dir;
+	/**
+	 * [in] type of the sram allocation type
+	 */
+	enum cfa_p4_action_sram_entry_type type;
+	/**
+	 * [in] action record to set. The 'type' specified lists the
+	 *	record definition to use in the passed in record.
+	 */
+	union cfa_p4_action_sram_act_record record;
+	/**
+	 * [in] number of elements in act_data
+	 */
+	uint32_t act_size;
+	/**
+	 * [in] ptr to array of action data
+	 */
+	uint64_t *act_data;
+};
+
+/**
+ * EEM Key entry sizes
+ */
+#define CFA_P4_EEM_KEY_MAX_SIZE 52
+#define CFA_P4_EEM_KEY_RECORD_SIZE 64
+
+/**
+ * cfa_eem_entry_hdr
+ */
+struct cfa_p4_eem_entry_hdr {
+	uint32_t pointer;
+	uint32_t word1;  /*
+			  * The header is made up of two words,
+			  * this is the first word. This field has multiple
+			  * subfields, there is no suitable single name for
+			  * it so just going with word1.
+			  */
+#define CFA_P4_EEM_ENTRY_VALID_SHIFT 31
+#define CFA_P4_EEM_ENTRY_VALID_MASK 0x80000000
+#define CFA_P4_EEM_ENTRY_L1_CACHEABLE_SHIFT 30
+#define CFA_P4_EEM_ENTRY_L1_CACHEABLE_MASK 0x40000000
+#define CFA_P4_EEM_ENTRY_STRENGTH_SHIFT 28
+#define CFA_P4_EEM_ENTRY_STRENGTH_MASK 0x30000000
+#define CFA_P4_EEM_ENTRY_RESERVED_SHIFT 17
+#define CFA_P4_EEM_ENTRY_RESERVED_MASK 0x0FFE0000
+#define CFA_P4_EEM_ENTRY_KEY_SIZE_SHIFT 8
+#define CFA_P4_EEM_ENTRY_KEY_SIZE_MASK 0x0001FF00
+#define CFA_P4_EEM_ENTRY_ACT_REC_SIZE_SHIFT 3
+#define CFA_P4_EEM_ENTRY_ACT_REC_SIZE_MASK 0x000000F8
+#define CFA_P4_EEM_ENTRY_ACT_REC_INT_SHIFT 2
+#define CFA_P4_EEM_ENTRY_ACT_REC_INT_MASK 0x00000004
+#define CFA_P4_EEM_ENTRY_EXT_FLOW_CTR_SHIFT 1
+#define CFA_P4_EEM_ENTRY_EXT_FLOW_CTR_MASK 0x00000002
+#define CFA_P4_EEM_ENTRY_ACT_PTR_MSB_SHIFT 0
+#define CFA_P4_EEM_ENTRY_ACT_PTR_MSB_MASK 0x00000001
+};
+
+/**
+ *  cfa_p4_eem_key_entry
+ */
+struct cfa_p4_eem_64b_entry {
+	/** Key is 448 bits - 56 bytes */
+	uint8_t key[CFA_P4_EEM_KEY_RECORD_SIZE - sizeof(struct cfa_p4_eem_entry_hdr)];
+	/** Header is 8 bytes long */
+	struct cfa_p4_eem_entry_hdr hdr;
+};
+
+#endif /* _CFA_HW_P4_H_ */
diff --git a/drivers/net/bnxt/meson.build b/drivers/net/bnxt/meson.build
index 1f7df9d..4994d4c 100644
--- a/drivers/net/bnxt/meson.build
+++ b/drivers/net/bnxt/meson.build
@@ -43,6 +43,9 @@ sources = files('bnxt_cpr.c',
 	'tf_core/tf_util.c',
 	'tf_core/tf_rm_new.c',
 
+	'hcapi/hcapi_cfa_common.c',
+	'hcapi/hcapi_cfa_p4.c',
+
 	'tf_ulp/bnxt_ulp.c',
 	'tf_ulp/ulp_mark_mgr.c',
 	'tf_ulp/ulp_flow_db.c',
diff --git a/drivers/net/bnxt/tf_core/tf_em.c b/drivers/net/bnxt/tf_core/tf_em.c
index 91cbc62..da1f4d4 100644
--- a/drivers/net/bnxt/tf_core/tf_em.c
+++ b/drivers/net/bnxt/tf_core/tf_em.c
@@ -189,7 +189,7 @@ void *tf_em_get_table_page(struct tf_tbl_scope_cb *tbl_scope_cb,
 	if (dir != TF_DIR_RX && dir != TF_DIR_TX)
 		return NULL;
 
-	if (table_type < KEY0_TABLE || table_type > EFC_TABLE)
+	if (table_type < TF_KEY0_TABLE || table_type > TF_EFC_TABLE)
 		return NULL;
 
 	/*
@@ -325,7 +325,7 @@ static int tf_em_select_inject_table(struct tf_tbl_scope_cb *tbl_scope_cb,
 	key0_entry = tf_em_entry_exists(tbl_scope_cb,
 					 entry,
 					 key0_hash,
-					 KEY0_TABLE,
+					 TF_KEY0_TABLE,
 					 dir);
 
 	/*
@@ -334,23 +334,23 @@ static int tf_em_select_inject_table(struct tf_tbl_scope_cb *tbl_scope_cb,
 	key1_entry = tf_em_entry_exists(tbl_scope_cb,
 					 entry,
 					 key1_hash,
-					 KEY1_TABLE,
+					 TF_KEY1_TABLE,
 					 dir);
 
 	if (key0_entry == -EEXIST) {
-		*table = KEY0_TABLE;
+		*table = TF_KEY0_TABLE;
 		*index = key0_hash;
 		return -EEXIST;
 	} else if (key1_entry == -EEXIST) {
-		*table = KEY1_TABLE;
+		*table = TF_KEY1_TABLE;
 		*index = key1_hash;
 		return -EEXIST;
 	} else if (key0_entry == 0) {
-		*table = KEY0_TABLE;
+		*table = TF_KEY0_TABLE;
 		*index = key0_hash;
 		return 0;
 	} else if (key1_entry == 0) {
-		*table = KEY1_TABLE;
+		*table = TF_KEY1_TABLE;
 		*index = key1_hash;
 		return 0;
 	}
@@ -384,7 +384,7 @@ int tf_insert_eem_entry(struct tf_session *session,
 	int		   num_of_entry;
 
 	/* Get mask to use on hash */
-	mask = tf_em_get_key_mask(tbl_scope_cb->em_ctx_info[parms->dir].em_tables[KEY0_TABLE].num_entries);
+	mask = tf_em_get_key_mask(tbl_scope_cb->em_ctx_info[parms->dir].em_tables[TF_KEY0_TABLE].num_entries);
 
 	if (!mask)
 		return -EINVAL;
@@ -420,14 +420,14 @@ int tf_insert_eem_entry(struct tf_session *session,
 				      key1_index,
 				      &index,
 				      &table_type) == 0) {
-		if (table_type == KEY0_TABLE) {
+		if (table_type == TF_KEY0_TABLE) {
 			TF_SET_GFID(gfid,
 				    key0_index,
-				    KEY0_TABLE);
+				    TF_KEY0_TABLE);
 		} else {
 			TF_SET_GFID(gfid,
 				    key1_index,
-				    KEY1_TABLE);
+				    TF_KEY1_TABLE);
 		}
 
 		/*
diff --git a/drivers/net/bnxt/tf_core/tf_tbl.c b/drivers/net/bnxt/tf_core/tf_tbl.c
index f9bfae7..07c3469 100644
--- a/drivers/net/bnxt/tf_core/tf_tbl.c
+++ b/drivers/net/bnxt/tf_core/tf_tbl.c
@@ -285,8 +285,8 @@ tf_em_setup_page_table(struct tf_em_table *tbl)
 		tf_em_link_page_table(tp, tp_next, set_pte_last);
 	}
 
-	tbl->l0_addr = tbl->pg_tbl[PT_LVL_0].pg_va_tbl[0];
-	tbl->l0_dma_addr = tbl->pg_tbl[PT_LVL_0].pg_pa_tbl[0];
+	tbl->l0_addr = tbl->pg_tbl[TF_PT_LVL_0].pg_va_tbl[0];
+	tbl->l0_dma_addr = tbl->pg_tbl[TF_PT_LVL_0].pg_pa_tbl[0];
 }
 
 /**
@@ -317,7 +317,7 @@ tf_em_size_page_tbl_lvl(uint32_t page_size,
 			uint64_t *num_data_pages)
 {
 	uint64_t lvl_data_size = page_size;
-	int lvl = PT_LVL_0;
+	int lvl = TF_PT_LVL_0;
 	uint64_t data_size;
 
 	*num_data_pages = 0;
@@ -326,10 +326,10 @@ tf_em_size_page_tbl_lvl(uint32_t page_size,
 	while (lvl_data_size < data_size) {
 		lvl++;
 
-		if (lvl == PT_LVL_1)
+		if (lvl == TF_PT_LVL_1)
 			lvl_data_size = (uint64_t)MAX_PAGE_PTRS(page_size) *
 				page_size;
-		else if (lvl == PT_LVL_2)
+		else if (lvl == TF_PT_LVL_2)
 			lvl_data_size = (uint64_t)MAX_PAGE_PTRS(page_size) *
 				MAX_PAGE_PTRS(page_size) * page_size;
 		else
@@ -386,18 +386,18 @@ tf_em_size_page_tbls(int max_lvl,
 		     uint32_t page_size,
 		     uint32_t *page_cnt)
 {
-	if (max_lvl == PT_LVL_0) {
-		page_cnt[PT_LVL_0] = num_data_pages;
-	} else if (max_lvl == PT_LVL_1) {
-		page_cnt[PT_LVL_1] = num_data_pages;
-		page_cnt[PT_LVL_0] =
-		tf_em_page_tbl_pgcnt(page_cnt[PT_LVL_1], page_size);
-	} else if (max_lvl == PT_LVL_2) {
-		page_cnt[PT_LVL_2] = num_data_pages;
-		page_cnt[PT_LVL_1] =
-		tf_em_page_tbl_pgcnt(page_cnt[PT_LVL_2], page_size);
-		page_cnt[PT_LVL_0] =
-		tf_em_page_tbl_pgcnt(page_cnt[PT_LVL_1], page_size);
+	if (max_lvl == TF_PT_LVL_0) {
+		page_cnt[TF_PT_LVL_0] = num_data_pages;
+	} else if (max_lvl == TF_PT_LVL_1) {
+		page_cnt[TF_PT_LVL_1] = num_data_pages;
+		page_cnt[TF_PT_LVL_0] =
+		tf_em_page_tbl_pgcnt(page_cnt[TF_PT_LVL_1], page_size);
+	} else if (max_lvl == TF_PT_LVL_2) {
+		page_cnt[TF_PT_LVL_2] = num_data_pages;
+		page_cnt[TF_PT_LVL_1] =
+		tf_em_page_tbl_pgcnt(page_cnt[TF_PT_LVL_2], page_size);
+		page_cnt[TF_PT_LVL_0] =
+		tf_em_page_tbl_pgcnt(page_cnt[TF_PT_LVL_1], page_size);
 	} else {
 		return;
 	}
@@ -434,7 +434,7 @@ tf_em_size_table(struct tf_em_table *tbl)
 	/* Determine number of page table levels and the number
 	 * of data pages needed to process the given eem table.
 	 */
-	if (tbl->type == RECORD_TABLE) {
+	if (tbl->type == TF_RECORD_TABLE) {
 		/*
 		 * For action records just a memory size is provided. Work
 		 * backwards to resolve to number of entries
@@ -480,9 +480,9 @@ tf_em_size_table(struct tf_em_table *tbl)
 		    max_lvl + 1,
 		    (uint64_t)num_data_pages * TF_EM_PAGE_SIZE,
 		    num_data_pages,
-		    page_cnt[PT_LVL_0],
-		    page_cnt[PT_LVL_1],
-		    page_cnt[PT_LVL_2]);
+		    page_cnt[TF_PT_LVL_0],
+		    page_cnt[TF_PT_LVL_1],
+		    page_cnt[TF_PT_LVL_2]);
 
 	return 0;
 }
@@ -508,7 +508,7 @@ tf_em_ctx_unreg(struct tf *tfp,
 	struct tf_em_table *tbl;
 	int i;
 
-	for (i = KEY0_TABLE; i < MAX_TABLE; i++) {
+	for (i = TF_KEY0_TABLE; i < TF_MAX_TABLE; i++) {
 		tbl = &ctxp->em_tables[i];
 
 		if (tbl->num_entries != 0 && tbl->entry_size != 0) {
@@ -544,7 +544,7 @@ tf_em_ctx_reg(struct tf *tfp,
 	int rc = 0;
 	int i;
 
-	for (i = KEY0_TABLE; i < MAX_TABLE; i++) {
+	for (i = TF_KEY0_TABLE; i < TF_MAX_TABLE; i++) {
 		tbl = &ctxp->em_tables[i];
 
 		if (tbl->num_entries && tbl->entry_size) {
@@ -719,41 +719,41 @@ tf_em_validate_num_entries(struct tf_tbl_scope_cb *tbl_scope_cb,
 		return -EINVAL;
 	}
 	/* Rx */
-	tbl_scope_cb->em_ctx_info[TF_DIR_RX].em_tables[KEY0_TABLE].num_entries =
+	tbl_scope_cb->em_ctx_info[TF_DIR_RX].em_tables[TF_KEY0_TABLE].num_entries =
 		parms->rx_num_flows_in_k * TF_KILOBYTE;
-	tbl_scope_cb->em_ctx_info[TF_DIR_RX].em_tables[KEY0_TABLE].entry_size =
+	tbl_scope_cb->em_ctx_info[TF_DIR_RX].em_tables[TF_KEY0_TABLE].entry_size =
 		parms->rx_max_key_sz_in_bits / 8;
 
-	tbl_scope_cb->em_ctx_info[TF_DIR_RX].em_tables[KEY1_TABLE].num_entries =
+	tbl_scope_cb->em_ctx_info[TF_DIR_RX].em_tables[TF_KEY1_TABLE].num_entries =
 		parms->rx_num_flows_in_k * TF_KILOBYTE;
-	tbl_scope_cb->em_ctx_info[TF_DIR_RX].em_tables[KEY1_TABLE].entry_size =
+	tbl_scope_cb->em_ctx_info[TF_DIR_RX].em_tables[TF_KEY1_TABLE].entry_size =
 		parms->rx_max_key_sz_in_bits / 8;
 
-	tbl_scope_cb->em_ctx_info[TF_DIR_RX].em_tables[RECORD_TABLE].num_entries
+	tbl_scope_cb->em_ctx_info[TF_DIR_RX].em_tables[TF_RECORD_TABLE].num_entries
 		= parms->rx_num_flows_in_k * TF_KILOBYTE;
-	tbl_scope_cb->em_ctx_info[TF_DIR_RX].em_tables[RECORD_TABLE].entry_size
+	tbl_scope_cb->em_ctx_info[TF_DIR_RX].em_tables[TF_RECORD_TABLE].entry_size
 		= parms->rx_max_action_entry_sz_in_bits / 8;
 
-	tbl_scope_cb->em_ctx_info[TF_DIR_RX].em_tables[EFC_TABLE].num_entries
+	tbl_scope_cb->em_ctx_info[TF_DIR_RX].em_tables[TF_EFC_TABLE].num_entries
 		= 0;
 
 	/* Tx */
-	tbl_scope_cb->em_ctx_info[TF_DIR_TX].em_tables[KEY0_TABLE].num_entries =
+	tbl_scope_cb->em_ctx_info[TF_DIR_TX].em_tables[TF_KEY0_TABLE].num_entries =
 		parms->tx_num_flows_in_k * TF_KILOBYTE;
-	tbl_scope_cb->em_ctx_info[TF_DIR_TX].em_tables[KEY0_TABLE].entry_size =
+	tbl_scope_cb->em_ctx_info[TF_DIR_TX].em_tables[TF_KEY0_TABLE].entry_size =
 		parms->tx_max_key_sz_in_bits / 8;
 
-	tbl_scope_cb->em_ctx_info[TF_DIR_TX].em_tables[KEY1_TABLE].num_entries =
+	tbl_scope_cb->em_ctx_info[TF_DIR_TX].em_tables[TF_KEY1_TABLE].num_entries =
 		parms->tx_num_flows_in_k * TF_KILOBYTE;
-	tbl_scope_cb->em_ctx_info[TF_DIR_TX].em_tables[KEY1_TABLE].entry_size =
+	tbl_scope_cb->em_ctx_info[TF_DIR_TX].em_tables[TF_KEY1_TABLE].entry_size =
 		parms->tx_max_key_sz_in_bits / 8;
 
-	tbl_scope_cb->em_ctx_info[TF_DIR_TX].em_tables[RECORD_TABLE].num_entries
+	tbl_scope_cb->em_ctx_info[TF_DIR_TX].em_tables[TF_RECORD_TABLE].num_entries
 		= parms->tx_num_flows_in_k * TF_KILOBYTE;
-	tbl_scope_cb->em_ctx_info[TF_DIR_TX].em_tables[RECORD_TABLE].entry_size
+	tbl_scope_cb->em_ctx_info[TF_DIR_TX].em_tables[TF_RECORD_TABLE].entry_size
 		= parms->tx_max_action_entry_sz_in_bits / 8;
 
-	tbl_scope_cb->em_ctx_info[TF_DIR_TX].em_tables[EFC_TABLE].num_entries
+	tbl_scope_cb->em_ctx_info[TF_DIR_TX].em_tables[TF_EFC_TABLE].num_entries
 		= 0;
 
 	return 0;
@@ -1572,11 +1572,11 @@ tf_alloc_eem_tbl_scope(struct tf *tfp,
 
 		em_tables = tbl_scope_cb->em_ctx_info[dir].em_tables;
 		rc = tf_msg_em_cfg(tfp,
-				   em_tables[KEY0_TABLE].num_entries,
-				   em_tables[KEY0_TABLE].ctx_id,
-				   em_tables[KEY1_TABLE].ctx_id,
-				   em_tables[RECORD_TABLE].ctx_id,
-				   em_tables[EFC_TABLE].ctx_id,
+				   em_tables[TF_KEY0_TABLE].num_entries,
+				   em_tables[TF_KEY0_TABLE].ctx_id,
+				   em_tables[TF_KEY1_TABLE].ctx_id,
+				   em_tables[TF_RECORD_TABLE].ctx_id,
+				   em_tables[TF_EFC_TABLE].ctx_id,
 				   parms->hw_flow_cache_flush_timer,
 				   dir);
 		if (rc) {
@@ -1601,8 +1601,8 @@ tf_alloc_eem_tbl_scope(struct tf *tfp,
 		 */
 		rc = tf_create_tbl_pool_external(dir,
 					    tbl_scope_cb,
-					    em_tables[RECORD_TABLE].num_entries,
-					    em_tables[RECORD_TABLE].entry_size);
+					    em_tables[TF_RECORD_TABLE].num_entries,
+					    em_tables[TF_RECORD_TABLE].entry_size);
 		if (rc) {
 			PMD_DRV_LOG(ERR,
 				    "%d TBL: Unable to allocate idx pools %s\n",
@@ -1672,7 +1672,7 @@ tf_set_tbl_entry(struct tf *tfp,
 		base_addr = tf_em_get_table_page(tbl_scope_cb,
 						 parms->dir,
 						 offset,
-						 RECORD_TABLE);
+						 TF_RECORD_TABLE);
 		if (base_addr == NULL) {
 			PMD_DRV_LOG(ERR,
 				    "dir:%d, Base address lookup failed\n",
@@ -1972,7 +1972,7 @@ void tf_dump_dma(struct tf *tfp, uint32_t tbl_scope_id)
 	for (dir = 0; dir < TF_DIR_MAX; dir++) {
 		printf("Direction %s:\n", (dir == TF_DIR_RX ? "Rx" : "Tx"));
 
-		for (j = KEY0_TABLE; j < MAX_TABLE; j++) {
+		for (j = TF_KEY0_TABLE; j < TF_MAX_TABLE; j++) {
 			tbl = &tbl_scope_cb->em_ctx_info[dir].em_tables[j];
 			printf
 	("Table: j:%d type:%d num_entries:%d entry_size:0x%x num_lvl:%d ",
diff --git a/drivers/net/bnxt/tf_core/tf_tbl.h b/drivers/net/bnxt/tf_core/tf_tbl.h
index b335a9c..d78e4fe 100644
--- a/drivers/net/bnxt/tf_core/tf_tbl.h
+++ b/drivers/net/bnxt/tf_core/tf_tbl.h
@@ -14,18 +14,18 @@
 struct tf_session;
 
 enum tf_pg_tbl_lvl {
-	PT_LVL_0,
-	PT_LVL_1,
-	PT_LVL_2,
-	PT_LVL_MAX
+	TF_PT_LVL_0,
+	TF_PT_LVL_1,
+	TF_PT_LVL_2,
+	TF_PT_LVL_MAX
 };
 
 enum tf_em_table_type {
-	KEY0_TABLE,
-	KEY1_TABLE,
-	RECORD_TABLE,
-	EFC_TABLE,
-	MAX_TABLE
+	TF_KEY0_TABLE,
+	TF_KEY1_TABLE,
+	TF_RECORD_TABLE,
+	TF_EFC_TABLE,
+	TF_MAX_TABLE
 };
 
 struct tf_em_page_tbl {
@@ -41,15 +41,15 @@ struct tf_em_table {
 	uint16_t			ctx_id;
 	uint32_t			entry_size;
 	int				num_lvl;
-	uint32_t			page_cnt[PT_LVL_MAX];
+	uint32_t			page_cnt[TF_PT_LVL_MAX];
 	uint64_t			num_data_pages;
 	void				*l0_addr;
 	uint64_t			l0_dma_addr;
-	struct tf_em_page_tbl pg_tbl[PT_LVL_MAX];
+	struct tf_em_page_tbl pg_tbl[TF_PT_LVL_MAX];
 };
 
 struct tf_em_ctx_mem_info {
-	struct tf_em_table		em_tables[MAX_TABLE];
+	struct tf_em_table		em_tables[TF_MAX_TABLE];
 };
 
 /** table scope control block content */
-- 
2.7.4


^ permalink raw reply	[flat|nested] 271+ messages in thread

* [dpdk-dev] [PATCH 16/50] net/bnxt: add core changes for EM and EEM lookups
  2020-06-12 13:28 [dpdk-dev] [PATCH 00/50] add features for host-based flow management Somnath Kotur
                   ` (14 preceding siblings ...)
  2020-06-12 13:28 ` [dpdk-dev] [PATCH 15/50] net/bnxt: add HCAPI interface support Somnath Kotur
@ 2020-06-12 13:29 ` Somnath Kotur
  2020-06-12 13:29 ` [dpdk-dev] [PATCH 17/50] net/bnxt: implement support for TCAM access Somnath Kotur
                   ` (34 subsequent siblings)
  50 siblings, 0 replies; 271+ messages in thread
From: Somnath Kotur @ 2020-06-12 13:29 UTC (permalink / raw)
  To: dev; +Cc: ferruh.yigit

From: Randy Schacher <stuart.schacher@broadcom.com>

- Move External Exact and Exact Match to device module using HCAPI
  to add and delete entries
- Make EM active through the device interface.

Signed-off-by: Randy Schacher <stuart.schacher@broadcom.com>
Reviewed-by: Randy Schacher <stuart.schacher@broadcom.com>
Reviewed-by: Shahaji Bhosle <shahaji.bhosle@broadcom.com>
Signed-off-by: Venkat Duvvuru <venkatkumar.duvvuru@broadcom.com>
---
 drivers/net/bnxt/Makefile                 |   3 +-
 drivers/net/bnxt/hcapi/Makefile           |   5 +-
 drivers/net/bnxt/hcapi/cfa_p40_hw.h       | 688 ++++++++++++++++++++++++++++++
 drivers/net/bnxt/hcapi/cfa_p40_tbl.h      | 250 +++++++++++
 drivers/net/bnxt/hcapi/hcapi_cfa_common.c |  92 ----
 drivers/net/bnxt/hcapi/hcapi_cfa_defs.h   |  26 +-
 drivers/net/bnxt/hcapi/hcapi_cfa_p4.c     |  14 +-
 drivers/net/bnxt/hcapi/hcapi_cfa_p4.h     |   2 +-
 drivers/net/bnxt/meson.build              |   1 -
 drivers/net/bnxt/tf_core/Makefile         |   8 +
 drivers/net/bnxt/tf_core/hwrm_tf.h        |  24 +-
 drivers/net/bnxt/tf_core/stack.c          |   2 +-
 drivers/net/bnxt/tf_core/tf_core.c        | 441 ++++++++++---------
 drivers/net/bnxt/tf_core/tf_core.h        | 141 +++---
 drivers/net/bnxt/tf_core/tf_device.h      |  32 ++
 drivers/net/bnxt/tf_core/tf_device_p4.c   |   3 +
 drivers/net/bnxt/tf_core/tf_em.c          | 567 +++++++-----------------
 drivers/net/bnxt/tf_core/tf_em.h          |  72 +---
 drivers/net/bnxt/tf_core/tf_msg.c         |  22 +-
 drivers/net/bnxt/tf_core/tf_msg.h         |   4 +-
 drivers/net/bnxt/tf_core/tf_resources.h   |  25 +-
 drivers/net/bnxt/tf_core/tf_rm.c          | 163 ++++---
 drivers/net/bnxt/tf_core/tf_tbl.c         | 440 ++++++++-----------
 drivers/net/bnxt/tf_core/tf_tbl.h         |  49 +--
 24 files changed, 1816 insertions(+), 1258 deletions(-)
 create mode 100644 drivers/net/bnxt/hcapi/cfa_p40_hw.h
 create mode 100644 drivers/net/bnxt/hcapi/cfa_p40_tbl.h
 delete mode 100644 drivers/net/bnxt/hcapi/hcapi_cfa_common.c

diff --git a/drivers/net/bnxt/Makefile b/drivers/net/bnxt/Makefile
index 3656274..349b09c 100644
--- a/drivers/net/bnxt/Makefile
+++ b/drivers/net/bnxt/Makefile
@@ -46,9 +46,10 @@ SRCS-$(CONFIG_RTE_LIBRTE_BNXT_PMD) += bnxt_rxtx_vec_sse.c
 endif
 
 ifeq ($(CONFIG_RTE_LIBRTE_BNXT_PMD), y)
-CFLAGS += -I$(SRCDIR) -I$(SRCDIR)/tf_ulp -I$(SRCDIR)/tf_core
+CFLAGS += -I$(SRCDIR) -I$(SRCDIR)/tf_ulp -I$(SRCDIR)/tf_core -I$(SRCDIR)/hcapi
 include $(SRCDIR)/tf_ulp/Makefile
 include $(SRCDIR)/tf_core/Makefile
+include $(SRCDIR)/hcapi/Makefile
 endif
 
 #
diff --git a/drivers/net/bnxt/hcapi/Makefile b/drivers/net/bnxt/hcapi/Makefile
index c4c91b6..65cddd7 100644
--- a/drivers/net/bnxt/hcapi/Makefile
+++ b/drivers/net/bnxt/hcapi/Makefile
@@ -2,6 +2,9 @@
 # Copyright(c) 2019-2020 Broadcom Limited.
 # All rights reserved.
 
-SRCS-$(CONFIG_RTE_LIBRTE_BNXT_PMD) += hcapi/hcapi_cfa_common.c
 SRCS-$(CONFIG_RTE_LIBRTE_BNXT_PMD) += hcapi/hcapi_cfa_p4.c
 
+SYMLINK-$(CONFIG_RTE_LIBRTE_BNXT_PMD)-include += hcapi/hcapi_cfa.h
+SYMLINK-$(CONFIG_RTE_LIBRTE_BNXT_PMD)-include += hcapi/hcapi_cfa_defs.h
+SYMLINK-$(CONFIG_RTE_LIBRTE_BNXT_PMD)-include += hcapi/hcapi_cfa_p4.h
+SYMLINK-$(CONFIG_RTE_LIBRTE_BNXT_PMD)-include += hcapi/cfa_p40_hw.h
diff --git a/drivers/net/bnxt/hcapi/cfa_p40_hw.h b/drivers/net/bnxt/hcapi/cfa_p40_hw.h
new file mode 100644
index 0000000..1c51da8
--- /dev/null
+++ b/drivers/net/bnxt/hcapi/cfa_p40_hw.h
@@ -0,0 +1,688 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2019-2020 Broadcom
+ * All rights reserved.
+ */
+/*
+ * Name:  cfa_p40_hw.h
+ *
+ * Description: header for SWE based on TFLIB2.0
+ *
+ * Date:  taken from 12/16/19 17:18:12
+ *
+ * Note:  This file was first generated using  tflib_decode.py.
+ *
+ *        Changes have been made due to lack of availability of xml for
+ *        addtional tables at this time (EEM Record and union table fields)
+ *        Changes not autogenerated are noted in comments.
+ */
+
+#ifndef _CFA_P40_HW_H_
+#define _CFA_P40_HW_H_
+
+/**
+ * Valid TCAM entry. (for idx 5 ...)
+ */
+#define CFA_P40_PROF_L2_CTXT_TCAM_VALID_BITPOS   166
+#define CFA_P40_PROF_L2_CTXT_TCAM_VALID_NUM_BITS 1
+/**
+ * Key type (pass). (for idx 5 ...)
+ */
+#define CFA_P40_PROF_L2_CTXT_TCAM_KEY_TYPE_BITPOS 164
+#define CFA_P40_PROF_L2_CTXT_TCAM_KEY_TYPE_NUM_BITS 2
+/**
+ * Tunnel HDR type. (for idx 5 ...)
+ */
+#define CFA_P40_PROF_L2_CTXT_TCAM_TUN_HDR_TYPE_BITPOS 160
+#define CFA_P40_PROF_L2_CTXT_TCAM_TUN_HDR_TYPE_NUM_BITS 4
+/**
+ * Number of VLAN tags in tunnel l2 header. (for idx 4 ...)
+ */
+#define CFA_P40_PROF_L2_CTXT_TCAM_T_L2_NUMTAGS_BITPOS 158
+#define CFA_P40_PROF_L2_CTXT_TCAM_T_L2_NUMTAGS_NUM_BITS 2
+/**
+ * Number of VLAN tags in l2 header. (for idx 4 ...)
+ */
+#define CFA_P40_PROF_L2_CTXT_TCAM_L2_NUMTAGS_BITPOS 156
+#define CFA_P40_PROF_L2_CTXT_TCAM_L2_NUMTAGS_NUM_BITS 2
+/**
+ * Tunnel/Inner Source/Dest. MAC Address.
+ */
+#define CFA_P40_PROF_L2_CTXT_TCAM_MAC1_BITPOS    108
+#define CFA_P40_PROF_L2_CTXT_TCAM_MAC1_NUM_BITS  48
+/**
+ * Tunnel Outer VLAN Tag ID. (for idx 3 ...)
+ */
+#define CFA_P40_PROF_L2_CTXT_TCAM_T_OVID_BITPOS  96
+#define CFA_P40_PROF_L2_CTXT_TCAM_T_OVID_NUM_BITS 12
+/**
+ * Tunnel Inner VLAN Tag ID. (for idx 2 ...)
+ */
+#define CFA_P40_PROF_L2_CTXT_TCAM_T_IVID_BITPOS  84
+#define CFA_P40_PROF_L2_CTXT_TCAM_T_IVID_NUM_BITS 12
+/**
+ * Source Partition. (for idx 2 ...)
+ */
+#define CFA_P40_PROF_L2_CTXT_TCAM_SPARIF_BITPOS  80
+#define CFA_P40_PROF_L2_CTXT_TCAM_SPARIF_NUM_BITS 4
+/**
+ * Source Virtual I/F. (for idx 2 ...)
+ */
+#define CFA_P40_PROF_L2_CTXT_TCAM_SVIF_BITPOS    72
+#define CFA_P40_PROF_L2_CTXT_TCAM_SVIF_NUM_BITS  8
+/**
+ * Tunnel/Inner Source/Dest. MAC Address.
+ */
+#define CFA_P40_PROF_L2_CTXT_TCAM_MAC0_BITPOS    24
+#define CFA_P40_PROF_L2_CTXT_TCAM_MAC0_NUM_BITS  48
+/**
+ * Outer VLAN Tag ID.
+ */
+#define CFA_P40_PROF_L2_CTXT_TCAM_OVID_BITPOS    12
+#define CFA_P40_PROF_L2_CTXT_TCAM_OVID_NUM_BITS  12
+/**
+ * Inner VLAN Tag ID.
+ */
+#define CFA_P40_PROF_L2_CTXT_TCAM_IVID_BITPOS    0
+#define CFA_P40_PROF_L2_CTXT_TCAM_IVID_NUM_BITS  12
+
+enum cfa_p40_prof_l2_ctxt_tcam_flds {
+	CFA_P40_PROF_L2_CTXT_TCAM_VALID_FLD = 0,
+	CFA_P40_PROF_L2_CTXT_TCAM_KEY_TYPE_FLD = 1,
+	CFA_P40_PROF_L2_CTXT_TCAM_TUN_HDR_TYPE_FLD = 2,
+	CFA_P40_PROF_L2_CTXT_TCAM_T_L2_NUMTAGS_FLD = 3,
+	CFA_P40_PROF_L2_CTXT_TCAM_L2_NUMTAGS_FLD = 4,
+	CFA_P40_PROF_L2_CTXT_TCAM_MAC1_FLD = 5,
+	CFA_P40_PROF_L2_CTXT_TCAM_T_OVID_FLD = 6,
+	CFA_P40_PROF_L2_CTXT_TCAM_T_IVID_FLD = 7,
+	CFA_P40_PROF_L2_CTXT_TCAM_SPARIF_FLD = 8,
+	CFA_P40_PROF_L2_CTXT_TCAM_SVIF_FLD = 9,
+	CFA_P40_PROF_L2_CTXT_TCAM_MAC0_FLD = 10,
+	CFA_P40_PROF_L2_CTXT_TCAM_OVID_FLD = 11,
+	CFA_P40_PROF_L2_CTXT_TCAM_IVID_FLD = 12,
+	CFA_P40_PROF_L2_CTXT_TCAM_MAX_FLD
+};
+
+#define CFA_P40_PROF_L2_CTXT_TCAM_TOTAL_NUM_BITS 167
+
+/**
+ * Valid entry. (for idx 2 ...)
+ */
+#define CFA_P40_ACT_VEB_TCAM_VALID_BITPOS        79
+#define CFA_P40_ACT_VEB_TCAM_VALID_NUM_BITS      1
+/**
+ * reserved program to 0. (for idx 2 ...)
+ */
+#define CFA_P40_ACT_VEB_TCAM_RESERVED_BITPOS     78
+#define CFA_P40_ACT_VEB_TCAM_RESERVED_NUM_BITS   1
+/**
+ * PF Parif Number. (for idx 2 ...)
+ */
+#define CFA_P40_ACT_VEB_TCAM_PARIF_IN_BITPOS     74
+#define CFA_P40_ACT_VEB_TCAM_PARIF_IN_NUM_BITS   4
+/**
+ * Number of VLAN Tags. (for idx 2 ...)
+ */
+#define CFA_P40_ACT_VEB_TCAM_NUM_VTAGS_BITPOS    72
+#define CFA_P40_ACT_VEB_TCAM_NUM_VTAGS_NUM_BITS  2
+/**
+ * Dest. MAC Address.
+ */
+#define CFA_P40_ACT_VEB_TCAM_MAC_BITPOS          24
+#define CFA_P40_ACT_VEB_TCAM_MAC_NUM_BITS        48
+/**
+ * Outer VLAN Tag ID.
+ */
+#define CFA_P40_ACT_VEB_TCAM_OVID_BITPOS         12
+#define CFA_P40_ACT_VEB_TCAM_OVID_NUM_BITS       12
+/**
+ * Inner VLAN Tag ID.
+ */
+#define CFA_P40_ACT_VEB_TCAM_IVID_BITPOS         0
+#define CFA_P40_ACT_VEB_TCAM_IVID_NUM_BITS       12
+
+enum cfa_p40_act_veb_tcam_flds {
+	CFA_P40_ACT_VEB_TCAM_VALID_FLD = 0,
+	CFA_P40_ACT_VEB_TCAM_RESERVED_FLD = 1,
+	CFA_P40_ACT_VEB_TCAM_PARIF_IN_FLD = 2,
+	CFA_P40_ACT_VEB_TCAM_NUM_VTAGS_FLD = 3,
+	CFA_P40_ACT_VEB_TCAM_MAC_FLD = 4,
+	CFA_P40_ACT_VEB_TCAM_OVID_FLD = 5,
+	CFA_P40_ACT_VEB_TCAM_IVID_FLD = 6,
+	CFA_P40_ACT_VEB_TCAM_MAX_FLD
+};
+
+#define CFA_P40_ACT_VEB_TCAM_TOTAL_NUM_BITS 80
+
+/**
+ * Entry is valid.
+ */
+#define CFA_P40_LKUP_TCAM_RECORD_MEM_VALID_BITPOS 18
+#define CFA_P40_LKUP_TCAM_RECORD_MEM_VALID_NUM_BITS 1
+/**
+ * Action Record Pointer
+ */
+#define CFA_P40_LKUP_TCAM_RECORD_MEM_ACT_REC_PTR_BITPOS 2
+#define CFA_P40_LKUP_TCAM_RECORD_MEM_ACT_REC_PTR_NUM_BITS 16
+/**
+ * for resolving TCAM/EM conflicts
+ */
+#define CFA_P40_LKUP_TCAM_RECORD_MEM_STRENGTH_BITPOS 0
+#define CFA_P40_LKUP_TCAM_RECORD_MEM_STRENGTH_NUM_BITS 2
+
+enum cfa_p40_lkup_tcam_record_mem_flds {
+	CFA_P40_LKUP_TCAM_RECORD_MEM_VALID_FLD = 0,
+	CFA_P40_LKUP_TCAM_RECORD_MEM_ACT_REC_PTR_FLD = 1,
+	CFA_P40_LKUP_TCAM_RECORD_MEM_STRENGTH_FLD = 2,
+	CFA_P40_LKUP_TCAM_RECORD_MEM_MAX_FLD
+};
+
+#define CFA_P40_LKUP_TCAM_RECORD_MEM_TOTAL_NUM_BITS 19
+
+/**
+ * (for idx 1 ...)
+ */
+#define CFA_P40_PROF_CTXT_REMAP_MEM_TPID_ANTI_SPOOF_CTL_BITPOS 62
+#define CFA_P40_PROF_CTXT_REMAP_MEM_TPID_ANTI_SPOOF_CTL_NUM_BITS 2
+enum cfa_p40_prof_ctxt_remap_mem_tpid_anti_spoof_ctl {
+	CFA_P40_PROF_CTXT_REMAP_MEM_TPID_IGNORE = 0x0UL,
+
+	CFA_P40_PROF_CTXT_REMAP_MEM_TPID_DROP = 0x1UL,
+
+	CFA_P40_PROF_CTXT_REMAP_MEM_TPID_DEFAULT = 0x2UL,
+
+	CFA_P40_PROF_CTXT_REMAP_MEM_TPID_SPIF = 0x3UL,
+	CFA_P40_PROF_CTXT_REMAP_MEM_TPID_MAX = 0x3UL
+};
+/**
+ * (for idx 1 ...)
+ */
+#define CFA_P40_PROF_CTXT_REMAP_MEM_PRI_ANTI_SPOOF_CTL_BITPOS 60
+#define CFA_P40_PROF_CTXT_REMAP_MEM_PRI_ANTI_SPOOF_CTL_NUM_BITS 2
+enum cfa_p40_prof_ctxt_remap_mem_pri_anti_spoof_ctl {
+	CFA_P40_PROF_CTXT_REMAP_MEM_PRI_IGNORE = 0x0UL,
+
+	CFA_P40_PROF_CTXT_REMAP_MEM_PRI_DROP = 0x1UL,
+
+	CFA_P40_PROF_CTXT_REMAP_MEM_PRI_DEFAULT = 0x2UL,
+
+	CFA_P40_PROF_CTXT_REMAP_MEM_PRI_SPIF = 0x3UL,
+	CFA_P40_PROF_CTXT_REMAP_MEM_PRI_MAX = 0x3UL
+};
+/**
+ * Bypass Source Properties Lookup. (for idx 1 ...)
+ */
+#define CFA_P40_PROF_CTXT_REMAP_MEM_BYP_SP_LKUP_BITPOS 59
+#define CFA_P40_PROF_CTXT_REMAP_MEM_BYP_SP_LKUP_NUM_BITS 1
+/**
+ * SP Record Pointer. (for idx 1 ...)
+ */
+#define CFA_P40_PROF_CTXT_REMAP_MEM_SP_REC_PTR_BITPOS 43
+#define CFA_P40_PROF_CTXT_REMAP_MEM_SP_REC_PTR_NUM_BITS 16
+/**
+ * BD Action pointer passing enable. (for idx 1 ...)
+ */
+#define CFA_P40_PROF_CTXT_REMAP_MEM_BD_ACT_EN_BITPOS 42
+#define CFA_P40_PROF_CTXT_REMAP_MEM_BD_ACT_EN_NUM_BITS 1
+/**
+ * Default VLAN TPID. (for idx 1 ...)
+ */
+#define CFA_P40_PROF_CTXT_REMAP_MEM_DEFAULT_TPID_BITPOS 39
+#define CFA_P40_PROF_CTXT_REMAP_MEM_DEFAULT_TPID_NUM_BITS 3
+/**
+ * Allowed VLAN TPIDs. (for idx 1 ...)
+ */
+#define CFA_P40_PROF_CTXT_REMAP_MEM_ALLOWED_TPID_BITPOS 33
+#define CFA_P40_PROF_CTXT_REMAP_MEM_ALLOWED_TPID_NUM_BITS 6
+/**
+ * Default VLAN PRI.
+ */
+#define CFA_P40_PROF_CTXT_REMAP_MEM_DEFAULT_PRI_BITPOS 30
+#define CFA_P40_PROF_CTXT_REMAP_MEM_DEFAULT_PRI_NUM_BITS 3
+/**
+ * Allowed VLAN PRIs.
+ */
+#define CFA_P40_PROF_CTXT_REMAP_MEM_ALLOWED_PRI_BITPOS 22
+#define CFA_P40_PROF_CTXT_REMAP_MEM_ALLOWED_PRI_NUM_BITS 8
+/**
+ * Partition.
+ */
+#define CFA_P40_PROF_CTXT_REMAP_MEM_PARIF_BITPOS 18
+#define CFA_P40_PROF_CTXT_REMAP_MEM_PARIF_NUM_BITS 4
+/**
+ * Bypass Lookup.
+ */
+#define CFA_P40_PROF_CTXT_REMAP_MEM_BYP_LKUP_EN_BITPOS 17
+#define CFA_P40_PROF_CTXT_REMAP_MEM_BYP_LKUP_EN_NUM_BITS 1
+
+/**
+ * L2 Context Remap Data. Action bypass mode (1) {7'd0,prof_vnic[9:0]} Note:
+ * should also set byp_lkup_en. Action bypass mode (0) byp_lkup_en(0) -
+ * {prof_func[6:0],l2_context[9:0]} byp_lkup_en(1) - {1'b0,act_rec_ptr[15:0]}
+ */
+
+#define CFA_P40_PROF_CTXT_REMAP_MEM_PROF_VNIC_BITPOS 0
+#define CFA_P40_PROF_CTXT_REMAP_MEM_PROF_VNIC_NUM_BITS 12
+
+#define CFA_P40_PROF_CTXT_REMAP_MEM_PROF_FUNC_BITPOS 10
+#define CFA_P40_PROF_CTXT_REMAP_MEM_PROF_FUNC_NUM_BITS 7
+
+#define CFA_P40_PROF_CTXT_REMAP_MEM_L2_CTXT_BITPOS 0
+#define CFA_P40_PROF_CTXT_REMAP_MEM_L2_CTXT_NUM_BITS 10
+
+#define CFA_P40_PROF_CTXT_REMAP_MEM_ARP_BITPOS 0
+#define CFA_P40_PROF_CTXT_REMAP_MEM_ARP_NUM_BITS 16
+
+enum cfa_p40_prof_ctxt_remap_mem_flds {
+	CFA_P40_PROF_CTXT_REMAP_MEM_TPID_ANTI_SPOOF_CTL_FLD = 0,
+	CFA_P40_PROF_CTXT_REMAP_MEM_PRI_ANTI_SPOOF_CTL_FLD = 1,
+	CFA_P40_PROF_CTXT_REMAP_MEM_BYP_SP_LKUP_FLD = 2,
+	CFA_P40_PROF_CTXT_REMAP_MEM_SP_REC_PTR_FLD = 3,
+	CFA_P40_PROF_CTXT_REMAP_MEM_BD_ACT_EN_FLD = 4,
+	CFA_P40_PROF_CTXT_REMAP_MEM_DEFAULT_TPID_FLD = 5,
+	CFA_P40_PROF_CTXT_REMAP_MEM_ALLOWED_TPID_FLD = 6,
+	CFA_P40_PROF_CTXT_REMAP_MEM_DEFAULT_PRI_FLD = 7,
+	CFA_P40_PROF_CTXT_REMAP_MEM_ALLOWED_PRI_FLD = 8,
+	CFA_P40_PROF_CTXT_REMAP_MEM_PARIF_FLD = 9,
+	CFA_P40_PROF_CTXT_REMAP_MEM_BYP_LKUP_EN_FLD = 10,
+	CFA_P40_PROF_CTXT_REMAP_MEM_PROF_VNIC_FLD = 11,
+	CFA_P40_PROF_CTXT_REMAP_MEM_PROF_FUNC_FLD = 12,
+	CFA_P40_PROF_CTXT_REMAP_MEM_L2_CTXT_FLD = 13,
+	CFA_P40_PROF_CTXT_REMAP_MEM_ARP_FLD = 14,
+	CFA_P40_PROF_CTXT_REMAP_MEM_MAX_FLD
+};
+
+#define CFA_P40_PROF_CTXT_REMAP_MEM_TOTAL_NUM_BITS 64
+
+/**
+ * Bypass action pointer look up (for idx 1 ...)
+ */
+#define CFA_P40_PROF_PROFILE_TCAM_REMAP_MEM_PL_BYP_LKUP_EN_BITPOS 37
+#define CFA_P40_PROF_PROFILE_TCAM_REMAP_MEM_PL_BYP_LKUP_EN_NUM_BITS 1
+/**
+ * Exact match search enable (for idx 1 ...)
+ */
+#define CFA_P40_PROF_PROFILE_TCAM_REMAP_MEM_EM_SEARCH_ENB_BITPOS 36
+#define CFA_P40_PROF_PROFILE_TCAM_REMAP_MEM_EM_SEARCH_ENB_NUM_BITS 1
+/**
+ * Exact match profile
+ */
+#define CFA_P40_PROF_PROFILE_TCAM_REMAP_MEM_EM_PROFILE_ID_BITPOS 28
+#define CFA_P40_PROF_PROFILE_TCAM_REMAP_MEM_EM_PROFILE_ID_NUM_BITS 8
+/**
+ * Exact match key format
+ */
+#define CFA_P40_PROF_PROFILE_TCAM_REMAP_MEM_EM_KEY_ID_BITPOS 23
+#define CFA_P40_PROF_PROFILE_TCAM_REMAP_MEM_EM_KEY_ID_NUM_BITS 5
+/**
+ * Exact match key mask
+ */
+#define CFA_P40_PROF_PROFILE_TCAM_REMAP_MEM_EM_KEY_MASK_BITPOS 13
+#define CFA_P40_PROF_PROFILE_TCAM_REMAP_MEM_EM_KEY_MASK_NUM_BITS 10
+/**
+ * TCAM search enable
+ */
+#define CFA_P40_PROF_PROFILE_TCAM_REMAP_MEM_TCAM_SEARCH_ENB_BITPOS 12
+#define CFA_P40_PROF_PROFILE_TCAM_REMAP_MEM_TCAM_SEARCH_ENB_NUM_BITS 1
+/**
+ * TCAM profile
+ */
+#define CFA_P40_PROF_PROFILE_TCAM_REMAP_MEM_TCAM_PROFILE_ID_BITPOS 4
+#define CFA_P40_PROF_PROFILE_TCAM_REMAP_MEM_TCAM_PROFILE_ID_NUM_BITS 8
+/**
+ * TCAM key format
+ */
+#define CFA_P40_PROF_PROFILE_TCAM_REMAP_MEM_TCAM_KEY_ID_BITPOS 0
+#define CFA_P40_PROF_PROFILE_TCAM_REMAP_MEM_TCAM_KEY_ID_NUM_BITS 4
+
+enum cfa_p40_prof_profile_tcam_remap_mem_flds {
+	CFA_P40_PROF_PROFILE_TCAM_REMAP_MEM_PL_BYP_LKUP_EN_FLD = 0,
+	CFA_P40_PROF_PROFILE_TCAM_REMAP_MEM_EM_SEARCH_ENB_FLD = 1,
+	CFA_P40_PROF_PROFILE_TCAM_REMAP_MEM_EM_PROFILE_ID_FLD = 2,
+	CFA_P40_PROF_PROFILE_TCAM_REMAP_MEM_EM_KEY_ID_FLD = 3,
+	CFA_P40_PROF_PROFILE_TCAM_REMAP_MEM_EM_KEY_MASK_FLD = 4,
+	CFA_P40_PROF_PROFILE_TCAM_REMAP_MEM_TCAM_SEARCH_ENB_FLD = 5,
+	CFA_P40_PROF_PROFILE_TCAM_REMAP_MEM_TCAM_PROFILE_ID_FLD = 6,
+	CFA_P40_PROF_PROFILE_TCAM_REMAP_MEM_TCAM_KEY_ID_FLD = 7,
+	CFA_P40_PROF_PROFILE_TCAM_REMAP_MEM_MAX_FLD
+};
+
+#define CFA_P40_PROF_PROFILE_TCAM_REMAP_MEM_TOTAL_NUM_BITS 38
+
+/**
+ * Valid TCAM entry (for idx 2 ...)
+ */
+#define CFA_P40_PROF_PROFILE_TCAM_VALID_BITPOS   80
+#define CFA_P40_PROF_PROFILE_TCAM_VALID_NUM_BITS 1
+/**
+ * Packet type (for idx 2 ...)
+ */
+#define CFA_P40_PROF_PROFILE_TCAM_PKT_TYPE_BITPOS 76
+#define CFA_P40_PROF_PROFILE_TCAM_PKT_TYPE_NUM_BITS 4
+/**
+ * Pass through CFA (for idx 2 ...)
+ */
+#define CFA_P40_PROF_PROFILE_TCAM_RECYCLE_CNT_BITPOS 74
+#define CFA_P40_PROF_PROFILE_TCAM_RECYCLE_CNT_NUM_BITS 2
+/**
+ * Aggregate error (for idx 2 ...)
+ */
+#define CFA_P40_PROF_PROFILE_TCAM_AGG_ERROR_BITPOS 73
+#define CFA_P40_PROF_PROFILE_TCAM_AGG_ERROR_NUM_BITS 1
+/**
+ * Profile function (for idx 2 ...)
+ */
+#define CFA_P40_PROF_PROFILE_TCAM_PROF_FUNC_BITPOS 66
+#define CFA_P40_PROF_PROFILE_TCAM_PROF_FUNC_NUM_BITS 7
+/**
+ * Reserved for future use. Set to 0.
+ */
+#define CFA_P40_PROF_PROFILE_TCAM_RESERVED_BITPOS 57
+#define CFA_P40_PROF_PROFILE_TCAM_RESERVED_NUM_BITS 9
+/**
+ * non-tunnel(0)/tunneled(1) packet (for idx 1 ...)
+ */
+#define CFA_P40_PROF_PROFILE_TCAM_HREC_NEXT_BITPOS 56
+#define CFA_P40_PROF_PROFILE_TCAM_HREC_NEXT_NUM_BITS 1
+/**
+ * Tunnel L2 tunnel valid (for idx 1 ...)
+ */
+#define CFA_P40_PROF_PROFILE_TCAM_TL2_HDR_VALID_BITPOS 55
+#define CFA_P40_PROF_PROFILE_TCAM_TL2_HDR_VALID_NUM_BITS 1
+/**
+ * Tunnel L2 header type (for idx 1 ...)
+ */
+#define CFA_P40_PROF_PROFILE_TCAM_TL2_HDR_TYPE_BITPOS 53
+#define CFA_P40_PROF_PROFILE_TCAM_TL2_HDR_TYPE_NUM_BITS 2
+/**
+ * Remapped tunnel L2 dest_type UC(0)/MC(2)/BC(3) (for idx 1 ...)
+ */
+#define CFA_P40_PROF_PROFILE_TCAM_TL2_UC_MC_BC_BITPOS 51
+#define CFA_P40_PROF_PROFILE_TCAM_TL2_UC_MC_BC_NUM_BITS 2
+/**
+ * Tunnel L2 1+ VLAN tags present (for idx 1 ...)
+ */
+#define CFA_P40_PROF_PROFILE_TCAM_TL2_VTAG_PRESENT_BITPOS 50
+#define CFA_P40_PROF_PROFILE_TCAM_TL2_VTAG_PRESENT_NUM_BITS 1
+/**
+ * Tunnel L2 2 VLAN tags present (for idx 1 ...)
+ */
+#define CFA_P40_PROF_PROFILE_TCAM_TL2_TWO_VTAGS_BITPOS 49
+#define CFA_P40_PROF_PROFILE_TCAM_TL2_TWO_VTAGS_NUM_BITS 1
+/**
+ * Tunnel L3 valid (for idx 1 ...)
+ */
+#define CFA_P40_PROF_PROFILE_TCAM_TL3_VALID_BITPOS 48
+#define CFA_P40_PROF_PROFILE_TCAM_TL3_VALID_NUM_BITS 1
+/**
+ * Tunnel L3 error (for idx 1 ...)
+ */
+#define CFA_P40_PROF_PROFILE_TCAM_TL3_ERROR_BITPOS 47
+#define CFA_P40_PROF_PROFILE_TCAM_TL3_ERROR_NUM_BITS 1
+/**
+ * Tunnel L3 header type (for idx 1 ...)
+ */
+#define CFA_P40_PROF_PROFILE_TCAM_TL3_HDR_TYPE_BITPOS 43
+#define CFA_P40_PROF_PROFILE_TCAM_TL3_HDR_TYPE_NUM_BITS 4
+/**
+ * Tunnel L3 header is IPV4 or IPV6. (for idx 1 ...)
+ */
+#define CFA_P40_PROF_PROFILE_TCAM_TL3_HDR_ISIP_BITPOS 42
+#define CFA_P40_PROF_PROFILE_TCAM_TL3_HDR_ISIP_NUM_BITS 1
+/**
+ * Tunnel L3 IPV6 src address is compressed (for idx 1 ...)
+ */
+#define CFA_P40_PROF_PROFILE_TCAM_TL3_IPV6_CMP_SRC_BITPOS 41
+#define CFA_P40_PROF_PROFILE_TCAM_TL3_IPV6_CMP_SRC_NUM_BITS 1
+/**
+ * Tunnel L3 IPV6 dest address is compressed (for idx 1 ...)
+ */
+#define CFA_P40_PROF_PROFILE_TCAM_TL3_IPV6_CMP_DEST_BITPOS 40
+#define CFA_P40_PROF_PROFILE_TCAM_TL3_IPV6_CMP_DEST_NUM_BITS 1
+/**
+ * Tunnel L4 valid (for idx 1 ...)
+ */
+#define CFA_P40_PROF_PROFILE_TCAM_TL4_HDR_VALID_BITPOS 39
+#define CFA_P40_PROF_PROFILE_TCAM_TL4_HDR_VALID_NUM_BITS 1
+/**
+ * Tunnel L4 error (for idx 1 ...)
+ */
+#define CFA_P40_PROF_PROFILE_TCAM_TL4_HDR_ERROR_BITPOS 38
+#define CFA_P40_PROF_PROFILE_TCAM_TL4_HDR_ERROR_NUM_BITS 1
+/**
+ * Tunnel L4 header type (for idx 1 ...)
+ */
+#define CFA_P40_PROF_PROFILE_TCAM_TL4_HDR_TYPE_BITPOS 34
+#define CFA_P40_PROF_PROFILE_TCAM_TL4_HDR_TYPE_NUM_BITS 4
+/**
+ * Tunnel L4 header is UDP or TCP (for idx 1 ...)
+ */
+#define CFA_P40_PROF_PROFILE_TCAM_TL4_HDR_IS_UDP_TCP_BITPOS 33
+#define CFA_P40_PROF_PROFILE_TCAM_TL4_HDR_IS_UDP_TCP_NUM_BITS 1
+/**
+ * Tunnel valid (for idx 1 ...)
+ */
+#define CFA_P40_PROF_PROFILE_TCAM_TUN_HDR_VALID_BITPOS 32
+#define CFA_P40_PROF_PROFILE_TCAM_TUN_HDR_VALID_NUM_BITS 1
+/**
+ * Tunnel error
+ */
+#define CFA_P40_PROF_PROFILE_TCAM_TUN_HDR_ERR_BITPOS 31
+#define CFA_P40_PROF_PROFILE_TCAM_TUN_HDR_ERR_NUM_BITS 1
+/**
+ * Tunnel header type
+ */
+#define CFA_P40_PROF_PROFILE_TCAM_TUN_HDR_TYPE_BITPOS 27
+#define CFA_P40_PROF_PROFILE_TCAM_TUN_HDR_TYPE_NUM_BITS 4
+/**
+ * Tunnel header flags
+ */
+#define CFA_P40_PROF_PROFILE_TCAM_TUN_HDR_FLAGS_BITPOS 24
+#define CFA_P40_PROF_PROFILE_TCAM_TUN_HDR_FLAGS_NUM_BITS 3
+/**
+ * L2 header valid
+ */
+#define CFA_P40_PROF_PROFILE_TCAM_L2_HDR_VALID_BITPOS 23
+#define CFA_P40_PROF_PROFILE_TCAM_L2_HDR_VALID_NUM_BITS 1
+/**
+ * L2 header error
+ */
+#define CFA_P40_PROF_PROFILE_TCAM_L2_HDR_ERROR_BITPOS 22
+#define CFA_P40_PROF_PROFILE_TCAM_L2_HDR_ERROR_NUM_BITS 1
+/**
+ * L2 header type
+ */
+#define CFA_P40_PROF_PROFILE_TCAM_L2_HDR_TYPE_BITPOS 20
+#define CFA_P40_PROF_PROFILE_TCAM_L2_HDR_TYPE_NUM_BITS 2
+/**
+ * Remapped L2 dest_type UC(0)/MC(2)/BC(3)
+ */
+#define CFA_P40_PROF_PROFILE_TCAM_L2_UC_MC_BC_BITPOS 18
+#define CFA_P40_PROF_PROFILE_TCAM_L2_UC_MC_BC_NUM_BITS 2
+/**
+ * L2 header 1+ VLAN tags present
+ */
+#define CFA_P40_PROF_PROFILE_TCAM_L2_VTAG_PRESENT_BITPOS 17
+#define CFA_P40_PROF_PROFILE_TCAM_L2_VTAG_PRESENT_NUM_BITS 1
+/**
+ * L2 header 2 VLAN tags present
+ */
+#define CFA_P40_PROF_PROFILE_TCAM_L2_TWO_VTAGS_BITPOS 16
+#define CFA_P40_PROF_PROFILE_TCAM_L2_TWO_VTAGS_NUM_BITS 1
+/**
+ * L3 header valid
+ */
+#define CFA_P40_PROF_PROFILE_TCAM_L3_VALID_BITPOS 15
+#define CFA_P40_PROF_PROFILE_TCAM_L3_VALID_NUM_BITS 1
+/**
+ * L3 header error
+ */
+#define CFA_P40_PROF_PROFILE_TCAM_L3_ERROR_BITPOS 14
+#define CFA_P40_PROF_PROFILE_TCAM_L3_ERROR_NUM_BITS 1
+/**
+ * L3 header type
+ */
+#define CFA_P40_PROF_PROFILE_TCAM_L3_HDR_TYPE_BITPOS 10
+#define CFA_P40_PROF_PROFILE_TCAM_L3_HDR_TYPE_NUM_BITS 4
+/**
+ * L3 header is IPV4 or IPV6.
+ */
+#define CFA_P40_PROF_PROFILE_TCAM_L3_HDR_ISIP_BITPOS 9
+#define CFA_P40_PROF_PROFILE_TCAM_L3_HDR_ISIP_NUM_BITS 1
+/**
+ * L3 header IPV6 src address is compressed
+ */
+#define CFA_P40_PROF_PROFILE_TCAM_L3_IPV6_CMP_SRC_BITPOS 8
+#define CFA_P40_PROF_PROFILE_TCAM_L3_IPV6_CMP_SRC_NUM_BITS 1
+/**
+ * L3 header IPV6 dest address is compressed
+ */
+#define CFA_P40_PROF_PROFILE_TCAM_L3_IPV6_CMP_DEST_BITPOS 7
+#define CFA_P40_PROF_PROFILE_TCAM_L3_IPV6_CMP_DEST_NUM_BITS 1
+/**
+ * L4 header valid
+ */
+#define CFA_P40_PROF_PROFILE_TCAM_L4_HDR_VALID_BITPOS 6
+#define CFA_P40_PROF_PROFILE_TCAM_L4_HDR_VALID_NUM_BITS 1
+/**
+ * L4 header error
+ */
+#define CFA_P40_PROF_PROFILE_TCAM_L4_HDR_ERROR_BITPOS 5
+#define CFA_P40_PROF_PROFILE_TCAM_L4_HDR_ERROR_NUM_BITS 1
+/**
+ * L4 header type
+ */
+#define CFA_P40_PROF_PROFILE_TCAM_L4_HDR_TYPE_BITPOS 1
+#define CFA_P40_PROF_PROFILE_TCAM_L4_HDR_TYPE_NUM_BITS 4
+/**
+ * L4 header is UDP or TCP
+ */
+#define CFA_P40_PROF_PROFILE_TCAM_L4_HDR_IS_UDP_TCP_BITPOS 0
+#define CFA_P40_PROF_PROFILE_TCAM_L4_HDR_IS_UDP_TCP_NUM_BITS 1
+
+enum cfa_p40_prof_profile_tcam_flds {
+	CFA_P40_PROF_PROFILE_TCAM_VALID_FLD = 0,
+	CFA_P40_PROF_PROFILE_TCAM_PKT_TYPE_FLD = 1,
+	CFA_P40_PROF_PROFILE_TCAM_RECYCLE_CNT_FLD = 2,
+	CFA_P40_PROF_PROFILE_TCAM_AGG_ERROR_FLD = 3,
+	CFA_P40_PROF_PROFILE_TCAM_PROF_FUNC_FLD = 4,
+	CFA_P40_PROF_PROFILE_TCAM_RESERVED_FLD = 5,
+	CFA_P40_PROF_PROFILE_TCAM_HREC_NEXT_FLD = 6,
+	CFA_P40_PROF_PROFILE_TCAM_TL2_HDR_VALID_FLD = 7,
+	CFA_P40_PROF_PROFILE_TCAM_TL2_HDR_TYPE_FLD = 8,
+	CFA_P40_PROF_PROFILE_TCAM_TL2_UC_MC_BC_FLD = 9,
+	CFA_P40_PROF_PROFILE_TCAM_TL2_VTAG_PRESENT_FLD = 10,
+	CFA_P40_PROF_PROFILE_TCAM_TL2_TWO_VTAGS_FLD = 11,
+	CFA_P40_PROF_PROFILE_TCAM_TL3_VALID_FLD = 12,
+	CFA_P40_PROF_PROFILE_TCAM_TL3_ERROR_FLD = 13,
+	CFA_P40_PROF_PROFILE_TCAM_TL3_HDR_TYPE_FLD = 14,
+	CFA_P40_PROF_PROFILE_TCAM_TL3_HDR_ISIP_FLD = 15,
+	CFA_P40_PROF_PROFILE_TCAM_TL3_IPV6_CMP_SRC_FLD = 16,
+	CFA_P40_PROF_PROFILE_TCAM_TL3_IPV6_CMP_DEST_FLD = 17,
+	CFA_P40_PROF_PROFILE_TCAM_TL4_HDR_VALID_FLD = 18,
+	CFA_P40_PROF_PROFILE_TCAM_TL4_HDR_ERROR_FLD = 19,
+	CFA_P40_PROF_PROFILE_TCAM_TL4_HDR_TYPE_FLD = 20,
+	CFA_P40_PROF_PROFILE_TCAM_TL4_HDR_IS_UDP_TCP_FLD = 21,
+	CFA_P40_PROF_PROFILE_TCAM_TUN_HDR_VALID_FLD = 22,
+	CFA_P40_PROF_PROFILE_TCAM_TUN_HDR_ERR_FLD = 23,
+	CFA_P40_PROF_PROFILE_TCAM_TUN_HDR_TYPE_FLD = 24,
+	CFA_P40_PROF_PROFILE_TCAM_TUN_HDR_FLAGS_FLD = 25,
+	CFA_P40_PROF_PROFILE_TCAM_L2_HDR_VALID_FLD = 26,
+	CFA_P40_PROF_PROFILE_TCAM_L2_HDR_ERROR_FLD = 27,
+	CFA_P40_PROF_PROFILE_TCAM_L2_HDR_TYPE_FLD = 28,
+	CFA_P40_PROF_PROFILE_TCAM_L2_UC_MC_BC_FLD = 29,
+	CFA_P40_PROF_PROFILE_TCAM_L2_VTAG_PRESENT_FLD = 30,
+	CFA_P40_PROF_PROFILE_TCAM_L2_TWO_VTAGS_FLD = 31,
+	CFA_P40_PROF_PROFILE_TCAM_L3_VALID_FLD = 32,
+	CFA_P40_PROF_PROFILE_TCAM_L3_ERROR_FLD = 33,
+	CFA_P40_PROF_PROFILE_TCAM_L3_HDR_TYPE_FLD = 34,
+	CFA_P40_PROF_PROFILE_TCAM_L3_HDR_ISIP_FLD = 35,
+	CFA_P40_PROF_PROFILE_TCAM_L3_IPV6_CMP_SRC_FLD = 36,
+	CFA_P40_PROF_PROFILE_TCAM_L3_IPV6_CMP_DEST_FLD = 37,
+	CFA_P40_PROF_PROFILE_TCAM_L4_HDR_VALID_FLD = 38,
+	CFA_P40_PROF_PROFILE_TCAM_L4_HDR_ERROR_FLD = 39,
+	CFA_P40_PROF_PROFILE_TCAM_L4_HDR_TYPE_FLD = 40,
+	CFA_P40_PROF_PROFILE_TCAM_L4_HDR_IS_UDP_TCP_FLD = 41,
+	CFA_P40_PROF_PROFILE_TCAM_MAX_FLD
+};
+
+#define CFA_P40_PROF_PROFILE_TCAM_TOTAL_NUM_BITS 81
+
+/**
+ * CFA flexible key layout definition
+ */
+enum cfa_p40_key_fld_id {
+	CFA_P40_KEY_FLD_ID_MAX
+};
+
+/**************************************************************************/
+/**
+ * Non-autogenerated fields
+ */
+
+/**
+ * Valid
+ */
+#define CFA_P40_EEM_KEY_TBL_VALID_BITPOS 0
+#define CFA_P40_EEM_KEY_TBL_VALID_NUM_BITS 1
+
+/**
+ * L1 Cacheable
+ */
+#define CFA_P40_EEM_KEY_TBL_L1_CACHEABLE_BITPOS 1
+#define CFA_P40_EEM_KEY_TBL_L1_CACHEABLE_NUM_BITS 1
+
+/**
+ * Strength
+ */
+#define CFA_P40_EEM_KEY_TBL_STRENGTH_BITPOS 2
+#define CFA_P40_EEM_KEY_TBL_STRENGTH_NUM_BITS 2
+
+/**
+ * Key Size
+ */
+#define CFA_P40_EEM_KEY_TBL_KEY_SZ_BITPOS 15
+#define CFA_P40_EEM_KEY_TBL_KEY_SZ_NUM_BITS 9
+
+/**
+ * Record Size
+ */
+#define CFA_P40_EEM_KEY_TBL_REC_SZ_BITPOS 24
+#define CFA_P40_EEM_KEY_TBL_REC_SZ_NUM_BITS 5
+
+/**
+ * Action Record Internal
+ */
+#define CFA_P40_EEM_KEY_TBL_ACT_REC_INT_BITPOS 29
+#define CFA_P40_EEM_KEY_TBL_ACT_REC_INT_NUM_BITS 1
+
+/**
+ * External Flow Counter
+ */
+#define CFA_P40_EEM_KEY_TBL_EXT_FLOW_CTR_BITPOS 30
+#define CFA_P40_EEM_KEY_TBL_EXT_FLOW_CTR_NUM_BITS 1
+
+/**
+ * Action Record Pointer
+ */
+#define CFA_P40_EEM_KEY_TBL_AR_PTR_BITPOS 31
+#define CFA_P40_EEM_KEY_TBL_AR_PTR_NUM_BITS 33
+
+/**
+ * EEM Key omitted - create using keybuilder
+ * Fields here cannot be larger than a uint64_t
+ */
+
+#define CFA_P40_EEM_KEY_TBL_TOTAL_NUM_BITS 64
+
+enum cfa_p40_eem_key_tbl_flds {
+	CFA_P40_EEM_KEY_TBL_VALID_FLD = 0,
+	CFA_P40_EEM_KEY_TBL_L1_CACHEABLE_FLD = 1,
+	CFA_P40_EEM_KEY_TBL_STRENGTH_FLD = 2,
+	CFA_P40_EEM_KEY_TBL_KEY_SZ_FLD = 3,
+	CFA_P40_EEM_KEY_TBL_REC_SZ_FLD = 4,
+	CFA_P40_EEM_KEY_TBL_ACT_REC_INT_FLD = 5,
+	CFA_P40_EEM_KEY_TBL_EXT_FLOW_CTR_FLD = 6,
+	CFA_P40_EEM_KEY_TBL_AR_PTR_FLD = 7,
+	CFA_P40_EEM_KEY_TBL_MAX_FLD
+};
+#endif /* _CFA_P40_HW_H_ */
diff --git a/drivers/net/bnxt/hcapi/cfa_p40_tbl.h b/drivers/net/bnxt/hcapi/cfa_p40_tbl.h
new file mode 100644
index 0000000..4238561
--- /dev/null
+++ b/drivers/net/bnxt/hcapi/cfa_p40_tbl.h
@@ -0,0 +1,250 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2019-2020 Broadcom
+ * All rights reserved.
+ */
+/*
+ * Name:  cfa_p40_tbl.h
+ *
+ * Description: header for SWE based on TFLIB2.0
+ *
+ * Date:  12/16/19 17:18:12
+ *
+ * Note:  This file was originally generated by tflib_decode.py.
+ *        Remainder is hand coded due to lack of availability of xml for
+ *        addtional tables at this time (EEM Record and union fields)
+ *
+ **/
+#ifndef _CFA_P40_TBL_H_
+#define _CFA_P40_TBL_H_
+
+#include "cfa_p40_hw.h"
+
+#include "hcapi_cfa_defs.h"
+
+const struct hcapi_cfa_field cfa_p40_prof_l2_ctxt_tcam_layout[] = {
+	{CFA_P40_PROF_L2_CTXT_TCAM_VALID_BITPOS,
+	 CFA_P40_PROF_L2_CTXT_TCAM_VALID_NUM_BITS},
+	{CFA_P40_PROF_L2_CTXT_TCAM_KEY_TYPE_BITPOS,
+	 CFA_P40_PROF_L2_CTXT_TCAM_KEY_TYPE_NUM_BITS},
+	{CFA_P40_PROF_L2_CTXT_TCAM_TUN_HDR_TYPE_BITPOS,
+	 CFA_P40_PROF_L2_CTXT_TCAM_TUN_HDR_TYPE_NUM_BITS},
+	{CFA_P40_PROF_L2_CTXT_TCAM_T_L2_NUMTAGS_BITPOS,
+	 CFA_P40_PROF_L2_CTXT_TCAM_T_L2_NUMTAGS_NUM_BITS},
+	{CFA_P40_PROF_L2_CTXT_TCAM_L2_NUMTAGS_BITPOS,
+	 CFA_P40_PROF_L2_CTXT_TCAM_L2_NUMTAGS_NUM_BITS},
+	{CFA_P40_PROF_L2_CTXT_TCAM_MAC1_BITPOS,
+	 CFA_P40_PROF_L2_CTXT_TCAM_MAC1_NUM_BITS},
+	{CFA_P40_PROF_L2_CTXT_TCAM_T_OVID_BITPOS,
+	 CFA_P40_PROF_L2_CTXT_TCAM_T_OVID_NUM_BITS},
+	{CFA_P40_PROF_L2_CTXT_TCAM_T_IVID_BITPOS,
+	 CFA_P40_PROF_L2_CTXT_TCAM_T_IVID_NUM_BITS},
+	{CFA_P40_PROF_L2_CTXT_TCAM_SPARIF_BITPOS,
+	 CFA_P40_PROF_L2_CTXT_TCAM_SPARIF_NUM_BITS},
+	{CFA_P40_PROF_L2_CTXT_TCAM_SVIF_BITPOS,
+	 CFA_P40_PROF_L2_CTXT_TCAM_SVIF_NUM_BITS},
+	{CFA_P40_PROF_L2_CTXT_TCAM_MAC0_BITPOS,
+	 CFA_P40_PROF_L2_CTXT_TCAM_MAC0_NUM_BITS},
+	{CFA_P40_PROF_L2_CTXT_TCAM_OVID_BITPOS,
+	 CFA_P40_PROF_L2_CTXT_TCAM_OVID_NUM_BITS},
+	{CFA_P40_PROF_L2_CTXT_TCAM_IVID_BITPOS,
+	 CFA_P40_PROF_L2_CTXT_TCAM_IVID_NUM_BITS},
+};
+
+const struct hcapi_cfa_field cfa_p40_act_veb_tcam_layout[] = {
+	{CFA_P40_ACT_VEB_TCAM_VALID_BITPOS,
+	 CFA_P40_ACT_VEB_TCAM_VALID_NUM_BITS},
+	{CFA_P40_ACT_VEB_TCAM_RESERVED_BITPOS,
+	 CFA_P40_ACT_VEB_TCAM_RESERVED_NUM_BITS},
+	{CFA_P40_ACT_VEB_TCAM_PARIF_IN_BITPOS,
+	 CFA_P40_ACT_VEB_TCAM_PARIF_IN_NUM_BITS},
+	{CFA_P40_ACT_VEB_TCAM_NUM_VTAGS_BITPOS,
+	 CFA_P40_ACT_VEB_TCAM_NUM_VTAGS_NUM_BITS},
+	{CFA_P40_ACT_VEB_TCAM_MAC_BITPOS,
+	 CFA_P40_ACT_VEB_TCAM_MAC_NUM_BITS},
+	{CFA_P40_ACT_VEB_TCAM_OVID_BITPOS,
+	 CFA_P40_ACT_VEB_TCAM_OVID_NUM_BITS},
+	{CFA_P40_ACT_VEB_TCAM_IVID_BITPOS,
+	 CFA_P40_ACT_VEB_TCAM_IVID_NUM_BITS},
+};
+
+const struct hcapi_cfa_field cfa_p40_lkup_tcam_record_mem_layout[] = {
+	{CFA_P40_LKUP_TCAM_RECORD_MEM_VALID_BITPOS,
+	 CFA_P40_LKUP_TCAM_RECORD_MEM_VALID_NUM_BITS},
+	{CFA_P40_LKUP_TCAM_RECORD_MEM_ACT_REC_PTR_BITPOS,
+	 CFA_P40_LKUP_TCAM_RECORD_MEM_ACT_REC_PTR_NUM_BITS},
+	{CFA_P40_LKUP_TCAM_RECORD_MEM_STRENGTH_BITPOS,
+	 CFA_P40_LKUP_TCAM_RECORD_MEM_STRENGTH_NUM_BITS},
+};
+
+const struct hcapi_cfa_field cfa_p40_prof_ctxt_remap_mem_layout[] = {
+	{CFA_P40_PROF_CTXT_REMAP_MEM_TPID_ANTI_SPOOF_CTL_BITPOS,
+	 CFA_P40_PROF_CTXT_REMAP_MEM_TPID_ANTI_SPOOF_CTL_NUM_BITS},
+	{CFA_P40_PROF_CTXT_REMAP_MEM_PRI_ANTI_SPOOF_CTL_BITPOS,
+	 CFA_P40_PROF_CTXT_REMAP_MEM_PRI_ANTI_SPOOF_CTL_NUM_BITS},
+	{CFA_P40_PROF_CTXT_REMAP_MEM_BYP_SP_LKUP_BITPOS,
+	 CFA_P40_PROF_CTXT_REMAP_MEM_BYP_SP_LKUP_NUM_BITS},
+	{CFA_P40_PROF_CTXT_REMAP_MEM_SP_REC_PTR_BITPOS,
+	 CFA_P40_PROF_CTXT_REMAP_MEM_SP_REC_PTR_NUM_BITS},
+	{CFA_P40_PROF_CTXT_REMAP_MEM_BD_ACT_EN_BITPOS,
+	 CFA_P40_PROF_CTXT_REMAP_MEM_BD_ACT_EN_NUM_BITS},
+	{CFA_P40_PROF_CTXT_REMAP_MEM_DEFAULT_TPID_BITPOS,
+	 CFA_P40_PROF_CTXT_REMAP_MEM_DEFAULT_TPID_NUM_BITS},
+	{CFA_P40_PROF_CTXT_REMAP_MEM_ALLOWED_TPID_BITPOS,
+	 CFA_P40_PROF_CTXT_REMAP_MEM_ALLOWED_TPID_NUM_BITS},
+	{CFA_P40_PROF_CTXT_REMAP_MEM_DEFAULT_PRI_BITPOS,
+	 CFA_P40_PROF_CTXT_REMAP_MEM_DEFAULT_PRI_NUM_BITS},
+	{CFA_P40_PROF_CTXT_REMAP_MEM_ALLOWED_PRI_BITPOS,
+	 CFA_P40_PROF_CTXT_REMAP_MEM_ALLOWED_PRI_NUM_BITS},
+	{CFA_P40_PROF_CTXT_REMAP_MEM_PARIF_BITPOS,
+	 CFA_P40_PROF_CTXT_REMAP_MEM_PARIF_NUM_BITS},
+	{CFA_P40_PROF_CTXT_REMAP_MEM_BYP_LKUP_EN_BITPOS,
+	 CFA_P40_PROF_CTXT_REMAP_MEM_BYP_LKUP_EN_NUM_BITS},
+	/* Fields below not generated through automation */
+	{CFA_P40_PROF_CTXT_REMAP_MEM_PROF_VNIC_BITPOS,
+	 CFA_P40_PROF_CTXT_REMAP_MEM_PROF_VNIC_NUM_BITS},
+	{CFA_P40_PROF_CTXT_REMAP_MEM_PROF_FUNC_BITPOS,
+	 CFA_P40_PROF_CTXT_REMAP_MEM_PROF_FUNC_NUM_BITS},
+	{CFA_P40_PROF_CTXT_REMAP_MEM_L2_CTXT_BITPOS,
+	 CFA_P40_PROF_CTXT_REMAP_MEM_L2_CTXT_NUM_BITS},
+	{CFA_P40_PROF_CTXT_REMAP_MEM_ARP_BITPOS,
+	 CFA_P40_PROF_CTXT_REMAP_MEM_ARP_NUM_BITS},
+};
+
+const struct hcapi_cfa_field cfa_p40_prof_profile_tcam_remap_mem_layout[] = {
+	{CFA_P40_PROF_PROFILE_TCAM_REMAP_MEM_PL_BYP_LKUP_EN_BITPOS,
+	 CFA_P40_PROF_PROFILE_TCAM_REMAP_MEM_PL_BYP_LKUP_EN_NUM_BITS},
+	{CFA_P40_PROF_PROFILE_TCAM_REMAP_MEM_EM_SEARCH_ENB_BITPOS,
+	 CFA_P40_PROF_PROFILE_TCAM_REMAP_MEM_EM_SEARCH_ENB_NUM_BITS},
+	{CFA_P40_PROF_PROFILE_TCAM_REMAP_MEM_EM_PROFILE_ID_BITPOS,
+	 CFA_P40_PROF_PROFILE_TCAM_REMAP_MEM_EM_PROFILE_ID_NUM_BITS},
+	{CFA_P40_PROF_PROFILE_TCAM_REMAP_MEM_EM_KEY_ID_BITPOS,
+	 CFA_P40_PROF_PROFILE_TCAM_REMAP_MEM_EM_KEY_ID_NUM_BITS},
+	{CFA_P40_PROF_PROFILE_TCAM_REMAP_MEM_EM_KEY_MASK_BITPOS,
+	 CFA_P40_PROF_PROFILE_TCAM_REMAP_MEM_EM_KEY_MASK_NUM_BITS},
+	{CFA_P40_PROF_PROFILE_TCAM_REMAP_MEM_TCAM_SEARCH_ENB_BITPOS,
+	 CFA_P40_PROF_PROFILE_TCAM_REMAP_MEM_TCAM_SEARCH_ENB_NUM_BITS},
+	{CFA_P40_PROF_PROFILE_TCAM_REMAP_MEM_TCAM_PROFILE_ID_BITPOS,
+	 CFA_P40_PROF_PROFILE_TCAM_REMAP_MEM_TCAM_PROFILE_ID_NUM_BITS},
+	{CFA_P40_PROF_PROFILE_TCAM_REMAP_MEM_TCAM_KEY_ID_BITPOS,
+	 CFA_P40_PROF_PROFILE_TCAM_REMAP_MEM_TCAM_KEY_ID_NUM_BITS},
+};
+
+const struct hcapi_cfa_field cfa_p40_prof_profile_tcam_layout[] = {
+	{CFA_P40_PROF_PROFILE_TCAM_VALID_BITPOS,
+	 CFA_P40_PROF_PROFILE_TCAM_VALID_NUM_BITS},
+	{CFA_P40_PROF_PROFILE_TCAM_PKT_TYPE_BITPOS,
+	 CFA_P40_PROF_PROFILE_TCAM_PKT_TYPE_NUM_BITS},
+	{CFA_P40_PROF_PROFILE_TCAM_RECYCLE_CNT_BITPOS,
+	 CFA_P40_PROF_PROFILE_TCAM_RECYCLE_CNT_NUM_BITS},
+	{CFA_P40_PROF_PROFILE_TCAM_AGG_ERROR_BITPOS,
+	 CFA_P40_PROF_PROFILE_TCAM_AGG_ERROR_NUM_BITS},
+	{CFA_P40_PROF_PROFILE_TCAM_PROF_FUNC_BITPOS,
+	 CFA_P40_PROF_PROFILE_TCAM_PROF_FUNC_NUM_BITS},
+	{CFA_P40_PROF_PROFILE_TCAM_RESERVED_BITPOS,
+	 CFA_P40_PROF_PROFILE_TCAM_RESERVED_NUM_BITS},
+	{CFA_P40_PROF_PROFILE_TCAM_HREC_NEXT_BITPOS,
+	 CFA_P40_PROF_PROFILE_TCAM_HREC_NEXT_NUM_BITS},
+	{CFA_P40_PROF_PROFILE_TCAM_TL2_HDR_VALID_BITPOS,
+	 CFA_P40_PROF_PROFILE_TCAM_TL2_HDR_VALID_NUM_BITS},
+	{CFA_P40_PROF_PROFILE_TCAM_TL2_HDR_TYPE_BITPOS,
+	 CFA_P40_PROF_PROFILE_TCAM_TL2_HDR_TYPE_NUM_BITS},
+	{CFA_P40_PROF_PROFILE_TCAM_TL2_UC_MC_BC_BITPOS,
+	 CFA_P40_PROF_PROFILE_TCAM_TL2_UC_MC_BC_NUM_BITS},
+	{CFA_P40_PROF_PROFILE_TCAM_TL2_VTAG_PRESENT_BITPOS,
+	 CFA_P40_PROF_PROFILE_TCAM_TL2_VTAG_PRESENT_NUM_BITS},
+	{CFA_P40_PROF_PROFILE_TCAM_TL2_TWO_VTAGS_BITPOS,
+	 CFA_P40_PROF_PROFILE_TCAM_TL2_TWO_VTAGS_NUM_BITS},
+	{CFA_P40_PROF_PROFILE_TCAM_TL3_VALID_BITPOS,
+	 CFA_P40_PROF_PROFILE_TCAM_TL3_VALID_NUM_BITS},
+	{CFA_P40_PROF_PROFILE_TCAM_TL3_ERROR_BITPOS,
+	 CFA_P40_PROF_PROFILE_TCAM_TL3_ERROR_NUM_BITS},
+	{CFA_P40_PROF_PROFILE_TCAM_TL3_HDR_TYPE_BITPOS,
+	 CFA_P40_PROF_PROFILE_TCAM_TL3_HDR_TYPE_NUM_BITS},
+	{CFA_P40_PROF_PROFILE_TCAM_TL3_HDR_ISIP_BITPOS,
+	 CFA_P40_PROF_PROFILE_TCAM_TL3_HDR_ISIP_NUM_BITS},
+	{CFA_P40_PROF_PROFILE_TCAM_TL3_IPV6_CMP_SRC_BITPOS,
+	 CFA_P40_PROF_PROFILE_TCAM_TL3_IPV6_CMP_SRC_NUM_BITS},
+	{CFA_P40_PROF_PROFILE_TCAM_TL3_IPV6_CMP_DEST_BITPOS,
+	 CFA_P40_PROF_PROFILE_TCAM_TL3_IPV6_CMP_DEST_NUM_BITS},
+	{CFA_P40_PROF_PROFILE_TCAM_TL4_HDR_VALID_BITPOS,
+	 CFA_P40_PROF_PROFILE_TCAM_TL4_HDR_VALID_NUM_BITS},
+	{CFA_P40_PROF_PROFILE_TCAM_TL4_HDR_ERROR_BITPOS,
+	 CFA_P40_PROF_PROFILE_TCAM_TL4_HDR_ERROR_NUM_BITS},
+	{CFA_P40_PROF_PROFILE_TCAM_TL4_HDR_TYPE_BITPOS,
+	 CFA_P40_PROF_PROFILE_TCAM_TL4_HDR_TYPE_NUM_BITS},
+	{CFA_P40_PROF_PROFILE_TCAM_TL4_HDR_IS_UDP_TCP_BITPOS,
+	 CFA_P40_PROF_PROFILE_TCAM_TL4_HDR_IS_UDP_TCP_NUM_BITS},
+	{CFA_P40_PROF_PROFILE_TCAM_TUN_HDR_VALID_BITPOS,
+	 CFA_P40_PROF_PROFILE_TCAM_TUN_HDR_VALID_NUM_BITS},
+	{CFA_P40_PROF_PROFILE_TCAM_TUN_HDR_ERR_BITPOS,
+	 CFA_P40_PROF_PROFILE_TCAM_TUN_HDR_ERR_NUM_BITS},
+	{CFA_P40_PROF_PROFILE_TCAM_TUN_HDR_TYPE_BITPOS,
+	 CFA_P40_PROF_PROFILE_TCAM_TUN_HDR_TYPE_NUM_BITS},
+	{CFA_P40_PROF_PROFILE_TCAM_TUN_HDR_FLAGS_BITPOS,
+	 CFA_P40_PROF_PROFILE_TCAM_TUN_HDR_FLAGS_NUM_BITS},
+	{CFA_P40_PROF_PROFILE_TCAM_L2_HDR_VALID_BITPOS,
+	 CFA_P40_PROF_PROFILE_TCAM_L2_HDR_VALID_NUM_BITS},
+	{CFA_P40_PROF_PROFILE_TCAM_L2_HDR_ERROR_BITPOS,
+	 CFA_P40_PROF_PROFILE_TCAM_L2_HDR_ERROR_NUM_BITS},
+	{CFA_P40_PROF_PROFILE_TCAM_L2_HDR_TYPE_BITPOS,
+	 CFA_P40_PROF_PROFILE_TCAM_L2_HDR_TYPE_NUM_BITS},
+	{CFA_P40_PROF_PROFILE_TCAM_L2_UC_MC_BC_BITPOS,
+	 CFA_P40_PROF_PROFILE_TCAM_L2_UC_MC_BC_NUM_BITS},
+	{CFA_P40_PROF_PROFILE_TCAM_L2_VTAG_PRESENT_BITPOS,
+	 CFA_P40_PROF_PROFILE_TCAM_L2_VTAG_PRESENT_NUM_BITS},
+	{CFA_P40_PROF_PROFILE_TCAM_L2_TWO_VTAGS_BITPOS,
+	 CFA_P40_PROF_PROFILE_TCAM_L2_TWO_VTAGS_NUM_BITS},
+	{CFA_P40_PROF_PROFILE_TCAM_L3_VALID_BITPOS,
+	 CFA_P40_PROF_PROFILE_TCAM_L3_VALID_NUM_BITS},
+	{CFA_P40_PROF_PROFILE_TCAM_L3_ERROR_BITPOS,
+	 CFA_P40_PROF_PROFILE_TCAM_L3_ERROR_NUM_BITS},
+	{CFA_P40_PROF_PROFILE_TCAM_L3_HDR_TYPE_BITPOS,
+	 CFA_P40_PROF_PROFILE_TCAM_L3_HDR_TYPE_NUM_BITS},
+	{CFA_P40_PROF_PROFILE_TCAM_L3_HDR_ISIP_BITPOS,
+	 CFA_P40_PROF_PROFILE_TCAM_L3_HDR_ISIP_NUM_BITS},
+	{CFA_P40_PROF_PROFILE_TCAM_L3_IPV6_CMP_SRC_BITPOS,
+	 CFA_P40_PROF_PROFILE_TCAM_L3_IPV6_CMP_SRC_NUM_BITS},
+	{CFA_P40_PROF_PROFILE_TCAM_L3_IPV6_CMP_DEST_BITPOS,
+	 CFA_P40_PROF_PROFILE_TCAM_L3_IPV6_CMP_DEST_NUM_BITS},
+	{CFA_P40_PROF_PROFILE_TCAM_L4_HDR_VALID_BITPOS,
+	 CFA_P40_PROF_PROFILE_TCAM_L4_HDR_VALID_NUM_BITS},
+	{CFA_P40_PROF_PROFILE_TCAM_L4_HDR_ERROR_BITPOS,
+	 CFA_P40_PROF_PROFILE_TCAM_L4_HDR_ERROR_NUM_BITS},
+	{CFA_P40_PROF_PROFILE_TCAM_L4_HDR_TYPE_BITPOS,
+	 CFA_P40_PROF_PROFILE_TCAM_L4_HDR_TYPE_NUM_BITS},
+	{CFA_P40_PROF_PROFILE_TCAM_L4_HDR_IS_UDP_TCP_BITPOS,
+	 CFA_P40_PROF_PROFILE_TCAM_L4_HDR_IS_UDP_TCP_NUM_BITS},
+};
+
+/**************************************************************************/
+/**
+ * Non-autogenerated fields
+ */
+
+const struct hcapi_cfa_field cfa_p40_eem_key_tbl_layout[] = {
+	{CFA_P40_EEM_KEY_TBL_VALID_BITPOS,
+	 CFA_P40_EEM_KEY_TBL_VALID_NUM_BITS},
+
+	{CFA_P40_EEM_KEY_TBL_L1_CACHEABLE_BITPOS,
+	 CFA_P40_EEM_KEY_TBL_L1_CACHEABLE_NUM_BITS},
+
+	{CFA_P40_EEM_KEY_TBL_STRENGTH_BITPOS,
+	 CFA_P40_EEM_KEY_TBL_STRENGTH_NUM_BITS},
+
+	{CFA_P40_EEM_KEY_TBL_KEY_SZ_BITPOS,
+	 CFA_P40_EEM_KEY_TBL_KEY_SZ_NUM_BITS},
+
+	{CFA_P40_EEM_KEY_TBL_REC_SZ_BITPOS,
+	 CFA_P40_EEM_KEY_TBL_REC_SZ_NUM_BITS},
+
+	{CFA_P40_EEM_KEY_TBL_ACT_REC_INT_BITPOS,
+	 CFA_P40_EEM_KEY_TBL_ACT_REC_INT_NUM_BITS},
+
+	{CFA_P40_EEM_KEY_TBL_EXT_FLOW_CTR_BITPOS,
+	 CFA_P40_EEM_KEY_TBL_EXT_FLOW_CTR_NUM_BITS},
+
+	{CFA_P40_EEM_KEY_TBL_AR_PTR_BITPOS,
+	 CFA_P40_EEM_KEY_TBL_AR_PTR_NUM_BITS},
+
+};
+#endif /* _CFA_P40_TBL_H_ */
diff --git a/drivers/net/bnxt/hcapi/hcapi_cfa_common.c b/drivers/net/bnxt/hcapi/hcapi_cfa_common.c
deleted file mode 100644
index 39afd4d..0000000
--- a/drivers/net/bnxt/hcapi/hcapi_cfa_common.c
+++ /dev/null
@@ -1,92 +0,0 @@
-/*
- *   Copyright(c) 2019-2020 Broadcom Limited.
- *   All rights reserved.
- */
-
-#include "bitstring.h"
-#include "hcapi_cfa_defs.h"
-#include <errno.h>
-#include "assert.h"
-
-/* HCAPI CFA common PUT APIs */
-int hcapi_cfa_put_field(uint64_t *data_buf,
-			const struct hcapi_cfa_layout *layout,
-			uint16_t field_id, uint64_t val)
-{
-	assert(layout);
-
-	if (field_id > layout->array_sz)
-		/* Invalid field_id */
-		return -EINVAL;
-
-	if (layout->is_msb_order)
-		bs_put_msb(data_buf,
-			   layout->field_array[field_id].bitpos,
-			   layout->field_array[field_id].bitlen, val);
-	else
-		bs_put_lsb(data_buf,
-			   layout->field_array[field_id].bitpos,
-			   layout->field_array[field_id].bitlen, val);
-	return 0;
-}
-
-int hcapi_cfa_put_fields(uint64_t *obj_data,
-			 const struct hcapi_cfa_layout *layout,
-			 struct hcapi_cfa_data_obj *field_tbl,
-			 uint16_t field_tbl_sz)
-{
-	int i;
-	uint16_t bitpos;
-	uint8_t bitlen;
-	uint16_t field_id;
-
-	assert(layout);
-	assert(field_tbl);
-
-	if (layout->is_msb_order) {
-		for (i = 0; i < field_tbl_sz; i++) {
-			field_id = field_tbl[i].field_id;
-			if (field_id > layout->array_sz)
-				return -EINVAL;
-			bitpos = layout->field_array[field_id].bitpos;
-			bitlen = layout->field_array[field_id].bitlen;
-			bs_put_msb(obj_data, bitpos, bitlen,
-				   field_tbl[i].val);
-		}
-	} else {
-		for (i = 0; i < field_tbl_sz; i++) {
-			field_id = field_tbl[i].field_id;
-			if (field_id > layout->array_sz)
-				return -EINVAL;
-			bitpos = layout->field_array[field_id].bitpos;
-			bitlen = layout->field_array[field_id].bitlen;
-			bs_put_lsb(obj_data, bitpos, bitlen,
-				   field_tbl[i].val);
-		}
-	}
-	return 0;
-}
-
-/* HCAPI CFA common GET APIs */
-int hcapi_cfa_get_field(uint64_t *obj_data,
-			const struct hcapi_cfa_layout *layout,
-			uint16_t field_id,
-			uint64_t *val)
-{
-	assert(layout);
-	assert(val);
-
-	if (field_id > layout->array_sz)
-		/* Invalid field_id */
-		return -EINVAL;
-
-	if (layout->is_msb_order)
-		*val = bs_get_msb(obj_data,
-				  layout->field_array[field_id].bitpos,
-				  layout->field_array[field_id].bitlen);
-	else
-		*val = bs_get_lsb(obj_data,
-				  layout->field_array[field_id].bitpos,
-				  layout->field_array[field_id].bitlen);
-	return 0;
-}
diff --git a/drivers/net/bnxt/hcapi/hcapi_cfa_defs.h b/drivers/net/bnxt/hcapi/hcapi_cfa_defs.h
index ca562d2..0b7f98f 100644
--- a/drivers/net/bnxt/hcapi/hcapi_cfa_defs.h
+++ b/drivers/net/bnxt/hcapi/hcapi_cfa_defs.h
@@ -174,7 +174,7 @@ struct hcapi_cfa_data {
 	union {
 		uint32_t index;
 		uint32_t byte_offset;
-	};
+	} u;
 	/** [in] HW data buffer pointer */
 	uint8_t *data;
 	/** [in] HW data mask buffer pointer */
@@ -185,18 +185,18 @@ struct hcapi_cfa_data {
 
 /*********************** Truflow start ***************************/
 enum hcapi_cfa_pg_tbl_lvl {
-	PT_LVL_0,
-	PT_LVL_1,
-	PT_LVL_2,
-	PT_LVL_MAX
+	TF_PT_LVL_0,
+	TF_PT_LVL_1,
+	TF_PT_LVL_2,
+	TF_PT_LVL_MAX
 };
 
 enum hcapi_cfa_em_table_type {
-	KEY0_TABLE,
-	KEY1_TABLE,
-	RECORD_TABLE,
-	EFC_TABLE,
-	MAX_TABLE
+	TF_KEY0_TABLE,
+	TF_KEY1_TABLE,
+	TF_RECORD_TABLE,
+	TF_EFC_TABLE,
+	TF_MAX_TABLE
 };
 
 struct hcapi_cfa_em_page_tbl {
@@ -212,15 +212,15 @@ struct hcapi_cfa_em_table {
 	uint16_t			ctx_id;
 	uint32_t			entry_size;
 	int				num_lvl;
-	uint32_t			page_cnt[PT_LVL_MAX];
+	uint32_t			page_cnt[TF_PT_LVL_MAX];
 	uint64_t			num_data_pages;
 	void				*l0_addr;
 	uint64_t			l0_dma_addr;
-	struct hcapi_cfa_em_page_tbl    pg_tbl[PT_LVL_MAX];
+	struct hcapi_cfa_em_page_tbl    pg_tbl[TF_PT_LVL_MAX];
 };
 
 struct hcapi_cfa_em_ctx_mem_info {
-	struct hcapi_cfa_em_table		em_tables[MAX_TABLE];
+	struct hcapi_cfa_em_table		em_tables[TF_MAX_TABLE];
 };
 
 /*********************** Truflow end ****************************/
diff --git a/drivers/net/bnxt/hcapi/hcapi_cfa_p4.c b/drivers/net/bnxt/hcapi/hcapi_cfa_p4.c
index 5b5cac8..89c91ea 100644
--- a/drivers/net/bnxt/hcapi/hcapi_cfa_p4.c
+++ b/drivers/net/bnxt/hcapi/hcapi_cfa_p4.c
@@ -222,7 +222,7 @@ uint64_t hcapi_get_table_page(struct hcapi_cfa_em_table *mem,
 	 */
 	level = mem->num_lvl - 1;
 
-	addr = (uint64_t)mem->pg_tbl[level].pg_va_tbl[page];
+	addr = (uintptr_t)mem->pg_tbl[level].pg_va_tbl[page];
 
 #if 0
 	TFP_DRV_LOG(DEBUG, "dir:%d offset:0x%x level:%d page:%d addr:%p\n",
@@ -262,7 +262,7 @@ static int hcapi_cfa_key_hw_op_put(struct hcapi_cfa_hwop *op,
 {
 	int rc = 0;
 
-	memcpy((uint8_t *)op->hw.base_addr +
+	memcpy((uint8_t *)(uintptr_t)op->hw.base_addr +
 	       key_obj->offset,
 	       key_obj->data,
 	       key_obj->size);
@@ -276,7 +276,7 @@ static int hcapi_cfa_key_hw_op_get(struct hcapi_cfa_hwop *op,
 	int rc = 0;
 
 	memcpy(key_obj->data,
-	       (uint8_t *)op->hw.base_addr +
+	       (uint8_t *)(uintptr_t)op->hw.base_addr +
 	       key_obj->offset,
 	       key_obj->size);
 
@@ -293,7 +293,7 @@ static int hcapi_cfa_key_hw_op_add(struct hcapi_cfa_hwop *op,
 	 * Is entry free?
 	 */
 	memcpy(&table_entry,
-	       (uint8_t *)op->hw.base_addr +
+	       (uint8_t *)(uintptr_t)op->hw.base_addr +
 	       key_obj->offset,
 	       key_obj->size);
 
@@ -303,7 +303,7 @@ static int hcapi_cfa_key_hw_op_add(struct hcapi_cfa_hwop *op,
 	if (table_entry.hdr.word1 & (1 << CFA_P4_EEM_ENTRY_VALID_SHIFT))
 		return -1;
 
-	memcpy((uint8_t *)op->hw.base_addr +
+	memcpy((uint8_t *)(uintptr_t)op->hw.base_addr +
 	       key_obj->offset,
 	       key_obj->data,
 	       key_obj->size);
@@ -321,7 +321,7 @@ static int hcapi_cfa_key_hw_op_del(struct hcapi_cfa_hwop *op,
 	 * Read entry
 	 */
 	memcpy(&table_entry,
-	       (uint8_t *)op->hw.base_addr +
+	       (uint8_t *)(uintptr_t)op->hw.base_addr +
 	       key_obj->offset,
 	       key_obj->size);
 
@@ -347,7 +347,7 @@ static int hcapi_cfa_key_hw_op_del(struct hcapi_cfa_hwop *op,
 	/*
 	 * Delete entry
 	 */
-	memset((uint8_t *)op->hw.base_addr +
+	memset((uint8_t *)(uintptr_t)op->hw.base_addr +
 	       key_obj->offset,
 	       0,
 	       key_obj->size);
diff --git a/drivers/net/bnxt/hcapi/hcapi_cfa_p4.h b/drivers/net/bnxt/hcapi/hcapi_cfa_p4.h
index 0c11876..2c1bcad 100644
--- a/drivers/net/bnxt/hcapi/hcapi_cfa_p4.h
+++ b/drivers/net/bnxt/hcapi/hcapi_cfa_p4.h
@@ -4,7 +4,7 @@
   */
 
 #ifndef _HCAPI_CFA_P4_H_
-#define _HCPAI_CFA_P4_H_
+#define _HCAPI_CFA_P4_H_
 
 #include "cfa_p40_hw.h"
 
diff --git a/drivers/net/bnxt/meson.build b/drivers/net/bnxt/meson.build
index 4994d4c..33e6ebd 100644
--- a/drivers/net/bnxt/meson.build
+++ b/drivers/net/bnxt/meson.build
@@ -43,7 +43,6 @@ sources = files('bnxt_cpr.c',
 	'tf_core/tf_util.c',
 	'tf_core/tf_rm_new.c',
 
-	'hcapi/hcapi_cfa_common.c',
 	'hcapi/hcapi_cfa_p4.c',
 
 	'tf_ulp/bnxt_ulp.c',
diff --git a/drivers/net/bnxt/tf_core/Makefile b/drivers/net/bnxt/tf_core/Makefile
index 7191c7f..ecd5aac 100644
--- a/drivers/net/bnxt/tf_core/Makefile
+++ b/drivers/net/bnxt/tf_core/Makefile
@@ -25,3 +25,11 @@ SRCS-$(CONFIG_RTE_LIBRTE_BNXT_PMD) += tf_core/tf_tcam.c
 SRCS-$(CONFIG_RTE_LIBRTE_BNXT_PMD) += tf_core/tf_util.c
 SRCS-$(CONFIG_RTE_LIBRTE_BNXT_PMD) += tf_core/tf_rm_new.c
 
+
+SYMLINK-$(CONFIG_RTE_LIBRTE_BNXT_PMD)-include += tf_core/tf_core.h
+SYMLINK-$(CONFIG_RTE_LIBRTE_BNXT_PMD)-include += tf_core/tf_project.h
+SYMLINK-$(CONFIG_RTE_LIBRTE_BNXT_PMD)-include += tf_core/tf_device.h
+SYMLINK-$(CONFIG_RTE_LIBRTE_BNXT_PMD)-include += tf_core/tf_identifier.h
+SYMLINK-$(CONFIG_RTE_LIBRTE_BNXT_PMD)-include += tf_core/tf_tbl.h
+SYMLINK-$(CONFIG_RTE_LIBRTE_BNXT_PMD)-include += tf_core/stack.h
+SYMLINK-$(CONFIG_RTE_LIBRTE_BNXT_PMD)-include += tf_core/tf_tcam.h
diff --git a/drivers/net/bnxt/tf_core/hwrm_tf.h b/drivers/net/bnxt/tf_core/hwrm_tf.h
index c04d103..1e78296 100644
--- a/drivers/net/bnxt/tf_core/hwrm_tf.h
+++ b/drivers/net/bnxt/tf_core/hwrm_tf.h
@@ -27,8 +27,8 @@ typedef enum tf_subtype {
 	HWRM_TFT_REG_SET = 822,
 	HWRM_TFT_TBL_TYPE_SET = 823,
 	HWRM_TFT_TBL_TYPE_GET = 824,
-	HWRM_TFT_TBL_TYPE_GET_BULK = 825,
-	TF_SUBTYPE_LAST = HWRM_TFT_TBL_TYPE_GET_BULK,
+	HWRM_TFT_TBL_TYPE_BULK_GET = 825,
+	TF_SUBTYPE_LAST = HWRM_TFT_TBL_TYPE_BULK_GET,
 } tf_subtype_t;
 
 /* Request and Response compile time checking */
@@ -82,8 +82,8 @@ struct tf_session_sram_resc_flush_input;
 struct tf_tbl_type_set_input;
 struct tf_tbl_type_get_input;
 struct tf_tbl_type_get_output;
-struct tf_tbl_type_get_bulk_input;
-struct tf_tbl_type_get_bulk_output;
+struct tf_tbl_type_bulk_get_input;
+struct tf_tbl_type_bulk_get_output;
 /* Input params for session attach */
 typedef struct tf_session_attach_input {
 	/* Firmware session id returned when HWRM_TF_SESSION_OPEN is sent */
@@ -905,8 +905,6 @@ typedef struct tf_tbl_type_get_input {
 #define TF_TBL_TYPE_GET_INPUT_FLAGS_DIR_RX			(0x0)
 	/* When set to 1, indicates the get apply to TX */
 #define TF_TBL_TYPE_GET_INPUT_FLAGS_DIR_TX			(0x1)
-	/* When set to 1, indicates the clear entry on read */
-#define TF_TBL_TYPE_GET_INPUT_FLAGS_CLEAR_ON_READ	  (0x2)
 	/* Type of the object to set */
 	uint32_t			 type;
 	/* Index to get */
@@ -922,17 +920,17 @@ typedef struct tf_tbl_type_get_output {
 } tf_tbl_type_get_output_t, *ptf_tbl_type_get_output_t;
 
 /* Input params for table type get */
-typedef struct tf_tbl_type_get_bulk_input {
+typedef struct tf_tbl_type_bulk_get_input {
 	/* Session Id */
 	uint32_t			 fw_session_id;
 	/* flags */
 	uint16_t			 flags;
 	/* When set to 0, indicates the get apply to RX */
-#define TF_TBL_TYPE_GET_BULK_INPUT_FLAGS_DIR_RX	   (0x0)
+#define TF_TBL_TYPE_BULK_GET_INPUT_FLAGS_DIR_RX	   (0x0)
 	/* When set to 1, indicates the get apply to TX */
-#define TF_TBL_TYPE_GET_BULK_INPUT_FLAGS_DIR_TX	   (0x1)
+#define TF_TBL_TYPE_BULK_GET_INPUT_FLAGS_DIR_TX	   (0x1)
 	/* When set to 1, indicates the clear entry on read */
-#define TF_TBL_TYPE_GET_BULK_INPUT_FLAGS_CLEAR_ON_READ	  (0x2)
+#define TF_TBL_TYPE_BULK_GET_INPUT_FLAGS_CLEAR_ON_READ	  (0x2)
 	/* Type of the object to set */
 	uint32_t			 type;
 	/* Starting index to get from */
@@ -941,12 +939,12 @@ typedef struct tf_tbl_type_get_bulk_input {
 	uint32_t			 num_entries;
 	/* Host memory where data will be stored */
 	uint64_t			 host_addr;
-} tf_tbl_type_get_bulk_input_t, *ptf_tbl_type_get_bulk_input_t;
+} tf_tbl_type_bulk_get_input_t, *ptf_tbl_type_bulk_get_input_t;
 
 /* Output params for table type get */
-typedef struct tf_tbl_type_get_bulk_output {
+typedef struct tf_tbl_type_bulk_get_output {
 	/* Size of the total data read in bytes */
 	uint16_t			 size;
-} tf_tbl_type_get_bulk_output_t, *ptf_tbl_type_get_bulk_output_t;
+} tf_tbl_type_bulk_get_output_t, *ptf_tbl_type_bulk_get_output_t;
 
 #endif /* _HWRM_TF_H_ */
diff --git a/drivers/net/bnxt/tf_core/stack.c b/drivers/net/bnxt/tf_core/stack.c
index 9548063..240fbe2 100644
--- a/drivers/net/bnxt/tf_core/stack.c
+++ b/drivers/net/bnxt/tf_core/stack.c
@@ -81,7 +81,7 @@ int
 stack_pop(struct stack *st, uint32_t *x)
 {
 	if (stack_is_empty(st))
-		return -ENOENT;
+		return -ENODATA;
 
 	*x = st->items[st->top];
 	st->top--;
diff --git a/drivers/net/bnxt/tf_core/tf_core.c b/drivers/net/bnxt/tf_core/tf_core.c
index 7d9bca8..648d0d1 100644
--- a/drivers/net/bnxt/tf_core/tf_core.c
+++ b/drivers/net/bnxt/tf_core/tf_core.c
@@ -19,33 +19,41 @@
 #include "tf_common.h"
 #include "hwrm_tf.h"
 
-static inline uint32_t SWAP_WORDS32(uint32_t val32)
+static int tf_check_tcam_entry(enum tf_tcam_tbl_type tcam_tbl_type,
+			       enum tf_device_type device,
+			       uint16_t key_sz_in_bits,
+			       uint16_t *num_slice_per_row)
 {
-	return (((val32 & 0x0000ffff) << 16) |
-		((val32 & 0xffff0000) >> 16));
-}
+	uint16_t key_bytes;
+	uint16_t slice_sz = 0;
+
+#define CFA_P4_WC_TCAM_SLICES_PER_ROW 2
+#define CFA_P4_WC_TCAM_SLICE_SIZE     12
+
+	if (tcam_tbl_type == TF_TCAM_TBL_TYPE_WC_TCAM) {
+		key_bytes = TF_BITS2BYTES_WORD_ALIGN(key_sz_in_bits);
+		if (device == TF_DEVICE_TYPE_WH) {
+			slice_sz = CFA_P4_WC_TCAM_SLICE_SIZE;
+			*num_slice_per_row = CFA_P4_WC_TCAM_SLICES_PER_ROW;
+		} else {
+			TFP_DRV_LOG(ERR,
+				    "Unsupported device type %d\n",
+				    device);
+			return -ENOTSUP;
+		}
 
-static void tf_seeds_init(struct tf_session *session)
-{
-	int i;
-	uint32_t r;
-
-	/* Initialize the lfsr */
-	rand_init();
-
-	/* RX and TX use the same seed values */
-	session->lkup_lkup3_init_cfg[TF_DIR_RX] =
-		session->lkup_lkup3_init_cfg[TF_DIR_TX] =
-						SWAP_WORDS32(rand32());
-
-	for (i = 0; i < TF_LKUP_SEED_MEM_SIZE / 2; i++) {
-		r = SWAP_WORDS32(rand32());
-		session->lkup_em_seed_mem[TF_DIR_RX][i * 2] = r;
-		session->lkup_em_seed_mem[TF_DIR_TX][i * 2] = r;
-		r = SWAP_WORDS32(rand32());
-		session->lkup_em_seed_mem[TF_DIR_RX][i * 2 + 1] = (r & 0x1);
-		session->lkup_em_seed_mem[TF_DIR_TX][i * 2 + 1] = (r & 0x1);
+		if (key_bytes > *num_slice_per_row * slice_sz) {
+			TFP_DRV_LOG(ERR,
+				    "%s: Key size %d is not supported\n",
+				    tf_tcam_tbl_2_str(tcam_tbl_type),
+				    key_bytes);
+			return -ENOTSUP;
+		}
+	} else { /* for other type of tcam */
+		*num_slice_per_row = 1;
 	}
+
+	return 0;
 }
 
 /**
@@ -153,15 +161,18 @@ tf_open_session(struct tf                    *tfp,
 	uint8_t fw_session_id;
 	int dir;
 
-	if (tfp == NULL || parms == NULL)
-		return -EINVAL;
+	TF_CHECK_PARMS(tfp, parms);
 
 	/* Filter out any non-supported device types on the Core
 	 * side. It is assumed that the Firmware will be supported if
 	 * firmware open session succeeds.
 	 */
-	if (parms->device_type != TF_DEVICE_TYPE_WH)
+	if (parms->device_type != TF_DEVICE_TYPE_WH) {
+		TFP_DRV_LOG(ERR,
+			    "Unsupported device type %d\n",
+			    parms->device_type);
 		return -ENOTSUP;
+	}
 
 	/* Build the beginning of session_id */
 	rc = sscanf(parms->ctrl_chan_name,
@@ -171,7 +182,7 @@ tf_open_session(struct tf                    *tfp,
 		    &slot,
 		    &device);
 	if (rc != 4) {
-		PMD_DRV_LOG(ERR,
+		TFP_DRV_LOG(ERR,
 			    "Failed to scan device ctrl_chan_name\n");
 		return -EINVAL;
 	}
@@ -183,13 +194,13 @@ tf_open_session(struct tf                    *tfp,
 	if (rc) {
 		/* Log error */
 		if (rc == -EEXIST)
-			PMD_DRV_LOG(ERR,
-				    "Session is already open, rc:%d\n",
-				    rc);
+			TFP_DRV_LOG(ERR,
+				    "Session is already open, rc:%s\n",
+				    strerror(-rc));
 		else
-			PMD_DRV_LOG(ERR,
-				    "Open message send failed, rc:%d\n",
-				    rc);
+			TFP_DRV_LOG(ERR,
+				    "Open message send failed, rc:%s\n",
+				    strerror(-rc));
 
 		parms->session_id.id = TF_FW_SESSION_ID_INVALID;
 		return rc;
@@ -202,13 +213,13 @@ tf_open_session(struct tf                    *tfp,
 	rc = tfp_calloc(&alloc_parms);
 	if (rc) {
 		/* Log error */
-		PMD_DRV_LOG(ERR,
-			    "Failed to allocate session info, rc:%d\n",
-			    rc);
+		TFP_DRV_LOG(ERR,
+			    "Failed to allocate session info, rc:%s\n",
+			    strerror(-rc));
 		goto cleanup;
 	}
 
-	tfp->session = alloc_parms.mem_va;
+	tfp->session = (struct tf_session_info *)alloc_parms.mem_va;
 
 	/* Allocate core data for the session */
 	alloc_parms.nitems = 1;
@@ -217,9 +228,9 @@ tf_open_session(struct tf                    *tfp,
 	rc = tfp_calloc(&alloc_parms);
 	if (rc) {
 		/* Log error */
-		PMD_DRV_LOG(ERR,
-			    "Failed to allocate session data, rc:%d\n",
-			    rc);
+		TFP_DRV_LOG(ERR,
+			    "Failed to allocate session data, rc:%s\n",
+			    strerror(-rc));
 		goto cleanup;
 	}
 
@@ -240,12 +251,13 @@ tf_open_session(struct tf                    *tfp,
 	session->session_id.internal.device = device;
 	session->session_id.internal.fw_session_id = fw_session_id;
 
+	/* Query for Session Config
+	 */
 	rc = tf_msg_session_qcfg(tfp);
 	if (rc) {
-		/* Log error */
-		PMD_DRV_LOG(ERR,
-			    "Query config message send failed, rc:%d\n",
-			    rc);
+		TFP_DRV_LOG(ERR,
+			    "Query config message send failed, rc:%s\n",
+			    strerror(-rc));
 		goto cleanup_close;
 	}
 
@@ -256,9 +268,9 @@ tf_open_session(struct tf                    *tfp,
 #if (TF_SHADOW == 1)
 		rc = tf_rm_shadow_db_init(tfs);
 		if (rc)
-			PMD_DRV_LOG(ERR,
-				    "Shadow DB Initialization failed\n, rc:%d",
-				    rc);
+			TFP_DRV_LOG(ERR,
+				    "Shadow DB Initialization failed\n, rc:%s",
+				    strerror(-rc));
 		/* Add additional processing */
 #endif /* TF_SHADOW */
 	}
@@ -266,13 +278,12 @@ tf_open_session(struct tf                    *tfp,
 	/* Adjust the Session with what firmware allowed us to get */
 	rc = tf_rm_allocate_validate(tfp);
 	if (rc) {
-		/* Log error */
+		TFP_DRV_LOG(ERR,
+			    "Rm allocate validate failed, rc:%s\n",
+			    strerror(-rc));
 		goto cleanup_close;
 	}
 
-	/* Setup hash seeds */
-	tf_seeds_init(session);
-
 	/* Initialize EM pool */
 	for (dir = 0; dir < TF_DIR_MAX; dir++) {
 		rc = tf_create_em_pool(session,
@@ -290,11 +301,11 @@ tf_open_session(struct tf                    *tfp,
 	/* Return session ID */
 	parms->session_id = session->session_id;
 
-	PMD_DRV_LOG(INFO,
+	TFP_DRV_LOG(INFO,
 		    "Session created, session_id:%d\n",
 		    parms->session_id.id);
 
-	PMD_DRV_LOG(INFO,
+	TFP_DRV_LOG(INFO,
 		    "domain:%d, bus:%d, device:%d, fw_session_id:%d\n",
 		    parms->session_id.internal.domain,
 		    parms->session_id.internal.bus,
@@ -379,8 +390,7 @@ tf_attach_session(struct tf *tfp __rte_unused,
 #if (TF_SHARED == 1)
 	int rc;
 
-	if (tfp == NULL)
-		return -EINVAL;
+	TF_CHECK_PARMS_SESSION(tfp, parms);
 
 	/* - Open the shared memory for the attach_chan_name
 	 * - Point to the shared session for this Device instance
@@ -389,12 +399,10 @@ tf_attach_session(struct tf *tfp __rte_unused,
 	 *   than one client of the session.
 	 */
 
-	if (tfp->session) {
-		if (tfp->session->session_id.id != TF_SESSION_ID_INVALID) {
-			rc = tf_msg_session_attach(tfp,
-						   parms->ctrl_chan_name,
-						   parms->session_id);
-		}
+	if (tfp->session->session_id.id != TF_SESSION_ID_INVALID) {
+		rc = tf_msg_session_attach(tfp,
+					   parms->ctrl_chan_name,
+					   parms->session_id);
 	}
 #endif /* TF_SHARED */
 	return -1;
@@ -472,8 +480,7 @@ tf_close_session(struct tf *tfp)
 	union tf_session_id session_id;
 	int dir;
 
-	if (tfp == NULL || tfp->session == NULL)
-		return -EINVAL;
+	TF_CHECK_TFP_SESSION(tfp);
 
 	tfs = (struct tf_session *)(tfp->session->core_data);
 
@@ -487,9 +494,9 @@ tf_close_session(struct tf *tfp)
 		rc = tf_msg_session_close(tfp);
 		if (rc) {
 			/* Log error */
-			PMD_DRV_LOG(ERR,
-				    "Message send failed, rc:%d\n",
-				    rc);
+			TFP_DRV_LOG(ERR,
+				    "Message send failed, rc:%s\n",
+				    strerror(-rc));
 		}
 
 		/* Update the ref_count */
@@ -509,11 +516,11 @@ tf_close_session(struct tf *tfp)
 		tfp->session = NULL;
 	}
 
-	PMD_DRV_LOG(INFO,
+	TFP_DRV_LOG(INFO,
 		    "Session closed, session_id:%d\n",
 		    session_id.id);
 
-	PMD_DRV_LOG(INFO,
+	TFP_DRV_LOG(INFO,
 		    "domain:%d, bus:%d, device:%d, fw_session_id:%d\n",
 		    session_id.internal.domain,
 		    session_id.internal.bus,
@@ -565,27 +572,39 @@ tf_close_session_new(struct tf *tfp)
 int tf_insert_em_entry(struct tf *tfp,
 		       struct tf_insert_em_entry_parms *parms)
 {
-	struct tf_tbl_scope_cb     *tbl_scope_cb;
+	struct tf_session      *tfs;
+	struct tf_dev_info     *dev;
+	int rc;
 
-	if (tfp == NULL || parms == NULL)
-		return -EINVAL;
+	TF_CHECK_PARMS_SESSION(tfp, parms);
 
-	tbl_scope_cb = tbl_scope_cb_find(
-		(struct tf_session *)(tfp->session->core_data),
-		parms->tbl_scope_id);
-	if (tbl_scope_cb == NULL)
-		return -EINVAL;
+	/* Retrieve the session information */
+	rc = tf_session_get_session(tfp, &tfs);
+	if (rc) {
+		TFP_DRV_LOG(ERR,
+			    "%s: Failed to lookup session, rc:%s\n",
+			    tf_dir_2_str(parms->dir),
+			    strerror(-rc));
+		return rc;
+	}
 
-	/* Process the EM entry per Table Scope type */
-	if (parms->mem == TF_MEM_EXTERNAL) {
-		/* External EEM */
-		return tf_insert_eem_entry
-			((struct tf_session *)(tfp->session->core_data),
-			tbl_scope_cb,
-			parms);
-	} else if (parms->mem == TF_MEM_INTERNAL) {
-		/* Internal EM */
-		return tf_insert_em_internal_entry(tfp,	parms);
+	/* Retrieve the device information */
+	rc = tf_session_get_device(tfs, &dev);
+	if (rc) {
+		TFP_DRV_LOG(ERR,
+			    "%s: Failed to lookup device, rc:%s\n",
+			    tf_dir_2_str(parms->dir),
+			    strerror(-rc));
+		return rc;
+	}
+
+	rc = dev->ops->tf_dev_insert_em_entry(tfp, parms);
+	if (rc) {
+		TFP_DRV_LOG(ERR,
+			    "%s: EM insert failed, rc:%s\n",
+			    tf_dir_2_str(parms->dir),
+			    strerror(-rc));
+		return rc;
 	}
 
 	return -EINVAL;
@@ -600,27 +619,44 @@ int tf_insert_em_entry(struct tf *tfp,
 int tf_delete_em_entry(struct tf *tfp,
 		       struct tf_delete_em_entry_parms *parms)
 {
-	struct tf_tbl_scope_cb     *tbl_scope_cb;
+	struct tf_session      *tfs;
+	struct tf_dev_info     *dev;
+	int rc;
 
-	if (tfp == NULL || parms == NULL)
-		return -EINVAL;
+	TF_CHECK_PARMS_SESSION(tfp, parms);
 
-	tbl_scope_cb = tbl_scope_cb_find(
-		(struct tf_session *)(tfp->session->core_data),
-		parms->tbl_scope_id);
-	if (tbl_scope_cb == NULL)
-		return -EINVAL;
+	/* Retrieve the session information */
+	rc = tf_session_get_session(tfp, &tfs);
+	if (rc) {
+		TFP_DRV_LOG(ERR,
+			    "%s: Failed to lookup session, rc:%s\n",
+			    tf_dir_2_str(parms->dir),
+			    strerror(-rc));
+		return rc;
+	}
 
-	if (parms->mem == TF_MEM_EXTERNAL)
-		return tf_delete_eem_entry(tfp, parms);
-	else
-		return tf_delete_em_internal_entry(tfp, parms);
+	/* Retrieve the device information */
+	rc = tf_session_get_device(tfs, &dev);
+	if (rc) {
+		TFP_DRV_LOG(ERR,
+			    "%s: Failed to lookup device, rc:%s\n",
+			    tf_dir_2_str(parms->dir),
+			    strerror(-rc));
+		return rc;
+	}
+
+	rc = dev->ops->tf_dev_delete_em_entry(tfp, parms);
+	if (rc) {
+		TFP_DRV_LOG(ERR,
+			    "%s: EM delete failed, rc:%s\n",
+			    tf_dir_2_str(parms->dir),
+			    strerror(-rc));
+		return rc;
+	}
+
+	return rc;
 }
 
-/** allocate identifier resource
- *
- * Returns success or failure code.
- */
 int tf_alloc_identifier(struct tf *tfp,
 			struct tf_alloc_identifier_parms *parms)
 {
@@ -629,14 +665,7 @@ int tf_alloc_identifier(struct tf *tfp,
 	int id;
 	int rc;
 
-	if (parms == NULL || tfp == NULL)
-		return -EINVAL;
-
-	if (tfp->session == NULL || tfp->session->core_data == NULL) {
-		PMD_DRV_LOG(ERR, "%s: session error\n",
-			    tf_dir_2_str(parms->dir));
-		return -EINVAL;
-	}
+	TF_CHECK_PARMS_SESSION(tfp, parms);
 
 	tfs = (struct tf_session *)(tfp->session->core_data);
 
@@ -662,30 +691,31 @@ int tf_alloc_identifier(struct tf *tfp,
 				rc);
 		break;
 	case TF_IDENT_TYPE_L2_FUNC:
-		PMD_DRV_LOG(ERR, "%s: unsupported %s\n",
+		TFP_DRV_LOG(ERR, "%s: unsupported %s\n",
 			    tf_dir_2_str(parms->dir),
 			    tf_ident_2_str(parms->ident_type));
 		rc = -EOPNOTSUPP;
 		break;
 	default:
-		PMD_DRV_LOG(ERR, "%s: %s\n",
+		TFP_DRV_LOG(ERR, "%s: %s\n",
 			    tf_dir_2_str(parms->dir),
 			    tf_ident_2_str(parms->ident_type));
-		rc = -EINVAL;
+		rc = -EOPNOTSUPP;
 		break;
 	}
 
 	if (rc) {
-		PMD_DRV_LOG(ERR, "%s: identifier pool %s failure\n",
+		TFP_DRV_LOG(ERR, "%s: identifier pool %s failure, rc:%s\n",
 			    tf_dir_2_str(parms->dir),
-			    tf_ident_2_str(parms->ident_type));
+			    tf_ident_2_str(parms->ident_type),
+			    strerror(-rc));
 		return rc;
 	}
 
 	id = ba_alloc(session_pool);
 
 	if (id == BA_FAIL) {
-		PMD_DRV_LOG(ERR, "%s: %s: No resource available\n",
+		TFP_DRV_LOG(ERR, "%s: %s: No resource available\n",
 			    tf_dir_2_str(parms->dir),
 			    tf_ident_2_str(parms->ident_type));
 		return -ENOMEM;
@@ -763,14 +793,7 @@ int tf_free_identifier(struct tf *tfp,
 	int ba_rc;
 	struct tf_session *tfs;
 
-	if (parms == NULL || tfp == NULL)
-		return -EINVAL;
-
-	if (tfp->session == NULL || tfp->session->core_data == NULL) {
-		PMD_DRV_LOG(ERR, "%s: Session error\n",
-			    tf_dir_2_str(parms->dir));
-		return -EINVAL;
-	}
+	TF_CHECK_PARMS_SESSION(tfp, parms);
 
 	tfs = (struct tf_session *)(tfp->session->core_data);
 
@@ -796,29 +819,31 @@ int tf_free_identifier(struct tf *tfp,
 				rc);
 		break;
 	case TF_IDENT_TYPE_L2_FUNC:
-		PMD_DRV_LOG(ERR, "%s: unsupported %s\n",
+		TFP_DRV_LOG(ERR, "%s: unsupported %s\n",
 			    tf_dir_2_str(parms->dir),
 			    tf_ident_2_str(parms->ident_type));
 		rc = -EOPNOTSUPP;
 		break;
 	default:
-		PMD_DRV_LOG(ERR, "%s: invalid %s\n",
+		TFP_DRV_LOG(ERR, "%s: invalid %s\n",
 			    tf_dir_2_str(parms->dir),
 			    tf_ident_2_str(parms->ident_type));
-		rc = -EINVAL;
+		rc = -EOPNOTSUPP;
 		break;
 	}
 	if (rc) {
-		PMD_DRV_LOG(ERR, "%s: %s Identifier pool access failed\n",
+		TFP_DRV_LOG(ERR,
+			    "%s: %s Identifier pool access failed, rc:%s\n",
 			    tf_dir_2_str(parms->dir),
-			    tf_ident_2_str(parms->ident_type));
+			    tf_ident_2_str(parms->ident_type),
+			    strerror(-rc));
 		return rc;
 	}
 
 	ba_rc = ba_inuse(session_pool, (int)parms->id);
 
 	if (ba_rc == BA_FAIL || ba_rc == BA_ENTRY_FREE) {
-		PMD_DRV_LOG(ERR, "%s: %s: Entry %d already free",
+		TFP_DRV_LOG(ERR, "%s: %s: Entry %d already free",
 			    tf_dir_2_str(parms->dir),
 			    tf_ident_2_str(parms->ident_type),
 			    parms->id);
@@ -893,21 +918,30 @@ tf_alloc_tcam_entry(struct tf *tfp,
 		    struct tf_alloc_tcam_entry_parms *parms)
 {
 	int rc;
-	int index = 0;
+	int index;
 	struct tf_session *tfs;
 	struct bitalloc *session_pool;
+	uint16_t num_slice_per_row;
 
-	if (parms == NULL || tfp == NULL)
-		return -EINVAL;
+	/* TEMP, due to device design. When tcam is modularized device
+	 * should be retrieved from the session
+	 */
+	enum tf_device_type device_type;
+	/* TEMP */
+	device_type = TF_DEVICE_TYPE_WH;
 
-	if (tfp->session == NULL || tfp->session->core_data == NULL) {
-		PMD_DRV_LOG(ERR, "%s: session error\n",
-			    tf_dir_2_str(parms->dir));
-		return -EINVAL;
-	}
+	TF_CHECK_PARMS_SESSION(tfp, parms);
 
 	tfs = (struct tf_session *)(tfp->session->core_data);
 
+	rc = tf_check_tcam_entry(parms->tcam_tbl_type,
+				 device_type,
+				 parms->key_sz_in_bits,
+				 &num_slice_per_row);
+	/* Error logging handled by tf_check_tcam_entry */
+	if (rc)
+		return rc;
+
 	rc = tf_rm_lookup_tcam_type_pool(tfs,
 					 parms->dir,
 					 parms->tcam_tbl_type,
@@ -916,36 +950,16 @@ tf_alloc_tcam_entry(struct tf *tfp,
 	if (rc)
 		return rc;
 
-	/*
-	 * priority  0: allocate from top of the tcam i.e. high
-	 * priority !0: allocate index from bottom i.e lowest
-	 */
-	if (parms->priority) {
-		for (index = session_pool->size - 1; index >= 0; index--) {
-			if (ba_inuse(session_pool,
-					  index) == BA_ENTRY_FREE) {
-				break;
-			}
-		}
-		if (ba_alloc_index(session_pool,
-				   index) == BA_FAIL) {
-			TFP_DRV_LOG(ERR,
-				    "%s: %s: ba_alloc index %d failed\n",
-				    tf_dir_2_str(parms->dir),
-				    tf_tcam_tbl_2_str(parms->tcam_tbl_type),
-				    index);
-			return -ENOMEM;
-		}
-	} else {
-		index = ba_alloc(session_pool);
-		if (index == BA_FAIL) {
-			TFP_DRV_LOG(ERR, "%s: %s: Out of resource\n",
-				    tf_dir_2_str(parms->dir),
-				    tf_tcam_tbl_2_str(parms->tcam_tbl_type));
-			return -ENOMEM;
-		}
+	index = ba_alloc(session_pool);
+	if (index == BA_FAIL) {
+		TFP_DRV_LOG(ERR, "%s: %s: No resource available\n",
+			    tf_dir_2_str(parms->dir),
+			    tf_tcam_tbl_2_str(parms->tcam_tbl_type));
+		return -ENOMEM;
 	}
 
+	index *= num_slice_per_row;
+
 	parms->idx = index;
 	return 0;
 }
@@ -956,26 +970,29 @@ tf_set_tcam_entry(struct tf *tfp,
 {
 	int rc;
 	int id;
+	int index;
 	struct tf_session *tfs;
 	struct bitalloc *session_pool;
+	uint16_t num_slice_per_row;
 
-	if (tfp == NULL || parms == NULL) {
-		PMD_DRV_LOG(ERR, "Invalid parameters\n");
-		return -EINVAL;
-	}
+	/* TEMP, due to device design. When tcam is modularized device
+	 * should be retrieved from the session
+	 */
+	enum tf_device_type device_type;
+	/* TEMP */
+	device_type = TF_DEVICE_TYPE_WH;
 
-	if (tfp->session == NULL || tfp->session->core_data == NULL) {
-		PMD_DRV_LOG(ERR,
-			    "%s, Session info invalid\n",
-			    tf_dir_2_str(parms->dir));
-		return -EINVAL;
-	}
+	TF_CHECK_PARMS_SESSION(tfp, parms);
 
 	tfs = (struct tf_session *)(tfp->session->core_data);
 
-	/*
-	 * Each tcam send msg function should check for key sizes range
-	 */
+	rc = tf_check_tcam_entry(parms->tcam_tbl_type,
+				 device_type,
+				 parms->key_sz_in_bits,
+				 &num_slice_per_row);
+	/* Error logging handled by tf_check_tcam_entry */
+	if (rc)
+		return rc;
 
 	rc = tf_rm_lookup_tcam_type_pool(tfs,
 					 parms->dir,
@@ -985,11 +1002,12 @@ tf_set_tcam_entry(struct tf *tfp,
 	if (rc)
 		return rc;
 
-
 	/* Verify that the entry has been previously allocated */
-	id = ba_inuse(session_pool, parms->idx);
+	index = parms->idx / num_slice_per_row;
+
+	id = ba_inuse(session_pool, index);
 	if (id != 1) {
-		PMD_DRV_LOG(ERR,
+		TFP_DRV_LOG(ERR,
 		   "%s: %s: Invalid or not allocated index, idx:%d\n",
 		   tf_dir_2_str(parms->dir),
 		   tf_tcam_tbl_2_str(parms->tcam_tbl_type),
@@ -1006,21 +1024,8 @@ int
 tf_get_tcam_entry(struct tf *tfp __rte_unused,
 		  struct tf_get_tcam_entry_parms *parms __rte_unused)
 {
-	int rc = -EOPNOTSUPP;
-
-	if (tfp == NULL || parms == NULL) {
-		PMD_DRV_LOG(ERR, "Invalid parameters\n");
-		return -EINVAL;
-	}
-
-	if (tfp->session == NULL || tfp->session->core_data == NULL) {
-		PMD_DRV_LOG(ERR,
-			    "%s, Session info invalid\n",
-			    tf_dir_2_str(parms->dir));
-		return -EINVAL;
-	}
-
-	return rc;
+	TF_CHECK_PARMS_SESSION(tfp, parms);
+	return -EOPNOTSUPP;
 }
 
 int
@@ -1028,20 +1033,29 @@ tf_free_tcam_entry(struct tf *tfp,
 		   struct tf_free_tcam_entry_parms *parms)
 {
 	int rc;
+	int index;
 	struct tf_session *tfs;
 	struct bitalloc *session_pool;
+	uint16_t num_slice_per_row = 1;
 
-	if (parms == NULL || tfp == NULL)
-		return -EINVAL;
-
-	if (tfp->session == NULL || tfp->session->core_data == NULL) {
-		PMD_DRV_LOG(ERR, "%s: Session error\n",
-			    tf_dir_2_str(parms->dir));
-		return -EINVAL;
-	}
+	/* TEMP, due to device design. When tcam is modularized device
+	 * should be retrieved from the session
+	 */
+	enum tf_device_type device_type;
+	/* TEMP */
+	device_type = TF_DEVICE_TYPE_WH;
 
+	TF_CHECK_PARMS_SESSION(tfp, parms);
 	tfs = (struct tf_session *)(tfp->session->core_data);
 
+	rc = tf_check_tcam_entry(parms->tcam_tbl_type,
+				 device_type,
+				 0,
+				 &num_slice_per_row);
+	/* Error logging handled by tf_check_tcam_entry */
+	if (rc)
+		return rc;
+
 	rc = tf_rm_lookup_tcam_type_pool(tfs,
 					 parms->dir,
 					 parms->tcam_tbl_type,
@@ -1050,24 +1064,27 @@ tf_free_tcam_entry(struct tf *tfp,
 	if (rc)
 		return rc;
 
-	rc = ba_inuse(session_pool, (int)parms->idx);
+	index = parms->idx / num_slice_per_row;
+
+	rc = ba_inuse(session_pool, index);
 	if (rc == BA_FAIL || rc == BA_ENTRY_FREE) {
-		PMD_DRV_LOG(ERR, "%s: %s: Entry %d already free",
+		TFP_DRV_LOG(ERR, "%s: %s: Entry %d already free",
 			    tf_dir_2_str(parms->dir),
 			    tf_tcam_tbl_2_str(parms->tcam_tbl_type),
-			    parms->idx);
+			    index);
 		return -EINVAL;
 	}
 
-	ba_free(session_pool, (int)parms->idx);
+	ba_free(session_pool, index);
 
 	rc = tf_msg_tcam_entry_free(tfp, parms);
 	if (rc) {
 		/* Log error */
-		PMD_DRV_LOG(ERR, "%s: %s: Entry %d free failed",
+		TFP_DRV_LOG(ERR, "%s: %s: Entry %d free failed with err %s",
 			    tf_dir_2_str(parms->dir),
 			    tf_tcam_tbl_2_str(parms->tcam_tbl_type),
-			    parms->idx);
+			    parms->idx,
+			    strerror(-rc));
 	}
 
 	return rc;
diff --git a/drivers/net/bnxt/tf_core/tf_core.h b/drivers/net/bnxt/tf_core/tf_core.h
index f1ef00b..bb456bb 100644
--- a/drivers/net/bnxt/tf_core/tf_core.h
+++ b/drivers/net/bnxt/tf_core/tf_core.h
@@ -10,7 +10,7 @@
 #include <stdlib.h>
 #include <stdbool.h>
 #include <stdio.h>
-
+#include "hcapi/hcapi_cfa.h"
 #include "tf_project.h"
 
 /**
@@ -54,6 +54,7 @@ enum tf_mem {
 #define TF_ACT_REC_OFFSET_2_PTR(offset) ((offset) >> 4)
 #define TF_ACT_REC_PTR_2_OFFSET(offset) ((offset) << 4)
 
+
 /*
  * Helper Macros
  */
@@ -132,34 +133,40 @@ struct tf_session_version {
  */
 enum tf_device_type {
 	TF_DEVICE_TYPE_WH = 0, /**< Whitney+  */
-	TF_DEVICE_TYPE_BRD2,   /**< TBD       */
-	TF_DEVICE_TYPE_BRD3,   /**< TBD       */
-	TF_DEVICE_TYPE_BRD4,   /**< TBD       */
+	TF_DEVICE_TYPE_SR,     /**< Stingray  */
+	TF_DEVICE_TYPE_THOR,   /**< Thor      */
+	TF_DEVICE_TYPE_SR2,    /**< Stingray2 */
 	TF_DEVICE_TYPE_MAX     /**< Maximum   */
 };
 
-/** Identifier resource types
+/**
+ * Identifier resource types
  */
 enum tf_identifier_type {
-	/** The L2 Context is returned from the L2 Ctxt TCAM lookup
+	/**
+	 *  The L2 Context is returned from the L2 Ctxt TCAM lookup
 	 *  and can be used in WC TCAM or EM keys to virtualize further
 	 *  lookups.
 	 */
 	TF_IDENT_TYPE_L2_CTXT,
-	/** The WC profile func is returned from the L2 Ctxt TCAM lookup
+	/**
+	 *  The WC profile func is returned from the L2 Ctxt TCAM lookup
 	 *  to enable virtualization of the profile TCAM.
 	 */
 	TF_IDENT_TYPE_PROF_FUNC,
-	/** The WC profile ID is included in the WC lookup key
+	/**
+	 *  The WC profile ID is included in the WC lookup key
 	 *  to enable virtualization of the WC TCAM hardware.
 	 */
 	TF_IDENT_TYPE_WC_PROF,
-	/** The EM profile ID is included in the EM lookup key
+	/**
+	 *  The EM profile ID is included in the EM lookup key
 	 *  to enable virtualization of the EM hardware. (not required for SR2
 	 *  as it has table scope)
 	 */
 	TF_IDENT_TYPE_EM_PROF,
-	/** The L2 func is included in the ILT result and from recycling to
+	/**
+	 *  The L2 func is included in the ILT result and from recycling to
 	 *  enable virtualization of further lookups.
 	 */
 	TF_IDENT_TYPE_L2_FUNC,
@@ -239,7 +246,8 @@ enum tf_tbl_type {
 
 	/* External */
 
-	/** External table type - initially 1 poolsize entries.
+	/**
+	 * External table type - initially 1 poolsize entries.
 	 * All External table types are associated with a table
 	 * scope. Internal types are not.
 	 */
@@ -279,13 +287,17 @@ enum tf_em_tbl_type {
 	TF_EM_TBL_TYPE_MAX
 };
 
-/** TruFlow Session Information
+/**
+ * TruFlow Session Information
  *
  * Structure defining a TruFlow Session, also known as a Management
  * session. This structure is initialized at time of
  * tf_open_session(). It is passed to all of the TruFlow APIs as way
  * to prescribe and isolate resources between different TruFlow ULP
  * Applications.
+ *
+ * Ownership of the elements is split between ULP and TruFlow. Please
+ * see the individual elements.
  */
 struct tf_session_info {
 	/**
@@ -355,7 +367,8 @@ struct tf_session_info {
 	uint32_t              core_data_sz_bytes;
 };
 
-/** TruFlow handle
+/**
+ * TruFlow handle
  *
  * Contains a pointer to the session info. Allocated by ULP and passed
  * to TruFlow using tf_open_session(). TruFlow will populate the
@@ -405,7 +418,8 @@ struct tf_session_resources {
  * tf_open_session parameters definition.
  */
 struct tf_open_session_parms {
-	/** [in] ctrl_chan_name
+	/**
+	 * [in] ctrl_chan_name
 	 *
 	 * String containing name of control channel interface to be
 	 * used for this session to communicate with firmware.
@@ -417,7 +431,8 @@ struct tf_open_session_parms {
 	 * shared memory allocation.
 	 */
 	char ctrl_chan_name[TF_SESSION_NAME_MAX];
-	/** [in] shadow_copy
+	/**
+	 * [in] shadow_copy
 	 *
 	 * Boolean controlling the use and availability of shadow
 	 * copy. Shadow copy will allow the TruFlow to keep track of
@@ -430,7 +445,8 @@ struct tf_open_session_parms {
 	 * control channel.
 	 */
 	bool shadow_copy;
-	/** [in/out] session_id
+	/**
+	 * [in/out] session_id
 	 *
 	 * Session_id is unique per session.
 	 *
@@ -441,7 +457,8 @@ struct tf_open_session_parms {
 	 * The session_id allows a session to be shared between devices.
 	 */
 	union tf_session_id session_id;
-	/** [in] device type
+	/**
+	 * [in] device type
 	 *
 	 * Device type is passed, one of Wh+, SR, Thor, SR2
 	 */
@@ -484,7 +501,8 @@ int tf_open_session_new(struct tf *tfp,
 			struct tf_open_session_parms *parms);
 
 struct tf_attach_session_parms {
-	/** [in] ctrl_chan_name
+	/**
+	 * [in] ctrl_chan_name
 	 *
 	 * String containing name of control channel interface to be
 	 * used for this session to communicate with firmware.
@@ -497,7 +515,8 @@ struct tf_attach_session_parms {
 	 */
 	char ctrl_chan_name[TF_SESSION_NAME_MAX];
 
-	/** [in] attach_chan_name
+	/**
+	 * [in] attach_chan_name
 	 *
 	 * String containing name of attach channel interface to be
 	 * used for this session.
@@ -510,7 +529,8 @@ struct tf_attach_session_parms {
 	 */
 	char attach_chan_name[TF_SESSION_NAME_MAX];
 
-	/** [in] session_id
+	/**
+	 * [in] session_id
 	 *
 	 * Session_id is unique per session. For Attach the session_id
 	 * should be the session_id that was returned on the first
@@ -565,7 +585,8 @@ int tf_close_session_new(struct tf *tfp);
  *
  * @ref tf_free_identifier
  */
-/** tf_alloc_identifier parameter definition
+/**
+ * tf_alloc_identifier parameter definition
  */
 struct tf_alloc_identifier_parms {
 	/**
@@ -582,7 +603,8 @@ struct tf_alloc_identifier_parms {
 	uint16_t id;
 };
 
-/** tf_free_identifier parameter definition
+/**
+ * tf_free_identifier parameter definition
  */
 struct tf_free_identifier_parms {
 	/**
@@ -599,7 +621,8 @@ struct tf_free_identifier_parms {
 	uint16_t id;
 };
 
-/** allocate identifier resource
+/**
+ * allocate identifier resource
  *
  * TruFlow core will allocate a free id from the per identifier resource type
  * pool reserved for the session during tf_open().  No firmware is involved.
@@ -611,7 +634,8 @@ int tf_alloc_identifier(struct tf *tfp,
 int tf_alloc_identifier_new(struct tf *tfp,
 			    struct tf_alloc_identifier_parms *parms);
 
-/** free identifier resource
+/**
+ * free identifier resource
  *
  * TruFlow core will return an id back to the per identifier resource type pool
  * reserved for the session.  No firmware is involved.  During tf_close, the
@@ -639,7 +663,8 @@ int tf_free_identifier_new(struct tf *tfp,
  */
 
 
-/** tf_alloc_tbl_scope_parms definition
+/**
+ * tf_alloc_tbl_scope_parms definition
  */
 struct tf_alloc_tbl_scope_parms {
 	/**
@@ -662,7 +687,7 @@ struct tf_alloc_tbl_scope_parms {
 	 */
 	uint32_t rx_num_flows_in_k;
 	/**
-	 * [in] Brd4 only receive table access interface id
+	 * [in] SR2 only receive table access interface id
 	 */
 	uint32_t rx_tbl_if_id;
 	/**
@@ -684,7 +709,7 @@ struct tf_alloc_tbl_scope_parms {
 	 */
 	uint32_t tx_num_flows_in_k;
 	/**
-	 * [in] Brd4 only receive table access interface id
+	 * [in] SR2 only receive table access interface id
 	 */
 	uint32_t tx_tbl_if_id;
 	/**
@@ -709,7 +734,7 @@ struct tf_free_tbl_scope_parms {
 /**
  * allocate a table scope
  *
- * On Brd4 Firmware will allocate a scope ID.  On other devices, the scope
+ * On SR2 Firmware will allocate a scope ID.  On other devices, the scope
  * is a software construct to identify an EEM table.  This function will
  * divide the hash memory/buckets and records according to the device
  * device constraints based upon calculations using either the number of flows
@@ -719,7 +744,7 @@ struct tf_free_tbl_scope_parms {
  *
  * This API will allocate the table region in
  * DRAM, program the PTU page table entries, and program the number of static
- * buckets (if Brd4) in the RX and TX CFAs.  Buckets are assumed to start at
+ * buckets (if SR2) in the RX and TX CFAs.  Buckets are assumed to start at
  * 0 in the EM memory for the scope.  Upon successful completion of this API,
  * hash tables are fully initialized and ready for entries to be inserted.
  *
@@ -750,7 +775,7 @@ int tf_alloc_tbl_scope(struct tf *tfp,
  *
  * Firmware checks that the table scope ID is owned by the TruFlow
  * session, verifies that no references to this table scope remains
- * (Brd4 ILT) or Profile TCAM entries for either CFA (RX/TX) direction,
+ * (SR2 ILT) or Profile TCAM entries for either CFA (RX/TX) direction,
  * then frees the table scope ID.
  *
  * Returns success or failure code.
@@ -758,7 +783,6 @@ int tf_alloc_tbl_scope(struct tf *tfp,
 int tf_free_tbl_scope(struct tf *tfp,
 		      struct tf_free_tbl_scope_parms *parms);
 
-
 /**
  * @page tcam TCAM Access
  *
@@ -771,7 +795,9 @@ int tf_free_tbl_scope(struct tf *tfp,
  * @ref tf_free_tcam_entry
  */
 
-/** tf_alloc_tcam_entry parameter definition
+
+/**
+ * tf_alloc_tcam_entry parameter definition
  */
 struct tf_alloc_tcam_entry_parms {
 	/**
@@ -799,9 +825,7 @@ struct tf_alloc_tcam_entry_parms {
 	 */
 	uint8_t *mask;
 	/**
-	 * [in] Priority of entry requested
-	 * 0: index from top i.e. highest priority first
-	 * !0: index from bottom i.e lowest priority first
+	 * [in] Priority of entry requested (definition TBD)
 	 */
 	uint32_t priority;
 	/**
@@ -819,7 +843,8 @@ struct tf_alloc_tcam_entry_parms {
 	uint16_t idx;
 };
 
-/** allocate TCAM entry
+/**
+ * allocate TCAM entry
  *
  * Allocate a TCAM entry - one of these types:
  *
@@ -844,7 +869,8 @@ struct tf_alloc_tcam_entry_parms {
 int tf_alloc_tcam_entry(struct tf *tfp,
 			struct tf_alloc_tcam_entry_parms *parms);
 
-/** tf_set_tcam_entry parameter definition
+/**
+ * tf_set_tcam_entry parameter definition
  */
 struct	tf_set_tcam_entry_parms {
 	/**
@@ -881,7 +907,8 @@ struct	tf_set_tcam_entry_parms {
 	uint16_t result_sz_in_bits;
 };
 
-/** set TCAM entry
+/**
+ * set TCAM entry
  *
  * Program a TCAM table entry for a TruFlow session.
  *
@@ -892,7 +919,8 @@ struct	tf_set_tcam_entry_parms {
 int tf_set_tcam_entry(struct tf	*tfp,
 		      struct tf_set_tcam_entry_parms *parms);
 
-/** tf_get_tcam_entry parameter definition
+/**
+ * tf_get_tcam_entry parameter definition
  */
 struct tf_get_tcam_entry_parms {
 	/**
@@ -929,7 +957,7 @@ struct tf_get_tcam_entry_parms {
 	uint16_t result_sz_in_bits;
 };
 
-/*
+/**
  * get TCAM entry
  *
  * Program a TCAM table entry for a TruFlow session.
@@ -941,7 +969,7 @@ struct tf_get_tcam_entry_parms {
 int tf_get_tcam_entry(struct tf *tfp,
 		      struct tf_get_tcam_entry_parms *parms);
 
-/*
+/**
  * tf_free_tcam_entry parameter definition
  */
 struct tf_free_tcam_entry_parms {
@@ -963,7 +991,9 @@ struct tf_free_tcam_entry_parms {
 	uint16_t ref_cnt;
 };
 
-/*
+/**
+ * free TCAM entry
+ *
  * Free TCAM entry.
  *
  * Firmware checks to ensure the TCAM entries are owned by the TruFlow
@@ -989,6 +1019,7 @@ int tf_free_tcam_entry(struct tf *tfp,
  * @ref tf_get_tbl_entry
  */
 
+
 /**
  * tf_alloc_tbl_entry parameter definition
  */
@@ -1201,9 +1232,9 @@ int tf_get_tbl_entry(struct tf *tfp,
 		     struct tf_get_tbl_entry_parms *parms);
 
 /**
- * tf_get_bulk_tbl_entry parameter definition
+ * tf_bulk_get_tbl_entry parameter definition
  */
-struct tf_get_bulk_tbl_entry_parms {
+struct tf_bulk_get_tbl_entry_parms {
 	/**
 	 * [in] Receive or transmit direction
 	 */
@@ -1213,11 +1244,6 @@ struct tf_get_bulk_tbl_entry_parms {
 	 */
 	enum tf_tbl_type type;
 	/**
-	 * [in] Clear hardware entries on reads only
-	 * supported for TF_TBL_TYPE_ACT_STATS_64
-	 */
-	bool clear_on_read;
-	/**
 	 * [in] Starting index to read from
 	 */
 	uint32_t starting_idx;
@@ -1250,8 +1276,8 @@ struct tf_get_bulk_tbl_entry_parms {
  * Returns success or failure code. Failure will be returned if the
  * provided data buffer is too small for the data type requested.
  */
-int tf_get_bulk_tbl_entry(struct tf *tfp,
-		     struct tf_get_bulk_tbl_entry_parms *parms);
+int tf_bulk_get_tbl_entry(struct tf *tfp,
+		     struct tf_bulk_get_tbl_entry_parms *parms);
 
 /**
  * @page exact_match Exact Match Table
@@ -1280,7 +1306,7 @@ struct tf_insert_em_entry_parms {
 	 */
 	uint32_t tbl_scope_id;
 	/**
-	 * [in] ID of table interface to use (Brd4 only)
+	 * [in] ID of table interface to use (SR2 only)
 	 */
 	uint32_t tbl_if_id;
 	/**
@@ -1332,12 +1358,12 @@ struct tf_delete_em_entry_parms {
 	 */
 	uint32_t tbl_scope_id;
 	/**
-	 * [in] ID of table interface to use (Brd4 only)
+	 * [in] ID of table interface to use (SR2 only)
 	 */
 	uint32_t tbl_if_id;
 	/**
 	 * [in] epoch group IDs of entry to delete
-	 * 2 element array with 2 ids. (Brd4 only)
+	 * 2 element array with 2 ids. (SR2 only)
 	 */
 	uint16_t *epochs;
 	/**
@@ -1366,7 +1392,7 @@ struct tf_search_em_entry_parms {
 	 */
 	uint32_t tbl_scope_id;
 	/**
-	 * [in] ID of table interface to use (Brd4 only)
+	 * [in] ID of table interface to use (SR2 only)
 	 */
 	uint32_t tbl_if_id;
 	/**
@@ -1387,7 +1413,7 @@ struct tf_search_em_entry_parms {
 	uint16_t em_record_sz_in_bits;
 	/**
 	 * [in] epoch group IDs of entry to lookup
-	 * 2 element array with 2 ids. (Brd4 only)
+	 * 2 element array with 2 ids. (SR2 only)
 	 */
 	uint16_t *epochs;
 	/**
@@ -1415,7 +1441,7 @@ struct tf_search_em_entry_parms {
  * specified direction and table scope.
  *
  * When inserting an entry into an exact match table, the TruFlow library may
- * need to allocate a dynamic bucket for the entry (Brd4 only).
+ * need to allocate a dynamic bucket for the entry (SR2 only).
  *
  * The insertion of duplicate entries in an EM table is not permitted.	If a
  * TruFlow application can guarantee that it will never insert duplicates, it
@@ -1490,4 +1516,5 @@ int tf_delete_em_entry(struct tf *tfp,
  */
 int tf_search_em_entry(struct tf *tfp,
 		       struct tf_search_em_entry_parms *parms);
+
 #endif /* _TF_CORE_H_ */
diff --git a/drivers/net/bnxt/tf_core/tf_device.h b/drivers/net/bnxt/tf_core/tf_device.h
index 6aeb6fe..1501b20 100644
--- a/drivers/net/bnxt/tf_core/tf_device.h
+++ b/drivers/net/bnxt/tf_core/tf_device.h
@@ -366,6 +366,38 @@ struct tf_dev_ops {
 	 */
 	int (*tf_dev_get_tcam)(struct tf *tfp,
 			       struct tf_tcam_get_parms *parms);
+
+	/**
+	 * Insert EM hash entry API
+	 *
+	 * [in] tfp
+	 *   Pointer to TF handle
+	 *
+	 * [in] parms
+	 *   Pointer to E/EM insert parameters
+	 *
+	 *  Returns:
+	 *    0       - Success
+	 *    -EINVAL - Error
+	 */
+	int (*tf_dev_insert_em_entry)(struct tf *tfp,
+				      struct tf_insert_em_entry_parms *parms);
+
+	/**
+	 * Delete EM hash entry API
+	 *
+	 * [in] tfp
+	 *   Pointer to TF handle
+	 *
+	 * [in] parms
+	 *   Pointer to E/EM delete parameters
+	 *
+	 *    returns:
+	 *    0       - Success
+	 *    -EINVAL - Error
+	 */
+	int (*tf_dev_delete_em_entry)(struct tf *tfp,
+				      struct tf_delete_em_entry_parms *parms);
 };
 
 /**
diff --git a/drivers/net/bnxt/tf_core/tf_device_p4.c b/drivers/net/bnxt/tf_core/tf_device_p4.c
index c235976..f4bd95f 100644
--- a/drivers/net/bnxt/tf_core/tf_device_p4.c
+++ b/drivers/net/bnxt/tf_core/tf_device_p4.c
@@ -10,6 +10,7 @@
 #include "tf_identifier.h"
 #include "tf_tbl_type.h"
 #include "tf_tcam.h"
+#include "tf_em.h"
 
 /**
  * Device specific function that retrieves the MAX number of HCAPI
@@ -89,4 +90,6 @@ const struct tf_dev_ops tf_dev_ops_p4 = {
 	.tf_dev_alloc_search_tcam = tf_tcam_alloc_search,
 	.tf_dev_set_tcam = tf_tcam_set,
 	.tf_dev_get_tcam = tf_tcam_get,
+	.tf_dev_insert_em_entry = tf_em_insert_entry,
+	.tf_dev_delete_em_entry = tf_em_delete_entry,
 };
diff --git a/drivers/net/bnxt/tf_core/tf_em.c b/drivers/net/bnxt/tf_core/tf_em.c
index da1f4d4..7b430fa 100644
--- a/drivers/net/bnxt/tf_core/tf_em.c
+++ b/drivers/net/bnxt/tf_core/tf_em.c
@@ -17,11 +17,6 @@
 
 #include "bnxt.h"
 
-/* Enable EEM table dump
- */
-#define TF_EEM_DUMP
-
-static struct tf_eem_64b_entry zero_key_entry;
 
 static uint32_t tf_em_get_key_mask(int num_entries)
 {
@@ -36,326 +31,22 @@ static uint32_t tf_em_get_key_mask(int num_entries)
 	return mask;
 }
 
-/* CRC32i support for Key0 hash */
-#define ucrc32(ch, crc) (crc32tbl[((crc) ^ (ch)) & 0xff] ^ ((crc) >> 8))
-#define crc32(x, y) crc32i(~0, x, y)
-
-static const uint32_t crc32tbl[] = {	/* CRC polynomial 0xedb88320 */
-0x00000000, 0x77073096, 0xee0e612c, 0x990951ba,
-0x076dc419, 0x706af48f, 0xe963a535, 0x9e6495a3,
-0x0edb8832, 0x79dcb8a4, 0xe0d5e91e, 0x97d2d988,
-0x09b64c2b, 0x7eb17cbd, 0xe7b82d07, 0x90bf1d91,
-0x1db71064, 0x6ab020f2, 0xf3b97148, 0x84be41de,
-0x1adad47d, 0x6ddde4eb, 0xf4d4b551, 0x83d385c7,
-0x136c9856, 0x646ba8c0, 0xfd62f97a, 0x8a65c9ec,
-0x14015c4f, 0x63066cd9, 0xfa0f3d63, 0x8d080df5,
-0x3b6e20c8, 0x4c69105e, 0xd56041e4, 0xa2677172,
-0x3c03e4d1, 0x4b04d447, 0xd20d85fd, 0xa50ab56b,
-0x35b5a8fa, 0x42b2986c, 0xdbbbc9d6, 0xacbcf940,
-0x32d86ce3, 0x45df5c75, 0xdcd60dcf, 0xabd13d59,
-0x26d930ac, 0x51de003a, 0xc8d75180, 0xbfd06116,
-0x21b4f4b5, 0x56b3c423, 0xcfba9599, 0xb8bda50f,
-0x2802b89e, 0x5f058808, 0xc60cd9b2, 0xb10be924,
-0x2f6f7c87, 0x58684c11, 0xc1611dab, 0xb6662d3d,
-0x76dc4190, 0x01db7106, 0x98d220bc, 0xefd5102a,
-0x71b18589, 0x06b6b51f, 0x9fbfe4a5, 0xe8b8d433,
-0x7807c9a2, 0x0f00f934, 0x9609a88e, 0xe10e9818,
-0x7f6a0dbb, 0x086d3d2d, 0x91646c97, 0xe6635c01,
-0x6b6b51f4, 0x1c6c6162, 0x856530d8, 0xf262004e,
-0x6c0695ed, 0x1b01a57b, 0x8208f4c1, 0xf50fc457,
-0x65b0d9c6, 0x12b7e950, 0x8bbeb8ea, 0xfcb9887c,
-0x62dd1ddf, 0x15da2d49, 0x8cd37cf3, 0xfbd44c65,
-0x4db26158, 0x3ab551ce, 0xa3bc0074, 0xd4bb30e2,
-0x4adfa541, 0x3dd895d7, 0xa4d1c46d, 0xd3d6f4fb,
-0x4369e96a, 0x346ed9fc, 0xad678846, 0xda60b8d0,
-0x44042d73, 0x33031de5, 0xaa0a4c5f, 0xdd0d7cc9,
-0x5005713c, 0x270241aa, 0xbe0b1010, 0xc90c2086,
-0x5768b525, 0x206f85b3, 0xb966d409, 0xce61e49f,
-0x5edef90e, 0x29d9c998, 0xb0d09822, 0xc7d7a8b4,
-0x59b33d17, 0x2eb40d81, 0xb7bd5c3b, 0xc0ba6cad,
-0xedb88320, 0x9abfb3b6, 0x03b6e20c, 0x74b1d29a,
-0xead54739, 0x9dd277af, 0x04db2615, 0x73dc1683,
-0xe3630b12, 0x94643b84, 0x0d6d6a3e, 0x7a6a5aa8,
-0xe40ecf0b, 0x9309ff9d, 0x0a00ae27, 0x7d079eb1,
-0xf00f9344, 0x8708a3d2, 0x1e01f268, 0x6906c2fe,
-0xf762575d, 0x806567cb, 0x196c3671, 0x6e6b06e7,
-0xfed41b76, 0x89d32be0, 0x10da7a5a, 0x67dd4acc,
-0xf9b9df6f, 0x8ebeeff9, 0x17b7be43, 0x60b08ed5,
-0xd6d6a3e8, 0xa1d1937e, 0x38d8c2c4, 0x4fdff252,
-0xd1bb67f1, 0xa6bc5767, 0x3fb506dd, 0x48b2364b,
-0xd80d2bda, 0xaf0a1b4c, 0x36034af6, 0x41047a60,
-0xdf60efc3, 0xa867df55, 0x316e8eef, 0x4669be79,
-0xcb61b38c, 0xbc66831a, 0x256fd2a0, 0x5268e236,
-0xcc0c7795, 0xbb0b4703, 0x220216b9, 0x5505262f,
-0xc5ba3bbe, 0xb2bd0b28, 0x2bb45a92, 0x5cb36a04,
-0xc2d7ffa7, 0xb5d0cf31, 0x2cd99e8b, 0x5bdeae1d,
-0x9b64c2b0, 0xec63f226, 0x756aa39c, 0x026d930a,
-0x9c0906a9, 0xeb0e363f, 0x72076785, 0x05005713,
-0x95bf4a82, 0xe2b87a14, 0x7bb12bae, 0x0cb61b38,
-0x92d28e9b, 0xe5d5be0d, 0x7cdcefb7, 0x0bdbdf21,
-0x86d3d2d4, 0xf1d4e242, 0x68ddb3f8, 0x1fda836e,
-0x81be16cd, 0xf6b9265b, 0x6fb077e1, 0x18b74777,
-0x88085ae6, 0xff0f6a70, 0x66063bca, 0x11010b5c,
-0x8f659eff, 0xf862ae69, 0x616bffd3, 0x166ccf45,
-0xa00ae278, 0xd70dd2ee, 0x4e048354, 0x3903b3c2,
-0xa7672661, 0xd06016f7, 0x4969474d, 0x3e6e77db,
-0xaed16a4a, 0xd9d65adc, 0x40df0b66, 0x37d83bf0,
-0xa9bcae53, 0xdebb9ec5, 0x47b2cf7f, 0x30b5ffe9,
-0xbdbdf21c, 0xcabac28a, 0x53b39330, 0x24b4a3a6,
-0xbad03605, 0xcdd70693, 0x54de5729, 0x23d967bf,
-0xb3667a2e, 0xc4614ab8, 0x5d681b02, 0x2a6f2b94,
-0xb40bbe37, 0xc30c8ea1, 0x5a05df1b, 0x2d02ef8d
-};
-
-static uint32_t crc32i(uint32_t crc, const uint8_t *buf, size_t len)
-{
-	int l;
-
-	for (l = (len - 1); l >= 0; l--)
-		crc = ucrc32(buf[l], crc);
-
-	return ~crc;
-}
-
-static uint32_t tf_em_lkup_get_crc32_hash(struct tf_session *session,
-					  uint8_t *key,
-					  enum tf_dir dir)
-{
-	int i;
-	uint32_t index;
-	uint32_t val1, val2;
-	uint8_t temp[4];
-	uint8_t *kptr = key;
-
-	/* Do byte-wise XOR of the 52-byte HASH key first. */
-	index = *key;
-	kptr--;
-
-	for (i = TF_HW_EM_KEY_MAX_SIZE - 2; i >= 0; i--) {
-		index = index ^ *kptr;
-		kptr--;
-	}
-
-	/* Get seeds */
-	val1 = session->lkup_em_seed_mem[dir][index * 2];
-	val2 = session->lkup_em_seed_mem[dir][index * 2 + 1];
-
-	temp[3] = (uint8_t)(val1 >> 24);
-	temp[2] = (uint8_t)(val1 >> 16);
-	temp[1] = (uint8_t)(val1 >> 8);
-	temp[0] = (uint8_t)(val1 & 0xff);
-	val1 = 0;
-
-	/* Start with seed */
-	if (!(val2 & 0x1))
-		val1 = crc32i(~val1, temp, 4);
-
-	val1 = crc32i(~val1,
-		      (key - (TF_HW_EM_KEY_MAX_SIZE - 1)),
-		      TF_HW_EM_KEY_MAX_SIZE);
-
-	/* End with seed */
-	if (val2 & 0x1)
-		val1 = crc32i(~val1, temp, 4);
-
-	return val1;
-}
-
-static uint32_t tf_em_lkup_get_lookup3_hash(uint32_t lookup3_init_value,
-					    uint8_t *in_key)
-{
-	uint32_t val1;
-
-	val1 = hashword(((uint32_t *)in_key) + 1,
-			 TF_HW_EM_KEY_MAX_SIZE / (sizeof(uint32_t)),
-			 lookup3_init_value);
-
-	return val1;
-}
-
-void *tf_em_get_table_page(struct tf_tbl_scope_cb *tbl_scope_cb,
-			   enum tf_dir dir,
-			   uint32_t offset,
-			   enum tf_em_table_type table_type)
-{
-	int level = 0;
-	int page = offset / TF_EM_PAGE_SIZE;
-	void *addr = NULL;
-	struct tf_em_ctx_mem_info *ctx = &tbl_scope_cb->em_ctx_info[dir];
-
-	if (ctx == NULL)
-		return NULL;
-
-	if (dir != TF_DIR_RX && dir != TF_DIR_TX)
-		return NULL;
-
-	if (table_type < TF_KEY0_TABLE || table_type > TF_EFC_TABLE)
-		return NULL;
-
-	/*
-	 * Use the level according to the num_level of page table
-	 */
-	level = ctx->em_tables[table_type].num_lvl - 1;
-
-	addr = (void *)ctx->em_tables[table_type].pg_tbl[level].pg_va_tbl[page];
-
-	return addr;
-}
-
-/** Read Key table entry
- *
- * Entry is read in to entry
- */
-static int tf_em_read_entry(struct tf_tbl_scope_cb *tbl_scope_cb,
-				 struct tf_eem_64b_entry *entry,
-				 uint32_t entry_size,
-				 uint32_t index,
-				 enum tf_em_table_type table_type,
-				 enum tf_dir dir)
-{
-	void *page;
-	uint32_t entry_offset = (index * entry_size) % TF_EM_PAGE_SIZE;
-
-	page = tf_em_get_table_page(tbl_scope_cb,
-				    dir,
-				    (index * entry_size),
-				    table_type);
-
-	if (page == NULL)
-		return -EINVAL;
-
-	memcpy((uint8_t *)entry, (uint8_t *)page + entry_offset, entry_size);
-	return 0;
-}
-
-static int tf_em_write_entry(struct tf_tbl_scope_cb *tbl_scope_cb,
-				 struct tf_eem_64b_entry *entry,
-				 uint32_t entry_size,
-				 uint32_t index,
-				 enum tf_em_table_type table_type,
-				 enum tf_dir dir)
-{
-	void *page;
-	uint32_t entry_offset = (index * entry_size) % TF_EM_PAGE_SIZE;
-
-	page = tf_em_get_table_page(tbl_scope_cb,
-				    dir,
-				    (index * entry_size),
-				    table_type);
-
-	if (page == NULL)
-		return -EINVAL;
-
-	memcpy((uint8_t *)page + entry_offset, entry, entry_size);
-
-	return 0;
-}
-
-static int tf_em_entry_exists(struct tf_tbl_scope_cb *tbl_scope_cb,
-			       struct tf_eem_64b_entry *entry,
-			       uint32_t index,
-			       enum tf_em_table_type table_type,
-			       enum tf_dir dir)
-{
-	int rc;
-	struct tf_eem_64b_entry table_entry;
-
-	rc = tf_em_read_entry(tbl_scope_cb,
-			      &table_entry,
-			      TF_EM_KEY_RECORD_SIZE,
-			      index,
-			      table_type,
-			      dir);
-
-	if (rc != 0)
-		return -EINVAL;
-
-	if (table_entry.hdr.word1 & (1 << TF_LKUP_RECORD_VALID_SHIFT)) {
-		if (entry != NULL) {
-			if (memcmp(&table_entry,
-				   entry,
-				   TF_EM_KEY_RECORD_SIZE) == 0)
-				return -EEXIST;
-		} else {
-			return -EEXIST;
-		}
-
-		return -EBUSY;
-	}
-
-	return 0;
-}
-
-static void tf_em_create_key_entry(struct tf_eem_entry_hdr *result,
-				    uint8_t *in_key,
-				    struct tf_eem_64b_entry *key_entry)
+static void tf_em_create_key_entry(struct cfa_p4_eem_entry_hdr *result,
+				   uint8_t	       *in_key,
+				   struct cfa_p4_eem_64b_entry *key_entry)
 {
 	key_entry->hdr.word1 = result->word1;
 
-	if (result->word1 & TF_LKUP_RECORD_ACT_REC_INT_MASK)
+	if (result->word1 & CFA_P4_EEM_ENTRY_ACT_REC_INT_MASK)
 		key_entry->hdr.pointer = result->pointer;
 	else
 		key_entry->hdr.pointer = result->pointer;
 
 	memcpy(key_entry->key, in_key, TF_HW_EM_KEY_MAX_SIZE + 4);
-}
-
-/* tf_em_select_inject_table
- *
- * Returns:
- * 0 - Key does not exist in either table and can be inserted
- *		at "index" in table "table".
- * EEXIST  - Key does exist in table at "index" in table "table".
- * TF_ERR     - Something went horribly wrong.
- */
-static int tf_em_select_inject_table(struct tf_tbl_scope_cb *tbl_scope_cb,
-					  enum tf_dir dir,
-					  struct tf_eem_64b_entry *entry,
-					  uint32_t key0_hash,
-					  uint32_t key1_hash,
-					  uint32_t *index,
-					  enum tf_em_table_type *table)
-{
-	int key0_entry;
-	int key1_entry;
-
-	/*
-	 * Check KEY0 table.
-	 */
-	key0_entry = tf_em_entry_exists(tbl_scope_cb,
-					 entry,
-					 key0_hash,
-					 TF_KEY0_TABLE,
-					 dir);
 
-	/*
-	 * Check KEY1 table.
-	 */
-	key1_entry = tf_em_entry_exists(tbl_scope_cb,
-					 entry,
-					 key1_hash,
-					 TF_KEY1_TABLE,
-					 dir);
-
-	if (key0_entry == -EEXIST) {
-		*table = TF_KEY0_TABLE;
-		*index = key0_hash;
-		return -EEXIST;
-	} else if (key1_entry == -EEXIST) {
-		*table = TF_KEY1_TABLE;
-		*index = key1_hash;
-		return -EEXIST;
-	} else if (key0_entry == 0) {
-		*table = TF_KEY0_TABLE;
-		*index = key0_hash;
-		return 0;
-	} else if (key1_entry == 0) {
-		*table = TF_KEY1_TABLE;
-		*index = key1_hash;
-		return 0;
-	}
-
-	return -EINVAL;
+#ifdef TF_EEM_DEBUG
+	dump_raw((uint8_t *)key_entry, TF_EM_KEY_RECORD_SIZE, "Create raw:");
+#endif
 }
 
 /** insert EEM entry API
@@ -368,20 +59,24 @@ static int tf_em_select_inject_table(struct tf_tbl_scope_cb *tbl_scope_cb,
  *   0
  *   TF_ERR_EM_DUP  - key is already in table
  */
-int tf_insert_eem_entry(struct tf_session *session,
-			struct tf_tbl_scope_cb *tbl_scope_cb,
-			struct tf_insert_em_entry_parms *parms)
+static int tf_insert_eem_entry(struct tf_tbl_scope_cb	   *tbl_scope_cb,
+			       struct tf_insert_em_entry_parms *parms)
 {
 	uint32_t	   mask;
 	uint32_t	   key0_hash;
 	uint32_t	   key1_hash;
 	uint32_t	   key0_index;
 	uint32_t	   key1_index;
-	struct tf_eem_64b_entry key_entry;
+	struct cfa_p4_eem_64b_entry key_entry;
 	uint32_t	   index;
-	enum tf_em_table_type table_type;
+	enum hcapi_cfa_em_table_type table_type;
 	uint32_t	   gfid;
-	int		   num_of_entry;
+	struct hcapi_cfa_hwop op;
+	struct hcapi_cfa_key_tbl key_tbl;
+	struct hcapi_cfa_key_data key_obj;
+	struct hcapi_cfa_key_loc key_loc;
+	uint64_t big_hash;
+	int rc;
 
 	/* Get mask to use on hash */
 	mask = tf_em_get_key_mask(tbl_scope_cb->em_ctx_info[parms->dir].em_tables[TF_KEY0_TABLE].num_entries);
@@ -389,72 +84,84 @@ int tf_insert_eem_entry(struct tf_session *session,
 	if (!mask)
 		return -EINVAL;
 
-	num_of_entry = TF_HW_EM_KEY_MAX_SIZE + 4;
+#ifdef TF_EEM_DEBUG
+	dump_raw((uint8_t *)parms->key, TF_HW_EM_KEY_MAX_SIZE + 4, "In Key");
+#endif
 
-	key0_hash = tf_em_lkup_get_crc32_hash(session,
-				      &parms->key[num_of_entry] - 1,
-				      parms->dir);
-	key0_index = key0_hash & mask;
+	big_hash = hcapi_cfa_key_hash((uint64_t *)parms->key,
+				      (TF_HW_EM_KEY_MAX_SIZE + 4) * 8);
+	key0_hash = (uint32_t)(big_hash >> 32);
+	key1_hash = (uint32_t)(big_hash & 0xFFFFFFFF);
 
-	key1_hash =
-	   tf_em_lkup_get_lookup3_hash(session->lkup_lkup3_init_cfg[parms->dir],
-					parms->key);
+	key0_index = key0_hash & mask;
 	key1_index = key1_hash & mask;
 
+#ifdef TF_EEM_DEBUG
+	TFP_DRV_LOG(DEBUG, "Key0 hash:0x%08x\n", key0_hash);
+	TFP_DRV_LOG(DEBUG, "Key1 hash:0x%08x\n", key1_hash);
+#endif
 	/*
 	 * Use the "result" arg to populate all of the key entry then
 	 * store the byte swapped "raw" entry in a local copy ready
 	 * for insertion in to the table.
 	 */
-	tf_em_create_key_entry((struct tf_eem_entry_hdr *)parms->em_record,
+	tf_em_create_key_entry((struct cfa_p4_eem_entry_hdr *)parms->em_record,
 				((uint8_t *)parms->key),
 				&key_entry);
 
 	/*
-	 * Find which table to use
+	 * Try to add to Key0 table, if that does not work then
+	 * try the key1 table.
 	 */
-	if (tf_em_select_inject_table(tbl_scope_cb,
-				      parms->dir,
-				      &key_entry,
-				      key0_index,
-				      key1_index,
-				      &index,
-				      &table_type) == 0) {
-		if (table_type == TF_KEY0_TABLE) {
-			TF_SET_GFID(gfid,
-				    key0_index,
-				    TF_KEY0_TABLE);
-		} else {
-			TF_SET_GFID(gfid,
-				    key1_index,
-				    TF_KEY1_TABLE);
-		}
-
-		/*
-		 * Inject
-		 */
-		if (tf_em_write_entry(tbl_scope_cb,
-				      &key_entry,
-				      TF_EM_KEY_RECORD_SIZE,
-				      index,
-				      table_type,
-				      parms->dir) == 0) {
-			TF_SET_FLOW_ID(parms->flow_id,
-				       gfid,
-				       TF_GFID_TABLE_EXTERNAL,
-				       parms->dir);
-			TF_SET_FIELDS_IN_FLOW_HANDLE(parms->flow_handle,
-						     0,
-						     0,
-						     0,
-						     index,
-						     0,
-						     table_type);
-			return 0;
-		}
+	index = key0_index;
+	op.opcode = HCAPI_CFA_HWOPS_ADD;
+	key_tbl.base0 =
+		(uint8_t *)&tbl_scope_cb->em_ctx_info[parms->dir].em_tables[TF_KEY0_TABLE];
+	key_obj.offset = (index * TF_EM_KEY_RECORD_SIZE) % TF_EM_PAGE_SIZE;
+	key_obj.data = (uint8_t *)&key_entry;
+	key_obj.size = TF_EM_KEY_RECORD_SIZE;
+
+	rc = hcapi_cfa_key_hw_op(&op,
+				 &key_tbl,
+				 &key_obj,
+				 &key_loc);
+
+	if (rc == 0) {
+		table_type = TF_KEY0_TABLE;
+	} else {
+		index = key1_index;
+
+		key_tbl.base0 =
+			(uint8_t *)&tbl_scope_cb->em_ctx_info[parms->dir].em_tables[TF_KEY1_TABLE];
+		key_obj.offset =
+			(index * TF_EM_KEY_RECORD_SIZE) % TF_EM_PAGE_SIZE;
+
+		rc = hcapi_cfa_key_hw_op(&op,
+					 &key_tbl,
+					 &key_obj,
+					 &key_loc);
+		if (rc != 0)
+			return rc;
+
+		table_type = TF_KEY1_TABLE;
 	}
 
-	return -EINVAL;
+	TF_SET_GFID(gfid,
+		    index,
+		    table_type);
+	TF_SET_FLOW_ID(parms->flow_id,
+		       gfid,
+		       TF_GFID_TABLE_EXTERNAL,
+		       parms->dir);
+	TF_SET_FIELDS_IN_FLOW_HANDLE(parms->flow_handle,
+				     0,
+				     0,
+				     0,
+				     index,
+				     0,
+				     table_type);
+
+	return 0;
 }
 
 /**
@@ -463,8 +170,8 @@ int tf_insert_eem_entry(struct tf_session *session,
  *  returns:
  *     0 - Success
  */
-int tf_insert_em_internal_entry(struct tf *tfp,
-				struct tf_insert_em_entry_parms *parms)
+static int tf_insert_em_internal_entry(struct tf                       *tfp,
+				       struct tf_insert_em_entry_parms *parms)
 {
 	int       rc;
 	uint32_t  gfid;
@@ -494,7 +201,8 @@ int tf_insert_em_internal_entry(struct tf *tfp,
 	if (rc != 0)
 		return -1;
 
-	TFP_DRV_LOG(INFO,
+	PMD_DRV_LOG(
+		   ERR,
 		   "Internal entry @ Index:%d rptr_index:0x%x rptr_entry:0x%x num_of_entries:%d\n",
 		   index * TF_SESSION_EM_ENTRY_SIZE,
 		   rptr_index,
@@ -527,8 +235,8 @@ int tf_insert_em_internal_entry(struct tf *tfp,
  * 0
  * -EINVAL
  */
-int tf_delete_em_internal_entry(struct tf *tfp,
-				struct tf_delete_em_entry_parms *parms)
+static int tf_delete_em_internal_entry(struct tf                       *tfp,
+				       struct tf_delete_em_entry_parms *parms)
 {
 	int rc;
 	struct tf_session *session =
@@ -558,46 +266,95 @@ int tf_delete_em_internal_entry(struct tf *tfp,
  *   0
  *   TF_NO_EM_MATCH - entry not found
  */
-int tf_delete_eem_entry(struct tf *tfp,
-			struct tf_delete_em_entry_parms *parms)
+static int tf_delete_eem_entry(struct tf_tbl_scope_cb *tbl_scope_cb,
+			       struct tf_delete_em_entry_parms *parms)
 {
-	struct tf_session	   *session;
-	struct tf_tbl_scope_cb	   *tbl_scope_cb;
-	enum tf_em_table_type hash_type;
+	enum hcapi_cfa_em_table_type hash_type;
 	uint32_t index;
+	struct hcapi_cfa_hwop op;
+	struct hcapi_cfa_key_tbl key_tbl;
+	struct hcapi_cfa_key_data key_obj;
+	struct hcapi_cfa_key_loc key_loc;
+	int rc;
 
-	if (parms == NULL)
+	if (parms->flow_handle == 0)
 		return -EINVAL;
 
-	session = (struct tf_session *)tfp->session->core_data;
-	if (session == NULL)
-		return -EINVAL;
+	TF_GET_HASH_TYPE_FROM_FLOW_HANDLE(parms->flow_handle, hash_type);
+	TF_GET_INDEX_FROM_FLOW_HANDLE(parms->flow_handle, index);
 
-	tbl_scope_cb = tbl_scope_cb_find(session,
-					 parms->tbl_scope_id);
-	if (tbl_scope_cb == NULL)
-		return -EINVAL;
+	op.opcode = HCAPI_CFA_HWOPS_DEL;
+	key_tbl.base0 =
+		(uint8_t *)&tbl_scope_cb->em_ctx_info[parms->dir].em_tables[
+			(hash_type == 0 ? TF_KEY0_TABLE : TF_KEY1_TABLE)];
+	key_obj.offset = (index * TF_EM_KEY_RECORD_SIZE) % TF_EM_PAGE_SIZE;
+	key_obj.data = NULL;
+	key_obj.size = TF_EM_KEY_RECORD_SIZE;
 
-	if (parms->flow_handle == 0)
+	rc = hcapi_cfa_key_hw_op(&op,
+				 &key_tbl,
+				 &key_obj,
+				 &key_loc);
+
+	if (!rc)
+		return rc;
+
+	return 0;
+}
+
+/** insert EM hash entry API
+ *
+ *    returns:
+ *    0       - Success
+ *    -EINVAL - Error
+ */
+int tf_em_insert_entry(struct tf *tfp,
+		       struct tf_insert_em_entry_parms *parms)
+{
+	struct tf_tbl_scope_cb *tbl_scope_cb;
+
+	tbl_scope_cb = tbl_scope_cb_find
+		((struct tf_session *)(tfp->session->core_data),
+		parms->tbl_scope_id);
+	if (tbl_scope_cb == NULL) {
+		TFP_DRV_LOG(ERR, "Invalid tbl_scope_cb\n");
 		return -EINVAL;
+	}
 
-	TF_GET_HASH_TYPE_FROM_FLOW_HANDLE(parms->flow_handle, hash_type);
-	TF_GET_INDEX_FROM_FLOW_HANDLE(parms->flow_handle, index);
+	/* Process the EM entry per Table Scope type */
+	if (parms->mem == TF_MEM_EXTERNAL)
+		/* External EEM */
+		return tf_insert_eem_entry
+			(tbl_scope_cb, parms);
+	else if (parms->mem == TF_MEM_INTERNAL)
+		/* Internal EM */
+		return tf_insert_em_internal_entry(tfp,	parms);
 
-	if (tf_em_entry_exists(tbl_scope_cb,
-			       NULL,
-			       index,
-			       hash_type,
-			       parms->dir) == -EEXIST) {
-		tf_em_write_entry(tbl_scope_cb,
-				  &zero_key_entry,
-				  TF_EM_KEY_RECORD_SIZE,
-				  index,
-				  hash_type,
-				  parms->dir);
+	return -EINVAL;
+}
 
-		return 0;
+/** Delete EM hash entry API
+ *
+ *    returns:
+ *    0       - Success
+ *    -EINVAL - Error
+ */
+int tf_em_delete_entry(struct tf *tfp,
+		       struct tf_delete_em_entry_parms *parms)
+{
+	struct tf_tbl_scope_cb *tbl_scope_cb;
+
+	tbl_scope_cb = tbl_scope_cb_find
+		((struct tf_session *)(tfp->session->core_data),
+		parms->tbl_scope_id);
+	if (tbl_scope_cb == NULL) {
+		TFP_DRV_LOG(ERR, "Invalid tbl_scope_cb\n");
+		return -EINVAL;
 	}
+	if (parms->mem == TF_MEM_EXTERNAL)
+		return tf_delete_eem_entry(tbl_scope_cb, parms);
+	else if (parms->mem == TF_MEM_INTERNAL)
+		return tf_delete_em_internal_entry(tfp, parms);
 
 	return -EINVAL;
 }
diff --git a/drivers/net/bnxt/tf_core/tf_em.h b/drivers/net/bnxt/tf_core/tf_em.h
index c1805df..2262ae7 100644
--- a/drivers/net/bnxt/tf_core/tf_em.h
+++ b/drivers/net/bnxt/tf_core/tf_em.h
@@ -9,6 +9,13 @@
 #include "tf_core.h"
 #include "tf_session.h"
 
+#define SUPPORT_CFA_HW_P4 1
+#define SUPPORT_CFA_HW_P58 0
+#define SUPPORT_CFA_HW_P59 0
+#define SUPPORT_CFA_HW_ALL 0
+
+#include "hcapi/hcapi_cfa_defs.h"
+
 #define TF_HW_EM_KEY_MAX_SIZE 52
 #define TF_EM_KEY_RECORD_SIZE 64
 
@@ -26,56 +33,15 @@
 #define TF_EM_INTERNAL_INDEX_MASK 0xFFFC
 #define TF_EM_INTERNAL_ENTRY_MASK  0x3
 
-/** EEM Entry header
- *
- */
-struct tf_eem_entry_hdr {
-	uint32_t pointer;
-	uint32_t word1;  /*
-			  * The header is made up of two words,
-			  * this is the first word. This field has multiple
-			  * subfields, there is no suitable single name for
-			  * it so just going with word1.
-			  */
-#define TF_LKUP_RECORD_VALID_SHIFT 31
-#define TF_LKUP_RECORD_VALID_MASK 0x80000000
-#define TF_LKUP_RECORD_L1_CACHEABLE_SHIFT 30
-#define TF_LKUP_RECORD_L1_CACHEABLE_MASK 0x40000000
-#define TF_LKUP_RECORD_STRENGTH_SHIFT 28
-#define TF_LKUP_RECORD_STRENGTH_MASK 0x30000000
-#define TF_LKUP_RECORD_RESERVED_SHIFT 17
-#define TF_LKUP_RECORD_RESERVED_MASK 0x0FFE0000
-#define TF_LKUP_RECORD_KEY_SIZE_SHIFT 8
-#define TF_LKUP_RECORD_KEY_SIZE_MASK 0x0001FF00
-#define TF_LKUP_RECORD_ACT_REC_SIZE_SHIFT 3
-#define TF_LKUP_RECORD_ACT_REC_SIZE_MASK 0x000000F8
-#define TF_LKUP_RECORD_ACT_REC_INT_SHIFT 2
-#define TF_LKUP_RECORD_ACT_REC_INT_MASK 0x00000004
-#define TF_LKUP_RECORD_EXT_FLOW_CTR_SHIFT 1
-#define TF_LKUP_RECORD_EXT_FLOW_CTR_MASK 0x00000002
-#define TF_LKUP_RECORD_ACT_PTR_MSB_SHIFT 0
-#define TF_LKUP_RECORD_ACT_PTR_MSB_MASK 0x00000001
-};
-
-/** EEM Entry
- *  Each EEM entry is 512-bit (64-bytes)
- */
-struct tf_eem_64b_entry {
-	/** Key is 448 bits - 56 bytes */
-	uint8_t key[TF_EM_KEY_RECORD_SIZE - sizeof(struct tf_eem_entry_hdr)];
-	/** Header is 8 bytes long */
-	struct tf_eem_entry_hdr hdr;
-};
-
 /** EM Entry
  *  Each EM entry is 512-bit (64-bytes) but ordered differently to
  *  EEM.
  */
 struct tf_em_64b_entry {
 	/** Header is 8 bytes long */
-	struct tf_eem_entry_hdr hdr;
+	struct cfa_p4_eem_entry_hdr hdr;
 	/** Key is 448 bits - 56 bytes */
-	uint8_t key[TF_EM_KEY_RECORD_SIZE - sizeof(struct tf_eem_entry_hdr)];
+	uint8_t key[TF_EM_KEY_RECORD_SIZE - sizeof(struct cfa_p4_eem_entry_hdr)];
 };
 
 /**
@@ -127,22 +93,14 @@ int tf_free_eem_tbl_scope_cb(struct tf *tfp,
 struct tf_tbl_scope_cb *tbl_scope_cb_find(struct tf_session *session,
 					  uint32_t tbl_scope_id);
 
-int tf_insert_eem_entry(struct tf_session *session,
-			struct tf_tbl_scope_cb *tbl_scope_cb,
-			struct tf_insert_em_entry_parms *parms);
-
-int tf_insert_em_internal_entry(struct tf *tfp,
-				struct tf_insert_em_entry_parms *parms);
-
-int tf_delete_eem_entry(struct tf *tfp,
-			struct tf_delete_em_entry_parms *parms);
-
-int tf_delete_em_internal_entry(struct tf                       *tfp,
-				struct tf_delete_em_entry_parms *parms);
-
 void *tf_em_get_table_page(struct tf_tbl_scope_cb *tbl_scope_cb,
 			   enum tf_dir dir,
 			   uint32_t offset,
-			   enum tf_em_table_type table_type);
+			   enum hcapi_cfa_em_table_type table_type);
+
+int tf_em_insert_entry(struct tf *tfp,
+		       struct tf_insert_em_entry_parms *parms);
 
+int tf_em_delete_entry(struct tf *tfp,
+		       struct tf_delete_em_entry_parms *parms);
 #endif /* _TF_EM_H_ */
diff --git a/drivers/net/bnxt/tf_core/tf_msg.c b/drivers/net/bnxt/tf_core/tf_msg.c
index e08a96f..90e1acf 100644
--- a/drivers/net/bnxt/tf_core/tf_msg.c
+++ b/drivers/net/bnxt/tf_core/tf_msg.c
@@ -184,6 +184,10 @@ tf_msg_free_dma_buf(struct tf_msg_dma_buf *buf)
 }
 
 /**
+ * NEW HWRM direct messages
+ */
+
+/**
  * Sends session open request to TF Firmware
  */
 int
@@ -1259,8 +1263,8 @@ int tf_msg_insert_em_internal_entry(struct tf *tfp,
 		 HWRM_TF_EM_INSERT_INPUT_FLAGS_DIR_TX :
 		 HWRM_TF_EM_INSERT_INPUT_FLAGS_DIR_RX);
 	req.flags = tfp_cpu_to_le_16(flags);
-	req.strength = (em_result->hdr.word1 & TF_LKUP_RECORD_STRENGTH_MASK) >>
-		TF_LKUP_RECORD_STRENGTH_SHIFT;
+	req.strength = (em_result->hdr.word1 & CFA_P4_EEM_ENTRY_STRENGTH_MASK) >>
+		CFA_P4_EEM_ENTRY_STRENGTH_SHIFT;
 	req.em_key_bitlen = em_parms->key_sz_in_bits;
 	req.action_ptr = em_result->hdr.pointer;
 	req.em_record_idx = *rptr_index;
@@ -1436,22 +1440,20 @@ tf_msg_get_tbl_entry(struct tf *tfp,
 }
 
 int
-tf_msg_get_bulk_tbl_entry(struct tf *tfp,
-			  struct tf_get_bulk_tbl_entry_parms *params)
+tf_msg_bulk_get_tbl_entry(struct tf *tfp,
+			  struct tf_bulk_get_tbl_entry_parms *params)
 {
 	int rc;
 	struct tfp_send_msg_parms parms = { 0 };
-	struct tf_tbl_type_get_bulk_input req = { 0 };
-	struct tf_tbl_type_get_bulk_output resp = { 0 };
+	struct tf_tbl_type_bulk_get_input req = { 0 };
+	struct tf_tbl_type_bulk_get_output resp = { 0 };
 	struct tf_session *tfs = (struct tf_session *)(tfp->session->core_data);
 	int data_size = 0;
 
 	/* Populate the request */
 	req.fw_session_id =
 		tfp_cpu_to_le_32(tfs->session_id.internal.fw_session_id);
-	req.flags = tfp_cpu_to_le_16((params->dir) |
-		((params->clear_on_read) ?
-		 TF_TBL_TYPE_GET_BULK_INPUT_FLAGS_CLEAR_ON_READ : 0x0));
+	req.flags = tfp_cpu_to_le_16(params->dir);
 	req.type = tfp_cpu_to_le_32(params->type);
 	req.start_index = tfp_cpu_to_le_32(params->starting_idx);
 	req.num_entries = tfp_cpu_to_le_32(params->num_entries);
@@ -1462,7 +1464,7 @@ tf_msg_get_bulk_tbl_entry(struct tf *tfp,
 	MSG_PREP(parms,
 		 TF_KONG_MB,
 		 HWRM_TF,
-		 HWRM_TFT_TBL_TYPE_GET_BULK,
+		 HWRM_TFT_TBL_TYPE_BULK_GET,
 		 req,
 		 resp);
 
diff --git a/drivers/net/bnxt/tf_core/tf_msg.h b/drivers/net/bnxt/tf_core/tf_msg.h
index 06f52ef..1dad2b9 100644
--- a/drivers/net/bnxt/tf_core/tf_msg.h
+++ b/drivers/net/bnxt/tf_core/tf_msg.h
@@ -338,7 +338,7 @@ int tf_msg_get_tbl_entry(struct tf *tfp,
  * Returns:
  *  0 on Success else internal Truflow error
  */
-int tf_msg_get_bulk_tbl_entry(struct tf *tfp,
-			  struct tf_get_bulk_tbl_entry_parms *parms);
+int tf_msg_bulk_get_tbl_entry(struct tf *tfp,
+			  struct tf_bulk_get_tbl_entry_parms *parms);
 
 #endif  /* _TF_MSG_H_ */
diff --git a/drivers/net/bnxt/tf_core/tf_resources.h b/drivers/net/bnxt/tf_core/tf_resources.h
index 9b7f5a0..b7b4451 100644
--- a/drivers/net/bnxt/tf_core/tf_resources.h
+++ b/drivers/net/bnxt/tf_core/tf_resources.h
@@ -23,29 +23,27 @@
 					    * IDs
 					    */
 #define TF_NUM_WC_PROF_ID         256      /* < Number WC profile IDs */
-#define TF_NUM_WC_TCAM_ROW        256      /*  Number slices per row in WC
-					    * TCAM. A slices is a WC TCAM entry.
-					    */
+#define TF_NUM_WC_TCAM_ROW        512      /* < Number of rows in WC TCAM */
 #define TF_NUM_METER_PROF         256      /* < Number of meter profiles */
 #define TF_NUM_METER             1024      /* < Number of meter instances */
 #define TF_NUM_MIRROR               2      /* < Number of mirror instances */
 #define TF_NUM_UPAR                 2      /* < Number of UPAR instances */
 
-/* Wh+/Brd2 specific HW resources */
+/* Wh+/SR specific HW resources */
 #define TF_NUM_SP_TCAM            512      /* < Number of Source Property TCAM
 					    * entries
 					    */
 
-/* Brd2/Brd4 specific HW resources */
+/* SR/SR2 specific HW resources */
 #define TF_NUM_L2_FUNC            256      /* < Number of L2 Func */
 
 
-/* Brd3, Brd4 common HW resources */
+/* Thor, SR2 common HW resources */
 #define TF_NUM_FKB                  1      /* < Number of Flexible Key Builder
 					    * templates
 					    */
 
-/* Brd4 specific HW resources */
+/* SR2 specific HW resources */
 #define TF_NUM_TBL_SCOPE           16      /* < Number of TBL scopes */
 #define TF_NUM_EPOCH0               1      /* < Number of Epoch0 */
 #define TF_NUM_EPOCH1               1      /* < Number of Epoch1 */
@@ -149,10 +147,11 @@
 #define TF_RSVD_METER_INST_END_IDX_TX             0
 
 /* Mirror */
-#define TF_RSVD_MIRROR_RX                         1
+/* Not yet supported fully in the infra */
+#define TF_RSVD_MIRROR_RX                         0
 #define TF_RSVD_MIRROR_BEGIN_IDX_RX               0
 #define TF_RSVD_MIRROR_END_IDX_RX                 0
-#define TF_RSVD_MIRROR_TX                         1
+#define TF_RSVD_MIRROR_TX                         0
 #define TF_RSVD_MIRROR_BEGIN_IDX_TX               0
 #define TF_RSVD_MIRROR_END_IDX_TX                 0
 
@@ -501,13 +500,13 @@ enum tf_resource_type_hw {
 	TF_RESC_TYPE_HW_METER_INST,
 	TF_RESC_TYPE_HW_MIRROR,
 	TF_RESC_TYPE_HW_UPAR,
-	/* Wh+/Brd2 specific HW resources */
+	/* Wh+/SR specific HW resources */
 	TF_RESC_TYPE_HW_SP_TCAM,
-	/* Brd2/Brd4 specific HW resources */
+	/* SR/SR2 specific HW resources */
 	TF_RESC_TYPE_HW_L2_FUNC,
-	/* Brd3, Brd4 common HW resources */
+	/* Thor, SR2 common HW resources */
 	TF_RESC_TYPE_HW_FKB,
-	/* Brd4 specific HW resources */
+	/* SR2 specific HW resources */
 	TF_RESC_TYPE_HW_TBL_SCOPE,
 	TF_RESC_TYPE_HW_EPOCH0,
 	TF_RESC_TYPE_HW_EPOCH1,
diff --git a/drivers/net/bnxt/tf_core/tf_rm.c b/drivers/net/bnxt/tf_core/tf_rm.c
index 2264704..d6739b3 100644
--- a/drivers/net/bnxt/tf_core/tf_rm.c
+++ b/drivers/net/bnxt/tf_core/tf_rm.c
@@ -14,6 +14,7 @@
 #include "tf_resources.h"
 #include "tf_msg.h"
 #include "bnxt.h"
+#include "tfp.h"
 
 /**
  * Internal macro to perform HW resource allocation check between what
@@ -329,13 +330,13 @@ tf_rm_print_hw_qcaps_error(enum tf_dir dir,
 {
 	int i;
 
-	PMD_DRV_LOG(ERR, "QCAPS errors HW\n");
-	PMD_DRV_LOG(ERR, "  Direction: %s\n", tf_dir_2_str(dir));
-	PMD_DRV_LOG(ERR, "  Elements:\n");
+	TFP_DRV_LOG(ERR, "QCAPS errors HW\n");
+	TFP_DRV_LOG(ERR, "  Direction: %s\n", tf_dir_2_str(dir));
+	TFP_DRV_LOG(ERR, "  Elements:\n");
 
 	for (i = 0; i < TF_RESC_TYPE_HW_MAX; i++) {
 		if (*error_flag & 1 << i)
-			PMD_DRV_LOG(ERR, "    %s, %d elem available, req:%d\n",
+			TFP_DRV_LOG(ERR, "    %s, %d elem available, req:%d\n",
 				    tf_hcapi_hw_2_str(i),
 				    hw_query->hw_query[i].max,
 				    tf_rm_rsvd_hw_value(dir, i));
@@ -359,13 +360,13 @@ tf_rm_print_sram_qcaps_error(enum tf_dir dir,
 {
 	int i;
 
-	PMD_DRV_LOG(ERR, "QCAPS errors SRAM\n");
-	PMD_DRV_LOG(ERR, "  Direction: %s\n", tf_dir_2_str(dir));
-	PMD_DRV_LOG(ERR, "  Elements:\n");
+	TFP_DRV_LOG(ERR, "QCAPS errors SRAM\n");
+	TFP_DRV_LOG(ERR, "  Direction: %s\n", tf_dir_2_str(dir));
+	TFP_DRV_LOG(ERR, "  Elements:\n");
 
 	for (i = 0; i < TF_RESC_TYPE_SRAM_MAX; i++) {
 		if (*error_flag & 1 << i)
-			PMD_DRV_LOG(ERR, "    %s, %d elem available, req:%d\n",
+			TFP_DRV_LOG(ERR, "    %s, %d elem available, req:%d\n",
 				    tf_hcapi_sram_2_str(i),
 				    sram_query->sram_query[i].max,
 				    tf_rm_rsvd_sram_value(dir, i));
@@ -1700,7 +1701,7 @@ tf_rm_hw_alloc_validate(enum tf_dir dir,
 
 	for (i = 0; i < TF_RESC_TYPE_HW_MAX; i++) {
 		if (hw_entry[i].stride != hw_alloc->hw_num[i]) {
-			PMD_DRV_LOG(ERR,
+			TFP_DRV_LOG(ERR,
 				"%s, Alloc failed id:%d expect:%d got:%d\n",
 				tf_dir_2_str(dir),
 				i,
@@ -1727,7 +1728,7 @@ tf_rm_sram_alloc_validate(enum tf_dir dir __rte_unused,
 
 	for (i = 0; i < TF_RESC_TYPE_SRAM_MAX; i++) {
 		if (sram_entry[i].stride != sram_alloc->sram_num[i]) {
-			PMD_DRV_LOG(ERR,
+			TFP_DRV_LOG(ERR,
 				"%s, Alloc failed idx:%d expect:%d got:%d\n",
 				tf_dir_2_str(dir),
 				i,
@@ -1820,19 +1821,22 @@ tf_rm_allocate_validate_hw(struct tf *tfp,
 	rc = tf_msg_session_hw_resc_qcaps(tfp, dir, &hw_query);
 	if (rc) {
 		/* Log error */
-		PMD_DRV_LOG(ERR,
-			    "%s, HW qcaps message send failed\n",
-			    tf_dir_2_str(dir));
+		TFP_DRV_LOG(ERR,
+			    "%s, HW qcaps message send failed, rc:%s\n",
+			    tf_dir_2_str(dir),
+			    strerror(-rc));
 		goto cleanup;
 	}
 
 	rc = tf_rm_check_hw_qcaps_static(&hw_query, dir, &error_flag);
 	if (rc) {
 		/* Log error */
-		PMD_DRV_LOG(ERR,
-			"%s, HW QCAPS validation failed, error_flag:0x%x\n",
+		TFP_DRV_LOG(ERR,
+			"%s, HW QCAPS validation failed,"
+			"error_flag:0x%x, rc:%s\n",
 			tf_dir_2_str(dir),
-			error_flag);
+			error_flag,
+			strerror(-rc));
 		tf_rm_print_hw_qcaps_error(dir, &hw_query, &error_flag);
 		goto cleanup;
 	}
@@ -1845,9 +1849,10 @@ tf_rm_allocate_validate_hw(struct tf *tfp,
 	rc = tf_msg_session_hw_resc_alloc(tfp, dir, &hw_alloc, hw_entries);
 	if (rc) {
 		/* Log error */
-		PMD_DRV_LOG(ERR,
-			    "%s, HW alloc message send failed\n",
-			    tf_dir_2_str(dir));
+		TFP_DRV_LOG(ERR,
+			    "%s, HW alloc message send failed, rc:%s\n",
+			    tf_dir_2_str(dir),
+			    strerror(-rc));
 		goto cleanup;
 	}
 
@@ -1857,15 +1862,17 @@ tf_rm_allocate_validate_hw(struct tf *tfp,
 	rc = tf_rm_hw_alloc_validate(dir, &hw_alloc, hw_entries);
 	if (rc) {
 		/* Log error */
-		PMD_DRV_LOG(ERR,
-			    "%s, HW Resource validation failed\n",
-			    tf_dir_2_str(dir));
+		TFP_DRV_LOG(ERR,
+			    "%s, HW Resource validation failed, rc:%s\n",
+			    tf_dir_2_str(dir),
+			    strerror(-rc));
 		goto cleanup;
 	}
 
 	return 0;
 
  cleanup:
+
 	return -1;
 }
 
@@ -1903,19 +1910,22 @@ tf_rm_allocate_validate_sram(struct tf *tfp,
 	rc = tf_msg_session_sram_resc_qcaps(tfp, dir, &sram_query);
 	if (rc) {
 		/* Log error */
-		PMD_DRV_LOG(ERR,
-			    "%s, SRAM qcaps message send failed\n",
-			    tf_dir_2_str(dir));
+		TFP_DRV_LOG(ERR,
+			    "%s, SRAM qcaps message send failed, rc:%s\n",
+			    tf_dir_2_str(dir),
+			    strerror(-rc));
 		goto cleanup;
 	}
 
 	rc = tf_rm_check_sram_qcaps_static(&sram_query, dir, &error_flag);
 	if (rc) {
 		/* Log error */
-		PMD_DRV_LOG(ERR,
-			"%s, SRAM QCAPS validation failed, error_flag:%x\n",
+		TFP_DRV_LOG(ERR,
+			"%s, SRAM QCAPS validation failed,"
+			"error_flag:%x, rc:%s\n",
 			tf_dir_2_str(dir),
-			error_flag);
+			error_flag,
+			strerror(-rc));
 		tf_rm_print_sram_qcaps_error(dir, &sram_query, &error_flag);
 		goto cleanup;
 	}
@@ -1931,9 +1941,10 @@ tf_rm_allocate_validate_sram(struct tf *tfp,
 					    sram_entries);
 	if (rc) {
 		/* Log error */
-		PMD_DRV_LOG(ERR,
-			    "%s, SRAM alloc message send failed\n",
-			    tf_dir_2_str(dir));
+		TFP_DRV_LOG(ERR,
+			    "%s, SRAM alloc message send failed, rc:%s\n",
+			    tf_dir_2_str(dir),
+			    strerror(-rc));
 		goto cleanup;
 	}
 
@@ -1943,15 +1954,18 @@ tf_rm_allocate_validate_sram(struct tf *tfp,
 	rc = tf_rm_sram_alloc_validate(dir, &sram_alloc, sram_entries);
 	if (rc) {
 		/* Log error */
-		PMD_DRV_LOG(ERR,
-			    "%s, SRAM Resource allocation validation failed\n",
-			    tf_dir_2_str(dir));
+		TFP_DRV_LOG(ERR,
+			    "%s, SRAM Resource allocation validation failed,"
+			    " rc:%s\n",
+			    tf_dir_2_str(dir),
+			    strerror(-rc));
 		goto cleanup;
 	}
 
 	return 0;
 
  cleanup:
+
 	return -1;
 }
 
@@ -2177,7 +2191,7 @@ tf_rm_hw_to_flush(struct tf_session *tfs,
 		flush_entries[TF_RESC_TYPE_HW_TBL_SCOPE].start = 0;
 		flush_entries[TF_RESC_TYPE_HW_TBL_SCOPE].stride = 0;
 	} else {
-		PMD_DRV_LOG(ERR, "%s: TBL_SCOPE free_cnt:%d, entries:%d\n",
+		TFP_DRV_LOG(ERR, "%s, TBL_SCOPE free_cnt:%d, entries:%d\n",
 			    tf_dir_2_str(dir),
 			    free_cnt,
 			    hw_entries[TF_RESC_TYPE_HW_TBL_SCOPE].stride);
@@ -2538,8 +2552,8 @@ tf_rm_log_hw_flush(enum tf_dir dir,
 	 */
 	for (i = 0; i < TF_RESC_TYPE_HW_MAX; i++) {
 		if (hw_entries[i].stride != 0)
-			PMD_DRV_LOG(ERR,
-				    "%s: %s was not cleaned up\n",
+			TFP_DRV_LOG(ERR,
+				    "%s, %s was not cleaned up\n",
 				    tf_dir_2_str(dir),
 				    tf_hcapi_hw_2_str(i));
 	}
@@ -2564,8 +2578,8 @@ tf_rm_log_sram_flush(enum tf_dir dir,
 	 */
 	for (i = 0; i < TF_RESC_TYPE_SRAM_MAX; i++) {
 		if (sram_entries[i].stride != 0)
-			PMD_DRV_LOG(ERR,
-				    "%s: %s was not cleaned up\n",
+			TFP_DRV_LOG(ERR,
+				    "%s, %s was not cleaned up\n",
 				    tf_dir_2_str(dir),
 				    tf_hcapi_sram_2_str(i));
 	}
@@ -2777,9 +2791,10 @@ tf_rm_close(struct tf *tfp)
 		if (rc) {
 			rc_close = -ENOTEMPTY;
 			/* Log error */
-			PMD_DRV_LOG(ERR,
-				    "%s, lingering HW resources\n",
-				    tf_dir_2_str(i));
+			TFP_DRV_LOG(ERR,
+				    "%s, lingering HW resources, rc:%s\n",
+				    tf_dir_2_str(i),
+				    strerror(-rc));
 
 			/* Log the entries to be flushed */
 			tf_rm_log_hw_flush(i, hw_flush_entries);
@@ -2789,9 +2804,10 @@ tf_rm_close(struct tf *tfp)
 			if (rc) {
 				rc_close = rc;
 				/* Log error */
-				PMD_DRV_LOG(ERR,
-					    "%s, HW flush failed\n",
-					    tf_dir_2_str(i));
+				TFP_DRV_LOG(ERR,
+					    "%s, HW flush failed, rc:%s\n",
+					    tf_dir_2_str(i),
+					    strerror(-rc));
 			}
 		}
 
@@ -2805,9 +2821,10 @@ tf_rm_close(struct tf *tfp)
 		if (rc) {
 			rc_close = -ENOTEMPTY;
 			/* Log error */
-			PMD_DRV_LOG(ERR,
-				    "%s, lingering SRAM resources\n",
-				    tf_dir_2_str(i));
+			TFP_DRV_LOG(ERR,
+				    "%s, lingering SRAM resources, rc:%s\n",
+				    tf_dir_2_str(i),
+				    strerror(-rc));
 
 			/* Log the entries to be flushed */
 			tf_rm_log_sram_flush(i, sram_flush_entries);
@@ -2818,9 +2835,10 @@ tf_rm_close(struct tf *tfp)
 			if (rc) {
 				rc_close = rc;
 				/* Log error */
-				PMD_DRV_LOG(ERR,
-					    "%s, HW flush failed\n",
-					    tf_dir_2_str(i));
+				TFP_DRV_LOG(ERR,
+					    "%s, HW flush failed, rc:%s\n",
+					    tf_dir_2_str(i),
+					    strerror(-rc));
 			}
 		}
 
@@ -2828,18 +2846,20 @@ tf_rm_close(struct tf *tfp)
 		if (rc) {
 			rc_close = rc;
 			/* Log error */
-			PMD_DRV_LOG(ERR,
-				    "%s, HW free failed\n",
-				    tf_dir_2_str(i));
+			TFP_DRV_LOG(ERR,
+				    "%s, HW free failed, rc:%s\n",
+				    tf_dir_2_str(i),
+				    strerror(-rc));
 		}
 
 		rc = tf_msg_session_sram_resc_free(tfp, i, sram_entries);
 		if (rc) {
 			rc_close = rc;
 			/* Log error */
-			PMD_DRV_LOG(ERR,
-				    "%s, SRAM free failed\n",
-				    tf_dir_2_str(i));
+			TFP_DRV_LOG(ERR,
+				    "%s, SRAM free failed, rc:%s\n",
+				    tf_dir_2_str(i),
+				    strerror(-rc));
 		}
 	}
 
@@ -2890,14 +2910,14 @@ tf_rm_lookup_tcam_type_pool(struct tf_session *tfs,
 	}
 
 	if (rc == -EOPNOTSUPP) {
-		PMD_DRV_LOG(ERR,
-			    "dir:%d, Tcam type not supported, type:%d\n",
-			    dir,
+		TFP_DRV_LOG(ERR,
+			    "%s, Tcam type not supported, type:%d\n",
+			    tf_dir_2_str(dir),
 			    type);
 		return rc;
 	} else if (rc == -1) {
-		PMD_DRV_LOG(ERR,
-			    "%s:, Tcam type lookup failed, type:%d\n",
+		TFP_DRV_LOG(ERR,
+			    "%s, Tcam type lookup failed, type:%d\n",
 			    tf_dir_2_str(dir),
 			    type);
 		return rc;
@@ -3057,15 +3077,15 @@ tf_rm_lookup_tbl_type_pool(struct tf_session *tfs,
 	}
 
 	if (rc == -EOPNOTSUPP) {
-		PMD_DRV_LOG(ERR,
-			    "dir:%d, Table type not supported, type:%d\n",
-			    dir,
+		TFP_DRV_LOG(ERR,
+			    "%s, Table type not supported, type:%d\n",
+			    tf_dir_2_str(dir),
 			    type);
 		return rc;
 	} else if (rc == -1) {
-		PMD_DRV_LOG(ERR,
-			    "dir:%d, Table type lookup failed, type:%d\n",
-			    dir,
+		TFP_DRV_LOG(ERR,
+			    "%s, Table type lookup failed, type:%d\n",
+			    tf_dir_2_str(dir),
 			    type);
 		return rc;
 	}
@@ -3166,6 +3186,13 @@ tf_rm_convert_tbl_type(enum tf_tbl_type type,
 	return rc;
 }
 
+#if 0
+enum tf_rm_convert_type {
+	TF_RM_CONVERT_ADD_BASE,
+	TF_RM_CONVERT_RM_BASE
+};
+#endif
+
 int
 tf_rm_convert_index(struct tf_session *tfs,
 		    enum tf_dir dir,
diff --git a/drivers/net/bnxt/tf_core/tf_tbl.c b/drivers/net/bnxt/tf_core/tf_tbl.c
index 07c3469..7f37f4d 100644
--- a/drivers/net/bnxt/tf_core/tf_tbl.c
+++ b/drivers/net/bnxt/tf_core/tf_tbl.c
@@ -23,6 +23,7 @@
 #include "bnxt.h"
 #include "tf_resources.h"
 #include "tf_rm.h"
+#include "stack.h"
 #include "tf_common.h"
 
 #define PTU_PTE_VALID          0x1UL
@@ -53,14 +54,14 @@
  *   Pointer to the page table to free
  */
 static void
-tf_em_free_pg_tbl(struct tf_em_page_tbl *tp)
+tf_em_free_pg_tbl(struct hcapi_cfa_em_page_tbl *tp)
 {
 	uint32_t i;
 
 	for (i = 0; i < tp->pg_count; i++) {
 		if (!tp->pg_va_tbl[i]) {
-			PMD_DRV_LOG(WARNING,
-				    "No map for page %d table %016" PRIu64 "\n",
+			TFP_DRV_LOG(WARNING,
+				    "No mapping for page: %d table: %016" PRIu64 "\n",
 				    i,
 				    (uint64_t)(uintptr_t)tp);
 			continue;
@@ -84,15 +85,14 @@ tf_em_free_pg_tbl(struct tf_em_page_tbl *tp)
  *   Pointer to the EM table to free
  */
 static void
-tf_em_free_page_table(struct tf_em_table *tbl)
+tf_em_free_page_table(struct hcapi_cfa_em_table *tbl)
 {
-	struct tf_em_page_tbl *tp;
+	struct hcapi_cfa_em_page_tbl *tp;
 	int i;
 
 	for (i = 0; i < tbl->num_lvl; i++) {
 		tp = &tbl->pg_tbl[i];
-
-		PMD_DRV_LOG(INFO,
+		TFP_DRV_LOG(INFO,
 			   "EEM: Freeing page table: size %u lvl %d cnt %u\n",
 			   TF_EM_PAGE_SIZE,
 			    i,
@@ -124,7 +124,7 @@ tf_em_free_page_table(struct tf_em_table *tbl)
  *   -ENOMEM - Out of memory
  */
 static int
-tf_em_alloc_pg_tbl(struct tf_em_page_tbl *tp,
+tf_em_alloc_pg_tbl(struct hcapi_cfa_em_page_tbl *tp,
 		   uint32_t pg_count,
 		   uint32_t pg_size)
 {
@@ -183,9 +183,9 @@ tf_em_alloc_pg_tbl(struct tf_em_page_tbl *tp,
  *   -ENOMEM - Out of memory
  */
 static int
-tf_em_alloc_page_table(struct tf_em_table *tbl)
+tf_em_alloc_page_table(struct hcapi_cfa_em_table *tbl)
 {
-	struct tf_em_page_tbl *tp;
+	struct hcapi_cfa_em_page_tbl *tp;
 	int rc = 0;
 	int i;
 	uint32_t j;
@@ -197,14 +197,15 @@ tf_em_alloc_page_table(struct tf_em_table *tbl)
 					tbl->page_cnt[i],
 					TF_EM_PAGE_SIZE);
 		if (rc) {
-			PMD_DRV_LOG(WARNING,
-				"Failed to allocate page table: lvl: %d\n",
-				i);
+			TFP_DRV_LOG(WARNING,
+				"Failed to allocate page table: lvl: %d, rc:%s\n",
+				i,
+				strerror(-rc));
 			goto cleanup;
 		}
 
 		for (j = 0; j < tp->pg_count; j++) {
-			PMD_DRV_LOG(INFO,
+			TFP_DRV_LOG(INFO,
 				"EEM: Allocated page table: size %u lvl %d cnt"
 				" %u VA:%p PA:%p\n",
 				TF_EM_PAGE_SIZE,
@@ -234,8 +235,8 @@ tf_em_alloc_page_table(struct tf_em_table *tbl)
  *   Flag controlling if the page table is last
  */
 static void
-tf_em_link_page_table(struct tf_em_page_tbl *tp,
-		      struct tf_em_page_tbl *tp_next,
+tf_em_link_page_table(struct hcapi_cfa_em_page_tbl *tp,
+		      struct hcapi_cfa_em_page_tbl *tp_next,
 		      bool set_pte_last)
 {
 	uint64_t *pg_pa = tp_next->pg_pa_tbl;
@@ -270,10 +271,10 @@ tf_em_link_page_table(struct tf_em_page_tbl *tp,
  *   Pointer to EM page table
  */
 static void
-tf_em_setup_page_table(struct tf_em_table *tbl)
+tf_em_setup_page_table(struct hcapi_cfa_em_table *tbl)
 {
-	struct tf_em_page_tbl *tp_next;
-	struct tf_em_page_tbl *tp;
+	struct hcapi_cfa_em_page_tbl *tp_next;
+	struct hcapi_cfa_em_page_tbl *tp;
 	bool set_pte_last = 0;
 	int i;
 
@@ -415,7 +416,7 @@ tf_em_size_page_tbls(int max_lvl,
  *   - ENOMEM - Out of memory
  */
 static int
-tf_em_size_table(struct tf_em_table *tbl)
+tf_em_size_table(struct hcapi_cfa_em_table *tbl)
 {
 	uint64_t num_data_pages;
 	uint32_t *page_cnt;
@@ -456,11 +457,10 @@ tf_em_size_table(struct tf_em_table *tbl)
 					  tbl->num_entries,
 					  &num_data_pages);
 	if (max_lvl < 0) {
-		PMD_DRV_LOG(WARNING, "EEM: Failed to size page table levels\n");
-		PMD_DRV_LOG(WARNING,
+		TFP_DRV_LOG(WARNING, "EEM: Failed to size page table levels\n");
+		TFP_DRV_LOG(WARNING,
 			    "table: %d data-sz: %016" PRIu64 " page-sz: %u\n",
-			    tbl->type,
-			    (uint64_t)num_entries * tbl->entry_size,
+			    tbl->type, (uint64_t)num_entries * tbl->entry_size,
 			    TF_EM_PAGE_SIZE);
 		return -ENOMEM;
 	}
@@ -474,8 +474,8 @@ tf_em_size_table(struct tf_em_table *tbl)
 	tf_em_size_page_tbls(max_lvl, num_data_pages, TF_EM_PAGE_SIZE,
 				page_cnt);
 
-	PMD_DRV_LOG(INFO, "EEM: Sized page table: %d\n", tbl->type);
-	PMD_DRV_LOG(INFO,
+	TFP_DRV_LOG(INFO, "EEM: Sized page table: %d\n", tbl->type);
+	TFP_DRV_LOG(INFO,
 		    "EEM: lvls: %d sz: %016" PRIu64 " pgs: %016" PRIu64 " l0: %u l1: %u l2: %u\n",
 		    max_lvl + 1,
 		    (uint64_t)num_data_pages * TF_EM_PAGE_SIZE,
@@ -504,8 +504,8 @@ tf_em_ctx_unreg(struct tf *tfp,
 		struct tf_tbl_scope_cb *tbl_scope_cb,
 		int dir)
 {
-	struct tf_em_ctx_mem_info *ctxp = &tbl_scope_cb->em_ctx_info[dir];
-	struct tf_em_table *tbl;
+	struct hcapi_cfa_em_ctx_mem_info *ctxp = &tbl_scope_cb->em_ctx_info[dir];
+	struct hcapi_cfa_em_table *tbl;
 	int i;
 
 	for (i = TF_KEY0_TABLE; i < TF_MAX_TABLE; i++) {
@@ -539,8 +539,8 @@ tf_em_ctx_reg(struct tf *tfp,
 	      struct tf_tbl_scope_cb *tbl_scope_cb,
 	      int dir)
 {
-	struct tf_em_ctx_mem_info *ctxp = &tbl_scope_cb->em_ctx_info[dir];
-	struct tf_em_table *tbl;
+	struct hcapi_cfa_em_ctx_mem_info *ctxp = &tbl_scope_cb->em_ctx_info[dir];
+	struct hcapi_cfa_em_table *tbl;
 	int rc = 0;
 	int i;
 
@@ -601,7 +601,7 @@ tf_em_validate_num_entries(struct tf_tbl_scope_cb *tbl_scope_cb,
 					TF_MEGABYTE) / (key_b + action_b);
 
 		if (num_entries < TF_EM_MIN_ENTRIES) {
-			PMD_DRV_LOG(ERR, "EEM: Insufficient memory requested:"
+			TFP_DRV_LOG(ERR, "EEM: Insufficient memory requested:"
 				    "%uMB\n",
 				    parms->rx_mem_size_in_mb);
 			return -EINVAL;
@@ -613,7 +613,7 @@ tf_em_validate_num_entries(struct tf_tbl_scope_cb *tbl_scope_cb,
 			cnt *= 2;
 
 		if (cnt > TF_EM_MAX_ENTRIES) {
-			PMD_DRV_LOG(ERR, "EEM: Invalid number of Tx requested: "
+			TFP_DRV_LOG(ERR, "EEM: Invalid number of Tx requested: "
 				    "%u\n",
 		       (parms->tx_num_flows_in_k * TF_KILOBYTE));
 			return -EINVAL;
@@ -625,7 +625,7 @@ tf_em_validate_num_entries(struct tf_tbl_scope_cb *tbl_scope_cb,
 		    TF_EM_MIN_ENTRIES ||
 		    (parms->rx_num_flows_in_k * TF_KILOBYTE) >
 		    tbl_scope_cb->em_caps[TF_DIR_RX].max_entries_supported) {
-			PMD_DRV_LOG(ERR,
+			TFP_DRV_LOG(ERR,
 				    "EEM: Invalid number of Rx flows "
 				    "requested:%u max:%u\n",
 				    parms->rx_num_flows_in_k * TF_KILOBYTE,
@@ -642,7 +642,7 @@ tf_em_validate_num_entries(struct tf_tbl_scope_cb *tbl_scope_cb,
 			cnt *= 2;
 
 		if (cnt > TF_EM_MAX_ENTRIES) {
-			PMD_DRV_LOG(ERR,
+			TFP_DRV_LOG(ERR,
 				    "EEM: Invalid number of Rx requested: %u\n",
 				    (parms->rx_num_flows_in_k * TF_KILOBYTE));
 			return -EINVAL;
@@ -658,7 +658,7 @@ tf_em_validate_num_entries(struct tf_tbl_scope_cb *tbl_scope_cb,
 			(key_b + action_b);
 
 		if (num_entries < TF_EM_MIN_ENTRIES) {
-			PMD_DRV_LOG(ERR,
+			TFP_DRV_LOG(ERR,
 				    "EEM: Insufficient memory requested:%uMB\n",
 				    parms->rx_mem_size_in_mb);
 			return -EINVAL;
@@ -670,7 +670,7 @@ tf_em_validate_num_entries(struct tf_tbl_scope_cb *tbl_scope_cb,
 			cnt *= 2;
 
 		if (cnt > TF_EM_MAX_ENTRIES) {
-			PMD_DRV_LOG(ERR,
+			TFP_DRV_LOG(ERR,
 				    "EEM: Invalid number of Tx requested: %u\n",
 		       (parms->tx_num_flows_in_k * TF_KILOBYTE));
 			return -EINVAL;
@@ -682,7 +682,7 @@ tf_em_validate_num_entries(struct tf_tbl_scope_cb *tbl_scope_cb,
 		    TF_EM_MIN_ENTRIES ||
 		    (parms->tx_num_flows_in_k * TF_KILOBYTE) >
 		    tbl_scope_cb->em_caps[TF_DIR_TX].max_entries_supported) {
-			PMD_DRV_LOG(ERR,
+			TFP_DRV_LOG(ERR,
 				    "EEM: Invalid number of Tx flows "
 				    "requested:%u max:%u\n",
 				    (parms->tx_num_flows_in_k * TF_KILOBYTE),
@@ -696,24 +696,24 @@ tf_em_validate_num_entries(struct tf_tbl_scope_cb *tbl_scope_cb,
 			cnt *= 2;
 
 		if (cnt > TF_EM_MAX_ENTRIES) {
-			PMD_DRV_LOG(ERR,
+			TFP_DRV_LOG(ERR,
 				    "EEM: Invalid number of Tx requested: %u\n",
 		       (parms->tx_num_flows_in_k * TF_KILOBYTE));
 			return -EINVAL;
 		}
 	}
 
-	if (parms->rx_num_flows_in_k != 0 &&
+	if ((parms->rx_num_flows_in_k != 0) &&
 	    (parms->rx_max_key_sz_in_bits / 8 == 0)) {
-		PMD_DRV_LOG(ERR,
+		TFP_DRV_LOG(ERR,
 			    "EEM: Rx key size required: %u\n",
 			    (parms->rx_max_key_sz_in_bits));
 		return -EINVAL;
 	}
 
-	if (parms->tx_num_flows_in_k != 0 &&
+	if ((parms->tx_num_flows_in_k != 0) &&
 	    (parms->tx_max_key_sz_in_bits / 8 == 0)) {
-		PMD_DRV_LOG(ERR,
+		TFP_DRV_LOG(ERR,
 			    "EEM: Tx key size required: %u\n",
 			    (parms->tx_max_key_sz_in_bits));
 		return -EINVAL;
@@ -795,11 +795,10 @@ tf_set_tbl_entry_internal(struct tf *tfp,
 
 	if (parms->type != TF_TBL_TYPE_FULL_ACT_RECORD &&
 	    parms->type != TF_TBL_TYPE_ACT_SP_SMAC_IPV4 &&
-	    parms->type != TF_TBL_TYPE_MIRROR_CONFIG &&
 	    parms->type != TF_TBL_TYPE_ACT_STATS_64) {
-		PMD_DRV_LOG(ERR,
-			    "dir:%d, Type not supported, type:%d\n",
-			    parms->dir,
+		TFP_DRV_LOG(ERR,
+			    "%s, Type not supported, type:%d\n",
+			    tf_dir_2_str(parms->dir),
 			    parms->type);
 		return -EOPNOTSUPP;
 	}
@@ -817,9 +816,9 @@ tf_set_tbl_entry_internal(struct tf *tfp,
 	/* Verify that the entry has been previously allocated */
 	id = ba_inuse(session_pool, index);
 	if (id != 1) {
-		PMD_DRV_LOG(ERR,
-		   "dir:%d, Invalid or not allocated index, type:%d, idx:%d\n",
-		   parms->dir,
+		TFP_DRV_LOG(ERR,
+		   "%s, Invalid or not allocated index, type:%d, idx:%d\n",
+		   tf_dir_2_str(parms->dir),
 		   parms->type,
 		   index);
 		return -EINVAL;
@@ -833,11 +832,11 @@ tf_set_tbl_entry_internal(struct tf *tfp,
 				  parms->data,
 				  parms->idx);
 	if (rc) {
-		PMD_DRV_LOG(ERR,
-			    "dir:%d, Set failed, type:%d, rc:%d\n",
-			    parms->dir,
+		TFP_DRV_LOG(ERR,
+			    "%s, Set failed, type:%d, rc:%s\n",
+			    tf_dir_2_str(parms->dir),
 			    parms->type,
-			    rc);
+			    strerror(-rc));
 	}
 
 	return rc;
@@ -891,9 +890,9 @@ tf_get_tbl_entry_internal(struct tf *tfp,
 	/* Verify that the entry has been previously allocated */
 	id = ba_inuse(session_pool, index);
 	if (id != 1) {
-		PMD_DRV_LOG(ERR,
-		   "dir:%d, Invalid or not allocated index, type:%d, idx:%d\n",
-		   parms->dir,
+		TFP_DRV_LOG(ERR,
+		   "%s, Invalid or not allocated index, type:%d, idx:%d\n",
+		   tf_dir_2_str(parms->dir),
 		   parms->type,
 		   index);
 		return -EINVAL;
@@ -907,11 +906,11 @@ tf_get_tbl_entry_internal(struct tf *tfp,
 				  parms->data,
 				  parms->idx);
 	if (rc) {
-		PMD_DRV_LOG(ERR,
-			    "dir:%d, Get failed, type:%d, rc:%d\n",
-			    parms->dir,
+		TFP_DRV_LOG(ERR,
+			    "%s, Get failed, type:%d, rc:%s\n",
+			    tf_dir_2_str(parms->dir),
 			    parms->type,
-			    rc);
+			    strerror(-rc));
 	}
 
 	return rc;
@@ -932,8 +931,8 @@ tf_get_tbl_entry_internal(struct tf *tfp,
  *   -EINVAL - Parameter error
  */
 static int
-tf_get_bulk_tbl_entry_internal(struct tf *tfp,
-			  struct tf_get_bulk_tbl_entry_parms *parms)
+tf_bulk_get_tbl_entry_internal(struct tf *tfp,
+			  struct tf_bulk_get_tbl_entry_parms *parms)
 {
 	int rc;
 	int id;
@@ -975,7 +974,7 @@ tf_get_bulk_tbl_entry_internal(struct tf *tfp,
 	}
 
 	/* Get the entry */
-	rc = tf_msg_get_bulk_tbl_entry(tfp, parms);
+	rc = tf_msg_bulk_get_tbl_entry(tfp, parms);
 	if (rc) {
 		TFP_DRV_LOG(ERR,
 			    "%s, Bulk get failed, type:%d, rc:%s\n",
@@ -1006,10 +1005,9 @@ static int
 tf_alloc_tbl_entry_shadow(struct tf_session *tfs __rte_unused,
 			  struct tf_alloc_tbl_entry_parms *parms __rte_unused)
 {
-	PMD_DRV_LOG(ERR,
-		    "dir:%d, Entry Alloc with search not supported\n",
-		    parms->dir);
-
+	TFP_DRV_LOG(ERR,
+		    "%s, Entry Alloc with search not supported\n",
+		    tf_dir_2_str(parms->dir));
 
 	return -EOPNOTSUPP;
 }
@@ -1032,9 +1030,9 @@ static int
 tf_free_tbl_entry_shadow(struct tf_session *tfs,
 			 struct tf_free_tbl_entry_parms *parms)
 {
-	PMD_DRV_LOG(ERR,
-		    "dir:%d, Entry Free with search not supported\n",
-		    parms->dir);
+	TFP_DRV_LOG(ERR,
+		    "%s, Entry Free with search not supported\n",
+		    tf_dir_2_str(parms->dir));
 
 	return -EOPNOTSUPP;
 }
@@ -1074,8 +1072,8 @@ tf_create_tbl_pool_external(enum tf_dir dir,
 	parms.alignment = 0;
 
 	if (tfp_calloc(&parms) != 0) {
-		PMD_DRV_LOG(ERR, "%d: TBL: external pool failure %s\n",
-			    dir, strerror(-ENOMEM));
+		TFP_DRV_LOG(ERR, "%s: TBL: external pool failure %s\n",
+			    tf_dir_2_str(dir), strerror(ENOMEM));
 		return -ENOMEM;
 	}
 
@@ -1084,8 +1082,8 @@ tf_create_tbl_pool_external(enum tf_dir dir,
 	rc = stack_init(num_entries, parms.mem_va, pool);
 
 	if (rc != 0) {
-		PMD_DRV_LOG(ERR, "%d: TBL: stack init failure %s\n",
-			    dir, strerror(-rc));
+		TFP_DRV_LOG(ERR, "%s: TBL: stack init failure %s\n",
+			    tf_dir_2_str(dir), strerror(-rc));
 		goto cleanup;
 	}
 
@@ -1101,13 +1099,13 @@ tf_create_tbl_pool_external(enum tf_dir dir,
 	for (i = 0; i < num_entries; i++) {
 		rc = stack_push(pool, j);
 		if (rc != 0) {
-			PMD_DRV_LOG(ERR, "%s TBL: stack failure %s\n",
+			TFP_DRV_LOG(ERR, "%s TBL: stack failure %s\n",
 				    tf_dir_2_str(dir), strerror(-rc));
 			goto cleanup;
 		}
 
 		if (j < 0) {
-			PMD_DRV_LOG(ERR, "%d TBL: invalid offset (%d)\n",
+			TFP_DRV_LOG(ERR, "%d TBL: invalid offset (%d)\n",
 				    dir, j);
 			goto cleanup;
 		}
@@ -1116,8 +1114,8 @@ tf_create_tbl_pool_external(enum tf_dir dir,
 
 	if (!stack_is_full(pool)) {
 		rc = -EINVAL;
-		PMD_DRV_LOG(ERR, "%d TBL: stack failure %s\n",
-			    dir, strerror(-rc));
+		TFP_DRV_LOG(ERR, "%s TBL: stack failure %s\n",
+			    tf_dir_2_str(dir), strerror(-rc));
 		goto cleanup;
 	}
 	return 0;
@@ -1168,18 +1166,7 @@ tf_alloc_tbl_entry_pool_external(struct tf *tfp,
 	struct tf_tbl_scope_cb *tbl_scope_cb;
 	struct stack *pool;
 
-	/* Check parameters */
-	if (tfp == NULL || parms == NULL) {
-		PMD_DRV_LOG(ERR, "Invalid parameters\n");
-		return -EINVAL;
-	}
-
-	if (tfp->session == NULL || tfp->session->core_data == NULL) {
-		PMD_DRV_LOG(ERR,
-			    "dir:%d, Session info invalid\n",
-			    parms->dir);
-		return -EINVAL;
-	}
+	TF_CHECK_PARMS_SESSION(tfp, parms);
 
 	tfs = (struct tf_session *)(tfp->session->core_data);
 
@@ -1188,9 +1175,9 @@ tf_alloc_tbl_entry_pool_external(struct tf *tfp,
 	tbl_scope_cb = tbl_scope_cb_find(tfs, parms->tbl_scope_id);
 
 	if (tbl_scope_cb == NULL) {
-		PMD_DRV_LOG(ERR,
-					"%s, table scope not allocated\n",
-					tf_dir_2_str(parms->dir));
+		TFP_DRV_LOG(ERR,
+			    "%s, table scope not allocated\n",
+			    tf_dir_2_str(parms->dir));
 		return -EINVAL;
 	}
 	pool = &tbl_scope_cb->ext_act_pool[parms->dir];
@@ -1200,9 +1187,9 @@ tf_alloc_tbl_entry_pool_external(struct tf *tfp,
 	rc = stack_pop(pool, &index);
 
 	if (rc != 0) {
-		PMD_DRV_LOG(ERR,
-		   "dir:%d, Allocation failed, type:%d\n",
-		   parms->dir,
+		TFP_DRV_LOG(ERR,
+		   "%s, Allocation failed, type:%d\n",
+		   tf_dir_2_str(parms->dir),
 		   parms->type);
 		return rc;
 	}
@@ -1233,18 +1220,7 @@ tf_alloc_tbl_entry_pool_internal(struct tf *tfp,
 	struct bitalloc *session_pool;
 	struct tf_session *tfs;
 
-	/* Check parameters */
-	if (tfp == NULL || parms == NULL) {
-		PMD_DRV_LOG(ERR, "Invalid parameters\n");
-		return -EINVAL;
-	}
-
-	if (tfp->session == NULL || tfp->session->core_data == NULL) {
-		PMD_DRV_LOG(ERR,
-			    "dir:%d, Session info invalid\n",
-			    parms->dir);
-		return -EINVAL;
-	}
+	TF_CHECK_PARMS_SESSION(tfp, parms);
 
 	tfs = (struct tf_session *)(tfp->session->core_data);
 
@@ -1254,11 +1230,10 @@ tf_alloc_tbl_entry_pool_internal(struct tf *tfp,
 	    parms->type != TF_TBL_TYPE_ACT_ENCAP_8B &&
 	    parms->type != TF_TBL_TYPE_ACT_ENCAP_16B &&
 	    parms->type != TF_TBL_TYPE_ACT_ENCAP_64B &&
-	    parms->type != TF_TBL_TYPE_MIRROR_CONFIG &&
 	    parms->type != TF_TBL_TYPE_ACT_STATS_64) {
-		PMD_DRV_LOG(ERR,
-			    "dir:%d, Type not supported, type:%d\n",
-			    parms->dir,
+		TFP_DRV_LOG(ERR,
+			    "%s, Type not supported, type:%d\n",
+			    tf_dir_2_str(parms->dir),
 			    parms->type);
 		return -EOPNOTSUPP;
 	}
@@ -1276,9 +1251,9 @@ tf_alloc_tbl_entry_pool_internal(struct tf *tfp,
 	if (id == -1) {
 		free_cnt = ba_free_count(session_pool);
 
-		PMD_DRV_LOG(ERR,
-		   "dir:%d, Allocation failed, type:%d, free:%d\n",
-		   parms->dir,
+		TFP_DRV_LOG(ERR,
+		   "%s, Allocation failed, type:%d, free:%d\n",
+		   tf_dir_2_str(parms->dir),
 		   parms->type,
 		   free_cnt);
 		return -ENOMEM;
@@ -1323,18 +1298,7 @@ tf_free_tbl_entry_pool_external(struct tf *tfp,
 	struct tf_tbl_scope_cb *tbl_scope_cb;
 	struct stack *pool;
 
-	/* Check parameters */
-	if (tfp == NULL || parms == NULL) {
-		PMD_DRV_LOG(ERR, "Invalid parameters\n");
-		return -EINVAL;
-	}
-
-	if (tfp->session == NULL || tfp->session->core_data == NULL) {
-		PMD_DRV_LOG(ERR,
-			    "dir:%d, Session info invalid\n",
-			    parms->dir);
-		return -EINVAL;
-	}
+	TF_CHECK_PARMS_SESSION(tfp, parms);
 
 	tfs = (struct tf_session *)(tfp->session->core_data);
 
@@ -1343,9 +1307,9 @@ tf_free_tbl_entry_pool_external(struct tf *tfp,
 	tbl_scope_cb = tbl_scope_cb_find(tfs, parms->tbl_scope_id);
 
 	if (tbl_scope_cb == NULL) {
-		PMD_DRV_LOG(ERR,
-			    "dir:%d, Session info invalid\n",
-			    parms->dir);
+		TFP_DRV_LOG(ERR,
+			    "%s, table scope error\n",
+			    tf_dir_2_str(parms->dir));
 		return -EINVAL;
 	}
 	pool = &tbl_scope_cb->ext_act_pool[parms->dir];
@@ -1355,9 +1319,9 @@ tf_free_tbl_entry_pool_external(struct tf *tfp,
 	rc = stack_push(pool, index);
 
 	if (rc != 0) {
-		PMD_DRV_LOG(ERR,
-		   "dir:%d, consistency error, stack full, type:%d, idx:%d\n",
-		   parms->dir,
+		TFP_DRV_LOG(ERR,
+		   "%s, consistency error, stack full, type:%d, idx:%d\n",
+		   tf_dir_2_str(parms->dir),
 		   parms->type,
 		   index);
 	}
@@ -1386,18 +1350,7 @@ tf_free_tbl_entry_pool_internal(struct tf *tfp,
 	struct tf_session *tfs;
 	uint32_t index;
 
-	/* Check parameters */
-	if (tfp == NULL || parms == NULL) {
-		PMD_DRV_LOG(ERR, "Invalid parameters\n");
-		return -EINVAL;
-	}
-
-	if (tfp->session == NULL || tfp->session->core_data == NULL) {
-		PMD_DRV_LOG(ERR,
-			    "dir:%d, Session info invalid\n",
-			    parms->dir);
-		return -EINVAL;
-	}
+	TF_CHECK_PARMS_SESSION(tfp, parms);
 
 	tfs = (struct tf_session *)(tfp->session->core_data);
 
@@ -1408,9 +1361,9 @@ tf_free_tbl_entry_pool_internal(struct tf *tfp,
 	    parms->type != TF_TBL_TYPE_ACT_ENCAP_16B &&
 	    parms->type != TF_TBL_TYPE_ACT_ENCAP_64B &&
 	    parms->type != TF_TBL_TYPE_ACT_STATS_64) {
-		PMD_DRV_LOG(ERR,
-			    "dir:%d, Type not supported, type:%d\n",
-			    parms->dir,
+		TFP_DRV_LOG(ERR,
+			    "%s, Type not supported, type:%d\n",
+			    tf_dir_2_str(parms->dir),
 			    parms->type);
 		return -EOPNOTSUPP;
 	}
@@ -1439,9 +1392,9 @@ tf_free_tbl_entry_pool_internal(struct tf *tfp,
 	/* Check if element was indeed allocated */
 	id = ba_inuse_free(session_pool, index);
 	if (id == -1) {
-		PMD_DRV_LOG(ERR,
-		   "dir:%d, Element not previously alloc'ed, type:%d, idx:%d\n",
-		   parms->dir,
+		TFP_DRV_LOG(ERR,
+		   "%s, Element not previously alloc'ed, type:%d, idx:%d\n",
+		   tf_dir_2_str(parms->dir),
 		   parms->type,
 		   index);
 		return -ENOMEM;
@@ -1485,8 +1438,10 @@ tf_free_eem_tbl_scope_cb(struct tf *tfp,
 	tbl_scope_cb = tbl_scope_cb_find(session,
 					 parms->tbl_scope_id);
 
-	if (tbl_scope_cb == NULL)
+	if (tbl_scope_cb == NULL) {
+		TFP_DRV_LOG(ERR, "Table scope error\n");
 		return -EINVAL;
+	}
 
 	/* Free Table control block */
 	ba_free(session->tbl_scope_pool_rx, tbl_scope_cb->index);
@@ -1516,23 +1471,17 @@ tf_alloc_eem_tbl_scope(struct tf *tfp,
 	int rc;
 	enum tf_dir dir;
 	struct tf_tbl_scope_cb *tbl_scope_cb;
-	struct tf_em_table *em_tables;
+	struct hcapi_cfa_em_table *em_tables;
 	int index;
 	struct tf_session *session;
 	struct tf_free_tbl_scope_parms free_parms;
 
-	/* check parameters */
-	if (parms == NULL || tfp->session == NULL) {
-		PMD_DRV_LOG(ERR, "TBL: Invalid parameters\n");
-		return -EINVAL;
-	}
-
 	session = (struct tf_session *)tfp->session->core_data;
 
 	/* Get Table Scope control block from the session pool */
 	index = ba_alloc(session->tbl_scope_pool_rx);
 	if (index == -1) {
-		PMD_DRV_LOG(ERR, "EEM: Unable to allocate table scope "
+		TFP_DRV_LOG(ERR, "EEM: Unable to allocate table scope "
 			    "Control Block\n");
 		return -ENOMEM;
 	}
@@ -1547,8 +1496,10 @@ tf_alloc_eem_tbl_scope(struct tf *tfp,
 				     dir,
 				     &tbl_scope_cb->em_caps[dir]);
 		if (rc) {
-			PMD_DRV_LOG(ERR,
-				"EEM: Unable to query for EEM capability\n");
+			TFP_DRV_LOG(ERR,
+				    "EEM: Unable to query for EEM capability,"
+				    " rc:%s\n",
+				    strerror(-rc));
 			goto cleanup;
 		}
 	}
@@ -1565,8 +1516,10 @@ tf_alloc_eem_tbl_scope(struct tf *tfp,
 		 */
 		rc = tf_em_ctx_reg(tfp, tbl_scope_cb, dir);
 		if (rc) {
-			PMD_DRV_LOG(ERR,
-				    "EEM: Unable to register for EEM ctx\n");
+			TFP_DRV_LOG(ERR,
+				    "EEM: Unable to register for EEM ctx,"
+				    " rc:%s\n",
+				    strerror(-rc));
 			goto cleanup;
 		}
 
@@ -1580,8 +1533,10 @@ tf_alloc_eem_tbl_scope(struct tf *tfp,
 				   parms->hw_flow_cache_flush_timer,
 				   dir);
 		if (rc) {
-			PMD_DRV_LOG(ERR,
-				"TBL: Unable to configure EEM in firmware\n");
+			TFP_DRV_LOG(ERR,
+				    "TBL: Unable to configure EEM in firmware"
+				    " rc:%s\n",
+				    strerror(-rc));
 			goto cleanup_full;
 		}
 
@@ -1590,8 +1545,10 @@ tf_alloc_eem_tbl_scope(struct tf *tfp,
 				  HWRM_TF_EXT_EM_OP_INPUT_OP_EXT_EM_ENABLE);
 
 		if (rc) {
-			PMD_DRV_LOG(ERR,
-				    "EEM: Unable to enable EEM in firmware\n");
+			TFP_DRV_LOG(ERR,
+				    "EEM: Unable to enable EEM in firmware"
+				    " rc:%s\n",
+				    strerror(-rc));
 			goto cleanup_full;
 		}
 
@@ -1604,9 +1561,9 @@ tf_alloc_eem_tbl_scope(struct tf *tfp,
 					    em_tables[TF_RECORD_TABLE].num_entries,
 					    em_tables[TF_RECORD_TABLE].entry_size);
 		if (rc) {
-			PMD_DRV_LOG(ERR,
-				    "%d TBL: Unable to allocate idx pools %s\n",
-				    dir,
+			TFP_DRV_LOG(ERR,
+				    "%s TBL: Unable to allocate idx pools %s\n",
+				    tf_dir_2_str(dir),
 				    strerror(-rc));
 			goto cleanup_full;
 		}
@@ -1634,13 +1591,12 @@ tf_set_tbl_entry(struct tf *tfp,
 	struct tf_tbl_scope_cb *tbl_scope_cb;
 	struct tf_session *session;
 
-	if (tfp == NULL || parms == NULL || parms->data == NULL)
-		return -EINVAL;
+	TF_CHECK_PARMS_SESSION(tfp, parms);
 
-	if (tfp->session == NULL || tfp->session->core_data == NULL) {
-		PMD_DRV_LOG(ERR,
-			    "dir:%d, Session info invalid\n",
-			    parms->dir);
+	if (parms->data == NULL) {
+		TFP_DRV_LOG(ERR,
+			    "%s, invalid parms->data\n",
+			    tf_dir_2_str(parms->dir));
 		return -EINVAL;
 	}
 
@@ -1654,9 +1610,9 @@ tf_set_tbl_entry(struct tf *tfp,
 		tbl_scope_id = parms->tbl_scope_id;
 
 		if (tbl_scope_id == TF_TBL_SCOPE_INVALID)  {
-			PMD_DRV_LOG(ERR,
-				    "dir:%d, Table scope not allocated\n",
-				    parms->dir);
+			TFP_DRV_LOG(ERR,
+				    "%s, Table scope not allocated\n",
+				    tf_dir_2_str(parms->dir));
 			return -EINVAL;
 		}
 
@@ -1665,18 +1621,22 @@ tf_set_tbl_entry(struct tf *tfp,
 		 */
 		tbl_scope_cb = tbl_scope_cb_find(session, tbl_scope_id);
 
-		if (tbl_scope_cb == NULL)
-			return -EINVAL;
+		if (tbl_scope_cb == NULL) {
+			TFP_DRV_LOG(ERR,
+				    "%s, table scope error\n",
+				    tf_dir_2_str(parms->dir));
+				return -EINVAL;
+		}
 
 		/* External table, implicitly the Action table */
-		base_addr = tf_em_get_table_page(tbl_scope_cb,
-						 parms->dir,
-						 offset,
-						 TF_RECORD_TABLE);
+		base_addr = (void *)(uintptr_t)hcapi_get_table_page(
+			&tbl_scope_cb->em_ctx_info[parms->dir].em_tables[TF_RECORD_TABLE],
+			offset);
+
 		if (base_addr == NULL) {
-			PMD_DRV_LOG(ERR,
-				    "dir:%d, Base address lookup failed\n",
-				    parms->dir);
+			TFP_DRV_LOG(ERR,
+				    "%s, Base address lookup failed\n",
+				    tf_dir_2_str(parms->dir));
 			return -EINVAL;
 		}
 
@@ -1688,11 +1648,11 @@ tf_set_tbl_entry(struct tf *tfp,
 		/* Internal table type processing */
 		rc = tf_set_tbl_entry_internal(tfp, parms);
 		if (rc) {
-			PMD_DRV_LOG(ERR,
-				    "dir:%d, Set failed, type:%d, rc:%d\n",
-				    parms->dir,
+			TFP_DRV_LOG(ERR,
+				    "%s, Set failed, type:%d, rc:%s\n",
+				    tf_dir_2_str(parms->dir),
 				    parms->type,
-				    rc);
+				    strerror(-rc));
 		}
 	}
 
@@ -1706,31 +1666,24 @@ tf_get_tbl_entry(struct tf *tfp,
 {
 	int rc = 0;
 
-	if (tfp == NULL || parms == NULL)
-		return -EINVAL;
-
-	if (tfp->session == NULL || tfp->session->core_data == NULL) {
-		PMD_DRV_LOG(ERR,
-			    "dir:%d, Session info invalid\n",
-			    parms->dir);
-		return -EINVAL;
-	}
+	TF_CHECK_PARMS_SESSION(tfp, parms);
 
 	if (parms->type == TF_TBL_TYPE_EXT) {
-		PMD_DRV_LOG(ERR,
-			    "dir:%d, External table type not supported\n",
-			    parms->dir);
+		/* Not supported, yet */
+		TFP_DRV_LOG(ERR,
+			    "%s, External table type not supported\n",
+			    tf_dir_2_str(parms->dir));
 
 		rc = -EOPNOTSUPP;
 	} else {
 		/* Internal table type processing */
 		rc = tf_get_tbl_entry_internal(tfp, parms);
 		if (rc)
-			PMD_DRV_LOG(ERR,
-				    "dir:%d, Get failed, type:%d, rc:%d\n",
-				    parms->dir,
+			TFP_DRV_LOG(ERR,
+				    "%s, Get failed, type:%d, rc:%s\n",
+				    tf_dir_2_str(parms->dir),
 				    parms->type,
-				    rc);
+				    strerror(-rc));
 	}
 
 	return rc;
@@ -1738,8 +1691,8 @@ tf_get_tbl_entry(struct tf *tfp,
 
 /* API defined in tf_core.h */
 int
-tf_get_bulk_tbl_entry(struct tf *tfp,
-		 struct tf_get_bulk_tbl_entry_parms *parms)
+tf_bulk_get_tbl_entry(struct tf *tfp,
+		 struct tf_bulk_get_tbl_entry_parms *parms)
 {
 	int rc = 0;
 
@@ -1754,7 +1707,7 @@ tf_get_bulk_tbl_entry(struct tf *tfp,
 		rc = -EOPNOTSUPP;
 	} else {
 		/* Internal table type processing */
-		rc = tf_get_bulk_tbl_entry_internal(tfp, parms);
+		rc = tf_bulk_get_tbl_entry_internal(tfp, parms);
 		if (rc)
 			TFP_DRV_LOG(ERR,
 				    "%s, Bulk get failed, type:%d, rc:%s\n",
@@ -1773,11 +1726,7 @@ tf_alloc_tbl_scope(struct tf *tfp,
 {
 	int rc;
 
-	/* check parameters */
-	if (parms == NULL || tfp == NULL) {
-		PMD_DRV_LOG(ERR, "TBL: Invalid parameters\n");
-		return -EINVAL;
-	}
+	TF_CHECK_PARMS_SESSION_NO_DIR(tfp, parms);
 
 	rc = tf_alloc_eem_tbl_scope(tfp, parms);
 
@@ -1791,11 +1740,7 @@ tf_free_tbl_scope(struct tf *tfp,
 {
 	int rc;
 
-	/* check parameters */
-	if (parms == NULL || tfp == NULL) {
-		PMD_DRV_LOG(ERR, "TBL: Invalid parameters\n");
-		return -EINVAL;
-	}
+	TF_CHECK_PARMS_SESSION_NO_DIR(tfp, parms);
 
 	/* free table scope and all associated resources */
 	rc = tf_free_eem_tbl_scope_cb(tfp, parms);
@@ -1813,11 +1758,7 @@ tf_alloc_tbl_entry(struct tf *tfp,
 	struct tf_session *tfs;
 #endif /* TF_SHADOW */
 
-	/* Check parameters */
-	if (parms == NULL || tfp == NULL) {
-		PMD_DRV_LOG(ERR, "TBL: Invalid parameters\n");
-		return -EINVAL;
-	}
+	TF_CHECK_PARMS_SESSION(tfp, parms);
 	/*
 	 * No shadow copy support for external tables, allocate and return
 	 */
@@ -1827,13 +1768,6 @@ tf_alloc_tbl_entry(struct tf *tfp,
 	}
 
 #if (TF_SHADOW == 1)
-	if (tfp->session == NULL || tfp->session->core_data == NULL) {
-		PMD_DRV_LOG(ERR,
-			    "dir:%d, Session info invalid\n",
-			    parms->dir);
-		return -EINVAL;
-	}
-
 	tfs = (struct tf_session *)(tfp->session->core_data);
 
 	/* Search the Shadow DB for requested element. If not found go
@@ -1849,9 +1783,9 @@ tf_alloc_tbl_entry(struct tf *tfp,
 
 	rc = tf_alloc_tbl_entry_pool_internal(tfp, parms);
 	if (rc)
-		PMD_DRV_LOG(ERR, "dir%d, Alloc failed, rc:%d\n",
-			    parms->dir,
-			    rc);
+		TFP_DRV_LOG(ERR, "%s, Alloc failed, rc:%s\n",
+			    tf_dir_2_str(parms->dir),
+			    strerror(-rc));
 
 	return rc;
 }
@@ -1866,11 +1800,8 @@ tf_free_tbl_entry(struct tf *tfp,
 	struct tf_session *tfs;
 #endif /* TF_SHADOW */
 
-	/* Check parameters */
-	if (parms == NULL || tfp == NULL) {
-		PMD_DRV_LOG(ERR, "TBL: Invalid parameters\n");
-		return -EINVAL;
-	}
+	TF_CHECK_PARMS_SESSION(tfp, parms);
+
 	/*
 	 * No shadow of external tables so just free the entry
 	 */
@@ -1880,13 +1811,6 @@ tf_free_tbl_entry(struct tf *tfp,
 	}
 
 #if (TF_SHADOW == 1)
-	if (tfp->session == NULL || tfp->session->core_data == NULL) {
-		PMD_DRV_LOG(ERR,
-			    "dir:%d, Session info invalid\n",
-			    parms->dir);
-		return -EINVAL;
-	}
-
 	tfs = (struct tf_session *)(tfp->session->core_data);
 
 	/* Search the Shadow DB for requested element. If not found go
@@ -1903,16 +1827,16 @@ tf_free_tbl_entry(struct tf *tfp,
 	rc = tf_free_tbl_entry_pool_internal(tfp, parms);
 
 	if (rc)
-		PMD_DRV_LOG(ERR, "dir:%d, Alloc failed, rc:%d\n",
-			    parms->dir,
-			    rc);
+		TFP_DRV_LOG(ERR, "%s, Alloc failed, rc:%s\n",
+			    tf_dir_2_str(parms->dir),
+			    strerror(-rc));
 	return rc;
 }
 
 
 static void
-tf_dump_link_page_table(struct tf_em_page_tbl *tp,
-			struct tf_em_page_tbl *tp_next)
+tf_dump_link_page_table(struct hcapi_cfa_em_page_tbl *tp,
+			struct hcapi_cfa_em_page_tbl *tp_next)
 {
 	uint64_t *pg_va;
 	uint32_t i;
@@ -1951,9 +1875,9 @@ void tf_dump_dma(struct tf *tfp, uint32_t tbl_scope_id)
 {
 	struct tf_session      *session;
 	struct tf_tbl_scope_cb *tbl_scope_cb;
-	struct tf_em_page_tbl *tp;
-	struct tf_em_page_tbl *tp_next;
-	struct tf_em_table *tbl;
+	struct hcapi_cfa_em_page_tbl *tp;
+	struct hcapi_cfa_em_page_tbl *tp_next;
+	struct hcapi_cfa_em_table *tbl;
 	int i;
 	int j;
 	int dir;
@@ -1967,7 +1891,7 @@ void tf_dump_dma(struct tf *tfp, uint32_t tbl_scope_id)
 	tbl_scope_cb = tbl_scope_cb_find(session,
 					 tbl_scope_id);
 	if (tbl_scope_cb == NULL)
-		TFP_DRV_LOG(ERR, "No table scope\n");
+		PMD_DRV_LOG(ERR, "No table scope\n");
 
 	for (dir = 0; dir < TF_DIR_MAX; dir++) {
 		printf("Direction %s:\n", (dir == TF_DIR_RX ? "Rx" : "Tx"));
diff --git a/drivers/net/bnxt/tf_core/tf_tbl.h b/drivers/net/bnxt/tf_core/tf_tbl.h
index d78e4fe..2b7456f 100644
--- a/drivers/net/bnxt/tf_core/tf_tbl.h
+++ b/drivers/net/bnxt/tf_core/tf_tbl.h
@@ -13,45 +13,6 @@
 
 struct tf_session;
 
-enum tf_pg_tbl_lvl {
-	TF_PT_LVL_0,
-	TF_PT_LVL_1,
-	TF_PT_LVL_2,
-	TF_PT_LVL_MAX
-};
-
-enum tf_em_table_type {
-	TF_KEY0_TABLE,
-	TF_KEY1_TABLE,
-	TF_RECORD_TABLE,
-	TF_EFC_TABLE,
-	TF_MAX_TABLE
-};
-
-struct tf_em_page_tbl {
-	uint32_t	pg_count;
-	uint32_t	pg_size;
-	void		**pg_va_tbl;
-	uint64_t	*pg_pa_tbl;
-};
-
-struct tf_em_table {
-	int				type;
-	uint32_t			num_entries;
-	uint16_t			ctx_id;
-	uint32_t			entry_size;
-	int				num_lvl;
-	uint32_t			page_cnt[TF_PT_LVL_MAX];
-	uint64_t			num_data_pages;
-	void				*l0_addr;
-	uint64_t			l0_dma_addr;
-	struct tf_em_page_tbl pg_tbl[TF_PT_LVL_MAX];
-};
-
-struct tf_em_ctx_mem_info {
-	struct tf_em_table		em_tables[TF_MAX_TABLE];
-};
-
 /** table scope control block content */
 struct tf_em_caps {
 	uint32_t flags;
@@ -74,18 +35,14 @@ struct tf_em_caps {
 struct tf_tbl_scope_cb {
 	uint32_t tbl_scope_id;
 	int index;
-	struct tf_em_ctx_mem_info  em_ctx_info[TF_DIR_MAX];
+	struct hcapi_cfa_em_ctx_mem_info  em_ctx_info[TF_DIR_MAX];
 	struct tf_em_caps          em_caps[TF_DIR_MAX];
 	struct stack               ext_act_pool[TF_DIR_MAX];
 	uint32_t                  *ext_act_pool_mem[TF_DIR_MAX];
 };
 
-/**
- * Hardware Page sizes supported for EEM:
- *   4K, 8K, 64K, 256K, 1M, 2M, 4M, 1G.
- *
- * Round-down other page sizes to the lower hardware page
- * size supported.
+/** Hardware Page sizes supported for EEM: 4K, 8K, 64K, 256K, 1M, 2M, 4M, 1G.
+ * Round-down other page sizes to the lower hardware page size supported.
  */
 #define TF_EM_PAGE_SIZE_4K 12
 #define TF_EM_PAGE_SIZE_8K 13
-- 
2.7.4


^ permalink raw reply	[flat|nested] 271+ messages in thread

* [dpdk-dev] [PATCH 17/50] net/bnxt: implement support for TCAM access
  2020-06-12 13:28 [dpdk-dev] [PATCH 00/50] add features for host-based flow management Somnath Kotur
                   ` (15 preceding siblings ...)
  2020-06-12 13:29 ` [dpdk-dev] [PATCH 16/50] net/bnxt: add core changes for EM and EEM lookups Somnath Kotur
@ 2020-06-12 13:29 ` Somnath Kotur
  2020-06-12 13:29 ` [dpdk-dev] [PATCH 18/50] net/bnxt: multiple device implementation Somnath Kotur
                   ` (33 subsequent siblings)
  50 siblings, 0 replies; 271+ messages in thread
From: Somnath Kotur @ 2020-06-12 13:29 UTC (permalink / raw)
  To: dev; +Cc: ferruh.yigit

From: Jay Ding <jay.ding@broadcom.com>

Implement TCAM alloc, free, bind, and unbind functions
Update tf_core, tf_msg, etc.

Signed-off-by: Jay Ding <jay.ding@broadcom.com>
Reviewed-by: Randy Schacher <stuart.schacher@broadcom.com>
Reviewed-by: Ajit Kumar Khaparde <ajit.khaparde@broadcom.com>
Signed-off-by: Venkat Duvvuru <venkatkumar.duvvuru@broadcom.com>
---
 drivers/net/bnxt/tf_core/tf_core.c      | 258 +++++++++++++----------------
 drivers/net/bnxt/tf_core/tf_device.h    |  14 +-
 drivers/net/bnxt/tf_core/tf_device_p4.c |  25 +--
 drivers/net/bnxt/tf_core/tf_msg.c       |  31 ++--
 drivers/net/bnxt/tf_core/tf_msg.h       |   4 +-
 drivers/net/bnxt/tf_core/tf_tcam.c      | 285 +++++++++++++++++++++++++++++++-
 drivers/net/bnxt/tf_core/tf_tcam.h      |  66 ++++++--
 7 files changed, 480 insertions(+), 203 deletions(-)

diff --git a/drivers/net/bnxt/tf_core/tf_core.c b/drivers/net/bnxt/tf_core/tf_core.c
index 648d0d1..29522c6 100644
--- a/drivers/net/bnxt/tf_core/tf_core.c
+++ b/drivers/net/bnxt/tf_core/tf_core.c
@@ -19,43 +19,6 @@
 #include "tf_common.h"
 #include "hwrm_tf.h"
 
-static int tf_check_tcam_entry(enum tf_tcam_tbl_type tcam_tbl_type,
-			       enum tf_device_type device,
-			       uint16_t key_sz_in_bits,
-			       uint16_t *num_slice_per_row)
-{
-	uint16_t key_bytes;
-	uint16_t slice_sz = 0;
-
-#define CFA_P4_WC_TCAM_SLICES_PER_ROW 2
-#define CFA_P4_WC_TCAM_SLICE_SIZE     12
-
-	if (tcam_tbl_type == TF_TCAM_TBL_TYPE_WC_TCAM) {
-		key_bytes = TF_BITS2BYTES_WORD_ALIGN(key_sz_in_bits);
-		if (device == TF_DEVICE_TYPE_WH) {
-			slice_sz = CFA_P4_WC_TCAM_SLICE_SIZE;
-			*num_slice_per_row = CFA_P4_WC_TCAM_SLICES_PER_ROW;
-		} else {
-			TFP_DRV_LOG(ERR,
-				    "Unsupported device type %d\n",
-				    device);
-			return -ENOTSUP;
-		}
-
-		if (key_bytes > *num_slice_per_row * slice_sz) {
-			TFP_DRV_LOG(ERR,
-				    "%s: Key size %d is not supported\n",
-				    tf_tcam_tbl_2_str(tcam_tbl_type),
-				    key_bytes);
-			return -ENOTSUP;
-		}
-	} else { /* for other type of tcam */
-		*num_slice_per_row = 1;
-	}
-
-	return 0;
-}
-
 /**
  * Create EM Tbl pool of memory indexes.
  *
@@ -918,49 +881,56 @@ tf_alloc_tcam_entry(struct tf *tfp,
 		    struct tf_alloc_tcam_entry_parms *parms)
 {
 	int rc;
-	int index;
 	struct tf_session *tfs;
-	struct bitalloc *session_pool;
-	uint16_t num_slice_per_row;
-
-	/* TEMP, due to device design. When tcam is modularized device
-	 * should be retrieved from the session
-	 */
-	enum tf_device_type device_type;
-	/* TEMP */
-	device_type = TF_DEVICE_TYPE_WH;
+	struct tf_dev_info *dev;
+	struct tf_tcam_alloc_parms aparms = { 0 };
 
-	TF_CHECK_PARMS_SESSION(tfp, parms);
+	TF_CHECK_PARMS2(tfp, parms);
 
-	tfs = (struct tf_session *)(tfp->session->core_data);
+	/* Retrieve the session information */
+	rc = tf_session_get_session(tfp, &tfs);
+	if (rc) {
+		TFP_DRV_LOG(ERR,
+			    "%s: Failed to lookup session, rc:%s\n",
+			    tf_dir_2_str(parms->dir),
+			    strerror(-rc));
+		return rc;
+	}
 
-	rc = tf_check_tcam_entry(parms->tcam_tbl_type,
-				 device_type,
-				 parms->key_sz_in_bits,
-				 &num_slice_per_row);
-	/* Error logging handled by tf_check_tcam_entry */
-	if (rc)
+	/* Retrieve the device information */
+	rc = tf_session_get_device(tfs, &dev);
+	if (rc) {
+		TFP_DRV_LOG(ERR,
+			    "%s: Failed to lookup device, rc:%s\n",
+			    tf_dir_2_str(parms->dir),
+			    strerror(-rc));
 		return rc;
+	}
 
-	rc = tf_rm_lookup_tcam_type_pool(tfs,
-					 parms->dir,
-					 parms->tcam_tbl_type,
-					 &session_pool);
-	/* Error logging handled by tf_rm_lookup_tcam_type_pool */
-	if (rc)
+	if (dev->ops->tf_dev_alloc_tcam == NULL) {
+		rc = -EOPNOTSUPP;
+		TFP_DRV_LOG(ERR,
+			    "%s: Operation not supported, rc:%s\n",
+			    tf_dir_2_str(parms->dir),
+			    strerror(-rc));
 		return rc;
+	}
 
-	index = ba_alloc(session_pool);
-	if (index == BA_FAIL) {
-		TFP_DRV_LOG(ERR, "%s: %s: No resource available\n",
+	aparms.dir = parms->dir;
+	aparms.type = parms->tcam_tbl_type;
+	aparms.key_size = TF_BITS2BYTES_WORD_ALIGN(parms->key_sz_in_bits);
+	aparms.priority = parms->priority;
+	rc = dev->ops->tf_dev_alloc_tcam(tfp, &aparms);
+	if (rc) {
+		TFP_DRV_LOG(ERR,
+			    "%s: TCAM allocation failed, rc:%s\n",
 			    tf_dir_2_str(parms->dir),
-			    tf_tcam_tbl_2_str(parms->tcam_tbl_type));
-		return -ENOMEM;
+			    strerror(-rc));
+		return rc;
 	}
 
-	index *= num_slice_per_row;
+	parms->idx = aparms.idx;
 
-	parms->idx = index;
 	return 0;
 }
 
@@ -969,55 +939,60 @@ tf_set_tcam_entry(struct tf *tfp,
 		  struct tf_set_tcam_entry_parms *parms)
 {
 	int rc;
-	int id;
-	int index;
 	struct tf_session *tfs;
-	struct bitalloc *session_pool;
-	uint16_t num_slice_per_row;
-
-	/* TEMP, due to device design. When tcam is modularized device
-	 * should be retrieved from the session
-	 */
-	enum tf_device_type device_type;
-	/* TEMP */
-	device_type = TF_DEVICE_TYPE_WH;
+	struct tf_dev_info *dev;
+	struct tf_tcam_set_parms sparms = { 0 };
 
-	TF_CHECK_PARMS_SESSION(tfp, parms);
+	TF_CHECK_PARMS2(tfp, parms);
 
-	tfs = (struct tf_session *)(tfp->session->core_data);
+	/* Retrieve the session information */
+	rc = tf_session_get_session(tfp, &tfs);
+	if (rc) {
+		TFP_DRV_LOG(ERR,
+			    "%s: Failed to lookup session, rc:%s\n",
+			    tf_dir_2_str(parms->dir),
+			    strerror(-rc));
+		return rc;
+	}
 
-	rc = tf_check_tcam_entry(parms->tcam_tbl_type,
-				 device_type,
-				 parms->key_sz_in_bits,
-				 &num_slice_per_row);
-	/* Error logging handled by tf_check_tcam_entry */
-	if (rc)
+	/* Retrieve the device information */
+	rc = tf_session_get_device(tfs, &dev);
+	if (rc) {
+		TFP_DRV_LOG(ERR,
+			    "%s: Failed to lookup device, rc:%s\n",
+			    tf_dir_2_str(parms->dir),
+			    strerror(-rc));
 		return rc;
+	}
 
-	rc = tf_rm_lookup_tcam_type_pool(tfs,
-					 parms->dir,
-					 parms->tcam_tbl_type,
-					 &session_pool);
-	/* Error logging handled by tf_rm_lookup_tcam_type_pool */
-	if (rc)
+	if (dev->ops->tf_dev_set_tcam == NULL) {
+		rc = -EOPNOTSUPP;
+		TFP_DRV_LOG(ERR,
+			    "%s: Operation not supported, rc:%s\n",
+			    tf_dir_2_str(parms->dir),
+			    strerror(-rc));
 		return rc;
+	}
 
-	/* Verify that the entry has been previously allocated */
-	index = parms->idx / num_slice_per_row;
+	sparms.dir = parms->dir;
+	sparms.type = parms->tcam_tbl_type;
+	sparms.idx = parms->idx;
+	sparms.key = parms->key;
+	sparms.mask = parms->mask;
+	sparms.key_size = TF_BITS2BYTES_WORD_ALIGN(parms->key_sz_in_bits);
+	sparms.result = parms->result;
+	sparms.result_size = TF_BITS2BYTES_WORD_ALIGN(parms->result_sz_in_bits);
 
-	id = ba_inuse(session_pool, index);
-	if (id != 1) {
+	rc = dev->ops->tf_dev_set_tcam(tfp, &sparms);
+	if (rc) {
 		TFP_DRV_LOG(ERR,
-		   "%s: %s: Invalid or not allocated index, idx:%d\n",
-		   tf_dir_2_str(parms->dir),
-		   tf_tcam_tbl_2_str(parms->tcam_tbl_type),
-		   parms->idx);
-		return -EINVAL;
+			    "%s: TCAM set failed, rc:%s\n",
+			    tf_dir_2_str(parms->dir),
+			    strerror(-rc));
+		return rc;
 	}
 
-	rc = tf_msg_tcam_entry_set(tfp, parms);
-
-	return rc;
+	return 0;
 }
 
 int
@@ -1033,59 +1008,52 @@ tf_free_tcam_entry(struct tf *tfp,
 		   struct tf_free_tcam_entry_parms *parms)
 {
 	int rc;
-	int index;
 	struct tf_session *tfs;
-	struct bitalloc *session_pool;
-	uint16_t num_slice_per_row = 1;
-
-	/* TEMP, due to device design. When tcam is modularized device
-	 * should be retrieved from the session
-	 */
-	enum tf_device_type device_type;
-	/* TEMP */
-	device_type = TF_DEVICE_TYPE_WH;
+	struct tf_dev_info *dev;
+	struct tf_tcam_free_parms fparms = { 0 };
 
-	TF_CHECK_PARMS_SESSION(tfp, parms);
-	tfs = (struct tf_session *)(tfp->session->core_data);
+	TF_CHECK_PARMS2(tfp, parms);
 
-	rc = tf_check_tcam_entry(parms->tcam_tbl_type,
-				 device_type,
-				 0,
-				 &num_slice_per_row);
-	/* Error logging handled by tf_check_tcam_entry */
-	if (rc)
+	/* Retrieve the session information */
+	rc = tf_session_get_session(tfp, &tfs);
+	if (rc) {
+		TFP_DRV_LOG(ERR,
+			    "%s: Failed to lookup session, rc:%s\n",
+			    tf_dir_2_str(parms->dir),
+			    strerror(-rc));
 		return rc;
+	}
 
-	rc = tf_rm_lookup_tcam_type_pool(tfs,
-					 parms->dir,
-					 parms->tcam_tbl_type,
-					 &session_pool);
-	/* Error logging handled by tf_rm_lookup_tcam_type_pool */
-	if (rc)
+	/* Retrieve the device information */
+	rc = tf_session_get_device(tfs, &dev);
+	if (rc) {
+		TFP_DRV_LOG(ERR,
+			    "%s: Failed to lookup device, rc:%s\n",
+			    tf_dir_2_str(parms->dir),
+			    strerror(-rc));
 		return rc;
+	}
 
-	index = parms->idx / num_slice_per_row;
-
-	rc = ba_inuse(session_pool, index);
-	if (rc == BA_FAIL || rc == BA_ENTRY_FREE) {
-		TFP_DRV_LOG(ERR, "%s: %s: Entry %d already free",
+	if (dev->ops->tf_dev_free_tcam == NULL) {
+		rc = -EOPNOTSUPP;
+		TFP_DRV_LOG(ERR,
+			    "%s: Operation not supported, rc:%s\n",
 			    tf_dir_2_str(parms->dir),
-			    tf_tcam_tbl_2_str(parms->tcam_tbl_type),
-			    index);
-		return -EINVAL;
+			    strerror(-rc));
+		return rc;
 	}
 
-	ba_free(session_pool, index);
-
-	rc = tf_msg_tcam_entry_free(tfp, parms);
+	fparms.dir = parms->dir;
+	fparms.type = parms->tcam_tbl_type;
+	fparms.idx = parms->idx;
+	rc = dev->ops->tf_dev_free_tcam(tfp, &fparms);
 	if (rc) {
-		/* Log error */
-		TFP_DRV_LOG(ERR, "%s: %s: Entry %d free failed with err %s",
+		TFP_DRV_LOG(ERR,
+			    "%s: TCAM allocation failed, rc:%s\n",
 			    tf_dir_2_str(parms->dir),
-			    tf_tcam_tbl_2_str(parms->tcam_tbl_type),
-			    parms->idx,
 			    strerror(-rc));
+		return rc;
 	}
 
-	return rc;
+	return 0;
 }
diff --git a/drivers/net/bnxt/tf_core/tf_device.h b/drivers/net/bnxt/tf_core/tf_device.h
index 1501b20..32d9a54 100644
--- a/drivers/net/bnxt/tf_core/tf_device.h
+++ b/drivers/net/bnxt/tf_core/tf_device.h
@@ -116,8 +116,11 @@ struct tf_dev_ops {
 	 * [in] tfp
 	 *   Pointer to TF handle
 	 *
-	 * [out] slice_size
-	 *   Pointer to slice size the device supports
+	 * [in] type
+	 *   TCAM table type
+	 *
+	 * [in] key_sz
+	 *   Key size
 	 *
 	 * [out] num_slices_per_row
 	 *   Pointer to number of slices per row the device supports
@@ -126,9 +129,10 @@ struct tf_dev_ops {
 	 *   - (0) if successful.
 	 *   - (-EINVAL) on failure.
 	 */
-	int (*tf_dev_get_wc_tcam_slices)(struct tf *tfp,
-					 uint16_t *slice_size,
-					 uint16_t *num_slices_per_row);
+	int (*tf_dev_get_tcam_slice_info)(struct tf *tfp,
+					  enum tf_tcam_tbl_type type,
+					  uint16_t key_sz,
+					  uint16_t *num_slices_per_row);
 
 	/**
 	 * Allocation of an identifier element.
diff --git a/drivers/net/bnxt/tf_core/tf_device_p4.c b/drivers/net/bnxt/tf_core/tf_device_p4.c
index f4bd95f..77fb693 100644
--- a/drivers/net/bnxt/tf_core/tf_device_p4.c
+++ b/drivers/net/bnxt/tf_core/tf_device_p4.c
@@ -56,18 +56,21 @@ tf_dev_p4_get_max_types(struct tf *tfp __rte_unused,
  *   - (-EINVAL) on failure.
  */
 static int
-tf_dev_p4_get_wc_tcam_slices(struct tf *tfp __rte_unused,
-			     uint16_t *slice_size,
-			     uint16_t *num_slices_per_row)
+tf_dev_p4_get_tcam_slice_info(struct tf *tfp __rte_unused,
+			      enum tf_tcam_tbl_type type,
+			      uint16_t key_sz,
+			      uint16_t *num_slices_per_row)
 {
-#define CFA_P4_WC_TCAM_SLICE_SIZE       12
-#define CFA_P4_WC_TCAM_SLICES_PER_ROW    2
+#define CFA_P4_WC_TCAM_SLICES_PER_ROW 2
+#define CFA_P4_WC_TCAM_SLICE_SIZE     12
 
-	if (slice_size == NULL || num_slices_per_row == NULL)
-		return -EINVAL;
-
-	*slice_size = CFA_P4_WC_TCAM_SLICE_SIZE;
-	*num_slices_per_row = CFA_P4_WC_TCAM_SLICES_PER_ROW;
+	if (type == TF_TCAM_TBL_TYPE_WC_TCAM) {
+		*num_slices_per_row = CFA_P4_WC_TCAM_SLICES_PER_ROW;
+		if (key_sz > *num_slices_per_row * CFA_P4_WC_TCAM_SLICE_SIZE)
+			return -ENOTSUP;
+	} else { /* for other type of tcam */
+		*num_slices_per_row = 1;
+	}
 
 	return 0;
 }
@@ -77,7 +80,7 @@ tf_dev_p4_get_wc_tcam_slices(struct tf *tfp __rte_unused,
  */
 const struct tf_dev_ops tf_dev_ops_p4 = {
 	.tf_dev_get_max_types = tf_dev_p4_get_max_types,
-	.tf_dev_get_wc_tcam_slices = tf_dev_p4_get_wc_tcam_slices,
+	.tf_dev_get_tcam_slice_info = tf_dev_p4_get_tcam_slice_info,
 	.tf_dev_alloc_ident = tf_ident_alloc,
 	.tf_dev_free_ident = tf_ident_free,
 	.tf_dev_alloc_tbl = tf_tbl_alloc,
diff --git a/drivers/net/bnxt/tf_core/tf_msg.c b/drivers/net/bnxt/tf_core/tf_msg.c
index 90e1acf..f223850 100644
--- a/drivers/net/bnxt/tf_core/tf_msg.c
+++ b/drivers/net/bnxt/tf_core/tf_msg.c
@@ -9,6 +9,7 @@
 #include <string.h>
 
 #include "tf_msg_common.h"
+#include "tf_device.h"
 #include "tf_msg.h"
 #include "tf_util.h"
 #include "tf_session.h"
@@ -1479,27 +1480,19 @@ tf_msg_bulk_get_tbl_entry(struct tf *tfp,
 	return tfp_le_to_cpu_32(parms.tf_resp_code);
 }
 
-#define TF_BYTES_PER_SLICE(tfp) 12
-#define NUM_SLICES(tfp, bytes) \
-	(((bytes) + TF_BYTES_PER_SLICE(tfp) - 1) / TF_BYTES_PER_SLICE(tfp))
-
 int
 tf_msg_tcam_entry_set(struct tf *tfp,
-		      struct tf_set_tcam_entry_parms *parms)
+		      struct tf_tcam_set_parms *parms)
 {
 	int rc;
 	struct tfp_send_msg_parms mparms = { 0 };
 	struct hwrm_tf_tcam_set_input req = { 0 };
 	struct hwrm_tf_tcam_set_output resp = { 0 };
-	uint16_t key_bytes =
-		TF_BITS2BYTES_WORD_ALIGN(parms->key_sz_in_bits);
-	uint16_t result_bytes =
-		TF_BITS2BYTES_WORD_ALIGN(parms->result_sz_in_bits);
 	struct tf_msg_dma_buf buf = { 0 };
 	uint8_t *data = NULL;
 	int data_size = 0;
 
-	rc = tf_tcam_tbl_2_hwrm(parms->tcam_tbl_type, &req.type);
+	rc = tf_tcam_tbl_2_hwrm(parms->type, &req.type);
 	if (rc != 0)
 		return rc;
 
@@ -1507,11 +1500,11 @@ tf_msg_tcam_entry_set(struct tf *tfp,
 	if (parms->dir == TF_DIR_TX)
 		req.flags |= HWRM_TF_TCAM_SET_INPUT_FLAGS_DIR_TX;
 
-	req.key_size = key_bytes;
-	req.mask_offset = key_bytes;
+	req.key_size = parms->key_size;
+	req.mask_offset = parms->key_size;
 	/* Result follows after key and mask, thus multiply by 2 */
-	req.result_offset = 2 * key_bytes;
-	req.result_size = result_bytes;
+	req.result_offset = 2 * parms->key_size;
+	req.result_size = parms->result_size;
 	data_size = 2 * req.key_size + req.result_size;
 
 	if (data_size <= TF_PCI_BUF_SIZE_MAX) {
@@ -1529,9 +1522,9 @@ tf_msg_tcam_entry_set(struct tf *tfp,
 			   sizeof(buf.pa_addr));
 	}
 
-	tfp_memcpy(&data[0], parms->key, key_bytes);
-	tfp_memcpy(&data[key_bytes], parms->mask, key_bytes);
-	tfp_memcpy(&data[req.result_offset], parms->result, result_bytes);
+	tfp_memcpy(&data[0], parms->key, parms->key_size);
+	tfp_memcpy(&data[parms->key_size], parms->mask, parms->key_size);
+	tfp_memcpy(&data[req.result_offset], parms->result, parms->result_size);
 
 	mparms.tf_type = HWRM_TF_TCAM_SET;
 	mparms.req_data = (uint32_t *)&req;
@@ -1553,7 +1546,7 @@ tf_msg_tcam_entry_set(struct tf *tfp,
 
 int
 tf_msg_tcam_entry_free(struct tf *tfp,
-		       struct tf_free_tcam_entry_parms *in_parms)
+		       struct tf_tcam_free_parms *in_parms)
 {
 	int rc;
 	struct hwrm_tf_tcam_free_input req =  { 0 };
@@ -1561,7 +1554,7 @@ tf_msg_tcam_entry_free(struct tf *tfp,
 	struct tfp_send_msg_parms parms = { 0 };
 
 	/* Populate the request */
-	rc = tf_tcam_tbl_2_hwrm(in_parms->tcam_tbl_type, &req.type);
+	rc = tf_tcam_tbl_2_hwrm(in_parms->type, &req.type);
 	if (rc != 0)
 		return rc;
 
diff --git a/drivers/net/bnxt/tf_core/tf_msg.h b/drivers/net/bnxt/tf_core/tf_msg.h
index 1dad2b9..a3e0f7b 100644
--- a/drivers/net/bnxt/tf_core/tf_msg.h
+++ b/drivers/net/bnxt/tf_core/tf_msg.h
@@ -247,7 +247,7 @@ int tf_msg_em_op(struct tf *tfp,
  *  0 on Success else internal Truflow error
  */
 int tf_msg_tcam_entry_set(struct tf *tfp,
-			  struct tf_set_tcam_entry_parms *parms);
+			  struct tf_tcam_set_parms *parms);
 
 /**
  * Sends tcam entry 'free' to the Firmware.
@@ -262,7 +262,7 @@ int tf_msg_tcam_entry_set(struct tf *tfp,
  *  0 on Success else internal Truflow error
  */
 int tf_msg_tcam_entry_free(struct tf *tfp,
-			   struct tf_free_tcam_entry_parms *parms);
+			   struct tf_tcam_free_parms *parms);
 
 /**
  * Sends Set message of a Table Type element to the firmware.
diff --git a/drivers/net/bnxt/tf_core/tf_tcam.c b/drivers/net/bnxt/tf_core/tf_tcam.c
index 3ad99dd..b9dba53 100644
--- a/drivers/net/bnxt/tf_core/tf_tcam.c
+++ b/drivers/net/bnxt/tf_core/tf_tcam.c
@@ -3,16 +3,24 @@
  * All rights reserved.
  */
 
+#include <string.h>
 #include <rte_common.h>
 
 #include "tf_tcam.h"
+#include "tf_common.h"
+#include "tf_util.h"
+#include "tf_rm_new.h"
+#include "tf_device.h"
+#include "tfp.h"
+#include "tf_session.h"
+#include "tf_msg.h"
 
 struct tf;
 
 /**
  * TCAM DBs.
  */
-/* static void *tcam_db[TF_DIR_MAX]; */
+static void *tcam_db[TF_DIR_MAX];
 
 /**
  * TCAM Shadow DBs
@@ -22,7 +30,7 @@ struct tf;
 /**
  * Init flag, set on bind and cleared on unbind
  */
-/* static uint8_t init; */
+static uint8_t init;
 
 /**
  * Shadow init flag, set on bind and cleared on unbind
@@ -33,19 +41,131 @@ int
 tf_tcam_bind(struct tf *tfp __rte_unused,
 	     struct tf_tcam_cfg_parms *parms __rte_unused)
 {
+	int rc;
+	int i;
+	struct tf_rm_create_db_parms db_cfg = { 0 };
+
+	TF_CHECK_PARMS2(tfp, parms);
+
+	if (init) {
+		TFP_DRV_LOG(ERR,
+			    "TCAM already initialized\n");
+		return -EINVAL;
+	}
+
+	db_cfg.num_elements = parms->num_elements;
+
+	for (i = 0; i < TF_DIR_MAX; i++) {
+		db_cfg.dir = i;
+		db_cfg.num_elements = parms->num_elements;
+		db_cfg.cfg = parms->cfg;
+		db_cfg.alloc_num = parms->resources->tcam_tbl_cnt[i];
+		db_cfg.rm_db = tcam_db[i];
+		rc = tf_rm_create_db(tfp, &db_cfg);
+		if (rc) {
+			TFP_DRV_LOG(ERR,
+				    "%s: TCAM DB creation failed\n",
+				    tf_dir_2_str(i));
+			return rc;
+		}
+	}
+
+	init = 1;
+
 	return 0;
 }
 
 int
 tf_tcam_unbind(struct tf *tfp __rte_unused)
 {
+	int rc;
+	int i;
+	struct tf_rm_free_db_parms fparms = { 0 };
+
+	TF_CHECK_PARMS1(tfp);
+
+	/* Bail if nothing has been initialized done silent as to
+	 * allow for creation cleanup.
+	 */
+	if (!init)
+		return -EINVAL;
+
+	for (i = 0; i < TF_DIR_MAX; i++) {
+		fparms.dir = i;
+		fparms.rm_db = tcam_db[i];
+		rc = tf_rm_free_db(tfp, &fparms);
+		if (rc)
+			return rc;
+
+		tcam_db[i] = NULL;
+	}
+
+	init = 0;
+
 	return 0;
 }
 
 int
-tf_tcam_alloc(struct tf *tfp __rte_unused,
-	      struct tf_tcam_alloc_parms *parms __rte_unused)
+tf_tcam_alloc(struct tf *tfp,
+	      struct tf_tcam_alloc_parms *parms)
 {
+	int rc;
+	struct tf_session *tfs;
+	struct tf_dev_info *dev;
+	struct tf_rm_allocate_parms aparms = { 0 };
+	uint16_t num_slice_per_row = 1;
+
+	TF_CHECK_PARMS2(tfp, parms);
+
+	if (!init) {
+		TFP_DRV_LOG(ERR,
+			    "%s: No TCAM DBs created\n",
+			    tf_dir_2_str(parms->dir));
+		return -EINVAL;
+	}
+
+	/* Retrieve the session information */
+	rc = tf_session_get_session(tfp, &tfs);
+	if (rc)
+		return rc;
+
+	/* Retrieve the device information */
+	rc = tf_session_get_device(tfs, &dev);
+	if (rc)
+		return rc;
+
+	if (dev->ops->tf_dev_get_tcam_slice_info == NULL) {
+		rc = -EOPNOTSUPP;
+		TFP_DRV_LOG(ERR,
+			    "%s: Operation not supported, rc:%s\n",
+			    tf_dir_2_str(parms->dir),
+			    strerror(-rc));
+		return rc;
+	}
+
+	/* Need to retrieve row size etc */
+	rc = dev->ops->tf_dev_get_tcam_slice_info(tfp,
+						  parms->type,
+						  parms->key_size,
+						  &num_slice_per_row);
+	if (rc)
+		return rc;
+
+	/* Allocate requested element */
+	aparms.rm_db = tcam_db[parms->dir];
+	aparms.db_index = parms->type;
+	aparms.index = (uint32_t *)&parms->idx;
+	rc = tf_rm_allocate(&aparms);
+	if (rc) {
+		TFP_DRV_LOG(ERR,
+			    "%s: Failed tcam, type:%d\n",
+			    tf_dir_2_str(parms->dir),
+			    parms->type);
+		return rc;
+	}
+
+	parms->idx *= num_slice_per_row;
+
 	return 0;
 }
 
@@ -53,6 +173,92 @@ int
 tf_tcam_free(struct tf *tfp __rte_unused,
 	     struct tf_tcam_free_parms *parms __rte_unused)
 {
+	int rc;
+	struct tf_session *tfs;
+	struct tf_dev_info *dev;
+	struct tf_rm_is_allocated_parms aparms = { 0 };
+	struct tf_rm_free_parms fparms = { 0 };
+	uint16_t num_slice_per_row = 1;
+	int allocated = 0;
+
+	TF_CHECK_PARMS2(tfp, parms);
+
+	if (!init) {
+		TFP_DRV_LOG(ERR,
+			    "%s: No TCAM DBs created\n",
+			    tf_dir_2_str(parms->dir));
+		return -EINVAL;
+	}
+
+	/* Retrieve the session information */
+	rc = tf_session_get_session(tfp, &tfs);
+	if (rc)
+		return rc;
+
+	/* Retrieve the device information */
+	rc = tf_session_get_device(tfs, &dev);
+	if (rc)
+		return rc;
+
+	if (dev->ops->tf_dev_get_tcam_slice_info == NULL) {
+		rc = -EOPNOTSUPP;
+		TFP_DRV_LOG(ERR,
+			    "%s: Operation not supported, rc:%s\n",
+			    tf_dir_2_str(parms->dir),
+			    strerror(-rc));
+		return rc;
+	}
+
+	/* Need to retrieve row size etc */
+	rc = dev->ops->tf_dev_get_tcam_slice_info(tfp,
+						  parms->type,
+						  0,
+						  &num_slice_per_row);
+	if (rc)
+		return rc;
+
+	/* Check if element is in use */
+	aparms.rm_db = tcam_db[parms->dir];
+	aparms.db_index = parms->type;
+	aparms.index = parms->idx / num_slice_per_row;
+	aparms.allocated = &allocated;
+	rc = tf_rm_is_allocated(&aparms);
+	if (rc)
+		return rc;
+
+	if (!allocated) {
+		TFP_DRV_LOG(ERR,
+			    "%s: Entry already free, type:%d, index:%d\n",
+			    tf_dir_2_str(parms->dir),
+			    parms->type,
+			    parms->idx);
+		return rc;
+	}
+
+	/* Free requested element */
+	fparms.rm_db = tcam_db[parms->dir];
+	fparms.db_index = parms->type;
+	fparms.index = parms->idx / num_slice_per_row;
+	rc = tf_rm_free(&fparms);
+	if (rc) {
+		TFP_DRV_LOG(ERR,
+			    "%s: Free failed, type:%d, index:%d\n",
+			    tf_dir_2_str(parms->dir),
+			    parms->type,
+			    parms->idx);
+		return rc;
+	}
+
+	rc = tf_msg_tcam_entry_free(tfp, parms);
+	if (rc) {
+		/* Log error */
+		TFP_DRV_LOG(ERR, "%s: %s: Entry %d free failed with err %s",
+			    tf_dir_2_str(parms->dir),
+			    tf_tcam_tbl_2_str(parms->type),
+			    parms->idx,
+			    strerror(-rc));
+	}
+
 	return 0;
 }
 
@@ -67,6 +273,77 @@ int
 tf_tcam_set(struct tf *tfp __rte_unused,
 	    struct tf_tcam_set_parms *parms __rte_unused)
 {
+	int rc;
+	struct tf_session *tfs;
+	struct tf_dev_info *dev;
+	struct tf_rm_is_allocated_parms aparms = { 0 };
+	uint16_t num_slice_per_row = 1;
+	int allocated = 0;
+
+	TF_CHECK_PARMS2(tfp, parms);
+
+	if (!init) {
+		TFP_DRV_LOG(ERR,
+			    "%s: No TCAM DBs created\n",
+			    tf_dir_2_str(parms->dir));
+		return -EINVAL;
+	}
+
+	/* Retrieve the session information */
+	rc = tf_session_get_session(tfp, &tfs);
+	if (rc)
+		return rc;
+
+	/* Retrieve the device information */
+	rc = tf_session_get_device(tfs, &dev);
+	if (rc)
+		return rc;
+
+	if (dev->ops->tf_dev_get_tcam_slice_info == NULL) {
+		rc = -EOPNOTSUPP;
+		TFP_DRV_LOG(ERR,
+			    "%s: Operation not supported, rc:%s\n",
+			    tf_dir_2_str(parms->dir),
+			    strerror(-rc));
+		return rc;
+	}
+
+	/* Need to retrieve row size etc */
+	rc = dev->ops->tf_dev_get_tcam_slice_info(tfp,
+						  parms->type,
+						  parms->key_size,
+						  &num_slice_per_row);
+	if (rc)
+		return rc;
+
+	/* Check if element is in use */
+	aparms.rm_db = tcam_db[parms->dir];
+	aparms.db_index = parms->type;
+	aparms.index = parms->idx / num_slice_per_row;
+	aparms.allocated = &allocated;
+	rc = tf_rm_is_allocated(&aparms);
+	if (rc)
+		return rc;
+
+	if (!allocated) {
+		TFP_DRV_LOG(ERR,
+			    "%s: Entry is not allocated, type:%d, index:%d\n",
+			    tf_dir_2_str(parms->dir),
+			    parms->type,
+			    parms->idx);
+		return rc;
+	}
+
+	rc = tf_msg_tcam_entry_set(tfp, parms);
+	if (rc) {
+		/* Log error */
+		TFP_DRV_LOG(ERR, "%s: %s: Entry %d free failed with err %s",
+			    tf_dir_2_str(parms->dir),
+			    tf_tcam_tbl_2_str(parms->type),
+			    parms->idx,
+			    strerror(-rc));
+	}
+
 	return 0;
 }
 
diff --git a/drivers/net/bnxt/tf_core/tf_tcam.h b/drivers/net/bnxt/tf_core/tf_tcam.h
index 68c25eb..67c3bcb 100644
--- a/drivers/net/bnxt/tf_core/tf_tcam.h
+++ b/drivers/net/bnxt/tf_core/tf_tcam.h
@@ -51,9 +51,17 @@ struct tf_tcam_alloc_parms {
 	 */
 	enum tf_tcam_tbl_type type;
 	/**
+	 * [in] key size
+	 */
+	uint16_t key_size;
+	/**
+	 * [in] Priority of entry requested (definition TBD)
+	 */
+	uint32_t priority;
+	/**
 	 * [out] Idx of allocated entry or found entry (if search_enable)
 	 */
-	uint32_t idx;
+	uint16_t idx;
 };
 
 /**
@@ -71,7 +79,7 @@ struct tf_tcam_free_parms {
 	/**
 	 * [in] Index to free
 	 */
-	uint32_t idx;
+	uint16_t idx;
 	/**
 	 * [out] Reference count after free, only valid if session has been
 	 * created with shadow_copy.
@@ -90,7 +98,7 @@ struct tf_tcam_alloc_search_parms {
 	/**
 	 * [in] TCAM table type
 	 */
-	enum tf_tcam_tbl_type tcam_tbl_type;
+	enum tf_tcam_tbl_type type;
 	/**
 	 * [in] Enable search for matching entry
 	 */
@@ -100,9 +108,9 @@ struct tf_tcam_alloc_search_parms {
 	 */
 	uint8_t *key;
 	/**
-	 * [in] key size in bits (if search)
+	 * [in] key size (if search)
 	 */
-	uint16_t key_sz_in_bits;
+	uint16_t key_size;
 	/**
 	 * [in] Mask data to match on (if search)
 	 */
@@ -139,17 +147,29 @@ struct tf_tcam_set_parms {
 	 */
 	enum tf_tcam_tbl_type type;
 	/**
-	 * [in] Entry data
+	 * [in] Entry index to write to
 	 */
-	uint8_t *data;
+	uint32_t idx;
 	/**
-	 * [in] Entry size
+	 * [in] array containing key
 	 */
-	uint16_t data_sz_in_bytes;
+	uint8_t *key;
 	/**
-	 * [in] Entry index to write to
+	 * [in] array containing mask fields
 	 */
-	uint32_t idx;
+	uint8_t *mask;
+	/**
+	 * [in] key size
+	 */
+	uint16_t key_size;
+	/**
+	 * [in] array containing result
+	 */
+	uint8_t *result;
+	/**
+	 * [in] result size
+	 */
+	uint16_t result_size;
 };
 
 /**
@@ -165,17 +185,29 @@ struct tf_tcam_get_parms {
 	 */
 	enum tf_tcam_tbl_type type;
 	/**
-	 * [out] Entry data
+	 * [in] Entry index to read
 	 */
-	uint8_t *data;
+	uint32_t idx;
 	/**
-	 * [out] Entry size
+	 * [out] array containing key
 	 */
-	uint16_t data_sz_in_bytes;
+	uint8_t *key;
 	/**
-	 * [in] Entry index to read
+	 * [out] array containing mask fields
 	 */
-	uint32_t idx;
+	uint8_t *mask;
+	/**
+	 * [out] key size
+	 */
+	uint16_t key_size;
+	/**
+	 * [out] array containing result
+	 */
+	uint8_t *result;
+	/**
+	 * [out] result size
+	 */
+	uint16_t result_size;
 };
 
 /**
-- 
2.7.4


^ permalink raw reply	[flat|nested] 271+ messages in thread

* [dpdk-dev] [PATCH 18/50] net/bnxt: multiple device implementation
  2020-06-12 13:28 [dpdk-dev] [PATCH 00/50] add features for host-based flow management Somnath Kotur
                   ` (16 preceding siblings ...)
  2020-06-12 13:29 ` [dpdk-dev] [PATCH 17/50] net/bnxt: implement support for TCAM access Somnath Kotur
@ 2020-06-12 13:29 ` Somnath Kotur
  2020-06-12 13:29 ` [dpdk-dev] [PATCH 19/50] net/bnxt: update identifier with remap support Somnath Kotur
                   ` (32 subsequent siblings)
  50 siblings, 0 replies; 271+ messages in thread
From: Somnath Kotur @ 2020-06-12 13:29 UTC (permalink / raw)
  To: dev; +Cc: ferruh.yigit

From: Michael Wildt <michael.wildt@broadcom.com>

Implement the Identifier, Table Type and the Resource Manager
modules.
Integrate Resource Manager with HCAPI.
Update open/close session.
Move to direct msgs for qcaps and resv messages.

Signed-off-by: Michael Wildt <michael.wildt@broadcom.com>
Reviewed-by: Randy Schacher <stuart.schacher@broadcom.com>
Signed-off-by: Venkat Duvvuru <venkatkumar.duvvuru@broadcom.com>
---
 drivers/net/bnxt/tf_core/tf_core.c       | 751 ++++++++++---------------------
 drivers/net/bnxt/tf_core/tf_core.h       |  97 ++--
 drivers/net/bnxt/tf_core/tf_device.c     |  10 +-
 drivers/net/bnxt/tf_core/tf_device.h     |   1 +
 drivers/net/bnxt/tf_core/tf_device_p4.c  |  26 +-
 drivers/net/bnxt/tf_core/tf_identifier.c |  29 +-
 drivers/net/bnxt/tf_core/tf_identifier.h |   4 +-
 drivers/net/bnxt/tf_core/tf_msg.c        |  45 +-
 drivers/net/bnxt/tf_core/tf_msg.h        |   1 +
 drivers/net/bnxt/tf_core/tf_rm.c         |   7 -
 drivers/net/bnxt/tf_core/tf_rm_new.c     | 225 +++++++--
 drivers/net/bnxt/tf_core/tf_rm_new.h     |  11 +-
 drivers/net/bnxt/tf_core/tf_session.c    |  52 +--
 drivers/net/bnxt/tf_core/tf_session.h    |   2 +-
 drivers/net/bnxt/tf_core/tf_tbl.c        | 610 -------------------------
 drivers/net/bnxt/tf_core/tf_tbl_type.c   | 252 ++++++++++-
 drivers/net/bnxt/tf_core/tf_tbl_type.h   |   2 +-
 drivers/net/bnxt/tf_core/tf_tcam.c       |  12 +-
 drivers/net/bnxt/tf_core/tf_util.h       |  45 +-
 drivers/net/bnxt/tf_core/tfp.c           |   4 +-
 20 files changed, 879 insertions(+), 1307 deletions(-)

diff --git a/drivers/net/bnxt/tf_core/tf_core.c b/drivers/net/bnxt/tf_core/tf_core.c
index 29522c6..3e23d05 100644
--- a/drivers/net/bnxt/tf_core/tf_core.c
+++ b/drivers/net/bnxt/tf_core/tf_core.c
@@ -19,284 +19,15 @@
 #include "tf_common.h"
 #include "hwrm_tf.h"
 
-/**
- * Create EM Tbl pool of memory indexes.
- *
- * [in] session
- *   Pointer to session
- * [in] dir
- *   direction
- * [in] num_entries
- *   number of entries to write
- *
- * Return:
- *  0       - Success, entry allocated - no search support
- *  -ENOMEM -EINVAL -EOPNOTSUPP
- *          - Failure, entry not allocated, out of resources
- */
-static int
-tf_create_em_pool(struct tf_session *session,
-		  enum tf_dir dir,
-		  uint32_t num_entries)
-{
-	struct tfp_calloc_parms parms;
-	uint32_t i, j;
-	int rc = 0;
-	struct stack *pool = &session->em_pool[dir];
-
-	parms.nitems = num_entries;
-	parms.size = sizeof(uint32_t);
-	parms.alignment = 0;
-
-	if (tfp_calloc(&parms) != 0) {
-		TFP_DRV_LOG(ERR, "EM pool allocation failure %s\n",
-			    strerror(-ENOMEM));
-		return -ENOMEM;
-	}
-
-	/* Create empty stack
-	 */
-	rc = stack_init(num_entries, (uint32_t *)parms.mem_va, pool);
-
-	if (rc != 0) {
-		TFP_DRV_LOG(ERR, "EM pool stack init failure %s\n",
-			    strerror(-rc));
-		goto cleanup;
-	}
-
-	/* Fill pool with indexes
-	 */
-	j = num_entries - 1;
-
-	for (i = 0; i < num_entries; i++) {
-		rc = stack_push(pool, j);
-		if (rc != 0) {
-			TFP_DRV_LOG(ERR, "EM pool stack push failure %s\n",
-				    strerror(-rc));
-			goto cleanup;
-		}
-		j--;
-	}
-
-	if (!stack_is_full(pool)) {
-		rc = -EINVAL;
-		TFP_DRV_LOG(ERR, "EM pool stack failure %s\n",
-			    strerror(-rc));
-		goto cleanup;
-	}
-
-	return 0;
-cleanup:
-	tfp_free((void *)parms.mem_va);
-	return rc;
-}
-
-/**
- * Create EM Tbl pool of memory indexes.
- *
- * [in] session
- *   Pointer to session
- * [in] dir
- *   direction
- *
- * Return:
- */
-static void
-tf_free_em_pool(struct tf_session *session,
-		enum tf_dir dir)
-{
-	struct stack *pool = &session->em_pool[dir];
-	uint32_t *ptr;
-
-	ptr = stack_items(pool);
-
-	tfp_free(ptr);
-}
-
 int
-tf_open_session(struct tf                    *tfp,
+tf_open_session(struct tf *tfp,
 		struct tf_open_session_parms *parms)
 {
 	int rc;
-	struct tf_session *session;
-	struct tfp_calloc_parms alloc_parms;
-	unsigned int domain, bus, slot, device;
-	uint8_t fw_session_id;
-	int dir;
-
-	TF_CHECK_PARMS(tfp, parms);
-
-	/* Filter out any non-supported device types on the Core
-	 * side. It is assumed that the Firmware will be supported if
-	 * firmware open session succeeds.
-	 */
-	if (parms->device_type != TF_DEVICE_TYPE_WH) {
-		TFP_DRV_LOG(ERR,
-			    "Unsupported device type %d\n",
-			    parms->device_type);
-		return -ENOTSUP;
-	}
-
-	/* Build the beginning of session_id */
-	rc = sscanf(parms->ctrl_chan_name,
-		    "%x:%x:%x.%d",
-		    &domain,
-		    &bus,
-		    &slot,
-		    &device);
-	if (rc != 4) {
-		TFP_DRV_LOG(ERR,
-			    "Failed to scan device ctrl_chan_name\n");
-		return -EINVAL;
-	}
-
-	/* open FW session and get a new session_id */
-	rc = tf_msg_session_open(tfp,
-				 parms->ctrl_chan_name,
-				 &fw_session_id);
-	if (rc) {
-		/* Log error */
-		if (rc == -EEXIST)
-			TFP_DRV_LOG(ERR,
-				    "Session is already open, rc:%s\n",
-				    strerror(-rc));
-		else
-			TFP_DRV_LOG(ERR,
-				    "Open message send failed, rc:%s\n",
-				    strerror(-rc));
-
-		parms->session_id.id = TF_FW_SESSION_ID_INVALID;
-		return rc;
-	}
-
-	/* Allocate session */
-	alloc_parms.nitems = 1;
-	alloc_parms.size = sizeof(struct tf_session_info);
-	alloc_parms.alignment = 0;
-	rc = tfp_calloc(&alloc_parms);
-	if (rc) {
-		/* Log error */
-		TFP_DRV_LOG(ERR,
-			    "Failed to allocate session info, rc:%s\n",
-			    strerror(-rc));
-		goto cleanup;
-	}
-
-	tfp->session = (struct tf_session_info *)alloc_parms.mem_va;
-
-	/* Allocate core data for the session */
-	alloc_parms.nitems = 1;
-	alloc_parms.size = sizeof(struct tf_session);
-	alloc_parms.alignment = 0;
-	rc = tfp_calloc(&alloc_parms);
-	if (rc) {
-		/* Log error */
-		TFP_DRV_LOG(ERR,
-			    "Failed to allocate session data, rc:%s\n",
-			    strerror(-rc));
-		goto cleanup;
-	}
-
-	tfp->session->core_data = alloc_parms.mem_va;
-
-	session = (struct tf_session *)tfp->session->core_data;
-	tfp_memcpy(session->ctrl_chan_name,
-		   parms->ctrl_chan_name,
-		   TF_SESSION_NAME_MAX);
-
-	/* Initialize Session */
-	session->dev = NULL;
-	tf_rm_init(tfp);
-
-	/* Construct the Session ID */
-	session->session_id.internal.domain = domain;
-	session->session_id.internal.bus = bus;
-	session->session_id.internal.device = device;
-	session->session_id.internal.fw_session_id = fw_session_id;
-
-	/* Query for Session Config
-	 */
-	rc = tf_msg_session_qcfg(tfp);
-	if (rc) {
-		TFP_DRV_LOG(ERR,
-			    "Query config message send failed, rc:%s\n",
-			    strerror(-rc));
-		goto cleanup_close;
-	}
-
-	/* Shadow DB configuration */
-	if (parms->shadow_copy) {
-		/* Ignore shadow_copy setting */
-		session->shadow_copy = 0;/* parms->shadow_copy; */
-#if (TF_SHADOW == 1)
-		rc = tf_rm_shadow_db_init(tfs);
-		if (rc)
-			TFP_DRV_LOG(ERR,
-				    "Shadow DB Initialization failed\n, rc:%s",
-				    strerror(-rc));
-		/* Add additional processing */
-#endif /* TF_SHADOW */
-	}
-
-	/* Adjust the Session with what firmware allowed us to get */
-	rc = tf_rm_allocate_validate(tfp);
-	if (rc) {
-		TFP_DRV_LOG(ERR,
-			    "Rm allocate validate failed, rc:%s\n",
-			    strerror(-rc));
-		goto cleanup_close;
-	}
-
-	/* Initialize EM pool */
-	for (dir = 0; dir < TF_DIR_MAX; dir++) {
-		rc = tf_create_em_pool(session,
-				       (enum tf_dir)dir,
-				       TF_SESSION_EM_POOL_SIZE);
-		if (rc) {
-			TFP_DRV_LOG(ERR,
-				    "EM Pool initialization failed\n");
-			goto cleanup_close;
-		}
-	}
-
-	session->ref_count++;
-
-	/* Return session ID */
-	parms->session_id = session->session_id;
-
-	TFP_DRV_LOG(INFO,
-		    "Session created, session_id:%d\n",
-		    parms->session_id.id);
-
-	TFP_DRV_LOG(INFO,
-		    "domain:%d, bus:%d, device:%d, fw_session_id:%d\n",
-		    parms->session_id.internal.domain,
-		    parms->session_id.internal.bus,
-		    parms->session_id.internal.device,
-		    parms->session_id.internal.fw_session_id);
-
-	return 0;
-
- cleanup:
-	tfp_free(tfp->session->core_data);
-	tfp_free(tfp->session);
-	tfp->session = NULL;
-	return rc;
-
- cleanup_close:
-	tf_close_session(tfp);
-	return -EINVAL;
-}
-
-int
-tf_open_session_new(struct tf *tfp,
-		    struct tf_open_session_parms *parms)
-{
-	int rc;
 	unsigned int domain, bus, slot, device;
 	struct tf_session_open_session_parms oparms;
 
-	TF_CHECK_PARMS(tfp, parms);
+	TF_CHECK_PARMS2(tfp, parms);
 
 	/* Filter out any non-supported device types on the Core
 	 * side. It is assumed that the Firmware will be supported if
@@ -347,33 +78,8 @@ tf_open_session_new(struct tf *tfp,
 }
 
 int
-tf_attach_session(struct tf *tfp __rte_unused,
-		  struct tf_attach_session_parms *parms __rte_unused)
-{
-#if (TF_SHARED == 1)
-	int rc;
-
-	TF_CHECK_PARMS_SESSION(tfp, parms);
-
-	/* - Open the shared memory for the attach_chan_name
-	 * - Point to the shared session for this Device instance
-	 * - Check that session is valid
-	 * - Attach to the firmware so it can record there is more
-	 *   than one client of the session.
-	 */
-
-	if (tfp->session->session_id.id != TF_SESSION_ID_INVALID) {
-		rc = tf_msg_session_attach(tfp,
-					   parms->ctrl_chan_name,
-					   parms->session_id);
-	}
-#endif /* TF_SHARED */
-	return -1;
-}
-
-int
-tf_attach_session_new(struct tf *tfp,
-		      struct tf_attach_session_parms *parms)
+tf_attach_session(struct tf *tfp,
+		  struct tf_attach_session_parms *parms)
 {
 	int rc;
 	unsigned int domain, bus, slot, device;
@@ -438,65 +144,6 @@ int
 tf_close_session(struct tf *tfp)
 {
 	int rc;
-	int rc_close = 0;
-	struct tf_session *tfs;
-	union tf_session_id session_id;
-	int dir;
-
-	TF_CHECK_TFP_SESSION(tfp);
-
-	tfs = (struct tf_session *)(tfp->session->core_data);
-
-	/* Cleanup if we're last user of the session */
-	if (tfs->ref_count == 1) {
-		/* Cleanup any outstanding resources */
-		rc_close = tf_rm_close(tfp);
-	}
-
-	if (tfs->session_id.id != TF_SESSION_ID_INVALID) {
-		rc = tf_msg_session_close(tfp);
-		if (rc) {
-			/* Log error */
-			TFP_DRV_LOG(ERR,
-				    "Message send failed, rc:%s\n",
-				    strerror(-rc));
-		}
-
-		/* Update the ref_count */
-		tfs->ref_count--;
-	}
-
-	session_id = tfs->session_id;
-
-	/* Final cleanup as we're last user of the session */
-	if (tfs->ref_count == 0) {
-		/* Free EM pool */
-		for (dir = 0; dir < TF_DIR_MAX; dir++)
-			tf_free_em_pool(tfs, (enum tf_dir)dir);
-
-		tfp_free(tfp->session->core_data);
-		tfp_free(tfp->session);
-		tfp->session = NULL;
-	}
-
-	TFP_DRV_LOG(INFO,
-		    "Session closed, session_id:%d\n",
-		    session_id.id);
-
-	TFP_DRV_LOG(INFO,
-		    "domain:%d, bus:%d, device:%d, fw_session_id:%d\n",
-		    session_id.internal.domain,
-		    session_id.internal.bus,
-		    session_id.internal.device,
-		    session_id.internal.fw_session_id);
-
-	return rc_close;
-}
-
-int
-tf_close_session_new(struct tf *tfp)
-{
-	int rc;
 	struct tf_session_close_session_parms cparms = { 0 };
 	union tf_session_id session_id = { 0 };
 	uint8_t ref_count;
@@ -620,76 +267,9 @@ int tf_delete_em_entry(struct tf *tfp,
 	return rc;
 }
 
-int tf_alloc_identifier(struct tf *tfp,
-			struct tf_alloc_identifier_parms *parms)
-{
-	struct bitalloc *session_pool;
-	struct tf_session *tfs;
-	int id;
-	int rc;
-
-	TF_CHECK_PARMS_SESSION(tfp, parms);
-
-	tfs = (struct tf_session *)(tfp->session->core_data);
-
-	switch (parms->ident_type) {
-	case TF_IDENT_TYPE_L2_CTXT:
-		TF_RM_GET_POOLS(tfs, parms->dir, &session_pool,
-				TF_L2_CTXT_REMAP_POOL_NAME,
-				rc);
-		break;
-	case TF_IDENT_TYPE_PROF_FUNC:
-		TF_RM_GET_POOLS(tfs, parms->dir, &session_pool,
-				TF_PROF_FUNC_POOL_NAME,
-				rc);
-		break;
-	case TF_IDENT_TYPE_EM_PROF:
-		TF_RM_GET_POOLS(tfs, parms->dir, &session_pool,
-				TF_EM_PROF_ID_POOL_NAME,
-				rc);
-		break;
-	case TF_IDENT_TYPE_WC_PROF:
-		TF_RM_GET_POOLS(tfs, parms->dir, &session_pool,
-				TF_WC_TCAM_PROF_ID_POOL_NAME,
-				rc);
-		break;
-	case TF_IDENT_TYPE_L2_FUNC:
-		TFP_DRV_LOG(ERR, "%s: unsupported %s\n",
-			    tf_dir_2_str(parms->dir),
-			    tf_ident_2_str(parms->ident_type));
-		rc = -EOPNOTSUPP;
-		break;
-	default:
-		TFP_DRV_LOG(ERR, "%s: %s\n",
-			    tf_dir_2_str(parms->dir),
-			    tf_ident_2_str(parms->ident_type));
-		rc = -EOPNOTSUPP;
-		break;
-	}
-
-	if (rc) {
-		TFP_DRV_LOG(ERR, "%s: identifier pool %s failure, rc:%s\n",
-			    tf_dir_2_str(parms->dir),
-			    tf_ident_2_str(parms->ident_type),
-			    strerror(-rc));
-		return rc;
-	}
-
-	id = ba_alloc(session_pool);
-
-	if (id == BA_FAIL) {
-		TFP_DRV_LOG(ERR, "%s: %s: No resource available\n",
-			    tf_dir_2_str(parms->dir),
-			    tf_ident_2_str(parms->ident_type));
-		return -ENOMEM;
-	}
-	parms->id = id;
-	return 0;
-}
-
 int
-tf_alloc_identifier_new(struct tf *tfp,
-			struct tf_alloc_identifier_parms *parms)
+tf_alloc_identifier(struct tf *tfp,
+		    struct tf_alloc_identifier_parms *parms)
 {
 	int rc;
 	struct tf_session *tfs;
@@ -732,7 +312,7 @@ tf_alloc_identifier_new(struct tf *tfp,
 	}
 
 	aparms.dir = parms->dir;
-	aparms.ident_type = parms->ident_type;
+	aparms.type = parms->ident_type;
 	aparms.id = &id;
 	rc = dev->ops->tf_dev_alloc_ident(tfp, &aparms);
 	if (rc) {
@@ -748,79 +328,9 @@ tf_alloc_identifier_new(struct tf *tfp,
 	return 0;
 }
 
-int tf_free_identifier(struct tf *tfp,
-		       struct tf_free_identifier_parms *parms)
-{
-	struct bitalloc *session_pool;
-	int rc;
-	int ba_rc;
-	struct tf_session *tfs;
-
-	TF_CHECK_PARMS_SESSION(tfp, parms);
-
-	tfs = (struct tf_session *)(tfp->session->core_data);
-
-	switch (parms->ident_type) {
-	case TF_IDENT_TYPE_L2_CTXT:
-		TF_RM_GET_POOLS(tfs, parms->dir, &session_pool,
-				TF_L2_CTXT_REMAP_POOL_NAME,
-				rc);
-		break;
-	case TF_IDENT_TYPE_PROF_FUNC:
-		TF_RM_GET_POOLS(tfs, parms->dir, &session_pool,
-				TF_PROF_FUNC_POOL_NAME,
-				rc);
-		break;
-	case TF_IDENT_TYPE_EM_PROF:
-		TF_RM_GET_POOLS(tfs, parms->dir, &session_pool,
-				TF_EM_PROF_ID_POOL_NAME,
-				rc);
-		break;
-	case TF_IDENT_TYPE_WC_PROF:
-		TF_RM_GET_POOLS(tfs, parms->dir, &session_pool,
-				TF_WC_TCAM_PROF_ID_POOL_NAME,
-				rc);
-		break;
-	case TF_IDENT_TYPE_L2_FUNC:
-		TFP_DRV_LOG(ERR, "%s: unsupported %s\n",
-			    tf_dir_2_str(parms->dir),
-			    tf_ident_2_str(parms->ident_type));
-		rc = -EOPNOTSUPP;
-		break;
-	default:
-		TFP_DRV_LOG(ERR, "%s: invalid %s\n",
-			    tf_dir_2_str(parms->dir),
-			    tf_ident_2_str(parms->ident_type));
-		rc = -EOPNOTSUPP;
-		break;
-	}
-	if (rc) {
-		TFP_DRV_LOG(ERR,
-			    "%s: %s Identifier pool access failed, rc:%s\n",
-			    tf_dir_2_str(parms->dir),
-			    tf_ident_2_str(parms->ident_type),
-			    strerror(-rc));
-		return rc;
-	}
-
-	ba_rc = ba_inuse(session_pool, (int)parms->id);
-
-	if (ba_rc == BA_FAIL || ba_rc == BA_ENTRY_FREE) {
-		TFP_DRV_LOG(ERR, "%s: %s: Entry %d already free",
-			    tf_dir_2_str(parms->dir),
-			    tf_ident_2_str(parms->ident_type),
-			    parms->id);
-		return -EINVAL;
-	}
-
-	ba_free(session_pool, (int)parms->id);
-
-	return 0;
-}
-
 int
-tf_free_identifier_new(struct tf *tfp,
-		       struct tf_free_identifier_parms *parms)
+tf_free_identifier(struct tf *tfp,
+		   struct tf_free_identifier_parms *parms)
 {
 	int rc;
 	struct tf_session *tfs;
@@ -862,12 +372,12 @@ tf_free_identifier_new(struct tf *tfp,
 	}
 
 	fparms.dir = parms->dir;
-	fparms.ident_type = parms->ident_type;
+	fparms.type = parms->ident_type;
 	fparms.id = parms->id;
 	rc = dev->ops->tf_dev_free_ident(tfp, &fparms);
 	if (rc) {
 		TFP_DRV_LOG(ERR,
-			    "%s: Identifier allocation failed, rc:%s\n",
+			    "%s: Identifier free failed, rc:%s\n",
 			    tf_dir_2_str(parms->dir),
 			    strerror(-rc));
 		return rc;
@@ -1057,3 +567,242 @@ tf_free_tcam_entry(struct tf *tfp,
 
 	return 0;
 }
+
+int
+tf_alloc_tbl_entry(struct tf *tfp,
+		   struct tf_alloc_tbl_entry_parms *parms)
+{
+	int rc;
+	struct tf_session *tfs;
+	struct tf_dev_info *dev;
+	struct tf_tbl_alloc_parms aparms;
+	uint32_t idx;
+
+	TF_CHECK_PARMS2(tfp, parms);
+
+	/* Can't do static initialization due to UT enum check */
+	memset(&aparms, 0, sizeof(struct tf_tbl_alloc_parms));
+
+	/* Retrieve the session information */
+	rc = tf_session_get_session(tfp, &tfs);
+	if (rc) {
+		TFP_DRV_LOG(ERR,
+			    "%s: Failed to lookup session, rc:%s\n",
+			    tf_dir_2_str(parms->dir),
+			    strerror(-rc));
+		return rc;
+	}
+
+	/* Retrieve the device information */
+	rc = tf_session_get_device(tfs, &dev);
+	if (rc) {
+		TFP_DRV_LOG(ERR,
+			    "%s: Failed to lookup device, rc:%s\n",
+			    tf_dir_2_str(parms->dir),
+			    strerror(-rc));
+		return rc;
+	}
+
+	if (dev->ops->tf_dev_alloc_tbl == NULL) {
+		rc = -EOPNOTSUPP;
+		TFP_DRV_LOG(ERR,
+			    "%s: Operation not supported, rc:%s\n",
+			    tf_dir_2_str(parms->dir),
+			    strerror(-rc));
+		return -EOPNOTSUPP;
+	}
+
+	aparms.dir = parms->dir;
+	aparms.type = parms->type;
+	aparms.idx = &idx;
+	rc = dev->ops->tf_dev_alloc_tbl(tfp, &aparms);
+	if (rc) {
+		TFP_DRV_LOG(ERR,
+			    "%s: Table allocation failed, rc:%s\n",
+			    tf_dir_2_str(parms->dir),
+			    strerror(-rc));
+		return rc;
+	}
+
+	parms->idx = idx;
+
+	return 0;
+}
+
+int
+tf_free_tbl_entry(struct tf *tfp,
+		  struct tf_free_tbl_entry_parms *parms)
+{
+	int rc;
+	struct tf_session *tfs;
+	struct tf_dev_info *dev;
+	struct tf_tbl_free_parms fparms;
+
+	TF_CHECK_PARMS2(tfp, parms);
+
+	/* Can't do static initialization due to UT enum check */
+	memset(&fparms, 0, sizeof(struct tf_tbl_free_parms));
+
+	/* Retrieve the session information */
+	rc = tf_session_get_session(tfp, &tfs);
+	if (rc) {
+		TFP_DRV_LOG(ERR,
+			    "%s: Failed to lookup session, rc:%s\n",
+			    tf_dir_2_str(parms->dir),
+			    strerror(-rc));
+		return rc;
+	}
+
+	/* Retrieve the device information */
+	rc = tf_session_get_device(tfs, &dev);
+	if (rc) {
+		TFP_DRV_LOG(ERR,
+			    "%s: Failed to lookup device, rc:%s\n",
+			    tf_dir_2_str(parms->dir),
+			    strerror(-rc));
+		return rc;
+	}
+
+	if (dev->ops->tf_dev_free_tbl == NULL) {
+		rc = -EOPNOTSUPP;
+		TFP_DRV_LOG(ERR,
+			    "%s: Operation not supported, rc:%s\n",
+			    tf_dir_2_str(parms->dir),
+			    strerror(-rc));
+		return -EOPNOTSUPP;
+	}
+
+	fparms.dir = parms->dir;
+	fparms.type = parms->type;
+	fparms.idx = parms->idx;
+	rc = dev->ops->tf_dev_free_tbl(tfp, &fparms);
+	if (rc) {
+		TFP_DRV_LOG(ERR,
+			    "%s: Table free failed, rc:%s\n",
+			    tf_dir_2_str(parms->dir),
+			    strerror(-rc));
+		return rc;
+	}
+
+	return 0;
+}
+
+int
+tf_set_tbl_entry(struct tf *tfp,
+		 struct tf_set_tbl_entry_parms *parms)
+{
+	int rc = 0;
+	struct tf_session *tfs;
+	struct tf_dev_info *dev;
+	struct tf_tbl_set_parms sparms;
+
+	TF_CHECK_PARMS3(tfp, parms, parms->data);
+
+	/* Can't do static initialization due to UT enum check */
+	memset(&sparms, 0, sizeof(struct tf_tbl_set_parms));
+
+	/* Retrieve the session information */
+	rc = tf_session_get_session(tfp, &tfs);
+	if (rc) {
+		TFP_DRV_LOG(ERR,
+			    "%s: Failed to lookup session, rc:%s\n",
+			    tf_dir_2_str(parms->dir),
+			    strerror(-rc));
+		return rc;
+	}
+
+	/* Retrieve the device information */
+	rc = tf_session_get_device(tfs, &dev);
+	if (rc) {
+		TFP_DRV_LOG(ERR,
+			    "%s: Failed to lookup device, rc:%s\n",
+			    tf_dir_2_str(parms->dir),
+			    strerror(-rc));
+		return rc;
+	}
+
+	if (dev->ops->tf_dev_set_tbl == NULL) {
+		rc = -EOPNOTSUPP;
+		TFP_DRV_LOG(ERR,
+			    "%s: Operation not supported, rc:%s\n",
+			    tf_dir_2_str(parms->dir),
+			    strerror(-rc));
+		return -EOPNOTSUPP;
+	}
+
+	sparms.dir = parms->dir;
+	sparms.type = parms->type;
+	sparms.data = parms->data;
+	sparms.data_sz_in_bytes = parms->data_sz_in_bytes;
+	sparms.idx = parms->idx;
+	rc = dev->ops->tf_dev_set_tbl(tfp, &sparms);
+	if (rc) {
+		TFP_DRV_LOG(ERR,
+			    "%s: Table set failed, rc:%s\n",
+			    tf_dir_2_str(parms->dir),
+			    strerror(-rc));
+		return rc;
+	}
+
+	return rc;
+}
+
+int
+tf_get_tbl_entry(struct tf *tfp,
+		 struct tf_get_tbl_entry_parms *parms)
+{
+	int rc = 0;
+	struct tf_session *tfs;
+	struct tf_dev_info *dev;
+	struct tf_tbl_get_parms gparms;
+
+	TF_CHECK_PARMS3(tfp, parms, parms->data);
+
+	/* Can't do static initialization due to UT enum check */
+	memset(&gparms, 0, sizeof(struct tf_tbl_get_parms));
+
+	/* Retrieve the session information */
+	rc = tf_session_get_session(tfp, &tfs);
+	if (rc) {
+		TFP_DRV_LOG(ERR,
+			    "%s: Failed to lookup session, rc:%s\n",
+			    tf_dir_2_str(parms->dir),
+			    strerror(-rc));
+		return rc;
+	}
+
+	/* Retrieve the device information */
+	rc = tf_session_get_device(tfs, &dev);
+	if (rc) {
+		TFP_DRV_LOG(ERR,
+			    "%s: Failed to lookup device, rc:%s\n",
+			    tf_dir_2_str(parms->dir),
+			    strerror(-rc));
+		return rc;
+	}
+
+	if (dev->ops->tf_dev_get_tbl == NULL) {
+		rc = -EOPNOTSUPP;
+		TFP_DRV_LOG(ERR,
+			    "%s: Operation not supported, rc:%s\n",
+			    tf_dir_2_str(parms->dir),
+			    strerror(-rc));
+		return -EOPNOTSUPP;
+	}
+
+	gparms.dir = parms->dir;
+	gparms.type = parms->type;
+	gparms.data = parms->data;
+	gparms.data_sz_in_bytes = parms->data_sz_in_bytes;
+	gparms.idx = parms->idx;
+	rc = dev->ops->tf_dev_get_tbl(tfp, &gparms);
+	if (rc) {
+		TFP_DRV_LOG(ERR,
+			    "%s: Table get failed, rc:%s\n",
+			    tf_dir_2_str(parms->dir),
+			    strerror(-rc));
+		return rc;
+	}
+
+	return rc;
+}
diff --git a/drivers/net/bnxt/tf_core/tf_core.h b/drivers/net/bnxt/tf_core/tf_core.h
index bb456bb..a7a7bd3 100644
--- a/drivers/net/bnxt/tf_core/tf_core.h
+++ b/drivers/net/bnxt/tf_core/tf_core.h
@@ -385,33 +385,86 @@ struct tf {
 };
 
 /**
+ * Identifier resource definition
+ */
+struct tf_identifier_resources {
+	/**
+	 * Array of TF Identifiers where each entry is expected to be
+	 * set to the requested resource number of that specific type.
+	 * The index used is tf_identifier_type.
+	 */
+	uint16_t cnt[TF_IDENT_TYPE_MAX];
+};
+
+/**
+ * Table type resource definition
+ */
+struct tf_tbl_resources {
+	/**
+	 * Array of TF Table types where each entry is expected to be
+	 * set to the requeste resource number of that specific
+	 * type. The index used is tf_tbl_type.
+	 */
+	uint16_t cnt[TF_TBL_TYPE_MAX];
+};
+
+/**
+ * TCAM type resource definition
+ */
+struct tf_tcam_resources {
+	/**
+	 * Array of TF TCAM types where each entry is expected to be
+	 * set to the requested resource number of that specific
+	 * type. The index used is tf_tcam_tbl_type.
+	 */
+	uint16_t cnt[TF_TCAM_TBL_TYPE_MAX];
+};
+
+/**
+ * EM type resource definition
+ */
+struct tf_em_resources {
+	/**
+	 * Array of TF EM table types where each entry is expected to
+	 * be set to the requested resource number of that specific
+	 * type. The index used is tf_em_tbl_type.
+	 */
+	uint16_t cnt[TF_EM_TBL_TYPE_MAX];
+};
+
+/**
  * tf_session_resources parameter definition.
  */
 struct tf_session_resources {
-	/** [in] Requested Identifier Resources
+	/**
+	 * [in] Requested Identifier Resources
 	 *
-	 * The number of identifier resources requested for the session.
-	 * The index used is tf_identifier_type.
+	 * Number of identifier resources requested for the
+	 * session.
 	 */
-	uint16_t identifier_cnt[TF_IDENT_TYPE_MAX][TF_DIR_MAX];
-	/** [in] Requested Index Table resource counts
+	struct tf_identifier_resources ident_cnt[TF_DIR_MAX];
+	/**
+	 * [in] Requested Index Table resource counts
 	 *
-	 * The number of index table resources requested for the session.
-	 * The index used is tf_tbl_type.
+	 * The number of index table resources requested for the
+	 * session.
 	 */
-	uint16_t tbl_cnt[TF_TBL_TYPE_MAX][TF_DIR_MAX];
-	/** [in] Requested TCAM Table resource counts
+	struct tf_tbl_resources tbl_cnt[TF_DIR_MAX];
+	/**
+	 * [in] Requested TCAM Table resource counts
 	 *
-	 * The number of TCAM table resources requested for the session.
-	 * The index used is tf_tcam_tbl_type.
+	 * The number of TCAM table resources requested for the
+	 * session.
 	 */
-	uint16_t tcam_tbl_cnt[TF_TCAM_TBL_TYPE_MAX][TF_DIR_MAX];
-	/** [in] Requested EM resource counts
+
+	struct tf_tcam_resources tcam_cnt[TF_DIR_MAX];
+	/**
+	 * [in] Requested EM resource counts
 	 *
-	 * The number of internal EM table resources requested for the session
-	 * The index used is tf_em_tbl_type.
+	 * The number of internal EM table resources requested for the
+	 * session.
 	 */
-	uint16_t em_tbl_cnt[TF_EM_TBL_TYPE_MAX][TF_DIR_MAX];
+	struct tf_em_resources em_cnt[TF_DIR_MAX];
 };
 
 /**
@@ -497,9 +550,6 @@ struct tf_open_session_parms {
 int tf_open_session(struct tf *tfp,
 		    struct tf_open_session_parms *parms);
 
-int tf_open_session_new(struct tf *tfp,
-			struct tf_open_session_parms *parms);
-
 struct tf_attach_session_parms {
 	/**
 	 * [in] ctrl_chan_name
@@ -565,8 +615,6 @@ struct tf_attach_session_parms {
  */
 int tf_attach_session(struct tf *tfp,
 		      struct tf_attach_session_parms *parms);
-int tf_attach_session_new(struct tf *tfp,
-			  struct tf_attach_session_parms *parms);
 
 /**
  * Closes an existing session. Cleans up all hardware and firmware
@@ -576,7 +624,6 @@ int tf_attach_session_new(struct tf *tfp,
  * Returns success or failure code.
  */
 int tf_close_session(struct tf *tfp);
-int tf_close_session_new(struct tf *tfp);
 
 /**
  * @page  ident Identity Management
@@ -631,8 +678,6 @@ struct tf_free_identifier_parms {
  */
 int tf_alloc_identifier(struct tf *tfp,
 			struct tf_alloc_identifier_parms *parms);
-int tf_alloc_identifier_new(struct tf *tfp,
-			    struct tf_alloc_identifier_parms *parms);
 
 /**
  * free identifier resource
@@ -645,8 +690,6 @@ int tf_alloc_identifier_new(struct tf *tfp,
  */
 int tf_free_identifier(struct tf *tfp,
 		       struct tf_free_identifier_parms *parms);
-int tf_free_identifier_new(struct tf *tfp,
-			   struct tf_free_identifier_parms *parms);
 
 /**
  * @page dram_table DRAM Table Scope Interface
@@ -1277,7 +1320,7 @@ struct tf_bulk_get_tbl_entry_parms {
  * provided data buffer is too small for the data type requested.
  */
 int tf_bulk_get_tbl_entry(struct tf *tfp,
-		     struct tf_bulk_get_tbl_entry_parms *parms);
+			  struct tf_bulk_get_tbl_entry_parms *parms);
 
 /**
  * @page exact_match Exact Match Table
diff --git a/drivers/net/bnxt/tf_core/tf_device.c b/drivers/net/bnxt/tf_core/tf_device.c
index 4c46cad..b474e8c 100644
--- a/drivers/net/bnxt/tf_core/tf_device.c
+++ b/drivers/net/bnxt/tf_core/tf_device.c
@@ -43,6 +43,14 @@ dev_bind_p4(struct tf *tfp,
 	struct tf_tbl_cfg_parms tbl_cfg;
 	struct tf_tcam_cfg_parms tcam_cfg;
 
+	dev_handle->type = TF_DEVICE_TYPE_WH;
+	/* Initial function initialization */
+	dev_handle->ops = &tf_dev_ops_p4_init;
+
+	dev_handle->type = TF_DEVICE_TYPE_WH;
+	/* Initial function initialization */
+	dev_handle->ops = &tf_dev_ops_p4_init;
+
 	/* Initialize the modules */
 
 	ident_cfg.num_elements = TF_IDENT_TYPE_MAX;
@@ -78,7 +86,7 @@ dev_bind_p4(struct tf *tfp,
 		goto fail;
 	}
 
-	dev_handle->type = TF_DEVICE_TYPE_WH;
+	/* Final function initialization */
 	dev_handle->ops = &tf_dev_ops_p4;
 
 	return 0;
diff --git a/drivers/net/bnxt/tf_core/tf_device.h b/drivers/net/bnxt/tf_core/tf_device.h
index 32d9a54..c31bf23 100644
--- a/drivers/net/bnxt/tf_core/tf_device.h
+++ b/drivers/net/bnxt/tf_core/tf_device.h
@@ -407,6 +407,7 @@ struct tf_dev_ops {
 /**
  * Supported device operation structures
  */
+extern const struct tf_dev_ops tf_dev_ops_p4_init;
 extern const struct tf_dev_ops tf_dev_ops_p4;
 
 #endif /* _TF_DEVICE_H_ */
diff --git a/drivers/net/bnxt/tf_core/tf_device_p4.c b/drivers/net/bnxt/tf_core/tf_device_p4.c
index 77fb693..9e332c5 100644
--- a/drivers/net/bnxt/tf_core/tf_device_p4.c
+++ b/drivers/net/bnxt/tf_core/tf_device_p4.c
@@ -78,6 +78,26 @@ tf_dev_p4_get_tcam_slice_info(struct tf *tfp __rte_unused,
 /**
  * Truflow P4 device specific functions
  */
+const struct tf_dev_ops tf_dev_ops_p4_init = {
+	.tf_dev_get_max_types = tf_dev_p4_get_max_types,
+	.tf_dev_get_tcam_slice_info = tf_dev_p4_get_tcam_slice_info,
+	.tf_dev_alloc_ident = NULL,
+	.tf_dev_free_ident = NULL,
+	.tf_dev_alloc_tbl = NULL,
+	.tf_dev_free_tbl = NULL,
+	.tf_dev_alloc_search_tbl = NULL,
+	.tf_dev_set_tbl = NULL,
+	.tf_dev_get_tbl = NULL,
+	.tf_dev_alloc_tcam = NULL,
+	.tf_dev_free_tcam = NULL,
+	.tf_dev_alloc_search_tcam = NULL,
+	.tf_dev_set_tcam = NULL,
+	.tf_dev_get_tcam = NULL,
+};
+
+/**
+ * Truflow P4 device specific functions
+ */
 const struct tf_dev_ops tf_dev_ops_p4 = {
 	.tf_dev_get_max_types = tf_dev_p4_get_max_types,
 	.tf_dev_get_tcam_slice_info = tf_dev_p4_get_tcam_slice_info,
@@ -85,14 +105,14 @@ const struct tf_dev_ops tf_dev_ops_p4 = {
 	.tf_dev_free_ident = tf_ident_free,
 	.tf_dev_alloc_tbl = tf_tbl_alloc,
 	.tf_dev_free_tbl = tf_tbl_free,
-	.tf_dev_alloc_search_tbl = tf_tbl_alloc_search,
+	.tf_dev_alloc_search_tbl = NULL,
 	.tf_dev_set_tbl = tf_tbl_set,
 	.tf_dev_get_tbl = tf_tbl_get,
 	.tf_dev_alloc_tcam = tf_tcam_alloc,
 	.tf_dev_free_tcam = tf_tcam_free,
-	.tf_dev_alloc_search_tcam = tf_tcam_alloc_search,
+	.tf_dev_alloc_search_tcam = NULL,
 	.tf_dev_set_tcam = tf_tcam_set,
-	.tf_dev_get_tcam = tf_tcam_get,
+	.tf_dev_get_tcam = NULL,
 	.tf_dev_insert_em_entry = tf_em_insert_entry,
 	.tf_dev_delete_em_entry = tf_em_delete_entry,
 };
diff --git a/drivers/net/bnxt/tf_core/tf_identifier.c b/drivers/net/bnxt/tf_core/tf_identifier.c
index e89f976..ee07a6a 100644
--- a/drivers/net/bnxt/tf_core/tf_identifier.c
+++ b/drivers/net/bnxt/tf_core/tf_identifier.c
@@ -45,19 +45,22 @@ tf_ident_bind(struct tf *tfp,
 		db_cfg.dir = i;
 		db_cfg.num_elements = parms->num_elements;
 		db_cfg.cfg = parms->cfg;
-		db_cfg.alloc_num = parms->resources->identifier_cnt[i];
-		db_cfg.rm_db = ident_db[i];
+		db_cfg.alloc_cnt = parms->resources->ident_cnt[i].cnt;
+		db_cfg.rm_db = &ident_db[i];
 		rc = tf_rm_create_db(tfp, &db_cfg);
 		if (rc) {
 			TFP_DRV_LOG(ERR,
 				    "%s: Identifier DB creation failed\n",
 				    tf_dir_2_str(i));
+
 			return rc;
 		}
 	}
 
 	init = 1;
 
+	printf("Identifier - initialized\n");
+
 	return 0;
 }
 
@@ -73,8 +76,11 @@ tf_ident_unbind(struct tf *tfp __rte_unused)
 	/* Bail if nothing has been initialized done silent as to
 	 * allow for creation cleanup.
 	 */
-	if (!init)
+	if (!init) {
+		TFP_DRV_LOG(ERR,
+			    "No Identifier DBs created\n");
 		return -EINVAL;
+	}
 
 	for (i = 0; i < TF_DIR_MAX; i++) {
 		fparms.dir = i;
@@ -96,6 +102,7 @@ tf_ident_alloc(struct tf *tfp __rte_unused,
 	       struct tf_ident_alloc_parms *parms)
 {
 	int rc;
+	uint32_t id;
 	struct tf_rm_allocate_parms aparms = { 0 };
 
 	TF_CHECK_PARMS2(tfp, parms);
@@ -109,17 +116,19 @@ tf_ident_alloc(struct tf *tfp __rte_unused,
 
 	/* Allocate requested element */
 	aparms.rm_db = ident_db[parms->dir];
-	aparms.db_index = parms->ident_type;
-	aparms.index = (uint32_t *)&parms->id;
+	aparms.db_index = parms->type;
+	aparms.index = &id;
 	rc = tf_rm_allocate(&aparms);
 	if (rc) {
 		TFP_DRV_LOG(ERR,
 			    "%s: Failed allocate, type:%d\n",
 			    tf_dir_2_str(parms->dir),
-			    parms->ident_type);
+			    parms->type);
 		return rc;
 	}
 
+	*parms->id = id;
+
 	return 0;
 }
 
@@ -143,7 +152,7 @@ tf_ident_free(struct tf *tfp __rte_unused,
 
 	/* Check if element is in use */
 	aparms.rm_db = ident_db[parms->dir];
-	aparms.db_index = parms->ident_type;
+	aparms.db_index = parms->type;
 	aparms.index = parms->id;
 	aparms.allocated = &allocated;
 	rc = tf_rm_is_allocated(&aparms);
@@ -154,21 +163,21 @@ tf_ident_free(struct tf *tfp __rte_unused,
 		TFP_DRV_LOG(ERR,
 			    "%s: Entry already free, type:%d, index:%d\n",
 			    tf_dir_2_str(parms->dir),
-			    parms->ident_type,
+			    parms->type,
 			    parms->id);
 		return rc;
 	}
 
 	/* Free requested element */
 	fparms.rm_db = ident_db[parms->dir];
-	fparms.db_index = parms->ident_type;
+	fparms.db_index = parms->type;
 	fparms.index = parms->id;
 	rc = tf_rm_free(&fparms);
 	if (rc) {
 		TFP_DRV_LOG(ERR,
 			    "%s: Free failed, type:%d, index:%d\n",
 			    tf_dir_2_str(parms->dir),
-			    parms->ident_type,
+			    parms->type,
 			    parms->id);
 		return rc;
 	}
diff --git a/drivers/net/bnxt/tf_core/tf_identifier.h b/drivers/net/bnxt/tf_core/tf_identifier.h
index 1c5319b..6e36c52 100644
--- a/drivers/net/bnxt/tf_core/tf_identifier.h
+++ b/drivers/net/bnxt/tf_core/tf_identifier.h
@@ -43,7 +43,7 @@ struct tf_ident_alloc_parms {
 	/**
 	 * [in] Identifier type
 	 */
-	enum tf_identifier_type ident_type;
+	enum tf_identifier_type type;
 	/**
 	 * [out] Identifier allocated
 	 */
@@ -61,7 +61,7 @@ struct tf_ident_free_parms {
 	/**
 	 * [in] Identifier type
 	 */
-	enum tf_identifier_type ident_type;
+	enum tf_identifier_type type;
 	/**
 	 * [in] ID to free
 	 */
diff --git a/drivers/net/bnxt/tf_core/tf_msg.c b/drivers/net/bnxt/tf_core/tf_msg.c
index f223850..5ec2f83 100644
--- a/drivers/net/bnxt/tf_core/tf_msg.c
+++ b/drivers/net/bnxt/tf_core/tf_msg.c
@@ -12,6 +12,7 @@
 #include "tf_device.h"
 #include "tf_msg.h"
 #include "tf_util.h"
+#include "tf_common.h"
 #include "tf_session.h"
 #include "tfp.h"
 #include "hwrm_tf.h"
@@ -935,13 +936,7 @@ tf_msg_session_resc_qcaps(struct tf *tfp,
 	struct tf_rm_resc_req_entry *data;
 	int dma_size;
 
-	if (size == 0 || query == NULL || resv_strategy == NULL) {
-		TFP_DRV_LOG(ERR,
-			    "%s: Resource QCAPS parameter error, rc:%s\n",
-			    tf_dir_2_str(dir),
-			    strerror(-EINVAL));
-		return -EINVAL;
-	}
+	TF_CHECK_PARMS3(tfp, query, resv_strategy);
 
 	rc = tf_session_get_fw_session_id(tfp, &fw_session_id);
 	if (rc) {
@@ -962,7 +957,7 @@ tf_msg_session_resc_qcaps(struct tf *tfp,
 	req.fw_session_id = tfp_cpu_to_le_32(fw_session_id);
 	req.flags = tfp_cpu_to_le_16(dir);
 	req.qcaps_size = size;
-	req.qcaps_addr = qcaps_buf.pa_addr;
+	req.qcaps_addr = tfp_cpu_to_le_64(qcaps_buf.pa_addr);
 
 	parms.tf_type = HWRM_TF_SESSION_RESC_QCAPS;
 	parms.req_data = (uint32_t *)&req;
@@ -980,18 +975,29 @@ tf_msg_session_resc_qcaps(struct tf *tfp,
 	 */
 	if (resp.size != size) {
 		TFP_DRV_LOG(ERR,
-			    "%s: QCAPS message error, rc:%s\n",
+			    "%s: QCAPS message size error, rc:%s\n",
 			    tf_dir_2_str(dir),
 			    strerror(-EINVAL));
 		return -EINVAL;
 	}
 
+	printf("size: %d\n", resp.size);
+
 	/* Post process the response */
 	data = (struct tf_rm_resc_req_entry *)qcaps_buf.va_addr;
+
+	printf("\nQCAPS\n");
 	for (i = 0; i < size; i++) {
 		query[i].type = tfp_cpu_to_le_32(data[i].type);
 		query[i].min = tfp_le_to_cpu_16(data[i].min);
 		query[i].max = tfp_le_to_cpu_16(data[i].max);
+
+		printf("type: %d(0x%x) %d %d\n",
+		       query[i].type,
+		       query[i].type,
+		       query[i].min,
+		       query[i].max);
+
 	}
 
 	*resv_strategy = resp.flags &
@@ -1021,6 +1027,8 @@ tf_msg_session_resc_alloc(struct tf *tfp,
 	struct tf_rm_resc_entry *resv_data;
 	int dma_size;
 
+	TF_CHECK_PARMS3(tfp, request, resv);
+
 	rc = tf_session_get_fw_session_id(tfp, &fw_session_id);
 	if (rc) {
 		TFP_DRV_LOG(ERR,
@@ -1053,8 +1061,8 @@ tf_msg_session_resc_alloc(struct tf *tfp,
 		req_data[i].max = tfp_cpu_to_le_16(request[i].max);
 	}
 
-	req.req_addr = req_buf.pa_addr;
-	req.resp_addr = resv_buf.pa_addr;
+	req.req_addr = tfp_cpu_to_le_64(req_buf.pa_addr);
+	req.resc_addr = tfp_cpu_to_le_64(resv_buf.pa_addr);
 
 	parms.tf_type = HWRM_TF_SESSION_RESC_ALLOC;
 	parms.req_data = (uint32_t *)&req;
@@ -1072,18 +1080,28 @@ tf_msg_session_resc_alloc(struct tf *tfp,
 	 */
 	if (resp.size != size) {
 		TFP_DRV_LOG(ERR,
-			    "%s: Alloc message error, rc:%s\n",
+			    "%s: Alloc message size error, rc:%s\n",
 			    tf_dir_2_str(dir),
 			    strerror(-EINVAL));
 		return -EINVAL;
 	}
 
+	printf("\nRESV\n");
+	printf("size: %d\n", resp.size);
+
 	/* Post process the response */
 	resv_data = (struct tf_rm_resc_entry *)resv_buf.va_addr;
 	for (i = 0; i < size; i++) {
 		resv[i].type = tfp_cpu_to_le_32(resv_data[i].type);
 		resv[i].start = tfp_cpu_to_le_16(resv_data[i].start);
 		resv[i].stride = tfp_cpu_to_le_16(resv_data[i].stride);
+
+		printf("%d type: %d(0x%x) %d %d\n",
+		       i,
+		       resv[i].type,
+		       resv[i].type,
+		       resv[i].start,
+		       resv[i].stride);
 	}
 
 	tf_msg_free_dma_buf(&req_buf);
@@ -1459,7 +1477,8 @@ tf_msg_bulk_get_tbl_entry(struct tf *tfp,
 	req.start_index = tfp_cpu_to_le_32(params->starting_idx);
 	req.num_entries = tfp_cpu_to_le_32(params->num_entries);
 
-	data_size = (params->num_entries * params->entry_sz_in_bytes);
+	data_size = params->num_entries * params->entry_sz_in_bytes;
+
 	req.host_addr = tfp_cpu_to_le_64(params->physical_mem_addr);
 
 	MSG_PREP(parms,
diff --git a/drivers/net/bnxt/tf_core/tf_msg.h b/drivers/net/bnxt/tf_core/tf_msg.h
index a3e0f7b..fb635f6 100644
--- a/drivers/net/bnxt/tf_core/tf_msg.h
+++ b/drivers/net/bnxt/tf_core/tf_msg.h
@@ -12,6 +12,7 @@
 #include "tf_tbl.h"
 #include "tf_rm.h"
 #include "tf_rm_new.h"
+#include "tf_tcam.h"
 
 struct tf;
 
diff --git a/drivers/net/bnxt/tf_core/tf_rm.c b/drivers/net/bnxt/tf_core/tf_rm.c
index d6739b3..b6fe2f1 100644
--- a/drivers/net/bnxt/tf_core/tf_rm.c
+++ b/drivers/net/bnxt/tf_core/tf_rm.c
@@ -3186,13 +3186,6 @@ tf_rm_convert_tbl_type(enum tf_tbl_type type,
 	return rc;
 }
 
-#if 0
-enum tf_rm_convert_type {
-	TF_RM_CONVERT_ADD_BASE,
-	TF_RM_CONVERT_RM_BASE
-};
-#endif
-
 int
 tf_rm_convert_index(struct tf_session *tfs,
 		    enum tf_dir dir,
diff --git a/drivers/net/bnxt/tf_core/tf_rm_new.c b/drivers/net/bnxt/tf_core/tf_rm_new.c
index 7cadb23..6abf79a 100644
--- a/drivers/net/bnxt/tf_core/tf_rm_new.c
+++ b/drivers/net/bnxt/tf_core/tf_rm_new.c
@@ -10,6 +10,7 @@
 #include <cfa_resource_types.h>
 
 #include "tf_rm_new.h"
+#include "tf_common.h"
 #include "tf_util.h"
 #include "tf_session.h"
 #include "tf_device.h"
@@ -65,6 +66,46 @@ struct tf_rm_new_db {
 	struct tf_rm_element *db;
 };
 
+/**
+ * Adjust an index according to the allocation information.
+ *
+ * All resources are controlled in a 0 based pool. Some resources, by
+ * design, are not 0 based, i.e. Full Action Records (SRAM) thus they
+ * need to be adjusted before they are handed out.
+ *
+ * [in] cfg
+ *   Pointer to the DB configuration
+ *
+ * [in] reservations
+ *   Pointer to the allocation values associated with the module
+ *
+ * [in] count
+ *   Number of DB configuration elements
+ *
+ * [out] valid_count
+ *   Number of HCAPI entries with a reservation value greater than 0
+ *
+ * Returns:
+ *     0          - Success
+ *   - EOPNOTSUPP - Operation not supported
+ */
+static void
+tf_rm_count_hcapi_reservations(struct tf_rm_element_cfg *cfg,
+			       uint16_t *reservations,
+			       uint16_t count,
+			       uint16_t *valid_count)
+{
+	int i;
+	uint16_t cnt = 0;
+
+	for (i = 0; i < count; i++) {
+		if (cfg[i].cfg_type == TF_RM_ELEM_CFG_HCAPI &&
+		    reservations[i] > 0)
+			cnt++;
+	}
+
+	*valid_count = cnt;
+}
 
 /**
  * Resource Manager Adjust of base index definitions.
@@ -132,6 +173,7 @@ tf_rm_create_db(struct tf *tfp,
 {
 	int rc;
 	int i;
+	int j;
 	struct tf_session *tfs;
 	struct tf_dev_info *dev;
 	uint16_t max_types;
@@ -143,6 +185,9 @@ tf_rm_create_db(struct tf *tfp,
 	struct tf_rm_new_db *rm_db;
 	struct tf_rm_element *db;
 	uint32_t pool_size;
+	uint16_t hcapi_items;
+
+	TF_CHECK_PARMS2(tfp, parms);
 
 	/* Retrieve the session information */
 	rc = tf_session_get_session(tfp, &tfs);
@@ -177,10 +222,19 @@ tf_rm_create_db(struct tf *tfp,
 	if (rc)
 		return rc;
 
-	/* Process capabilities against db requirements */
+	/* Process capabilities against DB requirements. However, as a
+	 * DB can hold elements that are not HCAPI we can reduce the
+	 * req msg content by removing those out of the request yet
+	 * the DB holds them all as to give a fast lookup. We can also
+	 * remove entries where there are no request for elements.
+	 */
+	tf_rm_count_hcapi_reservations(parms->cfg,
+				       parms->alloc_cnt,
+				       parms->num_elements,
+				       &hcapi_items);
 
 	/* Alloc request, alignment already set */
-	cparms.nitems = parms->num_elements;
+	cparms.nitems = (size_t)hcapi_items;
 	cparms.size = sizeof(struct tf_rm_resc_req_entry);
 	rc = tfp_calloc(&cparms);
 	if (rc)
@@ -195,15 +249,24 @@ tf_rm_create_db(struct tf *tfp,
 	resv = (struct tf_rm_resc_entry *)cparms.mem_va;
 
 	/* Build the request */
-	for (i = 0; i < parms->num_elements; i++) {
+	for (i = 0, j = 0; i < parms->num_elements; i++) {
 		/* Skip any non HCAPI cfg elements */
 		if (parms->cfg[i].cfg_type == TF_RM_ELEM_CFG_HCAPI) {
-			req[i].type = parms->cfg[i].hcapi_type;
-			/* Check that we can get the full amount allocated */
-			if (parms->alloc_num[i] <=
+			/* Only perform reservation for entries that
+			 * has been requested
+			 */
+			if (parms->alloc_cnt[i] == 0)
+				continue;
+
+			/* Verify that we can get the full amount
+			 * allocated per the qcaps availability.
+			 */
+			if (parms->alloc_cnt[i] <=
 			    query[parms->cfg[i].hcapi_type].max) {
-				req[i].min = parms->alloc_num[i];
-				req[i].max = parms->alloc_num[i];
+				req[j].type = parms->cfg[i].hcapi_type;
+				req[j].min = parms->alloc_cnt[i];
+				req[j].max = parms->alloc_cnt[i];
+				j++;
 			} else {
 				TFP_DRV_LOG(ERR,
 					    "%s: Resource failure, type:%d\n",
@@ -211,19 +274,16 @@ tf_rm_create_db(struct tf *tfp,
 					    parms->cfg[i].hcapi_type);
 				TFP_DRV_LOG(ERR,
 					"req:%d, avail:%d\n",
-					parms->alloc_num[i],
+					parms->alloc_cnt[i],
 					query[parms->cfg[i].hcapi_type].max);
 				return -EINVAL;
 			}
-		} else {
-			/* Skip the element */
-			req[i].type = CFA_RESOURCE_TYPE_INVALID;
 		}
 	}
 
 	rc = tf_msg_session_resc_alloc(tfp,
 				       parms->dir,
-				       parms->num_elements,
+				       hcapi_items,
 				       req,
 				       resv);
 	if (rc)
@@ -246,42 +306,74 @@ tf_rm_create_db(struct tf *tfp,
 	rm_db->db = (struct tf_rm_element *)cparms.mem_va;
 
 	db = rm_db->db;
-	for (i = 0; i < parms->num_elements; i++) {
-		/* If allocation failed for a single entry the DB
-		 * creation is considered a failure.
+	for (i = 0, j = 0; i < parms->num_elements; i++) {
+		db[i].cfg_type = parms->cfg[i].cfg_type;
+		db[i].hcapi_type = parms->cfg[i].hcapi_type;
+
+		/* Skip any non HCAPI types as we didn't include them
+		 * in the reservation request.
+		 */
+		if (parms->cfg[i].cfg_type != TF_RM_ELEM_CFG_HCAPI)
+			continue;
+
+		/* If the element didn't request an allocation no need
+		 * to create a pool nor verify if we got a reservation.
 		 */
-		if (parms->alloc_num[i] != resv[i].stride) {
+		if (parms->alloc_cnt[i] == 0)
+			continue;
+
+		/* If the element had requested an allocation and that
+		 * allocation was a success (full amount) then
+		 * allocate the pool.
+		 */
+		if (parms->alloc_cnt[i] == resv[j].stride) {
+			db[i].alloc.entry.start = resv[j].start;
+			db[i].alloc.entry.stride = resv[j].stride;
+
+			/* Create pool */
+			pool_size = (BITALLOC_SIZEOF(resv[j].stride) /
+				     sizeof(struct bitalloc));
+			/* Alloc request, alignment already set */
+			cparms.nitems = pool_size;
+			cparms.size = sizeof(struct bitalloc);
+			rc = tfp_calloc(&cparms);
+			if (rc) {
+				TFP_DRV_LOG(ERR,
+					    "%s: Pool alloc failed, type:%d\n",
+					    tf_dir_2_str(parms->dir),
+					    db[i].cfg_type);
+				goto fail;
+			}
+			db[i].pool = (struct bitalloc *)cparms.mem_va;
+
+			rc = ba_init(db[i].pool, resv[j].stride);
+			if (rc) {
+				TFP_DRV_LOG(ERR,
+					    "%s: Pool init failed, type:%d\n",
+					    tf_dir_2_str(parms->dir),
+					    db[i].cfg_type);
+				goto fail;
+			}
+			j++;
+		} else {
+			/* Bail out as we want what we requested for
+			 * all elements, not any less.
+			 */
 			TFP_DRV_LOG(ERR,
 				    "%s: Alloc failed, type:%d\n",
 				    tf_dir_2_str(parms->dir),
-				    i);
+				    db[i].cfg_type);
 			TFP_DRV_LOG(ERR,
 				    "req:%d, alloc:%d\n",
-				    parms->alloc_num[i],
-				    resv[i].stride);
+				    parms->alloc_cnt[i],
+				    resv[j].stride);
 			goto fail;
 		}
-
-		db[i].cfg_type = parms->cfg[i].cfg_type;
-		db[i].hcapi_type = parms->cfg[i].hcapi_type;
-		db[i].alloc.entry.start = resv[i].start;
-		db[i].alloc.entry.stride = resv[i].stride;
-
-		/* Create pool */
-		pool_size = (BITALLOC_SIZEOF(resv[i].stride) /
-			     sizeof(struct bitalloc));
-		/* Alloc request, alignment already set */
-		cparms.nitems = pool_size;
-		cparms.size = sizeof(struct bitalloc);
-		rc = tfp_calloc(&cparms);
-		if (rc)
-			return rc;
-		db[i].pool = (struct bitalloc *)cparms.mem_va;
 	}
 
 	rm_db->num_entries = i;
 	rm_db->dir = parms->dir;
-	parms->rm_db = (void *)rm_db;
+	*parms->rm_db = (void *)rm_db;
 
 	tfp_free((void *)req);
 	tfp_free((void *)resv);
@@ -307,13 +399,15 @@ tf_rm_free_db(struct tf *tfp __rte_unused,
 	int i;
 	struct tf_rm_new_db *rm_db;
 
+	TF_CHECK_PARMS1(parms);
+
 	/* Traverse the DB and clear each pool.
 	 * NOTE:
 	 *   Firmware is not cleared. It will be cleared on close only.
 	 */
 	rm_db = (struct tf_rm_new_db *)parms->rm_db;
 	for (i = 0; i < rm_db->num_entries; i++)
-		tfp_free((void *)rm_db->db->pool);
+		tfp_free((void *)rm_db->db[i].pool);
 
 	tfp_free((void *)parms->rm_db);
 
@@ -325,11 +419,11 @@ tf_rm_allocate(struct tf_rm_allocate_parms *parms)
 {
 	int rc = 0;
 	int id;
+	uint32_t index;
 	struct tf_rm_new_db *rm_db;
 	enum tf_rm_elem_cfg_type cfg_type;
 
-	if (parms == NULL || parms->rm_db == NULL)
-		return -EINVAL;
+	TF_CHECK_PARMS2(parms, parms->rm_db);
 
 	rm_db = (struct tf_rm_new_db *)parms->rm_db;
 	cfg_type = rm_db->db[parms->db_index].cfg_type;
@@ -339,6 +433,17 @@ tf_rm_allocate(struct tf_rm_allocate_parms *parms)
 	    cfg_type != TF_RM_ELEM_CFG_PRIVATE)
 		return -ENOTSUP;
 
+	/* Bail out if the pool is not valid, should never happen */
+	if (rm_db->db[parms->db_index].pool == NULL) {
+		rc = -ENOTSUP;
+		TFP_DRV_LOG(ERR,
+			    "%s: Invalid pool for this type:%d, rc:%s\n",
+			    tf_dir_2_str(rm_db->dir),
+			    parms->db_index,
+			    strerror(-rc));
+		return rc;
+	}
+
 	id = ba_alloc(rm_db->db[parms->db_index].pool);
 	if (id == BA_FAIL) {
 		TFP_DRV_LOG(ERR,
@@ -353,15 +458,17 @@ tf_rm_allocate(struct tf_rm_allocate_parms *parms)
 				TF_RM_ADJUST_ADD_BASE,
 				parms->db_index,
 				id,
-				parms->index);
+				&index);
 	if (rc) {
 		TFP_DRV_LOG(ERR,
 			    "%s: Alloc adjust of base index failed, rc:%s\n",
 			    tf_dir_2_str(rm_db->dir),
 			    strerror(-rc));
-		return -1;
+		return -EINVAL;
 	}
 
+	*parms->index = index;
+
 	return rc;
 }
 
@@ -373,8 +480,7 @@ tf_rm_free(struct tf_rm_free_parms *parms)
 	struct tf_rm_new_db *rm_db;
 	enum tf_rm_elem_cfg_type cfg_type;
 
-	if (parms == NULL || parms->rm_db == NULL)
-		return -EINVAL;
+	TF_CHECK_PARMS2(parms, parms->rm_db);
 
 	rm_db = (struct tf_rm_new_db *)parms->rm_db;
 	cfg_type = rm_db->db[parms->db_index].cfg_type;
@@ -384,6 +490,17 @@ tf_rm_free(struct tf_rm_free_parms *parms)
 	    cfg_type != TF_RM_ELEM_CFG_PRIVATE)
 		return -ENOTSUP;
 
+	/* Bail out if the pool is not valid, should never happen */
+	if (rm_db->db[parms->db_index].pool == NULL) {
+		rc = -ENOTSUP;
+		TFP_DRV_LOG(ERR,
+			    "%s: Invalid pool for this type:%d, rc:%s\n",
+			    tf_dir_2_str(rm_db->dir),
+			    parms->db_index,
+			    strerror(-rc));
+		return rc;
+	}
+
 	/* Adjust for any non zero start value */
 	rc = tf_rm_adjust_index(rm_db->db,
 				TF_RM_ADJUST_RM_BASE,
@@ -409,8 +526,7 @@ tf_rm_is_allocated(struct tf_rm_is_allocated_parms *parms)
 	struct tf_rm_new_db *rm_db;
 	enum tf_rm_elem_cfg_type cfg_type;
 
-	if (parms == NULL || parms->rm_db == NULL)
-		return -EINVAL;
+	TF_CHECK_PARMS2(parms, parms->rm_db);
 
 	rm_db = (struct tf_rm_new_db *)parms->rm_db;
 	cfg_type = rm_db->db[parms->db_index].cfg_type;
@@ -420,6 +536,17 @@ tf_rm_is_allocated(struct tf_rm_is_allocated_parms *parms)
 	    cfg_type != TF_RM_ELEM_CFG_PRIVATE)
 		return -ENOTSUP;
 
+	/* Bail out if the pool is not valid, should never happen */
+	if (rm_db->db[parms->db_index].pool == NULL) {
+		rc = -ENOTSUP;
+		TFP_DRV_LOG(ERR,
+			    "%s: Invalid pool for this type:%d, rc:%s\n",
+			    tf_dir_2_str(rm_db->dir),
+			    parms->db_index,
+			    strerror(-rc));
+		return rc;
+	}
+
 	/* Adjust for any non zero start value */
 	rc = tf_rm_adjust_index(rm_db->db,
 				TF_RM_ADJUST_RM_BASE,
@@ -442,8 +569,7 @@ tf_rm_get_info(struct tf_rm_get_alloc_info_parms *parms)
 	struct tf_rm_new_db *rm_db;
 	enum tf_rm_elem_cfg_type cfg_type;
 
-	if (parms == NULL || parms->rm_db == NULL)
-		return -EINVAL;
+	TF_CHECK_PARMS2(parms, parms->rm_db);
 
 	rm_db = (struct tf_rm_new_db *)parms->rm_db;
 	cfg_type = rm_db->db[parms->db_index].cfg_type;
@@ -465,8 +591,7 @@ tf_rm_get_hcapi_type(struct tf_rm_get_hcapi_parms *parms)
 	struct tf_rm_new_db *rm_db;
 	enum tf_rm_elem_cfg_type cfg_type;
 
-	if (parms == NULL || parms->rm_db == NULL)
-		return -EINVAL;
+	TF_CHECK_PARMS2(parms, parms->rm_db);
 
 	rm_db = (struct tf_rm_new_db *)parms->rm_db;
 	cfg_type = rm_db->db[parms->db_index].cfg_type;
diff --git a/drivers/net/bnxt/tf_core/tf_rm_new.h b/drivers/net/bnxt/tf_core/tf_rm_new.h
index 6d8234d..ebf38c4 100644
--- a/drivers/net/bnxt/tf_core/tf_rm_new.h
+++ b/drivers/net/bnxt/tf_core/tf_rm_new.h
@@ -135,13 +135,16 @@ struct tf_rm_create_db_parms {
 	 */
 	struct tf_rm_element_cfg *cfg;
 	/**
-	 * Allocation number array. Array size is num_elements.
+	 * Resource allocation count array. This array content
+	 * originates from the tf_session_resources that is passed in
+	 * on session open.
+	 * Array size is num_elements.
 	 */
-	uint16_t *alloc_num;
+	uint16_t *alloc_cnt;
 	/**
 	 * [out] RM DB Handle
 	 */
-	void *rm_db;
+	void **rm_db;
 };
 
 /**
@@ -382,7 +385,7 @@ int tf_rm_get_info(struct tf_rm_get_alloc_info_parms *parms);
 
 /**
  * Performs a lookup in the Resource Manager DB and retrives the
- * requested HCAPI type.
+ * requested HCAPI RM type.
  *
  * [in] parms
  *   Pointer to get hcapi parameters
diff --git a/drivers/net/bnxt/tf_core/tf_session.c b/drivers/net/bnxt/tf_core/tf_session.c
index 2f769d8..bac9c76 100644
--- a/drivers/net/bnxt/tf_core/tf_session.c
+++ b/drivers/net/bnxt/tf_core/tf_session.c
@@ -95,21 +95,11 @@ tf_session_open_session(struct tf *tfp,
 		      parms->open_cfg->device_type,
 		      session->shadow_copy,
 		      &parms->open_cfg->resources,
-		      session->dev);
+		      &session->dev);
 	/* Logging handled by dev_bind */
 	if (rc)
 		return rc;
 
-	/* Query for Session Config
-	 */
-	rc = tf_msg_session_qcfg(tfp);
-	if (rc) {
-		TFP_DRV_LOG(ERR,
-			    "Query config message send failed, rc:%s\n",
-			    strerror(-rc));
-		goto cleanup_close;
-	}
-
 	session->ref_count++;
 
 	return 0;
@@ -119,40 +109,12 @@ tf_session_open_session(struct tf *tfp,
 	tfp_free(tfp->session);
 	tfp->session = NULL;
 	return rc;
-
- cleanup_close:
-	tf_close_session(tfp);
-	return -EINVAL;
 }
 
 int
 tf_session_attach_session(struct tf *tfp __rte_unused,
 			  struct tf_session_attach_session_parms *parms __rte_unused)
 {
-#if 0
-
-	/* A shared session is similar to single session. It consists
-	 * of two parts the tf_session_info element which remains
-	 * private to the caller and the session within this element
-	 * which is shared. The session it self holds the dynamic
-	 * data, i.e. the device and its sub modules.
-	 *
-	 * Firmware side is updated about any sharing as well.
-	 */
-
-	/* - Open the shared memory for the attach_chan_name
-	 * - Point to the shared session for this Device instance
-	 * - Check that session is valid
-	 * - Attach to the firmware so it can record there is more
-	 *   than one client of the session.
-	 */
-
-	if (tfp->session->session_id.id != TF_SESSION_ID_INVALID) {
-		rc = tf_msg_session_attach(tfp,
-					   parms->ctrl_chan_name,
-					   parms->session_id);
-	}
-#endif /* 0 */
 	int rc = -EOPNOTSUPP;
 
 	TF_CHECK_PARMS2(tfp, parms);
@@ -255,17 +217,7 @@ int
 tf_session_get_device(struct tf_session *tfs,
 		      struct tf_dev_info **tfd)
 {
-	int rc;
-
-	if (tfs->dev == NULL) {
-		rc = -EINVAL;
-		TFP_DRV_LOG(ERR,
-			    "Device not created, rc:%s\n",
-			    strerror(-rc));
-		return rc;
-	}
-
-	*tfd = tfs->dev;
+	*tfd = &tfs->dev;
 
 	return 0;
 }
diff --git a/drivers/net/bnxt/tf_core/tf_session.h b/drivers/net/bnxt/tf_core/tf_session.h
index 9279251..705bb09 100644
--- a/drivers/net/bnxt/tf_core/tf_session.h
+++ b/drivers/net/bnxt/tf_core/tf_session.h
@@ -97,7 +97,7 @@ struct tf_session {
 	uint8_t ref_count;
 
 	/** Device handle */
-	struct tf_dev_info *dev;
+	struct tf_dev_info dev;
 
 	/** Session HW and SRAM resources */
 	struct tf_rm_db resc;
diff --git a/drivers/net/bnxt/tf_core/tf_tbl.c b/drivers/net/bnxt/tf_core/tf_tbl.c
index 7f37f4d..b0a932b 100644
--- a/drivers/net/bnxt/tf_core/tf_tbl.c
+++ b/drivers/net/bnxt/tf_core/tf_tbl.c
@@ -760,163 +760,6 @@ tf_em_validate_num_entries(struct tf_tbl_scope_cb *tbl_scope_cb,
 }
 
 /**
- * Internal function to set a Table Entry. Supports all internal Table Types
- *
- * [in] tfp
- *   Pointer to TruFlow handle
- *
- * [in] parms
- *   Pointer to input parameters
- *
- * Returns:
- *   0       - Success
- *   -EINVAL - Parameter error
- */
-static int
-tf_set_tbl_entry_internal(struct tf *tfp,
-			  struct tf_set_tbl_entry_parms *parms)
-{
-	int rc;
-	int id;
-	uint32_t index;
-	struct bitalloc *session_pool;
-	struct tf_session *tfs = (struct tf_session *)(tfp->session->core_data);
-
-	/* Lookup the pool using the table type of the element */
-	rc = tf_rm_lookup_tbl_type_pool(tfs,
-					parms->dir,
-					parms->type,
-					&session_pool);
-	/* Error logging handled by tf_rm_lookup_tbl_type_pool */
-	if (rc)
-		return rc;
-
-	index = parms->idx;
-
-	if (parms->type != TF_TBL_TYPE_FULL_ACT_RECORD &&
-	    parms->type != TF_TBL_TYPE_ACT_SP_SMAC_IPV4 &&
-	    parms->type != TF_TBL_TYPE_ACT_STATS_64) {
-		TFP_DRV_LOG(ERR,
-			    "%s, Type not supported, type:%d\n",
-			    tf_dir_2_str(parms->dir),
-			    parms->type);
-		return -EOPNOTSUPP;
-	}
-
-	/* Adjust the returned index/offset as there is no guarantee
-	 * that the start is 0 at time of RM allocation
-	 */
-	tf_rm_convert_index(tfs,
-			    parms->dir,
-			    parms->type,
-			    TF_RM_CONVERT_RM_BASE,
-			    parms->idx,
-			    &index);
-
-	/* Verify that the entry has been previously allocated */
-	id = ba_inuse(session_pool, index);
-	if (id != 1) {
-		TFP_DRV_LOG(ERR,
-		   "%s, Invalid or not allocated index, type:%d, idx:%d\n",
-		   tf_dir_2_str(parms->dir),
-		   parms->type,
-		   index);
-		return -EINVAL;
-	}
-
-	/* Set the entry */
-	rc = tf_msg_set_tbl_entry(tfp,
-				  parms->dir,
-				  parms->type,
-				  parms->data_sz_in_bytes,
-				  parms->data,
-				  parms->idx);
-	if (rc) {
-		TFP_DRV_LOG(ERR,
-			    "%s, Set failed, type:%d, rc:%s\n",
-			    tf_dir_2_str(parms->dir),
-			    parms->type,
-			    strerror(-rc));
-	}
-
-	return rc;
-}
-
-/**
- * Internal function to get a Table Entry. Supports all Table Types
- * except the TF_TBL_TYPE_EXT as that is handled as a table scope.
- *
- * [in] tfp
- *   Pointer to TruFlow handle
- *
- * [in] parms
- *   Pointer to input parameters
- *
- * Returns:
- *   0       - Success
- *   -EINVAL - Parameter error
- */
-static int
-tf_get_tbl_entry_internal(struct tf *tfp,
-			  struct tf_get_tbl_entry_parms *parms)
-{
-	int rc;
-	int id;
-	uint32_t index;
-	struct bitalloc *session_pool;
-	struct tf_session *tfs = (struct tf_session *)(tfp->session->core_data);
-
-	/* Lookup the pool using the table type of the element */
-	rc = tf_rm_lookup_tbl_type_pool(tfs,
-					parms->dir,
-					parms->type,
-					&session_pool);
-	/* Error logging handled by tf_rm_lookup_tbl_type_pool */
-	if (rc)
-		return rc;
-
-	index = parms->idx;
-
-	/* Adjust the returned index/offset as there is no guarantee
-	 * that the start is 0 at time of RM allocation
-	 */
-	tf_rm_convert_index(tfs,
-			    parms->dir,
-			    parms->type,
-			    TF_RM_CONVERT_RM_BASE,
-			    parms->idx,
-			    &index);
-
-	/* Verify that the entry has been previously allocated */
-	id = ba_inuse(session_pool, index);
-	if (id != 1) {
-		TFP_DRV_LOG(ERR,
-		   "%s, Invalid or not allocated index, type:%d, idx:%d\n",
-		   tf_dir_2_str(parms->dir),
-		   parms->type,
-		   index);
-		return -EINVAL;
-	}
-
-	/* Get the entry */
-	rc = tf_msg_get_tbl_entry(tfp,
-				  parms->dir,
-				  parms->type,
-				  parms->data_sz_in_bytes,
-				  parms->data,
-				  parms->idx);
-	if (rc) {
-		TFP_DRV_LOG(ERR,
-			    "%s, Get failed, type:%d, rc:%s\n",
-			    tf_dir_2_str(parms->dir),
-			    parms->type,
-			    strerror(-rc));
-	}
-
-	return rc;
-}
-
-/**
  * Internal function to get a Table Entry. Supports all Table Types
  * except the TF_TBL_TYPE_EXT as that is handled as a table scope.
  *
@@ -1143,266 +986,6 @@ tf_destroy_tbl_pool_external(enum tf_dir dir,
 	tfp_free(ext_act_pool_mem);
 }
 
-/**
- * Allocate External Tbl entry from the Session Pool.
- *
- * [in] tfp
- *   Pointer to Truflow Handle
- * [in] parms
- *   Allocation parameters
- *
- * Return:
- *  0       - Success, entry allocated - no search support
- *  -ENOMEM -EINVAL -EOPNOTSUPP
- *          - Failure, entry not allocated, out of resources
- */
-static int
-tf_alloc_tbl_entry_pool_external(struct tf *tfp,
-				 struct tf_alloc_tbl_entry_parms *parms)
-{
-	int rc;
-	uint32_t index;
-	struct tf_session *tfs;
-	struct tf_tbl_scope_cb *tbl_scope_cb;
-	struct stack *pool;
-
-	TF_CHECK_PARMS_SESSION(tfp, parms);
-
-	tfs = (struct tf_session *)(tfp->session->core_data);
-
-	/* Get the pool info from the table scope
-	 */
-	tbl_scope_cb = tbl_scope_cb_find(tfs, parms->tbl_scope_id);
-
-	if (tbl_scope_cb == NULL) {
-		TFP_DRV_LOG(ERR,
-			    "%s, table scope not allocated\n",
-			    tf_dir_2_str(parms->dir));
-		return -EINVAL;
-	}
-	pool = &tbl_scope_cb->ext_act_pool[parms->dir];
-
-	/* Allocate an element
-	 */
-	rc = stack_pop(pool, &index);
-
-	if (rc != 0) {
-		TFP_DRV_LOG(ERR,
-		   "%s, Allocation failed, type:%d\n",
-		   tf_dir_2_str(parms->dir),
-		   parms->type);
-		return rc;
-	}
-	parms->idx = index;
-	return rc;
-}
-
-/**
- * Allocate Internal Tbl entry from the Session Pool.
- *
- * [in] tfp
- *   Pointer to Truflow Handle
- * [in] parms
- *   Allocation parameters
- *
- * Return:
- *  0       - Success, entry found and ref count decremented
- *  -ENOMEM - Failure, entry not allocated, out of resources
- */
-static int
-tf_alloc_tbl_entry_pool_internal(struct tf *tfp,
-				 struct tf_alloc_tbl_entry_parms *parms)
-{
-	int rc;
-	int id;
-	int free_cnt;
-	uint32_t index;
-	struct bitalloc *session_pool;
-	struct tf_session *tfs;
-
-	TF_CHECK_PARMS_SESSION(tfp, parms);
-
-	tfs = (struct tf_session *)(tfp->session->core_data);
-
-	if (parms->type != TF_TBL_TYPE_FULL_ACT_RECORD &&
-	    parms->type != TF_TBL_TYPE_ACT_SP_SMAC &&
-	    parms->type != TF_TBL_TYPE_ACT_SP_SMAC_IPV4 &&
-	    parms->type != TF_TBL_TYPE_ACT_ENCAP_8B &&
-	    parms->type != TF_TBL_TYPE_ACT_ENCAP_16B &&
-	    parms->type != TF_TBL_TYPE_ACT_ENCAP_64B &&
-	    parms->type != TF_TBL_TYPE_ACT_STATS_64) {
-		TFP_DRV_LOG(ERR,
-			    "%s, Type not supported, type:%d\n",
-			    tf_dir_2_str(parms->dir),
-			    parms->type);
-		return -EOPNOTSUPP;
-	}
-
-	/* Lookup the pool using the table type of the element */
-	rc = tf_rm_lookup_tbl_type_pool(tfs,
-					parms->dir,
-					parms->type,
-					&session_pool);
-	/* Error logging handled by tf_rm_lookup_tbl_type_pool */
-	if (rc)
-		return rc;
-
-	id = ba_alloc(session_pool);
-	if (id == -1) {
-		free_cnt = ba_free_count(session_pool);
-
-		TFP_DRV_LOG(ERR,
-		   "%s, Allocation failed, type:%d, free:%d\n",
-		   tf_dir_2_str(parms->dir),
-		   parms->type,
-		   free_cnt);
-		return -ENOMEM;
-	}
-
-	/* Adjust the returned index/offset as there is no guarantee
-	 * that the start is 0 at time of RM allocation
-	 */
-	tf_rm_convert_index(tfs,
-			    parms->dir,
-			    parms->type,
-			    TF_RM_CONVERT_ADD_BASE,
-			    id,
-			    &index);
-	parms->idx = index;
-	return rc;
-}
-
-/**
- * Free External Tbl entry to the session pool.
- *
- * [in] tfp
- *   Pointer to Truflow Handle
- * [in] parms
- *   Allocation parameters
- *
- * Return:
- *  0       - Success, entry freed
- *
- * - Failure, entry not successfully freed for these reasons
- *  -ENOMEM
- *  -EOPNOTSUPP
- *  -EINVAL
- */
-static int
-tf_free_tbl_entry_pool_external(struct tf *tfp,
-				struct tf_free_tbl_entry_parms *parms)
-{
-	int rc = 0;
-	struct tf_session *tfs;
-	uint32_t index;
-	struct tf_tbl_scope_cb *tbl_scope_cb;
-	struct stack *pool;
-
-	TF_CHECK_PARMS_SESSION(tfp, parms);
-
-	tfs = (struct tf_session *)(tfp->session->core_data);
-
-	/* Get the pool info from the table scope
-	 */
-	tbl_scope_cb = tbl_scope_cb_find(tfs, parms->tbl_scope_id);
-
-	if (tbl_scope_cb == NULL) {
-		TFP_DRV_LOG(ERR,
-			    "%s, table scope error\n",
-			    tf_dir_2_str(parms->dir));
-		return -EINVAL;
-	}
-	pool = &tbl_scope_cb->ext_act_pool[parms->dir];
-
-	index = parms->idx;
-
-	rc = stack_push(pool, index);
-
-	if (rc != 0) {
-		TFP_DRV_LOG(ERR,
-		   "%s, consistency error, stack full, type:%d, idx:%d\n",
-		   tf_dir_2_str(parms->dir),
-		   parms->type,
-		   index);
-	}
-	return rc;
-}
-
-/**
- * Free Internal Tbl entry from the Session Pool.
- *
- * [in] tfp
- *   Pointer to Truflow Handle
- * [in] parms
- *   Allocation parameters
- *
- * Return:
- *  0       - Success, entry found and ref count decremented
- *  -ENOMEM - Failure, entry not allocated, out of resources
- */
-static int
-tf_free_tbl_entry_pool_internal(struct tf *tfp,
-		       struct tf_free_tbl_entry_parms *parms)
-{
-	int rc = 0;
-	int id;
-	struct bitalloc *session_pool;
-	struct tf_session *tfs;
-	uint32_t index;
-
-	TF_CHECK_PARMS_SESSION(tfp, parms);
-
-	tfs = (struct tf_session *)(tfp->session->core_data);
-
-	if (parms->type != TF_TBL_TYPE_FULL_ACT_RECORD &&
-	    parms->type != TF_TBL_TYPE_ACT_SP_SMAC &&
-	    parms->type != TF_TBL_TYPE_ACT_SP_SMAC_IPV4 &&
-	    parms->type != TF_TBL_TYPE_ACT_ENCAP_8B &&
-	    parms->type != TF_TBL_TYPE_ACT_ENCAP_16B &&
-	    parms->type != TF_TBL_TYPE_ACT_ENCAP_64B &&
-	    parms->type != TF_TBL_TYPE_ACT_STATS_64) {
-		TFP_DRV_LOG(ERR,
-			    "%s, Type not supported, type:%d\n",
-			    tf_dir_2_str(parms->dir),
-			    parms->type);
-		return -EOPNOTSUPP;
-	}
-
-	/* Lookup the pool using the table type of the element */
-	rc = tf_rm_lookup_tbl_type_pool(tfs,
-					parms->dir,
-					parms->type,
-					&session_pool);
-	/* Error logging handled by tf_rm_lookup_tbl_type_pool */
-	if (rc)
-		return rc;
-
-	index = parms->idx;
-
-	/* Adjust the returned index/offset as there is no guarantee
-	 * that the start is 0 at time of RM allocation
-	 */
-	tf_rm_convert_index(tfs,
-			    parms->dir,
-			    parms->type,
-			    TF_RM_CONVERT_RM_BASE,
-			    parms->idx,
-			    &index);
-
-	/* Check if element was indeed allocated */
-	id = ba_inuse_free(session_pool, index);
-	if (id == -1) {
-		TFP_DRV_LOG(ERR,
-		   "%s, Element not previously alloc'ed, type:%d, idx:%d\n",
-		   tf_dir_2_str(parms->dir),
-		   parms->type,
-		   index);
-		return -ENOMEM;
-	}
-
-	return rc;
-}
-
 /* API defined in tf_em.h */
 struct tf_tbl_scope_cb *
 tbl_scope_cb_find(struct tf_session *session,
@@ -1584,113 +1167,6 @@ tf_alloc_eem_tbl_scope(struct tf *tfp,
 
 /* API defined in tf_core.h */
 int
-tf_set_tbl_entry(struct tf *tfp,
-		 struct tf_set_tbl_entry_parms *parms)
-{
-	int rc = 0;
-	struct tf_tbl_scope_cb *tbl_scope_cb;
-	struct tf_session *session;
-
-	TF_CHECK_PARMS_SESSION(tfp, parms);
-
-	if (parms->data == NULL) {
-		TFP_DRV_LOG(ERR,
-			    "%s, invalid parms->data\n",
-			    tf_dir_2_str(parms->dir));
-		return -EINVAL;
-	}
-
-	if (parms->type == TF_TBL_TYPE_EXT) {
-		void *base_addr;
-		uint32_t offset = parms->idx;
-		uint32_t tbl_scope_id;
-
-		session = (struct tf_session *)(tfp->session->core_data);
-
-		tbl_scope_id = parms->tbl_scope_id;
-
-		if (tbl_scope_id == TF_TBL_SCOPE_INVALID)  {
-			TFP_DRV_LOG(ERR,
-				    "%s, Table scope not allocated\n",
-				    tf_dir_2_str(parms->dir));
-			return -EINVAL;
-		}
-
-		/* Get the table scope control block associated with the
-		 * external pool
-		 */
-		tbl_scope_cb = tbl_scope_cb_find(session, tbl_scope_id);
-
-		if (tbl_scope_cb == NULL) {
-			TFP_DRV_LOG(ERR,
-				    "%s, table scope error\n",
-				    tf_dir_2_str(parms->dir));
-				return -EINVAL;
-		}
-
-		/* External table, implicitly the Action table */
-		base_addr = (void *)(uintptr_t)hcapi_get_table_page(
-			&tbl_scope_cb->em_ctx_info[parms->dir].em_tables[TF_RECORD_TABLE],
-			offset);
-
-		if (base_addr == NULL) {
-			TFP_DRV_LOG(ERR,
-				    "%s, Base address lookup failed\n",
-				    tf_dir_2_str(parms->dir));
-			return -EINVAL;
-		}
-
-		offset %= TF_EM_PAGE_SIZE;
-		rte_memcpy((char *)base_addr + offset,
-			   parms->data,
-			   parms->data_sz_in_bytes);
-	} else {
-		/* Internal table type processing */
-		rc = tf_set_tbl_entry_internal(tfp, parms);
-		if (rc) {
-			TFP_DRV_LOG(ERR,
-				    "%s, Set failed, type:%d, rc:%s\n",
-				    tf_dir_2_str(parms->dir),
-				    parms->type,
-				    strerror(-rc));
-		}
-	}
-
-	return rc;
-}
-
-/* API defined in tf_core.h */
-int
-tf_get_tbl_entry(struct tf *tfp,
-		 struct tf_get_tbl_entry_parms *parms)
-{
-	int rc = 0;
-
-	TF_CHECK_PARMS_SESSION(tfp, parms);
-
-	if (parms->type == TF_TBL_TYPE_EXT) {
-		/* Not supported, yet */
-		TFP_DRV_LOG(ERR,
-			    "%s, External table type not supported\n",
-			    tf_dir_2_str(parms->dir));
-
-		rc = -EOPNOTSUPP;
-	} else {
-		/* Internal table type processing */
-		rc = tf_get_tbl_entry_internal(tfp, parms);
-		if (rc)
-			TFP_DRV_LOG(ERR,
-				    "%s, Get failed, type:%d, rc:%s\n",
-				    tf_dir_2_str(parms->dir),
-				    parms->type,
-				    strerror(-rc));
-	}
-
-	return rc;
-}
-
-/* API defined in tf_core.h */
-int
 tf_bulk_get_tbl_entry(struct tf *tfp,
 		 struct tf_bulk_get_tbl_entry_parms *parms)
 {
@@ -1748,92 +1224,6 @@ tf_free_tbl_scope(struct tf *tfp,
 	return rc;
 }
 
-/* API defined in tf_core.h */
-int
-tf_alloc_tbl_entry(struct tf *tfp,
-		   struct tf_alloc_tbl_entry_parms *parms)
-{
-	int rc;
-#if (TF_SHADOW == 1)
-	struct tf_session *tfs;
-#endif /* TF_SHADOW */
-
-	TF_CHECK_PARMS_SESSION(tfp, parms);
-	/*
-	 * No shadow copy support for external tables, allocate and return
-	 */
-	if (parms->type == TF_TBL_TYPE_EXT) {
-		rc = tf_alloc_tbl_entry_pool_external(tfp, parms);
-		return rc;
-	}
-
-#if (TF_SHADOW == 1)
-	tfs = (struct tf_session *)(tfp->session->core_data);
-
-	/* Search the Shadow DB for requested element. If not found go
-	 * allocate one from the Session Pool
-	 */
-	if (parms->search_enable && tfs->shadow_copy) {
-		rc = tf_alloc_tbl_entry_shadow(tfs, parms);
-		/* Entry found and parms populated with return data */
-		if (rc == 0)
-			return rc;
-	}
-#endif /* TF_SHADOW */
-
-	rc = tf_alloc_tbl_entry_pool_internal(tfp, parms);
-	if (rc)
-		TFP_DRV_LOG(ERR, "%s, Alloc failed, rc:%s\n",
-			    tf_dir_2_str(parms->dir),
-			    strerror(-rc));
-
-	return rc;
-}
-
-/* API defined in tf_core.h */
-int
-tf_free_tbl_entry(struct tf *tfp,
-		  struct tf_free_tbl_entry_parms *parms)
-{
-	int rc;
-#if (TF_SHADOW == 1)
-	struct tf_session *tfs;
-#endif /* TF_SHADOW */
-
-	TF_CHECK_PARMS_SESSION(tfp, parms);
-
-	/*
-	 * No shadow of external tables so just free the entry
-	 */
-	if (parms->type == TF_TBL_TYPE_EXT) {
-		rc = tf_free_tbl_entry_pool_external(tfp, parms);
-		return rc;
-	}
-
-#if (TF_SHADOW == 1)
-	tfs = (struct tf_session *)(tfp->session->core_data);
-
-	/* Search the Shadow DB for requested element. If not found go
-	 * allocate one from the Session Pool
-	 */
-	if (parms->search_enable && tfs->shadow_copy) {
-		rc = tf_free_tbl_entry_shadow(tfs, parms);
-		/* Entry free'ed and parms populated with return data */
-		if (rc == 0)
-			return rc;
-	}
-#endif /* TF_SHADOW */
-
-	rc = tf_free_tbl_entry_pool_internal(tfp, parms);
-
-	if (rc)
-		TFP_DRV_LOG(ERR, "%s, Alloc failed, rc:%s\n",
-			    tf_dir_2_str(parms->dir),
-			    strerror(-rc));
-	return rc;
-}
-
-
 static void
 tf_dump_link_page_table(struct hcapi_cfa_em_page_tbl *tp,
 			struct hcapi_cfa_em_page_tbl *tp_next)
diff --git a/drivers/net/bnxt/tf_core/tf_tbl_type.c b/drivers/net/bnxt/tf_core/tf_tbl_type.c
index b79706f..51f8f07 100644
--- a/drivers/net/bnxt/tf_core/tf_tbl_type.c
+++ b/drivers/net/bnxt/tf_core/tf_tbl_type.c
@@ -6,13 +6,18 @@
 #include <rte_common.h>
 
 #include "tf_tbl_type.h"
+#include "tf_common.h"
+#include "tf_rm_new.h"
+#include "tf_util.h"
+#include "tf_msg.h"
+#include "tfp.h"
 
 struct tf;
 
 /**
  * Table DBs.
  */
-/* static void *tbl_db[TF_DIR_MAX]; */
+static void *tbl_db[TF_DIR_MAX];
 
 /**
  * Table Shadow DBs
@@ -22,7 +27,7 @@ struct tf;
 /**
  * Init flag, set on bind and cleared on unbind
  */
-/* static uint8_t init; */
+static uint8_t init;
 
 /**
  * Shadow init flag, set on bind and cleared on unbind
@@ -30,29 +35,164 @@ struct tf;
 /* static uint8_t shadow_init; */
 
 int
-tf_tbl_bind(struct tf *tfp __rte_unused,
-	    struct tf_tbl_cfg_parms *parms __rte_unused)
+tf_tbl_bind(struct tf *tfp,
+	    struct tf_tbl_cfg_parms *parms)
 {
+	int rc;
+	int i;
+	struct tf_rm_create_db_parms db_cfg = { 0 };
+
+	TF_CHECK_PARMS2(tfp, parms);
+
+	if (init) {
+		TFP_DRV_LOG(ERR,
+			    "Table already initialized\n");
+		return -EINVAL;
+	}
+
+	db_cfg.num_elements = parms->num_elements;
+
+	for (i = 0; i < TF_DIR_MAX; i++) {
+		db_cfg.dir = i;
+		db_cfg.num_elements = parms->num_elements;
+		db_cfg.cfg = parms->cfg;
+		db_cfg.alloc_cnt = parms->resources->tbl_cnt[i].cnt;
+		db_cfg.rm_db = &tbl_db[i];
+		rc = tf_rm_create_db(tfp, &db_cfg);
+		if (rc) {
+			TFP_DRV_LOG(ERR,
+				    "%s: Table DB creation failed\n",
+				    tf_dir_2_str(i));
+
+			return rc;
+		}
+	}
+
+	init = 1;
+
+	printf("Table Type - initialized\n");
+
 	return 0;
 }
 
 int
 tf_tbl_unbind(struct tf *tfp __rte_unused)
 {
+	int rc;
+	int i;
+	struct tf_rm_free_db_parms fparms = { 0 };
+
+	TF_CHECK_PARMS1(tfp);
+
+	/* Bail if nothing has been initialized done silent as to
+	 * allow for creation cleanup.
+	 */
+	if (!init) {
+		TFP_DRV_LOG(ERR,
+			    "No Table DBs created\n");
+		return -EINVAL;
+	}
+
+	for (i = 0; i < TF_DIR_MAX; i++) {
+		fparms.dir = i;
+		fparms.rm_db = tbl_db[i];
+		rc = tf_rm_free_db(tfp, &fparms);
+		if (rc)
+			return rc;
+
+		tbl_db[i] = NULL;
+	}
+
+	init = 0;
+
 	return 0;
 }
 
 int
 tf_tbl_alloc(struct tf *tfp __rte_unused,
-	     struct tf_tbl_alloc_parms *parms __rte_unused)
+	     struct tf_tbl_alloc_parms *parms)
 {
+	int rc;
+	uint32_t idx;
+	struct tf_rm_allocate_parms aparms = { 0 };
+
+	TF_CHECK_PARMS2(tfp, parms);
+
+	if (!init) {
+		TFP_DRV_LOG(ERR,
+			    "%s: No Table DBs created\n",
+			    tf_dir_2_str(parms->dir));
+		return -EINVAL;
+	}
+
+	/* Allocate requested element */
+	aparms.rm_db = tbl_db[parms->dir];
+	aparms.db_index = parms->type;
+	aparms.index = &idx;
+	rc = tf_rm_allocate(&aparms);
+	if (rc) {
+		TFP_DRV_LOG(ERR,
+			    "%s: Failed allocate, type:%d\n",
+			    tf_dir_2_str(parms->dir),
+			    parms->type);
+		return rc;
+	}
+
+	*parms->idx = idx;
+
 	return 0;
 }
 
 int
 tf_tbl_free(struct tf *tfp __rte_unused,
-	    struct tf_tbl_free_parms *parms __rte_unused)
+	    struct tf_tbl_free_parms *parms)
 {
+	int rc;
+	struct tf_rm_is_allocated_parms aparms = { 0 };
+	struct tf_rm_free_parms fparms = { 0 };
+	int allocated = 0;
+
+	TF_CHECK_PARMS2(tfp, parms);
+
+	if (!init) {
+		TFP_DRV_LOG(ERR,
+			    "%s: No Table DBs created\n",
+			    tf_dir_2_str(parms->dir));
+		return -EINVAL;
+	}
+
+	/* Check if element is in use */
+	aparms.rm_db = tbl_db[parms->dir];
+	aparms.db_index = parms->type;
+	aparms.index = parms->idx;
+	aparms.allocated = &allocated;
+	rc = tf_rm_is_allocated(&aparms);
+	if (rc)
+		return rc;
+
+	if (!allocated) {
+		TFP_DRV_LOG(ERR,
+			    "%s: Entry already free, type:%d, index:%d\n",
+			    tf_dir_2_str(parms->dir),
+			    parms->type,
+			    parms->idx);
+		return rc;
+	}
+
+	/* Free requested element */
+	fparms.rm_db = tbl_db[parms->dir];
+	fparms.db_index = parms->type;
+	fparms.index = parms->idx;
+	rc = tf_rm_free(&fparms);
+	if (rc) {
+		TFP_DRV_LOG(ERR,
+			    "%s: Free failed, type:%d, index:%d\n",
+			    tf_dir_2_str(parms->dir),
+			    parms->type,
+			    parms->idx);
+		return rc;
+	}
+
 	return 0;
 }
 
@@ -64,15 +204,107 @@ tf_tbl_alloc_search(struct tf *tfp __rte_unused,
 }
 
 int
-tf_tbl_set(struct tf *tfp __rte_unused,
-	   struct tf_tbl_set_parms *parms __rte_unused)
+tf_tbl_set(struct tf *tfp,
+	   struct tf_tbl_set_parms *parms)
 {
+	int rc;
+	struct tf_rm_is_allocated_parms aparms;
+	int allocated = 0;
+
+	TF_CHECK_PARMS3(tfp, parms, parms->data);
+
+	if (!init) {
+		TFP_DRV_LOG(ERR,
+			    "%s: No Table DBs created\n",
+			    tf_dir_2_str(parms->dir));
+		return -EINVAL;
+	}
+
+	/* Verify that the entry has been previously allocated */
+	aparms.rm_db = tbl_db[parms->dir];
+	aparms.db_index = parms->type;
+	aparms.index = parms->idx;
+	aparms.allocated = &allocated;
+	rc = tf_rm_is_allocated(&aparms);
+	if (rc)
+		return rc;
+
+	if (!allocated) {
+		TFP_DRV_LOG(ERR,
+		   "%s, Invalid or not allocated index, type:%d, idx:%d\n",
+		   tf_dir_2_str(parms->dir),
+		   parms->type,
+		   parms->idx);
+		return -EINVAL;
+	}
+
+	/* Set the entry */
+	rc = tf_msg_set_tbl_entry(tfp,
+				  parms->dir,
+				  parms->type,
+				  parms->data_sz_in_bytes,
+				  parms->data,
+				  parms->idx);
+	if (rc) {
+		TFP_DRV_LOG(ERR,
+			    "%s, Set failed, type:%d, rc:%s\n",
+			    tf_dir_2_str(parms->dir),
+			    parms->type,
+			    strerror(-rc));
+	}
+
 	return 0;
 }
 
 int
-tf_tbl_get(struct tf *tfp __rte_unused,
-	   struct tf_tbl_get_parms *parms __rte_unused)
+tf_tbl_get(struct tf *tfp,
+	   struct tf_tbl_get_parms *parms)
 {
+	int rc;
+	struct tf_rm_is_allocated_parms aparms;
+	int allocated = 0;
+
+	TF_CHECK_PARMS3(tfp, parms, parms->data);
+
+	if (!init) {
+		TFP_DRV_LOG(ERR,
+			    "%s: No Table DBs created\n",
+			    tf_dir_2_str(parms->dir));
+		return -EINVAL;
+	}
+
+	/* Verify that the entry has been previously allocated */
+	aparms.rm_db = tbl_db[parms->dir];
+	aparms.db_index = parms->type;
+	aparms.index = parms->idx;
+	aparms.allocated = &allocated;
+	rc = tf_rm_is_allocated(&aparms);
+	if (rc)
+		return rc;
+
+	if (!allocated) {
+		TFP_DRV_LOG(ERR,
+		   "%s, Invalid or not allocated index, type:%d, idx:%d\n",
+		   tf_dir_2_str(parms->dir),
+		   parms->type,
+		   parms->idx);
+		return -EINVAL;
+	}
+
+	/* Get the entry */
+	rc = tf_msg_get_tbl_entry(tfp,
+				  parms->dir,
+				  parms->type,
+				  parms->data_sz_in_bytes,
+				  parms->data,
+				  parms->idx);
+	if (rc) {
+		TFP_DRV_LOG(ERR,
+			    "%s, Get failed, type:%d, rc:%s\n",
+			    tf_dir_2_str(parms->dir),
+			    parms->type,
+			    strerror(-rc));
+	}
+
 	return 0;
 }
diff --git a/drivers/net/bnxt/tf_core/tf_tbl_type.h b/drivers/net/bnxt/tf_core/tf_tbl_type.h
index 11f2aa3..3474489 100644
--- a/drivers/net/bnxt/tf_core/tf_tbl_type.h
+++ b/drivers/net/bnxt/tf_core/tf_tbl_type.h
@@ -55,7 +55,7 @@ struct tf_tbl_alloc_parms {
 	/**
 	 * [out] Idx of allocated entry or found entry (if search_enable)
 	 */
-	uint32_t idx;
+	uint32_t *idx;
 };
 
 /**
diff --git a/drivers/net/bnxt/tf_core/tf_tcam.c b/drivers/net/bnxt/tf_core/tf_tcam.c
index b9dba53..e0fac31 100644
--- a/drivers/net/bnxt/tf_core/tf_tcam.c
+++ b/drivers/net/bnxt/tf_core/tf_tcam.c
@@ -38,8 +38,8 @@ static uint8_t init;
 /* static uint8_t shadow_init; */
 
 int
-tf_tcam_bind(struct tf *tfp __rte_unused,
-	     struct tf_tcam_cfg_parms *parms __rte_unused)
+tf_tcam_bind(struct tf *tfp,
+	     struct tf_tcam_cfg_parms *parms)
 {
 	int rc;
 	int i;
@@ -59,8 +59,8 @@ tf_tcam_bind(struct tf *tfp __rte_unused,
 		db_cfg.dir = i;
 		db_cfg.num_elements = parms->num_elements;
 		db_cfg.cfg = parms->cfg;
-		db_cfg.alloc_num = parms->resources->tcam_tbl_cnt[i];
-		db_cfg.rm_db = tcam_db[i];
+		db_cfg.alloc_cnt = parms->resources->tcam_cnt[i].cnt;
+		db_cfg.rm_db = &tcam_db[i];
 		rc = tf_rm_create_db(tfp, &db_cfg);
 		if (rc) {
 			TFP_DRV_LOG(ERR,
@@ -72,11 +72,13 @@ tf_tcam_bind(struct tf *tfp __rte_unused,
 
 	init = 1;
 
+	printf("TCAM - initialized\n");
+
 	return 0;
 }
 
 int
-tf_tcam_unbind(struct tf *tfp __rte_unused)
+tf_tcam_unbind(struct tf *tfp)
 {
 	int rc;
 	int i;
diff --git a/drivers/net/bnxt/tf_core/tf_util.h b/drivers/net/bnxt/tf_core/tf_util.h
index 4099629..ad8edaf 100644
--- a/drivers/net/bnxt/tf_core/tf_util.h
+++ b/drivers/net/bnxt/tf_core/tf_util.h
@@ -10,32 +10,57 @@
 
 /**
  * Helper function converting direction to text string
+ *
+ * [in] dir
+ *   Receive or transmit direction identifier
+ *
+ * Returns:
+ *   Pointer to a char string holding the string for the direction
  */
-const char
-*tf_dir_2_str(enum tf_dir dir);
+const char *tf_dir_2_str(enum tf_dir dir);
 
 /**
  * Helper function converting identifier to text string
+ *
+ * [in] id_type
+ *   Identifier type
+ *
+ * Returns:
+ *   Pointer to a char string holding the string for the identifier
  */
-const char
-*tf_ident_2_str(enum tf_identifier_type id_type);
+const char *tf_ident_2_str(enum tf_identifier_type id_type);
 
 /**
  * Helper function converting tcam type to text string
+ *
+ * [in] tcam_type
+ *   TCAM type
+ *
+ * Returns:
+ *   Pointer to a char string holding the string for the tcam
  */
-const char
-*tf_tcam_tbl_2_str(enum tf_tcam_tbl_type tcam_type);
+const char *tf_tcam_tbl_2_str(enum tf_tcam_tbl_type tcam_type);
 
 /**
  * Helper function converting tbl type to text string
+ *
+ * [in] tbl_type
+ *   Table type
+ *
+ * Returns:
+ *   Pointer to a char string holding the string for the table type
  */
-const char
-*tf_tbl_type_2_str(enum tf_tbl_type tbl_type);
+const char *tf_tbl_type_2_str(enum tf_tbl_type tbl_type);
 
 /**
  * Helper function converting em tbl type to text string
+ *
+ * [in] em_type
+ *   EM type
+ *
+ * Returns:
+ *   Pointer to a char string holding the string for the EM type
  */
-const char
-*tf_em_tbl_type_2_str(enum tf_em_tbl_type em_type);
+const char *tf_em_tbl_type_2_str(enum tf_em_tbl_type em_type);
 
 #endif /* _TF_UTIL_H_ */
diff --git a/drivers/net/bnxt/tf_core/tfp.c b/drivers/net/bnxt/tf_core/tfp.c
index 3bce3ad..69d1c9a 100644
--- a/drivers/net/bnxt/tf_core/tfp.c
+++ b/drivers/net/bnxt/tf_core/tfp.c
@@ -102,13 +102,13 @@ tfp_calloc(struct tfp_calloc_parms *parms)
 				    (parms->nitems * parms->size),
 				    parms->alignment);
 	if (parms->mem_va == NULL) {
-		PMD_DRV_LOG(ERR, "Allocate failed mem_va\n");
+		TFP_DRV_LOG(ERR, "Allocate failed mem_va\n");
 		return -ENOMEM;
 	}
 
 	parms->mem_pa = (void *)((uintptr_t)rte_mem_virt2iova(parms->mem_va));
 	if (parms->mem_pa == (void *)((uintptr_t)RTE_BAD_IOVA)) {
-		PMD_DRV_LOG(ERR, "Allocate failed mem_pa\n");
+		TFP_DRV_LOG(ERR, "Allocate failed mem_pa\n");
 		return -ENOMEM;
 	}
 
-- 
2.7.4


^ permalink raw reply	[flat|nested] 271+ messages in thread

* [dpdk-dev] [PATCH 19/50] net/bnxt: update identifier with remap support
  2020-06-12 13:28 [dpdk-dev] [PATCH 00/50] add features for host-based flow management Somnath Kotur
                   ` (17 preceding siblings ...)
  2020-06-12 13:29 ` [dpdk-dev] [PATCH 18/50] net/bnxt: multiple device implementation Somnath Kotur
@ 2020-06-12 13:29 ` Somnath Kotur
  2020-06-12 13:29 ` [dpdk-dev] [PATCH 20/50] net/bnxt: update RM with residual checker Somnath Kotur
                   ` (31 subsequent siblings)
  50 siblings, 0 replies; 271+ messages in thread
From: Somnath Kotur @ 2020-06-12 13:29 UTC (permalink / raw)
  To: dev; +Cc: ferruh.yigit

From: Michael Wildt <michael.wildt@broadcom.com>

- Add Identifier L2 CTXT Remap to the P4 device and updated the
  cfa_resource_types.h to get the type support.

Signed-off-by: Michael Wildt <michael.wildt@broadcom.com>
Reviewed-by: Randy Schacher <stuart.schacher@broadcom.com>
Reviewed-by: Ajit Kumar Khaparde <ajit.khaparde@broadcom.com>
Signed-off-by: Venkat Duvvuru <venkatkumar.duvvuru@broadcom.com>
---
 drivers/net/bnxt/tf_core/cfa_resource_types.h | 110 ++++++++++++++----------