From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id AFFB245C9F; Thu, 7 Nov 2024 14:47:10 +0100 (CET) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 1F15B42F6F; Thu, 7 Nov 2024 14:46:44 +0100 (CET) Received: from mail-pj1-f66.google.com (mail-pj1-f66.google.com [209.85.216.66]) by mails.dpdk.org (Postfix) with ESMTP id 623AE42F6F for ; Thu, 7 Nov 2024 14:46:42 +0100 (CET) Received: by mail-pj1-f66.google.com with SMTP id 98e67ed59e1d1-2e56df894d4so751106a91.3 for ; Thu, 07 Nov 2024 05:46:42 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=broadcom.com; s=google; t=1730987201; x=1731592001; darn=dpdk.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=9yUgEBsBA7KawewB8flRb10LzxWS1XOu+XEH+kFt32g=; b=EK+Hc/uhSxwWBGoSK7moDErbOxU2LzQRYgeCSJHltx0p4oTQTsuy8u1jTGP0i0V1Px une1br06Qc2JmT2K+1wgs4yw09VNd+7FtsmudKocmksbQ6R2YOFIpeR0AcIqmX3xg5JV CW1vXwBixNvQZf64X6X3ot5tW4wTRzDoQQ+h4= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1730987201; x=1731592001; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=9yUgEBsBA7KawewB8flRb10LzxWS1XOu+XEH+kFt32g=; b=f+0QrNqgB2KhTVMabf7gNuxNvYNXA98Q7ubi3nbxEloNmvpzVMnfVAwYm+35T18TW6 dxZdDwDttQGw4ftsUCkJ5c5aCmbfY9OrD1zZ/78Nor7tOTBg/gEwJGApz7jRut+xY7pn ox3Ca2ReW4CU53s0DL8KNSgrOg2d5Ixg1/BYilnYvNFXynl2HDmjY9W99C7pJdjswHyi pg65vs260ESUwmxauWUa5Frui3IsPQWnr+3gmLQhcyUvdCzQuLt2OaHUDlyKS4+tT4ZC PbGG/16GxIxmClMQuDWS/tRa79jzMUGn2GkqQHrmjhwRWs1uLnShJV3EuC9gRuLGp40o 71jQ== X-Gm-Message-State: AOJu0YwjNYqH0NdHnsRakjZWDlIW0PbGjK/x96nJfmID4RQTsUakhNe/ lc152zxpplJD1wv1RfAM4eKMtLEzU2kaQ7ksJTmLb/UbOKU1pKYEfIYxrlFvUDJvqzBVeSGUdAl gVFYRjqSL+YtTmbQSDx2YcA30nLDJIBkWi2kSYXphdhwzj+aZnm4pGrUKQLiypl9ud+FNCrbHJp mTMUqmXUnL1Umyf6+Ah5b82XyQA/FxCrN6jCB83PywS3FH X-Google-Smtp-Source: AGHT+IHBNZ9Myci2oQeFppHbe/DDOy3QysSfdoltIQmgvDqKqZ7cXzUNg7Dj7Z7AxaQci89Rqofc2w== X-Received: by 2002:a17:90b:3b50:b0:2e2:857e:fcfb with SMTP id 98e67ed59e1d1-2e9af9d678dmr68070a91.19.1730987199239; Thu, 07 Nov 2024 05:46:39 -0800 (PST) Received: from dhcp-10-123-154-23.dhcp.broadcom.net ([192.19.234.250]) by smtp.gmail.com with ESMTPSA id 98e67ed59e1d1-2e9a5f52969sm1409878a91.2.2024.11.07.05.46.35 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 07 Nov 2024 05:46:38 -0800 (PST) From: Sriharsha Basavapatna To: dev@dpdk.org Cc: Peter Spreadborough , Sriharsha Basavapatna , Kishore Padmanabha , Ajit Khaparde Subject: [PATCH v8 38/47] net/bnxt: tf_ulp: inline utility functions and use likely/unlikely Date: Thu, 7 Nov 2024 19:22:45 +0530 Message-Id: <20241107135254.1611676-39-sriharsha.basavapatna@broadcom.com> X-Mailer: git-send-email 2.39.0.189.g4dbebc36b0 In-Reply-To: <20241107135254.1611676-1-sriharsha.basavapatna@broadcom.com> References: <20241107135254.1611676-1-sriharsha.basavapatna@broadcom.com> MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org From: Peter Spreadborough This change makes performance improvements by inlining frequently used utility functions and by helping branch prediction by using the likely/unlikely compiler hints. Signed-off-by: Peter Spreadborough Signed-off-by: Sriharsha Basavapatna Reviewed-by: Kishore Padmanabha Reviewed-by: Ajit Khaparde --- drivers/net/bnxt/bnxt_ethdev.c | 1 + drivers/net/bnxt/bnxt_hwrm.c | 1 + drivers/net/bnxt/bnxt_reps.c | 1 + drivers/net/bnxt/tf_ulp/bnxt_tf_common.h | 7 - drivers/net/bnxt/tf_ulp/bnxt_ulp.c | 1041 +--------------------- drivers/net/bnxt/tf_ulp/bnxt_ulp.h | 364 +------- drivers/net/bnxt/tf_ulp/bnxt_ulp_flow.c | 200 ++--- drivers/net/bnxt/tf_ulp/bnxt_ulp_meter.c | 1 + drivers/net/bnxt/tf_ulp/bnxt_ulp_tf.c | 1 + drivers/net/bnxt/tf_ulp/bnxt_ulp_tfc.c | 1 + drivers/net/bnxt/tf_ulp/bnxt_ulp_utils.h | 1038 +++++++++++++++++++++ drivers/net/bnxt/tf_ulp/meson.build | 1 - drivers/net/bnxt/tf_ulp/ulp_alloc_tbl.c | 1 + drivers/net/bnxt/tf_ulp/ulp_def_rules.c | 1 + drivers/net/bnxt/tf_ulp/ulp_fc_mgr.c | 1 + drivers/net/bnxt/tf_ulp/ulp_fc_mgr_tf.c | 1 + drivers/net/bnxt/tf_ulp/ulp_fc_mgr_tfc.c | 1 + drivers/net/bnxt/tf_ulp/ulp_flow_db.c | 2 +- drivers/net/bnxt/tf_ulp/ulp_gen_tbl.c | 2 +- drivers/net/bnxt/tf_ulp/ulp_ha_mgr.c | 1 + drivers/net/bnxt/tf_ulp/ulp_mapper.c | 721 +++++++-------- drivers/net/bnxt/tf_ulp/ulp_mapper_tf.c | 3 +- drivers/net/bnxt/tf_ulp/ulp_mapper_tfc.c | 233 ++--- drivers/net/bnxt/tf_ulp/ulp_mark_mgr.c | 1 + drivers/net/bnxt/tf_ulp/ulp_matcher.c | 1 + drivers/net/bnxt/tf_ulp/ulp_port_db.c | 1 + drivers/net/bnxt/tf_ulp/ulp_rte_parser.c | 75 +- drivers/net/bnxt/tf_ulp/ulp_tun.c | 1 + drivers/net/bnxt/tf_ulp/ulp_utils.h | 915 ++++++++++++++++--- 29 files changed, 2528 insertions(+), 2090 deletions(-) create mode 100644 drivers/net/bnxt/tf_ulp/bnxt_ulp_utils.h diff --git a/drivers/net/bnxt/bnxt_ethdev.c b/drivers/net/bnxt/bnxt_ethdev.c index 2a0b06c97b..b905554d91 100644 --- a/drivers/net/bnxt/bnxt_ethdev.c +++ b/drivers/net/bnxt/bnxt_ethdev.c @@ -33,6 +33,7 @@ #include "bnxt_tf_common.h" #include "ulp_flow_db.h" #include "rte_pmd_bnxt.h" +#include "bnxt_ulp_utils.h" #define DRV_MODULE_NAME "bnxt" static const char bnxt_version[] = diff --git a/drivers/net/bnxt/bnxt_hwrm.c b/drivers/net/bnxt/bnxt_hwrm.c index a043bd4f6a..1ac4b8cd58 100644 --- a/drivers/net/bnxt/bnxt_hwrm.c +++ b/drivers/net/bnxt/bnxt_hwrm.c @@ -23,6 +23,7 @@ #include "bnxt_txr.h" #include "bnxt_vnic.h" #include "hsi_struct_def_dpdk.h" +#include "bnxt_ulp_utils.h" struct bnxt_plcmodes_cfg { uint32_t flags; diff --git a/drivers/net/bnxt/bnxt_reps.c b/drivers/net/bnxt/bnxt_reps.c index 00900ad631..6c431c7dd8 100644 --- a/drivers/net/bnxt/bnxt_reps.c +++ b/drivers/net/bnxt/bnxt_reps.c @@ -13,6 +13,7 @@ #include "bnxt_hwrm.h" #include "hsi_struct_def_dpdk.h" #include "bnxt_tf_common.h" +#include "bnxt_ulp_utils.h" #include "ulp_port_db.h" #include "ulp_flow_db.h" diff --git a/drivers/net/bnxt/tf_ulp/bnxt_tf_common.h b/drivers/net/bnxt/tf_ulp/bnxt_tf_common.h index 453a189e16..4d9a1a7790 100644 --- a/drivers/net/bnxt/tf_ulp/bnxt_tf_common.h +++ b/drivers/net/bnxt/tf_ulp/bnxt_tf_common.h @@ -67,11 +67,4 @@ enum bnxt_ulp_intf_type { BNXT_ULP_INTF_TYPE_LAST }; -struct bnxt_ulp_mark_tbl * -bnxt_ulp_cntxt_ptr2_mark_db_get(struct bnxt_ulp_context *ulp_ctx); - -int32_t -bnxt_ulp_cntxt_ptr2_mark_db_set(struct bnxt_ulp_context *ulp_ctx, - struct bnxt_ulp_mark_tbl *mark_tbl); - #endif /* _BNXT_TF_COMMON_H_ */ diff --git a/drivers/net/bnxt/tf_ulp/bnxt_ulp.c b/drivers/net/bnxt/tf_ulp/bnxt_ulp.c index 2b5f1dc4fd..e28a481f5e 100644 --- a/drivers/net/bnxt/tf_ulp/bnxt_ulp.c +++ b/drivers/net/bnxt/tf_ulp/bnxt_ulp.c @@ -12,6 +12,7 @@ #include "bnxt.h" #include "bnxt_ulp.h" +#include "bnxt_ulp_utils.h" #include "bnxt_tf_common.h" #include "bnxt_hwrm.h" #include "hsi_struct_def_dpdk.h" @@ -30,7 +31,6 @@ #include "ulp_ha_mgr.h" #include "bnxt_tf_pmd_shim.h" #include "ulp_template_db_tbl.h" -#include "ulp_utils.h" /* Linked list of all TF sessions. */ STAILQ_HEAD(, bnxt_ulp_session_state) bnxt_ulp_session_list = @@ -49,7 +49,7 @@ static struct cntx_list_entry_list ulp_cntx_list = bool ulp_is_default_session_active(struct bnxt_ulp_context *ulp_ctx) { - if (ulp_ctx == NULL || ulp_ctx->g_tfp[0] == NULL) + if (unlikely(ulp_ctx == NULL || ulp_ctx->g_tfp[0] == NULL)) return false; return true; @@ -62,7 +62,7 @@ ulp_is_default_session_active(struct bnxt_ulp_context *ulp_ctx) bool ulp_ctx_deinit_allowed(struct bnxt_ulp_context *ulp_ctx) { - if (!ulp_ctx || !ulp_ctx->cfg_data) + if (unlikely(!ulp_ctx || !ulp_ctx->cfg_data)) return false; if (!ulp_ctx->cfg_data->ref_cnt) { @@ -73,249 +73,6 @@ ulp_ctx_deinit_allowed(struct bnxt_ulp_context *ulp_ctx) return false; } -int32_t -bnxt_ulp_devid_get(struct bnxt *bp, - enum bnxt_ulp_device_id *ulp_dev_id) -{ - if (BNXT_CHIP_P7(bp)) { - *ulp_dev_id = BNXT_ULP_DEVICE_ID_THOR2; - return 0; - } - - if (BNXT_CHIP_P5(bp)) { - *ulp_dev_id = BNXT_ULP_DEVICE_ID_THOR; - return 0; - } - - if (BNXT_STINGRAY(bp)) - *ulp_dev_id = BNXT_ULP_DEVICE_ID_STINGRAY; - else - /* Assuming Whitney */ - *ulp_dev_id = BNXT_ULP_DEVICE_ID_WH_PLUS; - - return 0; -} - -struct bnxt_ulp_app_capabilities_info * -bnxt_ulp_app_cap_list_get(uint32_t *num_entries) -{ - if (!num_entries) - return NULL; - *num_entries = BNXT_ULP_APP_CAP_TBL_MAX_SZ; - return ulp_app_cap_info_list; -} - -struct bnxt_ulp_shared_act_info * -bnxt_ulp_shared_act_info_get(uint32_t *num_entries) -{ - if (!num_entries) - return NULL; - - *num_entries = BNXT_ULP_GEN_TBL_MAX_SZ; - - return ulp_shared_act_info; -} - -struct bnxt_ulp_resource_resv_info * -bnxt_ulp_app_resource_resv_list_get(uint32_t *num_entries) -{ - if (num_entries == NULL) - return NULL; - *num_entries = BNXT_ULP_APP_RESOURCE_RESV_LIST_MAX_SZ; - return ulp_app_resource_resv_list; -} - -struct bnxt_ulp_resource_resv_info * -bnxt_ulp_resource_resv_list_get(uint32_t *num_entries) -{ - if (!num_entries) - return NULL; - *num_entries = BNXT_ULP_RESOURCE_RESV_LIST_MAX_SZ; - return ulp_resource_resv_list; -} - -struct bnxt_ulp_glb_resource_info * -bnxt_ulp_app_glb_resource_info_list_get(uint32_t *num_entries) -{ - if (!num_entries) - return NULL; - *num_entries = BNXT_ULP_APP_GLB_RESOURCE_TBL_MAX_SZ; - return ulp_app_glb_resource_tbl; -} - -/* Function to set the number for vxlan_ip (custom vxlan) port into the context */ -int -bnxt_ulp_cntxt_ecpri_udp_port_set(struct bnxt_ulp_context *ulp_ctx, - uint32_t ecpri_udp_port) -{ - if (!ulp_ctx || !ulp_ctx->cfg_data) - return -EINVAL; - - ulp_ctx->cfg_data->ecpri_udp_port = ecpri_udp_port; - - return 0; -} - -/* Function to retrieve the vxlan_ip (custom vxlan) port from the context. */ -unsigned int -bnxt_ulp_cntxt_ecpri_udp_port_get(struct bnxt_ulp_context *ulp_ctx) -{ - if (!ulp_ctx || !ulp_ctx->cfg_data) - return 0; - - return (unsigned int)ulp_ctx->cfg_data->ecpri_udp_port; -} - -/* Function to set the number for vxlan_ip (custom vxlan) port into the context */ -int -bnxt_ulp_cntxt_vxlan_ip_port_set(struct bnxt_ulp_context *ulp_ctx, - uint32_t vxlan_ip_port) -{ - if (!ulp_ctx || !ulp_ctx->cfg_data) - return -EINVAL; - - ulp_ctx->cfg_data->vxlan_ip_port = vxlan_ip_port; - if (vxlan_ip_port) - ulp_ctx->cfg_data->ulp_flags |= BNXT_ULP_STATIC_VXLAN_SUPPORT; - return 0; -} - -/* Function to retrieve the vxlan_ip (custom vxlan) port from the context. */ -unsigned int -bnxt_ulp_cntxt_vxlan_ip_port_get(struct bnxt_ulp_context *ulp_ctx) -{ - if (!ulp_ctx || !ulp_ctx->cfg_data) - return 0; - - return (unsigned int)ulp_ctx->cfg_data->vxlan_ip_port; -} - -/* Function to set the number for vxlan_gpe next_proto into the context */ -uint32_t -bnxt_ulp_vxlan_gpe_next_proto_set(struct bnxt_ulp_context *ulp_ctx, - uint8_t tunnel_next_proto) -{ - if (!ulp_ctx || !ulp_ctx->cfg_data) - return -EINVAL; - - ulp_ctx->cfg_data->tunnel_next_proto = tunnel_next_proto; - - return 0; -} - -/* Function to retrieve the vxlan_gpe next_proto from the context. */ -uint8_t -bnxt_ulp_vxlan_gpe_next_proto_get(struct bnxt_ulp_context *ulp_ctx) -{ - if (!ulp_ctx || !ulp_ctx->cfg_data) - return 0; - - return ulp_ctx->cfg_data->tunnel_next_proto; -} - -/* Function to set the number for vxlan port into the context */ -int -bnxt_ulp_cntxt_vxlan_port_set(struct bnxt_ulp_context *ulp_ctx, - uint32_t vxlan_port) -{ - if (!ulp_ctx || !ulp_ctx->cfg_data) - return -EINVAL; - - ulp_ctx->cfg_data->vxlan_port = vxlan_port; - if (vxlan_port) - ulp_ctx->cfg_data->ulp_flags |= BNXT_ULP_STATIC_VXLAN_SUPPORT; - - return 0; -} - -/* Function to retrieve the vxlan port from the context. */ -unsigned int -bnxt_ulp_cntxt_vxlan_port_get(struct bnxt_ulp_context *ulp_ctx) -{ - if (!ulp_ctx || !ulp_ctx->cfg_data) - return 0; - - return (unsigned int)ulp_ctx->cfg_data->vxlan_port; -} - -int -bnxt_ulp_default_app_priority_set(struct bnxt_ulp_context *ulp_ctx, - uint32_t prio) -{ - if (!ulp_ctx || !ulp_ctx->cfg_data) - return -EINVAL; - - ulp_ctx->cfg_data->default_priority = prio; - return 0; -} - -unsigned int -bnxt_ulp_default_app_priority_get(struct bnxt_ulp_context *ulp_ctx) -{ - if (!ulp_ctx || !ulp_ctx->cfg_data) - return 0; - - return (unsigned int)ulp_ctx->cfg_data->default_priority; -} - -int -bnxt_ulp_max_def_priority_set(struct bnxt_ulp_context *ulp_ctx, - uint32_t prio) -{ - if (!ulp_ctx || !ulp_ctx->cfg_data) - return -EINVAL; - - ulp_ctx->cfg_data->max_def_priority = prio; - return 0; -} - -unsigned int -bnxt_ulp_max_def_priority_get(struct bnxt_ulp_context *ulp_ctx) -{ - if (!ulp_ctx || !ulp_ctx->cfg_data) - return 0; - - return (unsigned int)ulp_ctx->cfg_data->max_def_priority; -} - -int -bnxt_ulp_min_flow_priority_set(struct bnxt_ulp_context *ulp_ctx, uint32_t prio) -{ - if (!ulp_ctx || !ulp_ctx->cfg_data) - return -EINVAL; - - ulp_ctx->cfg_data->min_flow_priority = prio; - return 0; -} - -unsigned int -bnxt_ulp_min_flow_priority_get(struct bnxt_ulp_context *ulp_ctx) -{ - if (!ulp_ctx || !ulp_ctx->cfg_data) - return 0; - - return ulp_ctx->cfg_data->min_flow_priority; -} - -int -bnxt_ulp_max_flow_priority_set(struct bnxt_ulp_context *ulp_ctx, uint32_t prio) -{ - if (!ulp_ctx || !ulp_ctx->cfg_data) - return -EINVAL; - - ulp_ctx->cfg_data->max_flow_priority = prio; - return 0; -} - -unsigned int -bnxt_ulp_max_flow_priority_get(struct bnxt_ulp_context *ulp_ctx) -{ - if (!ulp_ctx || !ulp_ctx->cfg_data) - return 0; - - return ulp_ctx->cfg_data->max_flow_priority; -} - /* The function to initialize bp flags with truflow features */ static int32_t ulp_dparms_dev_port_intf_update(struct bnxt *bp, @@ -323,7 +80,7 @@ ulp_dparms_dev_port_intf_update(struct bnxt *bp, { enum bnxt_ulp_flow_mem_type mtype; - if (bnxt_ulp_cntxt_mem_type_get(ulp_ctx, &mtype)) + if (unlikely(bnxt_ulp_cntxt_mem_type_get(ulp_ctx, &mtype))) return -EINVAL; /* Update the bp flag with gfid flag */ if (mtype == BNXT_ULP_FLOW_MEM_TYPE_EXT) @@ -355,7 +112,7 @@ ulp_ctx_mh_get_session_name(struct bnxt *bp, &bus, &slot, &device); - if (rc != 3) { + if (unlikely(rc != 3)) { BNXT_DRV_DBG(DEBUG, "Failed to scan device ctrl_chan_name\n"); return -EINVAL; @@ -500,9 +257,9 @@ bnxt_ulp_flush_port_flows(struct bnxt *bp) uint16_t func_id; /* it is assumed that port is either TVF or PF */ - if (ulp_port_db_port_func_id_get(bp->ulp_ctx, - bp->eth_dev->data->port_id, - &func_id)) { + if (unlikely(ulp_port_db_port_func_id_get(bp->ulp_ctx, + bp->eth_dev->data->port_id, + &func_id))) { BNXT_DRV_DBG(ERR, "Invalid argument\n"); return; } @@ -518,10 +275,11 @@ bnxt_ulp_destroy_vfr_default_rules(struct bnxt *bp, bool global) struct rte_eth_dev *vfr_eth_dev; struct bnxt_representor *vfr_bp; - if (!BNXT_TRUFLOW_EN(bp) || BNXT_ETH_DEV_IS_REPRESENTOR(bp->eth_dev)) + if (unlikely(!BNXT_TRUFLOW_EN(bp) || + BNXT_ETH_DEV_IS_REPRESENTOR(bp->eth_dev))) return; - if (!bp->ulp_ctx || !bp->ulp_ctx->cfg_data) + if (unlikely(!bp->ulp_ctx || !bp->ulp_ctx->cfg_data)) return; /* Delete default rules for all ports */ @@ -556,12 +314,12 @@ ulp_l2_etype_tunnel_alloc(struct bnxt *bp) if (bp->l2_etype_tunnel_cnt) { BNXT_DRV_DBG(DEBUG, "L2 ETYPE Custom Tunnel already allocated\n"); - return 0; + return rc; } rc = bnxt_tunnel_dst_port_alloc(bp, BNXT_L2_ETYPE_TUNNEL_ID, HWRM_TUNNEL_DST_PORT_ALLOC_INPUT_TUNNEL_TYPE_L2_ETYPE); - if (rc) + if (unlikely(rc)) BNXT_DRV_DBG(ERR, "Failed to set global L2 ETYPE Custom Tunnel\n"); else bp->l2_etype_tunnel_cnt++; @@ -577,7 +335,7 @@ bnxt_ulp_port_func_ops_get(struct bnxt *bp) const struct bnxt_ulp_core_ops *func_ops; rc = bnxt_ulp_devid_get(bp, &dev_id); - if (rc) + if (unlikely(rc)) return NULL; switch (dev_id) { @@ -624,25 +382,25 @@ bnxt_ulp_port_init(struct bnxt *bp) } rc = bnxt_ulp_devid_get(bp, &dev_id); - if (rc) { + if (unlikely(rc)) { BNXT_DRV_DBG(DEBUG, "Unsupported device %x\n", rc); return rc; } - if (bp->ulp_ctx) { + if (unlikely(bp->ulp_ctx)) { BNXT_DRV_DBG(DEBUG, "ulp ctx already allocated\n"); return rc; } bp->ulp_ctx = rte_zmalloc("bnxt_ulp_ctx", sizeof(struct bnxt_ulp_context), 0); - if (!bp->ulp_ctx) { + if (unlikely(!bp->ulp_ctx)) { BNXT_DRV_DBG(ERR, "Failed to allocate ulp ctx\n"); return -ENOMEM; } rc = bnxt_ulp_cntxt_bp_set(bp->ulp_ctx, bp); - if (rc) { + if (unlikely(rc)) { BNXT_DRV_DBG(ERR, "Failed to set bp in ulp_ctx\n"); rte_free(bp->ulp_ctx); return -EIO; @@ -650,7 +408,7 @@ bnxt_ulp_port_init(struct bnxt *bp) /* This shouldn't fail, unless we have a unknown device */ bp->ulp_ctx->ops = bnxt_ulp_port_func_ops_get(bp); - if (!bp->ulp_ctx->ops) { + if (unlikely(!bp->ulp_ctx->ops)) { BNXT_DRV_DBG(ERR, "Failed to get ulp ops\n"); rte_free(bp->ulp_ctx); return -EIO; @@ -662,7 +420,7 @@ bnxt_ulp_port_init(struct bnxt *bp) * the TF session. */ session = ulp_session_init(bp, &initialized); - if (!session) { + if (unlikely(!session)) { BNXT_DRV_DBG(ERR, "Failed to initialize the tf session\n"); rc = -EIO; goto jump_to_error; @@ -674,13 +432,13 @@ bnxt_ulp_port_init(struct bnxt *bp) * simply assign the ulp context to this rte_eth_dev. */ rc = bp->ulp_ctx->ops->ulp_ctx_attach(bp, session); - if (rc) { + if (unlikely(rc)) { BNXT_DRV_DBG(ERR, "Failed to attach the ulp context\n"); goto jump_to_error; } } else { rc = bp->ulp_ctx->ops->ulp_init(bp, session); - if (rc) { + if (unlikely(rc)) { BNXT_DRV_DBG(ERR, "Failed to initialize the ulp init\n"); goto jump_to_error; } @@ -688,38 +446,38 @@ bnxt_ulp_port_init(struct bnxt *bp) /* setup the l2 etype tunnel for custom l2 encap/decap */ rc = ulp_l2_etype_tunnel_alloc(bp); - if (rc) + if (unlikely(rc)) goto jump_to_error; /* Update bnxt driver flags */ rc = ulp_dparms_dev_port_intf_update(bp, bp->ulp_ctx); - if (rc) { + if (unlikely(rc)) { BNXT_DRV_DBG(ERR, "Failed to update driver flags\n"); goto jump_to_error; } /* update the port database for the given interface */ rc = ulp_port_db_port_update(bp->ulp_ctx, bp->eth_dev); - if (rc) { + if (unlikely(rc)) { BNXT_DRV_DBG(ERR, "Failed to update port database\n"); goto jump_to_error; } /* create the default rules */ rc = bnxt_ulp_create_df_rules(bp); - if (rc) { + if (unlikely(rc)) { BNXT_DRV_DBG(ERR, "Failed to create default flow\n"); goto jump_to_error; } /* set the unicast mode */ - if (bnxt_ulp_cntxt_ptr2_ulp_flags_get(bp->ulp_ctx, &ulp_flags)) { + if (unlikely(bnxt_ulp_cntxt_ptr2_ulp_flags_get(bp->ulp_ctx, &ulp_flags))) { BNXT_DRV_DBG(ERR, "Error in getting ULP context flags\n"); goto jump_to_error; } if (ulp_flags & BNXT_ULP_APP_UNICAST_ONLY) { - if (bnxt_pmd_set_unicast_rxmask(bp->eth_dev)) { + if (unlikely(bnxt_pmd_set_unicast_rxmask(bp->eth_dev))) { BNXT_DRV_DBG(ERR, "Error in setting unicast rxmode\n"); goto jump_to_error; } @@ -732,14 +490,14 @@ bnxt_ulp_port_init(struct bnxt *bp) rc = bnxt_hwrm_vnic_update(bp, vnic, HWRM_VNIC_UPDATE_INPUT_ENABLES_METADATA_FORMAT_TYPE_VALID); - if (rc) { + if (unlikely(rc)) { BNXT_DRV_DBG(ERR, "Failed to set metadata format\n"); goto jump_to_error; } } rc = ulp_l2_etype_tunnel_alloc(bp); - if (rc) + if (unlikely(rc)) goto jump_to_error; return rc; @@ -757,14 +515,14 @@ ulp_l2_etype_tunnel_free(struct bnxt *bp) if (!ULP_APP_L2_ETYPE_SUPPORT(bp->ulp_ctx)) return; - if (bp->l2_etype_tunnel_cnt == 0) { + if (unlikely(bp->l2_etype_tunnel_cnt == 0)) { BNXT_DRV_DBG(DEBUG, "L2 ETYPE Custom Tunnel already freed\n"); return; } rc = bnxt_tunnel_dst_port_free(bp, BNXT_L2_ETYPE_TUNNEL_ID, HWRM_TUNNEL_DST_PORT_ALLOC_INPUT_TUNNEL_TYPE_L2_ETYPE); - if (rc) + if (unlikely(rc)) BNXT_DRV_DBG(ERR, "Failed to clear L2 ETYPE Custom Tunnel\n"); bp->l2_etype_tunnel_cnt--; @@ -781,21 +539,21 @@ bnxt_ulp_port_deinit(struct bnxt *bp) struct rte_pci_device *pci_dev; struct rte_pci_addr *pci_addr; - if (!BNXT_TRUFLOW_EN(bp)) { + if (unlikely(!BNXT_TRUFLOW_EN(bp))) { BNXT_DRV_DBG(DEBUG, "Skip ULP deinit for port:%d, TF is not enabled\n", bp->eth_dev->data->port_id); return; } - if (!BNXT_PF(bp) && !BNXT_VF_IS_TRUSTED(bp)) { + if (unlikely(!BNXT_PF(bp) && !BNXT_VF_IS_TRUSTED(bp))) { BNXT_DRV_DBG(DEBUG, "Skip ULP deinit port:%d, not a TVF or PF\n", bp->eth_dev->data->port_id); return; } - if (!bp->ulp_ctx) { + if (unlikely(!bp->ulp_ctx)) { BNXT_DRV_DBG(DEBUG, "ulp ctx already de-allocated\n"); return; } @@ -811,7 +569,7 @@ bnxt_ulp_port_deinit(struct bnxt *bp) pthread_mutex_unlock(&bnxt_ulp_global_mutex); /* session not found then just exit */ - if (!session) { + if (unlikely(!session)) { /* Free the ulp context */ rte_free(bp->ulp_ctx); bp->ulp_ctx = NULL; @@ -860,584 +618,6 @@ bnxt_ulp_port_deinit(struct bnxt *bp) bp->ulp_ctx = NULL; } -/* Below are the access functions to access internal data of ulp context. */ -/* Function to set the Mark DB into the context */ -int32_t -bnxt_ulp_cntxt_ptr2_mark_db_set(struct bnxt_ulp_context *ulp_ctx, - struct bnxt_ulp_mark_tbl *mark_tbl) -{ - if (!ulp_ctx || !ulp_ctx->cfg_data) { - BNXT_DRV_DBG(ERR, "Invalid ulp context data\n"); - return -EINVAL; - } - - ulp_ctx->cfg_data->mark_tbl = mark_tbl; - - return 0; -} - -/* Function to retrieve the Mark DB from the context. */ -struct bnxt_ulp_mark_tbl * -bnxt_ulp_cntxt_ptr2_mark_db_get(struct bnxt_ulp_context *ulp_ctx) -{ - if (!ulp_ctx || !ulp_ctx->cfg_data) - return NULL; - - return ulp_ctx->cfg_data->mark_tbl; -} - -bool -bnxt_ulp_cntxt_shared_session_enabled(struct bnxt_ulp_context *ulp_ctx) -{ - return ULP_SHARED_SESSION_IS_ENABLED(ulp_ctx->cfg_data->ulp_flags); -} - -bool -bnxt_ulp_cntxt_multi_shared_session_enabled(struct bnxt_ulp_context *ulp_ctx) -{ - return ULP_MULTI_SHARED_IS_SUPPORTED(ulp_ctx); -} - -int32_t -bnxt_ulp_cntxt_app_id_set(struct bnxt_ulp_context *ulp_ctx, uint8_t app_id) -{ - if (!ulp_ctx) - return -EINVAL; - ulp_ctx->cfg_data->app_id = app_id; - return 0; -} - -int32_t -bnxt_ulp_cntxt_app_id_get(struct bnxt_ulp_context *ulp_ctx, uint8_t *app_id) -{ - /* Default APP id is zero */ - if (!ulp_ctx || !app_id) - return -EINVAL; - *app_id = ulp_ctx->cfg_data->app_id; - return 0; -} - -/* Function to set the device id of the hardware. */ -int32_t -bnxt_ulp_cntxt_dev_id_set(struct bnxt_ulp_context *ulp_ctx, - uint32_t dev_id) -{ - if (ulp_ctx && ulp_ctx->cfg_data) { - ulp_ctx->cfg_data->dev_id = dev_id; - return 0; - } - - return -EINVAL; -} - -/* Function to get the device id of the hardware. */ -int32_t -bnxt_ulp_cntxt_dev_id_get(struct bnxt_ulp_context *ulp_ctx, - uint32_t *dev_id) -{ - if (ulp_ctx && ulp_ctx->cfg_data) { - *dev_id = ulp_ctx->cfg_data->dev_id; - return 0; - } - *dev_id = BNXT_ULP_DEVICE_ID_LAST; - BNXT_DRV_DBG(ERR, "Failed to read dev_id from ulp ctxt\n"); - return -EINVAL; -} - -int32_t -bnxt_ulp_cntxt_mem_type_set(struct bnxt_ulp_context *ulp_ctx, - enum bnxt_ulp_flow_mem_type mem_type) -{ - if (ulp_ctx && ulp_ctx->cfg_data) { - ulp_ctx->cfg_data->mem_type = mem_type; - return 0; - } - BNXT_DRV_DBG(ERR, "Failed to write mem_type in ulp ctxt\n"); - return -EINVAL; -} - -int32_t -bnxt_ulp_cntxt_mem_type_get(struct bnxt_ulp_context *ulp_ctx, - enum bnxt_ulp_flow_mem_type *mem_type) -{ - if (ulp_ctx && ulp_ctx->cfg_data) { - *mem_type = ulp_ctx->cfg_data->mem_type; - return 0; - } - *mem_type = BNXT_ULP_FLOW_MEM_TYPE_LAST; - BNXT_DRV_DBG(ERR, "Failed to read mem_type in ulp ctxt\n"); - return -EINVAL; -} - -/* Function to get the table scope id of the EEM table. */ -int32_t -bnxt_ulp_cntxt_tbl_scope_id_get(struct bnxt_ulp_context *ulp_ctx, - uint32_t *tbl_scope_id) -{ - if (ulp_ctx && ulp_ctx->cfg_data) { - *tbl_scope_id = ulp_ctx->cfg_data->tbl_scope_id; - return 0; - } - - return -EINVAL; -} - -/* Function to set the table scope id of the EEM table. */ -int32_t -bnxt_ulp_cntxt_tbl_scope_id_set(struct bnxt_ulp_context *ulp_ctx, - uint32_t tbl_scope_id) -{ - if (ulp_ctx && ulp_ctx->cfg_data) { - ulp_ctx->cfg_data->tbl_scope_id = tbl_scope_id; - return 0; - } - - return -EINVAL; -} - -/* Function to set the v3 table scope id, only works for tfc objects */ -int32_t -bnxt_ulp_cntxt_tsid_set(struct bnxt_ulp_context *ulp_ctx, uint8_t tsid) -{ - if (ulp_ctx && ulp_ctx->tfo_type == BNXT_ULP_TFO_TYPE_TFC) { - ulp_ctx->tsid = tsid; - ULP_BITMAP_SET(ulp_ctx->tfo_flags, BNXT_ULP_TFO_TSID_FLAG); - return 0; - } - return -EINVAL; -} - -/* Function to reset the v3 table scope id, only works for tfc objects */ -void -bnxt_ulp_cntxt_tsid_reset(struct bnxt_ulp_context *ulp_ctx) -{ - if (ulp_ctx && ulp_ctx->tfo_type == BNXT_ULP_TFO_TYPE_TFC) - ULP_BITMAP_RESET(ulp_ctx->tfo_flags, BNXT_ULP_TFO_TSID_FLAG); -} - -/* Function to set the v3 table scope id, only works for tfc objects */ -int32_t -bnxt_ulp_cntxt_tsid_get(struct bnxt_ulp_context *ulp_ctx, uint8_t *tsid) -{ - if (ulp_ctx && tsid && - ulp_ctx->tfo_type == BNXT_ULP_TFO_TYPE_TFC && - ULP_BITMAP_ISSET(ulp_ctx->tfo_flags, BNXT_ULP_TFO_TSID_FLAG)) { - *tsid = ulp_ctx->tsid; - return 0; - } - return -EINVAL; -} - -/* Function to set the v3 session id, only works for tfc objects */ -int32_t -bnxt_ulp_cntxt_sid_set(struct bnxt_ulp_context *ulp_ctx, - uint16_t sid) -{ - if (ulp_ctx && ulp_ctx->tfo_type == BNXT_ULP_TFO_TYPE_TFC) { - ulp_ctx->sid = sid; - ULP_BITMAP_SET(ulp_ctx->tfo_flags, BNXT_ULP_TFO_SID_FLAG); - return 0; - } - return -EINVAL; -} - -/* - * Function to reset the v3 session id, only works for tfc objects - * There isn't a known invalid value for sid, so this is necessary - */ -void -bnxt_ulp_cntxt_sid_reset(struct bnxt_ulp_context *ulp_ctx) -{ - if (ulp_ctx && ulp_ctx->tfo_type == BNXT_ULP_TFO_TYPE_TFC) - ULP_BITMAP_RESET(ulp_ctx->tfo_flags, BNXT_ULP_TFO_SID_FLAG); -} - -/* Function to get the v3 session id, only works for tfc objects */ -int32_t -bnxt_ulp_cntxt_sid_get(struct bnxt_ulp_context *ulp_ctx, - uint16_t *sid) -{ - if (ulp_ctx && sid && - ulp_ctx->tfo_type == BNXT_ULP_TFO_TYPE_TFC && - ULP_BITMAP_ISSET(ulp_ctx->tfo_flags, BNXT_ULP_TFO_SID_FLAG)) { - *sid = ulp_ctx->sid; - return 0; - } - return -EINVAL; -} - -/* Function to get the number of shared clients attached */ -uint8_t -bnxt_ulp_cntxt_num_shared_clients_get(struct bnxt_ulp_context *ulp) -{ - if (ulp == NULL || ulp->cfg_data == NULL) { - BNXT_DRV_DBG(ERR, "Invalid arguments\n"); - return 0; - } - return ulp->cfg_data->num_shared_clients; -} - -/* Function to set the number of shared clients */ -int -bnxt_ulp_cntxt_num_shared_clients_set(struct bnxt_ulp_context *ulp, bool incr) -{ - if (ulp == NULL || ulp->cfg_data == NULL) { - BNXT_DRV_DBG(ERR, "Invalid arguments\n"); - return 0; - } - if (incr) - ulp->cfg_data->num_shared_clients++; - else if (ulp->cfg_data->num_shared_clients) - ulp->cfg_data->num_shared_clients--; - - BNXT_DRV_DBG(DEBUG, "%d:clients(%d)\n", incr, - ulp->cfg_data->num_shared_clients); - - return 0; -} - -int32_t -bnxt_ulp_cntxt_bp_set(struct bnxt_ulp_context *ulp, struct bnxt *bp) -{ - if (ulp == NULL) { - BNXT_DRV_DBG(ERR, "Invalid arguments\n"); - return -EINVAL; - } - ulp->bp = bp; - return 0; -} - -struct bnxt* -bnxt_ulp_cntxt_bp_get(struct bnxt_ulp_context *ulp) -{ - if (ulp == NULL) { - BNXT_DRV_DBG(ERR, "Invalid arguments\n"); - return NULL; - } - return ulp->bp; -} - -int32_t -bnxt_ulp_cntxt_fid_get(struct bnxt_ulp_context *ulp, uint16_t *fid) -{ - if (ulp == NULL || fid == NULL) - return -EINVAL; - - *fid = ulp->bp->fw_fid; - return 0; -} - -void -bnxt_ulp_cntxt_ptr2_default_class_bits_set(struct bnxt_ulp_context *ulp_ctx, - uint64_t bits) -{ - if (!ulp_ctx || !ulp_ctx->cfg_data) - return; - ulp_ctx->cfg_data->default_class_bits = bits; -} - -uint64_t -bnxt_ulp_cntxt_ptr2_default_class_bits_get(struct bnxt_ulp_context *ulp_ctx) -{ - if (!ulp_ctx || !ulp_ctx->cfg_data) - return 0; - return ulp_ctx->cfg_data->default_class_bits; -} - -void -bnxt_ulp_cntxt_ptr2_default_act_bits_set(struct bnxt_ulp_context *ulp_ctx, - uint64_t bits) -{ - if (!ulp_ctx || !ulp_ctx->cfg_data) - return; - ulp_ctx->cfg_data->default_act_bits = bits; -} - -uint64_t -bnxt_ulp_cntxt_ptr2_default_act_bits_get(struct bnxt_ulp_context *ulp_ctx) -{ - if (!ulp_ctx || !ulp_ctx->cfg_data) - return 0; - return ulp_ctx->cfg_data->default_act_bits; -} - -/* - * Get the device table entry based on the device id. - * - * dev_id [in] The device id of the hardware - * - * Returns the pointer to the device parameters. - */ -struct bnxt_ulp_device_params * -bnxt_ulp_device_params_get(uint32_t dev_id) -{ - if (dev_id < BNXT_ULP_MAX_NUM_DEVICES) - return &ulp_device_params[dev_id]; - return NULL; -} - -/* Function to set the flow database to the ulp context. */ -int32_t -bnxt_ulp_cntxt_ptr2_flow_db_set(struct bnxt_ulp_context *ulp_ctx, - struct bnxt_ulp_flow_db *flow_db) -{ - if (!ulp_ctx || !ulp_ctx->cfg_data) - return -EINVAL; - - ulp_ctx->cfg_data->flow_db = flow_db; - return 0; -} - -/* Function to get the flow database from the ulp context. */ -struct bnxt_ulp_flow_db * -bnxt_ulp_cntxt_ptr2_flow_db_get(struct bnxt_ulp_context *ulp_ctx) -{ - if (!ulp_ctx || !ulp_ctx->cfg_data) - return NULL; - - return ulp_ctx->cfg_data->flow_db; -} - -/* Function to get the tunnel cache table info from the ulp context. */ -struct bnxt_tun_cache_entry * -bnxt_ulp_cntxt_ptr2_tun_tbl_get(struct bnxt_ulp_context *ulp_ctx) -{ - if (!ulp_ctx || !ulp_ctx->cfg_data) - return NULL; - - return ulp_ctx->cfg_data->tun_tbl; -} - -/* Function to get the ulp context from eth device. */ -struct bnxt_ulp_context * -bnxt_ulp_eth_dev_ptr2_cntxt_get(struct rte_eth_dev *dev) -{ - struct bnxt *bp = (struct bnxt *)dev->data->dev_private; - - if (BNXT_ETH_DEV_IS_REPRESENTOR(dev)) { - struct bnxt_representor *vfr = dev->data->dev_private; - - bp = vfr->parent_dev->data->dev_private; - } - - if (!bp) { - BNXT_DRV_DBG(ERR, "Bnxt private data is not initialized\n"); - return NULL; - } - return bp->ulp_ctx; -} - -int32_t -bnxt_ulp_cntxt_ptr2_mapper_data_set(struct bnxt_ulp_context *ulp_ctx, - void *mapper_data) -{ - if (!ulp_ctx || !ulp_ctx->cfg_data) { - BNXT_DRV_DBG(ERR, "Invalid ulp context data\n"); - return -EINVAL; - } - - ulp_ctx->cfg_data->mapper_data = mapper_data; - return 0; -} - -void * -bnxt_ulp_cntxt_ptr2_mapper_data_get(struct bnxt_ulp_context *ulp_ctx) -{ - if (!ulp_ctx || !ulp_ctx->cfg_data) { - BNXT_DRV_DBG(ERR, "Invalid ulp context data\n"); - return NULL; - } - - return ulp_ctx->cfg_data->mapper_data; -} - -int32_t -bnxt_ulp_cntxt_ptr2_matcher_data_set(struct bnxt_ulp_context *ulp_ctx, - void *matcher_data) -{ - if (!ulp_ctx || !ulp_ctx->cfg_data) { - BNXT_DRV_DBG(ERR, "Invalid ulp context data\n"); - return -EINVAL; - } - - ulp_ctx->cfg_data->matcher_data = matcher_data; - return 0; -} - -void * -bnxt_ulp_cntxt_ptr2_matcher_data_get(struct bnxt_ulp_context *ulp_ctx) -{ - if (!ulp_ctx || !ulp_ctx->cfg_data) { - BNXT_DRV_DBG(ERR, "Invalid ulp context data\n"); - return NULL; - } - - return ulp_ctx->cfg_data->matcher_data; -} - -/* Function to set the port database to the ulp context. */ -int32_t -bnxt_ulp_cntxt_ptr2_port_db_set(struct bnxt_ulp_context *ulp_ctx, - struct bnxt_ulp_port_db *port_db) -{ - if (!ulp_ctx || !ulp_ctx->cfg_data) - return -EINVAL; - - ulp_ctx->cfg_data->port_db = port_db; - return 0; -} - -/* Function to get the port database from the ulp context. */ -struct bnxt_ulp_port_db * -bnxt_ulp_cntxt_ptr2_port_db_get(struct bnxt_ulp_context *ulp_ctx) -{ - if (!ulp_ctx || !ulp_ctx->cfg_data) - return NULL; - - return ulp_ctx->cfg_data->port_db; -} - -/* Function to set the flow counter info into the context */ -int32_t -bnxt_ulp_cntxt_ptr2_fc_info_set(struct bnxt_ulp_context *ulp_ctx, - struct bnxt_ulp_fc_info *ulp_fc_info) -{ - if (!ulp_ctx || !ulp_ctx->cfg_data) { - BNXT_DRV_DBG(ERR, "Invalid ulp context data\n"); - return -EINVAL; - } - - ulp_ctx->cfg_data->fc_info = ulp_fc_info; - - return 0; -} - -/* Function to retrieve the flow counter info from the context. */ -struct bnxt_ulp_fc_info * -bnxt_ulp_cntxt_ptr2_fc_info_get(struct bnxt_ulp_context *ulp_ctx) -{ - if (!ulp_ctx || !ulp_ctx->cfg_data) - return NULL; - - return ulp_ctx->cfg_data->fc_info; -} - -/* Function to get the ulp flags from the ulp context. */ -int32_t -bnxt_ulp_cntxt_ptr2_ulp_flags_get(struct bnxt_ulp_context *ulp_ctx, - uint32_t *flags) -{ - if (!ulp_ctx || !ulp_ctx->cfg_data) - return -1; - - *flags = ulp_ctx->cfg_data->ulp_flags; - return 0; -} - -/* Function to get the ulp vfr info from the ulp context. */ -struct bnxt_ulp_vfr_rule_info* -bnxt_ulp_cntxt_ptr2_ulp_vfr_info_get(struct bnxt_ulp_context *ulp_ctx, - uint32_t port_id) -{ - if (!ulp_ctx || !ulp_ctx->cfg_data || port_id >= RTE_MAX_ETHPORTS) - return NULL; - - return &ulp_ctx->cfg_data->vfr_rule_info[port_id]; -} - -/* Function to acquire the flow database lock from the ulp context. */ -int32_t -bnxt_ulp_cntxt_acquire_fdb_lock(struct bnxt_ulp_context *ulp_ctx) -{ - if (!ulp_ctx || !ulp_ctx->cfg_data) - return -1; - - if (pthread_mutex_lock(&ulp_ctx->cfg_data->flow_db_lock)) { - BNXT_DRV_DBG(ERR, "unable to acquire fdb lock\n"); - return -1; - } - return 0; -} - -/* Function to release the flow database lock from the ulp context. */ -void -bnxt_ulp_cntxt_release_fdb_lock(struct bnxt_ulp_context *ulp_ctx) -{ - if (!ulp_ctx || !ulp_ctx->cfg_data) - return; - - pthread_mutex_unlock(&ulp_ctx->cfg_data->flow_db_lock); -} - -/* Function to extract the action type from the shared action handle. */ -int32_t -bnxt_get_action_handle_type(const struct rte_flow_action_handle *handle, - uint32_t *action_handle_type) -{ - if (!action_handle_type) - return -EINVAL; - - *action_handle_type = (uint32_t)(((uint64_t)handle >> 32) & 0xffffffff); - if (*action_handle_type >= BNXT_ULP_GEN_TBL_MAX_SZ) - return -EINVAL; - - return 0; -} - -/* Function to extract the direction from the shared action handle. */ -int32_t -bnxt_get_action_handle_direction(const struct rte_flow_action_handle *handle, - uint32_t *dir) -{ - uint32_t shared_type; - int32_t ret = 0; - - ret = bnxt_get_action_handle_type(handle, &shared_type); - if (ret) - return ret; - - *dir = shared_type & 0x1 ? BNXT_ULP_DIR_EGRESS : BNXT_ULP_DIR_INGRESS; - - return ret; -} - -/* Function to extract the action index from the shared action handle. */ -uint32_t -bnxt_get_action_handle_index(const struct rte_flow_action_handle *handle) -{ - return (uint32_t)((uint64_t)handle & 0xffffffff); -} - -/* Function to set the ha info into the context */ -int32_t -bnxt_ulp_cntxt_ptr2_ha_info_set(struct bnxt_ulp_context *ulp_ctx, - struct bnxt_ulp_ha_mgr_info *ulp_ha_info) -{ - if (ulp_ctx == NULL || ulp_ctx->cfg_data == NULL) { - BNXT_DRV_DBG(ERR, "Invalid ulp context data\n"); - return -EINVAL; - } - ulp_ctx->cfg_data->ha_info = ulp_ha_info; - return 0; -} - -/* Function to retrieve the ha info from the context. */ -struct bnxt_ulp_ha_mgr_info * -bnxt_ulp_cntxt_ptr2_ha_info_get(struct bnxt_ulp_context *ulp_ctx) -{ - if (ulp_ctx == NULL || ulp_ctx->cfg_data == NULL) - return NULL; - return ulp_ctx->cfg_data->ha_info; -} - -bool -bnxt_ulp_cntxt_ha_enabled(struct bnxt_ulp_context *ulp_ctx) -{ - if (ulp_ctx == NULL || ulp_ctx->cfg_data == NULL) - return false; - return !!ULP_HIGH_AVAIL_IS_ENABLED(ulp_ctx->cfg_data->ulp_flags); -} - int32_t bnxt_ulp_cntxt_list_init(void) { @@ -1454,7 +634,7 @@ bnxt_ulp_cntxt_list_add(struct bnxt_ulp_context *ulp_ctx) struct ulp_context_list_entry *entry; entry = rte_zmalloc(NULL, sizeof(struct ulp_context_list_entry), 0); - if (entry == NULL) { + if (unlikely(entry == NULL)) { BNXT_DRV_DBG(ERR, "unable to allocate memory\n"); return -ENOMEM; } @@ -1503,150 +683,3 @@ bnxt_ulp_cntxt_entry_release(void) { rte_spinlock_unlock(&bnxt_ulp_ctxt_lock); } - -/* Function to get the app tunnel details from the ulp context. */ -struct bnxt_flow_app_tun_ent * -bnxt_ulp_cntxt_ptr2_app_tun_list_get(struct bnxt_ulp_context *ulp) -{ - if (!ulp || !ulp->cfg_data) - return NULL; - - return ulp->cfg_data->app_tun; -} - -/* Function to get the truflow app id. This defined in the build file */ -uint32_t -bnxt_ulp_default_app_id_get(void) -{ - return BNXT_TF_APP_ID; -} - -/* Function to convert ulp dev id to regular dev id. */ -uint32_t -bnxt_ulp_cntxt_convert_dev_id(uint32_t ulp_dev_id) -{ - enum tf_device_type type = 0; - - switch (ulp_dev_id) { - case BNXT_ULP_DEVICE_ID_WH_PLUS: - type = TF_DEVICE_TYPE_P4; - break; - case BNXT_ULP_DEVICE_ID_STINGRAY: - type = TF_DEVICE_TYPE_SR; - break; - case BNXT_ULP_DEVICE_ID_THOR: - type = TF_DEVICE_TYPE_P5; - break; - default: - BNXT_DRV_DBG(ERR, "Invalid device id\n"); - break; - } - return type; -} - -/* This function sets the IF table index for the - * Application to poll to get the hot upgrade state and count details from - * the firmware. - */ -int32_t -bnxt_ulp_cntxt_ha_reg_set(struct bnxt_ulp_context *ulp_ctx, - uint8_t state, uint8_t cnt) -{ - if (!ulp_ctx || !ulp_ctx->cfg_data) - return -EINVAL; - - if (ULP_MULTI_SHARED_IS_SUPPORTED(ulp_ctx)) { - ulp_ctx->cfg_data->hu_reg_state = state; - ulp_ctx->cfg_data->hu_reg_cnt = cnt; - } else { - ulp_ctx->cfg_data->hu_reg_state = ULP_HA_IF_TBL_IDX; - ulp_ctx->cfg_data->hu_reg_cnt = ULP_HA_CLIENT_CNT_IF_TBL_IDX; - } - return 0; -} - -/* This function gets the IF table index for the - * application to poll to get the application hot upgrade state from - * the firmware. - */ -uint32_t -bnxt_ulp_cntxt_ha_reg_state_get(struct bnxt_ulp_context *ulp_ctx) -{ - if (!ulp_ctx || !ulp_ctx->cfg_data) - return 0; - - return (uint32_t)ulp_ctx->cfg_data->hu_reg_state; -} - -/* This function gets the IF table index for the - * Application to poll to get the application count from - * the firmware. - */ -uint32_t -bnxt_ulp_cntxt_ha_reg_cnt_get(struct bnxt_ulp_context *ulp_ctx) -{ - if (!ulp_ctx || !ulp_ctx->cfg_data) - return 0; - - return (uint32_t)ulp_ctx->cfg_data->hu_reg_cnt; -} - -/* This function sets the number of key recipes supported - * Generally, this should be set to the number of flexible keys - * supported - */ -void -bnxt_ulp_num_key_recipes_set(struct bnxt_ulp_context *ulp_ctx, - uint16_t num_recipes) -{ - if (!ulp_ctx || !ulp_ctx->cfg_data) - return; - ulp_ctx->cfg_data->num_key_recipes_per_dir = num_recipes; -} - -/* This function gets the number of key recipes supported */ -int32_t -bnxt_ulp_num_key_recipes_get(struct bnxt_ulp_context *ulp_ctx) -{ - if (!ulp_ctx || !ulp_ctx->cfg_data) - return 0; - return ulp_ctx->cfg_data->num_key_recipes_per_dir; -} - -/* This function gets the feature bits */ -uint64_t -bnxt_ulp_feature_bits_get(struct bnxt_ulp_context *ulp_ctx) -{ - if (!ulp_ctx || !ulp_ctx->cfg_data) - return 0; - return ulp_ctx->cfg_data->feature_bits; -} - -/* Add the VF Rep endpoint to the session */ -int32_t -bnxt_ulp_vfr_session_fid_add(struct bnxt_ulp_context *ulp_ctx, - uint16_t vfr_fid) -{ - int32_t rc = 0; - - if (ulp_ctx == NULL || ulp_ctx->ops == NULL) - return -EINVAL; - if (ulp_ctx->ops->ulp_vfr_session_fid_add) - rc = ulp_ctx->ops->ulp_vfr_session_fid_add(ulp_ctx, vfr_fid); - - return rc; -} - -/* Remove the VF Rep endpoint from the session */ -int32_t -bnxt_ulp_vfr_session_fid_rem(struct bnxt_ulp_context *ulp_ctx, - uint16_t vfr_fid) -{ - int32_t rc = 0; - - if (ulp_ctx == NULL || ulp_ctx->ops == NULL) - return -EINVAL; - if (ulp_ctx->ops->ulp_vfr_session_fid_rem) - rc = ulp_ctx->ops->ulp_vfr_session_fid_rem(ulp_ctx, vfr_fid); - return rc; -} diff --git a/drivers/net/bnxt/tf_ulp/bnxt_ulp.h b/drivers/net/bnxt/tf_ulp/bnxt_ulp.h index da5c9bc61a..758b9deb63 100644 --- a/drivers/net/bnxt/tf_ulp/bnxt_ulp.h +++ b/drivers/net/bnxt/tf_ulp/bnxt_ulp.h @@ -243,10 +243,6 @@ struct bnxt_ulp_core_ops { extern const struct bnxt_ulp_core_ops bnxt_ulp_tf_core_ops; extern const struct bnxt_ulp_core_ops bnxt_ulp_tfc_core_ops; -int32_t -bnxt_ulp_devid_get(struct bnxt *bp, - enum bnxt_ulp_device_id *ulp_dev_id); - bool ulp_is_default_session_active(struct bnxt_ulp_context *ulp_ctx); @@ -257,141 +253,11 @@ ulp_is_default_session_active(struct bnxt_ulp_context *ulp_ctx); bool ulp_ctx_deinit_allowed(struct bnxt_ulp_context *ulp_ctx); -/* Function to set the device id of the hardware. */ -int32_t -bnxt_ulp_cntxt_dev_id_set(struct bnxt_ulp_context *ulp_ctx, uint32_t dev_id); - -/* Function to get the device id of the hardware. */ -int32_t -bnxt_ulp_cntxt_dev_id_get(struct bnxt_ulp_context *ulp_ctx, uint32_t *dev_id); - -/* Function to get whether or not ext mem is used for EM */ -int32_t -bnxt_ulp_cntxt_mem_type_get(struct bnxt_ulp_context *ulp_ctx, - enum bnxt_ulp_flow_mem_type *mem_type); - -/* Function to set whether or not ext mem is used for EM */ -int32_t -bnxt_ulp_cntxt_mem_type_set(struct bnxt_ulp_context *ulp_ctx, - enum bnxt_ulp_flow_mem_type mem_type); - -/* Function to set the table scope id of the EEM table. */ -int32_t -bnxt_ulp_cntxt_tbl_scope_id_set(struct bnxt_ulp_context *ulp_ctx, - uint32_t tbl_scope_id); - -/* Function to get the table scope id of the EEM table. */ -int32_t -bnxt_ulp_cntxt_tbl_scope_id_get(struct bnxt_ulp_context *ulp_ctx, - uint32_t *tbl_scope_id); - -/* Function to set the bp associated with the ulp_ctx */ -int32_t -bnxt_ulp_cntxt_tfp_set(struct bnxt_ulp_context *ulp, - enum bnxt_ulp_session_type s_type, - struct tf *tfp); - -/* Function to get the tfp session details from ulp context. */ -struct tf * -bnxt_ulp_cntxt_tfp_get(struct bnxt_ulp_context *ulp, - enum bnxt_ulp_session_type s_type); - -int32_t -bnxt_ulp_cntxt_bp_set(struct bnxt_ulp_context *ulp, struct bnxt *bp); - -/* Function to get the bp associated with the ulp_ctx */ -struct bnxt * -bnxt_ulp_cntxt_bp_get(struct bnxt_ulp_context *ulp); - -/* Function to set the v3 table scope id, only works for tfc objects */ -int32_t -bnxt_ulp_cntxt_tsid_set(struct bnxt_ulp_context *ulp_ctx, uint8_t tsid); - -/* - * Function to set the v3 table scope id, only works for tfc objects - * There isn't a known invalid value for tsid, so this is necessary in order to - * know that the tsid is not set. - */ void -bnxt_ulp_cntxt_tsid_reset(struct bnxt_ulp_context *ulp_ctx); - -/* Function to set the v3 table scope id, only works for tfc objects */ -int32_t -bnxt_ulp_cntxt_tsid_get(struct bnxt_ulp_context *ulp_ctx, uint8_t *tsid); - -/* Function to set the v3 session id, only works for tfc objects */ -int32_t -bnxt_ulp_cntxt_sid_set(struct bnxt_ulp_context *ulp_ctx, uint16_t session_id); - -/* - * Function to reset the v3 session id, only works for tfc objects - * There isn't a known invalid value for sid, so this is necessary in order to - * know that the sid is not set. - */ -void -bnxt_ulp_cntxt_sid_reset(struct bnxt_ulp_context *ulp_ctx); - -/* Function to get the v3 session id, only works for tfc objects */ -int32_t -bnxt_ulp_cntxt_sid_get(struct bnxt_ulp_context *ulp_ctx, uint16_t *sid); - -int32_t -bnxt_ulp_cntxt_fid_get(struct bnxt_ulp_context *ulp, uint16_t *fw_fid); - -/* Get the device table entry based on the device id. */ -struct bnxt_ulp_device_params * -bnxt_ulp_device_params_get(uint32_t dev_id); - -int32_t -bnxt_ulp_ctxt_ptr2_mark_db_set(struct bnxt_ulp_context *ulp_ctx, - struct bnxt_ulp_mark_tbl *mark_tbl); - -struct bnxt_ulp_mark_tbl * -bnxt_ulp_ctxt_ptr2_mark_db_get(struct bnxt_ulp_context *ulp_ctx); - -/* Function to set the flow database to the ulp context. */ -int32_t -bnxt_ulp_cntxt_ptr2_flow_db_set(struct bnxt_ulp_context *ulp_ctx, - struct bnxt_ulp_flow_db *flow_db); - -/* Function to get the flow database from the ulp context. */ -struct bnxt_ulp_flow_db * -bnxt_ulp_cntxt_ptr2_flow_db_get(struct bnxt_ulp_context *ulp_ctx); - -/* Function to get the tunnel cache table info from the ulp context. */ -struct bnxt_tun_cache_entry * -bnxt_ulp_cntxt_ptr2_tun_tbl_get(struct bnxt_ulp_context *ulp_ctx); - -/* Function to get the ulp context from eth device. */ -struct bnxt_ulp_context * -bnxt_ulp_eth_dev_ptr2_cntxt_get(struct rte_eth_dev *dev); - -/* Function to add the ulp mapper data to the ulp context */ -int32_t -bnxt_ulp_cntxt_ptr2_mapper_data_set(struct bnxt_ulp_context *ulp_ctx, - void *mapper_data); - -/* Function to get the ulp mapper data from the ulp context */ -void * -bnxt_ulp_cntxt_ptr2_mapper_data_get(struct bnxt_ulp_context *ulp_ctx); - -/* Function to add the ulp matcher data to the ulp context */ -int32_t -bnxt_ulp_cntxt_ptr2_matcher_data_set(struct bnxt_ulp_context *ulp_ctx, - void *matcher_data); - -/* Function to get the ulp matcher data from the ulp context */ -void * -bnxt_ulp_cntxt_ptr2_matcher_data_get(struct bnxt_ulp_context *ulp_ctx); +bnxt_ulp_destroy_vfr_default_rules(struct bnxt *bp, bool global); -/* Function to set the port database to the ulp context. */ int32_t -bnxt_ulp_cntxt_ptr2_port_db_set(struct bnxt_ulp_context *ulp_ctx, - struct bnxt_ulp_port_db *port_db); - -/* Function to get the port database from the ulp context. */ -struct bnxt_ulp_port_db * -bnxt_ulp_cntxt_ptr2_port_db_get(struct bnxt_ulp_context *ulp_ctx); +bnxt_flow_mtr_init(struct bnxt *bp __rte_unused); /* Function to create default flows. */ int32_t @@ -401,87 +267,23 @@ ulp_default_flow_create(struct rte_eth_dev *eth_dev, uint16_t port_id, uint32_t *flow_id); -/* Function to destroy default flows. */ -int32_t -ulp_default_flow_destroy(struct rte_eth_dev *eth_dev, - uint32_t flow_id); - int bnxt_ulp_flow_destroy(struct rte_eth_dev *dev, struct rte_flow *flow, struct rte_flow_error *error); +/* Function to destroy default flows. */ int32_t -bnxt_ulp_cntxt_ptr2_fc_info_set(struct bnxt_ulp_context *ulp_ctx, - struct bnxt_ulp_fc_info *ulp_fc_info); - -struct bnxt_ulp_fc_info * -bnxt_ulp_cntxt_ptr2_fc_info_get(struct bnxt_ulp_context *ulp_ctx); - -int32_t -bnxt_ulp_cntxt_ptr2_ulp_flags_get(struct bnxt_ulp_context *ulp_ctx, - uint32_t *flags); +ulp_default_flow_destroy(struct rte_eth_dev *eth_dev, + uint32_t flow_id); int32_t -bnxt_ulp_get_df_rule_info(uint16_t port_id, struct bnxt_ulp_context *ulp_ctx, - struct bnxt_ulp_df_rule_info *info); - -struct bnxt_ulp_vfr_rule_info* -bnxt_ulp_cntxt_ptr2_ulp_vfr_info_get(struct bnxt_ulp_context *ulp_ctx, - uint32_t port_id); +bnxt_ulp_cntxt_list_init(void); int32_t -bnxt_ulp_cntxt_acquire_fdb_lock(struct bnxt_ulp_context *ulp_ctx); +bnxt_ulp_cntxt_list_add(struct bnxt_ulp_context *ulp_ctx); void -bnxt_ulp_cntxt_release_fdb_lock(struct bnxt_ulp_context *ulp_ctx); - -int32_t -bnxt_get_action_handle_type(const struct rte_flow_action_handle *handle, - uint32_t *action_handle_type); - -struct bnxt_ulp_shared_act_info * -bnxt_ulp_shared_act_info_get(uint32_t *num_entries); - -int32_t -bnxt_get_action_handle_direction(const struct rte_flow_action_handle *handle, - uint32_t *dir); - -uint32_t -bnxt_get_action_handle_index(const struct rte_flow_action_handle *handle); - -struct bnxt_ulp_glb_resource_info * -bnxt_ulp_app_glb_resource_info_list_get(uint32_t *num_entries); - -int32_t -bnxt_ulp_cntxt_app_id_set(struct bnxt_ulp_context *ulp_ctx, uint8_t app_id); - -int32_t -bnxt_ulp_cntxt_app_id_get(struct bnxt_ulp_context *ulp_ctx, uint8_t *app_id); - -bool -bnxt_ulp_cntxt_shared_session_enabled(struct bnxt_ulp_context *ulp_ctx); - -bool -bnxt_ulp_cntxt_multi_shared_session_enabled(struct bnxt_ulp_context *ulp_ctx); - -struct bnxt_ulp_app_capabilities_info * -bnxt_ulp_app_cap_list_get(uint32_t *num_entries); - -struct bnxt_ulp_resource_resv_info * -bnxt_ulp_app_resource_resv_list_get(uint32_t *num_entries); - -struct bnxt_ulp_resource_resv_info * -bnxt_ulp_resource_resv_list_get(uint32_t *num_entries); - -int32_t -bnxt_ulp_cntxt_ptr2_ha_info_set(struct bnxt_ulp_context *ulp_ctx, - struct bnxt_ulp_ha_mgr_info *ulp_ha_info); - -struct bnxt_ulp_ha_mgr_info * -bnxt_ulp_cntxt_ptr2_ha_info_get(struct bnxt_ulp_context *ulp_ctx); - -bool -bnxt_ulp_cntxt_ha_enabled(struct bnxt_ulp_context *ulp_ctx); +bnxt_ulp_cntxt_list_del(struct bnxt_ulp_context *ulp_ctx); struct bnxt_ulp_context * bnxt_ulp_cntxt_entry_acquire(void *arg); @@ -489,154 +291,4 @@ bnxt_ulp_cntxt_entry_acquire(void *arg); void bnxt_ulp_cntxt_entry_release(void); -uint8_t -bnxt_ulp_cntxt_num_shared_clients_get(struct bnxt_ulp_context *ulp_ctx); - -int -bnxt_ulp_cntxt_num_shared_clients_set(struct bnxt_ulp_context *ulp_ctx, - bool incr); - -struct bnxt_flow_app_tun_ent * -bnxt_ulp_cntxt_ptr2_app_tun_list_get(struct bnxt_ulp_context *ulp); - -/* Function to get the truflow app id. This defined in the build file */ -uint32_t -bnxt_ulp_default_app_id_get(void); - -int -bnxt_ulp_vxlan_port_set(struct bnxt_ulp_context *ulp_ctx, - uint32_t vxlan_port); -unsigned int -bnxt_ulp_vxlan_port_get(struct bnxt_ulp_context *ulp_ctx); - -int -bnxt_ulp_vxlan_ip_port_set(struct bnxt_ulp_context *ulp_ctx, - uint32_t vxlan_ip_port); - -unsigned int -bnxt_ulp_vxlan_ip_port_get(struct bnxt_ulp_context *ulp_ctx); - -int -bnxt_ulp_ecpri_udp_port_set(struct bnxt_ulp_context *ulp_ctx, - uint32_t ecpri_udp_port); - -unsigned int -bnxt_ulp_ecpri_udp_port_get(struct bnxt_ulp_context *ulp_ctx); - -uint32_t -bnxt_ulp_vxlan_gpe_next_proto_set(struct bnxt_ulp_context *ulp_ctx, - uint8_t tunnel_next_proto); - -uint8_t -bnxt_ulp_vxlan_gpe_next_proto_get(struct bnxt_ulp_context *ulp_ctx); - -int -bnxt_ulp_cntxt_vxlan_port_set(struct bnxt_ulp_context *ulp_ctx, - uint32_t vxlan_port); -unsigned int -bnxt_ulp_cntxt_vxlan_port_get(struct bnxt_ulp_context *ulp_ctx); - -int -bnxt_ulp_default_app_priority_set(struct bnxt_ulp_context *ulp_ctx, - uint32_t prio); - -unsigned int -bnxt_ulp_default_app_priority_get(struct bnxt_ulp_context *ulp_ctx); - -int -bnxt_ulp_max_def_priority_set(struct bnxt_ulp_context *ulp_ctx, - uint32_t prio); - -unsigned int -bnxt_ulp_max_def_priority_get(struct bnxt_ulp_context *ulp_ctx); - -int -bnxt_ulp_min_flow_priority_set(struct bnxt_ulp_context *ulp_ctx, uint32_t prio); - -unsigned int -bnxt_ulp_min_flow_priority_get(struct bnxt_ulp_context *ulp_ctx); - -int -bnxt_ulp_max_flow_priority_set(struct bnxt_ulp_context *ulp_ctx, uint32_t prio); - -unsigned int -bnxt_ulp_max_flow_priority_get(struct bnxt_ulp_context *ulp_ctx); - -int -bnxt_ulp_cntxt_vxlan_ip_port_set(struct bnxt_ulp_context *ulp_ctx, - uint32_t vxlan_ip_port); -unsigned int -bnxt_ulp_cntxt_vxlan_ip_port_get(struct bnxt_ulp_context *ulp_ctx); -int -bnxt_ulp_cntxt_ecpri_udp_port_set(struct bnxt_ulp_context *ulp_ctx, - uint32_t ecpri_udp_port); -unsigned int -bnxt_ulp_cntxt_ecpri_udp_port_get(struct bnxt_ulp_context *ulp_ctx); - -uint32_t -bnxt_ulp_cntxt_convert_dev_id(uint32_t ulp_dev_id); - -int32_t -bnxt_ulp_ha_reg_set(struct bnxt_ulp_context *ulp_ctx, - uint8_t state, uint8_t cnt); - -uint32_t -bnxt_ulp_ha_reg_state_get(struct bnxt_ulp_context *ulp_ctx); - -uint32_t -bnxt_ulp_ha_reg_cnt_get(struct bnxt_ulp_context *ulp_ctx); - -struct tf* -bnxt_ulp_bp_tfp_get(struct bnxt *bp, enum bnxt_ulp_session_type type); - -int32_t -bnxt_ulp_cntxt_ha_reg_set(struct bnxt_ulp_context *ulp_ctx, - uint8_t state, uint8_t cnt); - -uint32_t -bnxt_ulp_cntxt_ha_reg_state_get(struct bnxt_ulp_context *ulp_ctx); - -uint32_t -bnxt_ulp_cntxt_ha_reg_cnt_get(struct bnxt_ulp_context *ulp_ctx); - -int32_t bnxt_ulp_cntxt_list_init(void); - -int32_t bnxt_ulp_cntxt_list_add(struct bnxt_ulp_context *ulp_ctx); - -void bnxt_ulp_cntxt_list_del(struct bnxt_ulp_context *ulp_ctx); - -void -bnxt_ulp_destroy_vfr_default_rules(struct bnxt *bp, bool global); - -void bnxt_ulp_num_key_recipes_set(struct bnxt_ulp_context *ulp_ctx, - uint16_t recipes); - -int32_t bnxt_ulp_num_key_recipes_get(struct bnxt_ulp_context *ulp_ctx); - -uint64_t -bnxt_ulp_feature_bits_get(struct bnxt_ulp_context *ulp_ctx); - -int32_t -bnxt_ulp_vfr_session_fid_add(struct bnxt_ulp_context *ulp_ctx, - uint16_t vfr_fid); -int32_t -bnxt_ulp_vfr_session_fid_rem(struct bnxt_ulp_context *ulp_ctx, - uint16_t vfr_fid); - -int32_t -bnxt_flow_mtr_init(struct bnxt *bp __rte_unused); - -void -bnxt_ulp_cntxt_ptr2_default_class_bits_set(struct bnxt_ulp_context *ulp_ctx, - uint64_t bits); - -uint64_t -bnxt_ulp_cntxt_ptr2_default_class_bits_get(struct bnxt_ulp_context *ulp_ctx); - -void -bnxt_ulp_cntxt_ptr2_default_act_bits_set(struct bnxt_ulp_context *ulp_ctx, - uint64_t bits); -uint64_t -bnxt_ulp_cntxt_ptr2_default_act_bits_get(struct bnxt_ulp_context *ulp_ctx); - #endif /* _BNXT_ULP_H_ */ diff --git a/drivers/net/bnxt/tf_ulp/bnxt_ulp_flow.c b/drivers/net/bnxt/tf_ulp/bnxt_ulp_flow.c index 4c99c785c0..eea05e129a 100644 --- a/drivers/net/bnxt/tf_ulp/bnxt_ulp_flow.c +++ b/drivers/net/bnxt/tf_ulp/bnxt_ulp_flow.c @@ -5,6 +5,7 @@ #include "bnxt.h" #include "bnxt_tf_common.h" +#include "bnxt_ulp_utils.h" #include "ulp_rte_parser.h" #include "ulp_matcher.h" #include "ulp_flow_db.h" @@ -24,10 +25,10 @@ bnxt_ulp_flow_validate_args(const struct rte_flow_attr *attr, struct rte_flow_error *error) { /* Perform the validation of the arguments for null */ - if (!error) + if (unlikely(!error)) return BNXT_TF_RC_ERROR; - if (!pattern) { + if (unlikely(!pattern)) { rte_flow_error_set(error, EINVAL, RTE_FLOW_ERROR_TYPE_ITEM_NUM, @@ -36,7 +37,7 @@ bnxt_ulp_flow_validate_args(const struct rte_flow_attr *attr, return BNXT_TF_RC_ERROR; } - if (!actions) { + if (unlikely(!actions)) { rte_flow_error_set(error, EINVAL, RTE_FLOW_ERROR_TYPE_ACTION_NUM, @@ -45,7 +46,7 @@ bnxt_ulp_flow_validate_args(const struct rte_flow_attr *attr, return BNXT_TF_RC_ERROR; } - if (!attr) { + if (unlikely(!attr)) { rte_flow_error_set(error, EINVAL, RTE_FLOW_ERROR_TYPE_ATTR, @@ -54,7 +55,7 @@ bnxt_ulp_flow_validate_args(const struct rte_flow_attr *attr, return BNXT_TF_RC_ERROR; } - if (attr->egress && attr->ingress) { + if (unlikely(attr->egress && attr->ingress)) { rte_flow_error_set(error, EINVAL, RTE_FLOW_ERROR_TYPE_ATTR, @@ -93,14 +94,14 @@ bnxt_ulp_set_prio_attribute(struct ulp_rte_parser_params *params, uint32_t min_p = bnxt_ulp_min_flow_priority_get(params->ulp_ctx); if (max_p < min_p) { - if (attr->priority > min_p || attr->priority < max_p) { + if (unlikely(attr->priority > min_p || attr->priority < max_p)) { BNXT_DRV_DBG(ERR, "invalid prio, not in range %u:%u\n", max_p, min_p); return -EINVAL; } params->priority = attr->priority; } else { - if (attr->priority > max_p || attr->priority < min_p) { + if (unlikely(attr->priority > max_p || attr->priority < min_p)) { BNXT_DRV_DBG(ERR, "invalid prio, not in range %u:%u\n", min_p, max_p); return -EINVAL; @@ -251,15 +252,15 @@ bnxt_ulp_init_mapper_params(struct bnxt_ulp_mapper_parms *mparms, uint16_t vport; /* Get the port db ifindex */ - if (ulp_port_db_dev_port_to_ulp_index(params->ulp_ctx, - params->port_id, - &ifindex)) { + if (unlikely(ulp_port_db_dev_port_to_ulp_index(params->ulp_ctx, + params->port_id, + &ifindex))) { BNXT_DRV_DBG(ERR, "Invalid port id %u\n", params->port_id); return; } /* Update the phy port of the other interface */ - if (ulp_port_db_vport_get(params->ulp_ctx, ifindex, &vport)) { + if (unlikely(ulp_port_db_vport_get(params->ulp_ctx, ifindex, &vport))) { BNXT_DRV_DBG(ERR, "Invalid port if index %u\n", ifindex); return; @@ -288,15 +289,15 @@ bnxt_ulp_flow_create(struct rte_eth_dev *dev, if (error != NULL) error->type = RTE_FLOW_ERROR_TYPE_NONE; - if (bnxt_ulp_flow_validate_args(attr, - pattern, actions, - error) == BNXT_TF_RC_ERROR) { + if (unlikely(bnxt_ulp_flow_validate_args(attr, + pattern, actions, + error) == BNXT_TF_RC_ERROR)) { BNXT_DRV_DBG(ERR, "Invalid arguments being passed\n"); goto flow_error; } ulp_ctx = bnxt_ulp_eth_dev_ptr2_cntxt_get(dev); - if (!ulp_ctx) { + if (unlikely(!ulp_ctx)) { BNXT_DRV_DBG(ERR, "ULP context is not initialized\n"); goto flow_error; } @@ -305,7 +306,7 @@ bnxt_ulp_flow_create(struct rte_eth_dev *dev, memset(¶ms, 0, sizeof(struct ulp_rte_parser_params)); params.ulp_ctx = ulp_ctx; - if (bnxt_ulp_cntxt_app_id_get(params.ulp_ctx, ¶ms.app_id)) { + if (unlikely(bnxt_ulp_cntxt_app_id_get(params.ulp_ctx, ¶ms.app_id))) { BNXT_DRV_DBG(ERR, "failed to get the app id\n"); goto flow_error; } @@ -313,21 +314,21 @@ bnxt_ulp_flow_create(struct rte_eth_dev *dev, /* Set the flow attributes */ bnxt_ulp_set_dir_attributes(¶ms, attr); - if (bnxt_ulp_set_prio_attribute(¶ms, attr)) + if (unlikely(bnxt_ulp_set_prio_attribute(¶ms, attr))) goto flow_error; bnxt_ulp_init_parser_cf_defaults(¶ms, dev->data->port_id); /* Get the function id */ - if (ulp_port_db_port_func_id_get(ulp_ctx, - dev->data->port_id, - &func_id)) { + if (unlikely(ulp_port_db_port_func_id_get(ulp_ctx, + dev->data->port_id, + &func_id))) { BNXT_DRV_DBG(ERR, "conversion of port to func id failed\n"); goto flow_error; } /* Protect flow creation */ - if (bnxt_ulp_cntxt_acquire_fdb_lock(ulp_ctx)) { + if (unlikely(bnxt_ulp_cntxt_acquire_fdb_lock(ulp_ctx))) { BNXT_DRV_DBG(ERR, "Flow db lock acquire failed\n"); goto flow_error; } @@ -338,19 +339,19 @@ bnxt_ulp_flow_create(struct rte_eth_dev *dev, */ rc = ulp_flow_db_fid_alloc(ulp_ctx, BNXT_ULP_FDB_TYPE_REGULAR, func_id, &fid); - if (rc) { + if (unlikely(rc)) { BNXT_DRV_DBG(ERR, "Unable to allocate flow table entry\n"); goto release_lock; } /* Parse the rte flow pattern */ ret = bnxt_ulp_rte_parser_hdr_parse(pattern, ¶ms); - if (ret != BNXT_TF_RC_SUCCESS) + if (unlikely(ret != BNXT_TF_RC_SUCCESS)) goto free_fid; /* Parse the rte flow action */ ret = bnxt_ulp_rte_parser_act_parse(actions, ¶ms); - if (ret != BNXT_TF_RC_SUCCESS) + if (unlikely(ret != BNXT_TF_RC_SUCCESS)) goto free_fid; mparms.flow_id = fid; @@ -362,15 +363,15 @@ bnxt_ulp_flow_create(struct rte_eth_dev *dev, /* do the tunnel offload process if any */ ret = ulp_tunnel_offload_process(¶ms); - if (ret == BNXT_TF_RC_ERROR) + if (unlikely(ret == BNXT_TF_RC_ERROR)) goto free_fid; ret = ulp_matcher_pattern_match(¶ms, ¶ms.class_id); - if (ret != BNXT_TF_RC_SUCCESS) + if (unlikely(ret != BNXT_TF_RC_SUCCESS)) goto free_fid; ret = ulp_matcher_action_match(¶ms, ¶ms.act_tmpl); - if (ret != BNXT_TF_RC_SUCCESS) + if (unlikely(ret != BNXT_TF_RC_SUCCESS)) goto free_fid; bnxt_ulp_init_mapper_params(&mparms, ¶ms, @@ -378,7 +379,7 @@ bnxt_ulp_flow_create(struct rte_eth_dev *dev, /* Call the ulp mapper to create the flow in the hardware. */ ret = ulp_mapper_flow_create(ulp_ctx, &mparms, (void *)error); - if (ret) + if (unlikely(ret)) goto free_fid; bnxt_ulp_cntxt_release_fdb_lock(ulp_ctx); @@ -391,8 +392,8 @@ bnxt_ulp_flow_create(struct rte_eth_dev *dev, release_lock: bnxt_ulp_cntxt_release_fdb_lock(ulp_ctx); flow_error: - if (error != NULL && - error->type == RTE_FLOW_ERROR_TYPE_NONE) + if (unlikely(error != NULL && + error->type == RTE_FLOW_ERROR_TYPE_NONE)) rte_flow_error_set(error, ret, RTE_FLOW_ERROR_TYPE_HANDLE, NULL, "Failed to create flow."); @@ -412,15 +413,15 @@ bnxt_ulp_flow_validate(struct rte_eth_dev *dev, uint32_t class_id, act_tmpl; int ret = BNXT_TF_RC_ERROR; - if (bnxt_ulp_flow_validate_args(attr, - pattern, actions, - error) == BNXT_TF_RC_ERROR) { + if (unlikely(bnxt_ulp_flow_validate_args(attr, + pattern, actions, + error) == BNXT_TF_RC_ERROR)) { BNXT_DRV_DBG(ERR, "Invalid arguments being passed\n"); goto parse_error; } ulp_ctx = bnxt_ulp_eth_dev_ptr2_cntxt_get(dev); - if (!ulp_ctx) { + if (unlikely(!ulp_ctx)) { BNXT_DRV_DBG(ERR, "ULP context is not initialized\n"); goto parse_error; } @@ -429,7 +430,7 @@ bnxt_ulp_flow_validate(struct rte_eth_dev *dev, memset(¶ms, 0, sizeof(struct ulp_rte_parser_params)); params.ulp_ctx = ulp_ctx; - if (bnxt_ulp_cntxt_app_id_get(params.ulp_ctx, ¶ms.app_id)) { + if (unlikely(bnxt_ulp_cntxt_app_id_get(params.ulp_ctx, ¶ms.app_id))) { BNXT_DRV_DBG(ERR, "failed to get the app id\n"); goto parse_error; } @@ -437,19 +438,19 @@ bnxt_ulp_flow_validate(struct rte_eth_dev *dev, /* Set the flow attributes */ bnxt_ulp_set_dir_attributes(¶ms, attr); - if (bnxt_ulp_set_prio_attribute(¶ms, attr)) + if (unlikely(bnxt_ulp_set_prio_attribute(¶ms, attr))) goto parse_error; bnxt_ulp_init_parser_cf_defaults(¶ms, dev->data->port_id); /* Parse the rte flow pattern */ ret = bnxt_ulp_rte_parser_hdr_parse(pattern, ¶ms); - if (ret != BNXT_TF_RC_SUCCESS) + if (unlikely(ret != BNXT_TF_RC_SUCCESS)) goto parse_error; /* Parse the rte flow action */ ret = bnxt_ulp_rte_parser_act_parse(actions, ¶ms); - if (ret != BNXT_TF_RC_SUCCESS) + if (unlikely(ret != BNXT_TF_RC_SUCCESS)) goto parse_error; /* Perform the rte flow post process */ @@ -457,16 +458,16 @@ bnxt_ulp_flow_validate(struct rte_eth_dev *dev, /* do the tunnel offload process if any */ ret = ulp_tunnel_offload_process(¶ms); - if (ret == BNXT_TF_RC_ERROR) + if (unlikely(ret == BNXT_TF_RC_ERROR)) goto parse_error; ret = ulp_matcher_pattern_match(¶ms, &class_id); - if (ret != BNXT_TF_RC_SUCCESS) + if (unlikely(ret != BNXT_TF_RC_SUCCESS)) goto parse_error; ret = ulp_matcher_action_match(¶ms, &act_tmpl); - if (ret != BNXT_TF_RC_SUCCESS) + if (unlikely(ret != BNXT_TF_RC_SUCCESS)) goto parse_error; /* all good return success */ @@ -493,7 +494,7 @@ bnxt_ulp_flow_destroy(struct rte_eth_dev *dev, error->type = RTE_FLOW_ERROR_TYPE_NONE; ulp_ctx = bnxt_ulp_eth_dev_ptr2_cntxt_get(dev); - if (!ulp_ctx) { + if (unlikely(!ulp_ctx)) { BNXT_DRV_DBG(ERR, "ULP context is not initialized\n"); if (error) rte_flow_error_set(error, EINVAL, @@ -504,9 +505,9 @@ bnxt_ulp_flow_destroy(struct rte_eth_dev *dev, flow_id = (uint32_t)(uintptr_t)flow; - if (ulp_port_db_port_func_id_get(ulp_ctx, - dev->data->port_id, - &func_id)) { + if (unlikely(ulp_port_db_port_func_id_get(ulp_ctx, + dev->data->port_id, + &func_id))) { BNXT_DRV_DBG(ERR, "conversion of port to func id failed\n"); if (error) rte_flow_error_set(error, EINVAL, @@ -515,8 +516,8 @@ bnxt_ulp_flow_destroy(struct rte_eth_dev *dev, return -EINVAL; } - if (ulp_flow_db_validate_flow_func(ulp_ctx, flow_id, func_id) == - false) { + if (unlikely(ulp_flow_db_validate_flow_func(ulp_ctx, flow_id, func_id) == + false)) { BNXT_DRV_DBG(ERR, "Incorrect device params\n"); if (error) rte_flow_error_set(error, EINVAL, @@ -525,13 +526,13 @@ bnxt_ulp_flow_destroy(struct rte_eth_dev *dev, return -EINVAL; } - if (bnxt_ulp_cntxt_acquire_fdb_lock(ulp_ctx)) { + if (unlikely(bnxt_ulp_cntxt_acquire_fdb_lock(ulp_ctx))) { BNXT_DRV_DBG(ERR, "Flow db lock acquire failed\n"); return -EINVAL; } ret = ulp_mapper_flow_destroy(ulp_ctx, BNXT_ULP_FDB_TYPE_REGULAR, flow_id, (void *)error); - if (ret) { + if (unlikely(ret)) { BNXT_DRV_DBG(ERR, "Failed to destroy flow.\n"); if (error != NULL && error->type == RTE_FLOW_ERROR_TYPE_NONE) @@ -554,9 +555,8 @@ bnxt_ulp_flow_flush(struct rte_eth_dev *eth_dev, uint16_t func_id; ulp_ctx = bnxt_ulp_eth_dev_ptr2_cntxt_get(eth_dev); - if (!ulp_ctx) { + if (unlikely(!ulp_ctx)) return ret; - } /* Free the resources for the last device */ if (ulp_ctx_deinit_allowed(ulp_ctx)) { @@ -570,7 +570,7 @@ bnxt_ulp_flow_flush(struct rte_eth_dev *eth_dev, else BNXT_DRV_DBG(ERR, "convert port to func id failed\n"); } - if (ret) + if (unlikely(ret)) rte_flow_error_set(error, ret, RTE_FLOW_ERROR_TYPE_HANDLE, NULL, "Failed to flush flow."); @@ -601,12 +601,12 @@ static int ulp_flow_query_rss_get(struct bnxt_ulp_context *ctxt, int rc = 0; bp = bnxt_ulp_cntxt_bp_get(ctxt); - if (!bp) { + if (unlikely(!bp)) { BNXT_DRV_DBG(ERR, "Failed to get bp from ulp cntxt\n"); return -EINVAL; } - if (bnxt_ulp_cntxt_acquire_fdb_lock(ctxt)) { + if (unlikely(bnxt_ulp_cntxt_acquire_fdb_lock(ctxt))) { BNXT_DRV_DBG(ERR, "Flow db lock acquire failed\n"); return -EINVAL; } @@ -652,7 +652,7 @@ bnxt_ulp_flow_query(struct rte_eth_dev *eth_dev, uint32_t flow_id; ulp_ctx = bnxt_ulp_eth_dev_ptr2_cntxt_get(eth_dev); - if (!ulp_ctx) { + if (unlikely(!ulp_ctx)) { BNXT_DRV_DBG(ERR, "ULP context is not initialized\n"); rte_flow_error_set(error, EINVAL, RTE_FLOW_ERROR_TYPE_HANDLE, NULL, @@ -666,7 +666,7 @@ bnxt_ulp_flow_query(struct rte_eth_dev *eth_dev, case RTE_FLOW_ACTION_TYPE_RSS: rss_conf = (struct rte_flow_action_rss *)data; rc = ulp_flow_query_rss_get(ulp_ctx, flow_id, rss_conf); - if (rc) { + if (unlikely(rc)) { rte_flow_error_set(error, EINVAL, RTE_FLOW_ERROR_TYPE_HANDLE, NULL, "Failed to query RSS info."); @@ -676,7 +676,7 @@ bnxt_ulp_flow_query(struct rte_eth_dev *eth_dev, case RTE_FLOW_ACTION_TYPE_COUNT: count = data; rc = ulp_fc_mgr_query_count_get(ulp_ctx, flow_id, count); - if (rc) { + if (unlikely(rc)) { rte_flow_error_set(error, EINVAL, RTE_FLOW_ERROR_TYPE_HANDLE, NULL, "Failed to query flow."); @@ -742,11 +742,11 @@ bnxt_ulp_action_handle_create(struct rte_eth_dev *dev, if (error != NULL) error->type = RTE_FLOW_ERROR_TYPE_NONE; - if (bnxt_ulp_action_handle_chk_args(action, conf) != BNXT_TF_RC_SUCCESS) + if (unlikely(bnxt_ulp_action_handle_chk_args(action, conf) != BNXT_TF_RC_SUCCESS)) goto parse_error; ulp_ctx = bnxt_ulp_eth_dev_ptr2_cntxt_get(dev); - if (!ulp_ctx) { + if (unlikely(!ulp_ctx)) { BNXT_DRV_DBG(ERR, "ULP context is not initialized\n"); goto parse_error; } @@ -761,14 +761,14 @@ bnxt_ulp_action_handle_create(struct rte_eth_dev *dev, bnxt_ulp_set_action_handle_dir_attr(¶ms, conf); /* perform the conversion from dpdk port to bnxt ifindex */ - if (ulp_port_db_dev_port_to_ulp_index(ulp_ctx, - dev->data->port_id, - &ifindex)) { + if (unlikely(ulp_port_db_dev_port_to_ulp_index(ulp_ctx, + dev->data->port_id, + &ifindex))) { BNXT_DRV_DBG(ERR, "Port id is not valid\n"); goto parse_error; } port_type = ulp_port_db_port_type_get(ulp_ctx, ifindex); - if (port_type == BNXT_ULP_INTF_TYPE_INVALID) { + if (unlikely(port_type == BNXT_ULP_INTF_TYPE_INVALID)) { BNXT_DRV_DBG(ERR, "Port type is not valid\n"); goto parse_error; } @@ -794,14 +794,14 @@ bnxt_ulp_action_handle_create(struct rte_eth_dev *dev, } /* perform the conversion from dpdk port to bnxt ifindex */ - if (ulp_port_db_dev_port_to_ulp_index(ulp_ctx, - dev->data->port_id, - &ifindex)) { + if (unlikely(ulp_port_db_dev_port_to_ulp_index(ulp_ctx, + dev->data->port_id, + &ifindex))) { BNXT_DRV_DBG(ERR, "Port id is not valid\n"); goto parse_error; } port_type = ulp_port_db_port_type_get(ulp_ctx, ifindex); - if (port_type == BNXT_ULP_INTF_TYPE_INVALID) { + if (unlikely(port_type == BNXT_ULP_INTF_TYPE_INVALID)) { BNXT_DRV_DBG(ERR, "Port type is not valid\n"); goto parse_error; } @@ -828,7 +828,7 @@ bnxt_ulp_action_handle_create(struct rte_eth_dev *dev, /* Parse the shared action */ ret = bnxt_ulp_rte_parser_act_parse(actions, ¶ms); - if (ret != BNXT_TF_RC_SUCCESS) + if (unlikely(ret != BNXT_TF_RC_SUCCESS)) goto parse_error; /* Perform the rte flow post process */ @@ -836,11 +836,11 @@ bnxt_ulp_action_handle_create(struct rte_eth_dev *dev, /* do the tunnel offload process if any */ ret = ulp_tunnel_offload_process(¶ms); - if (ret == BNXT_TF_RC_ERROR) + if (unlikely(ret == BNXT_TF_RC_ERROR)) goto parse_error; ret = ulp_matcher_action_match(¶ms, &act_tid); - if (ret != BNXT_TF_RC_SUCCESS) + if (unlikely(ret != BNXT_TF_RC_SUCCESS)) goto parse_error; bnxt_ulp_init_mapper_params(&mparms, ¶ms, @@ -848,15 +848,15 @@ bnxt_ulp_action_handle_create(struct rte_eth_dev *dev, mparms.act_tid = act_tid; /* Get the function id */ - if (ulp_port_db_port_func_id_get(ulp_ctx, - dev->data->port_id, - &func_id)) { + if (unlikely(ulp_port_db_port_func_id_get(ulp_ctx, + dev->data->port_id, + &func_id))) { BNXT_DRV_DBG(ERR, "conversion of port to func id failed\n"); goto parse_error; } /* Protect flow creation */ - if (bnxt_ulp_cntxt_acquire_fdb_lock(ulp_ctx)) { + if (unlikely(bnxt_ulp_cntxt_acquire_fdb_lock(ulp_ctx))) { BNXT_DRV_DBG(ERR, "Flow db lock acquire failed\n"); goto parse_error; } @@ -865,7 +865,7 @@ bnxt_ulp_action_handle_create(struct rte_eth_dev *dev, (void *)error); bnxt_ulp_cntxt_release_fdb_lock(ulp_ctx); - if (ret) + if (unlikely(ret)) goto parse_error; return (struct rte_flow_action_handle *)((uintptr_t)mparms.shared_hndl); @@ -898,12 +898,12 @@ bnxt_ulp_action_handle_destroy(struct rte_eth_dev *dev, error->type = RTE_FLOW_ERROR_TYPE_NONE; ulp_ctx = bnxt_ulp_eth_dev_ptr2_cntxt_get(dev); - if (!ulp_ctx) { + if (unlikely(!ulp_ctx)) { BNXT_DRV_DBG(ERR, "ULP context is not initialized\n"); goto parse_error; } - if (!shared_hndl) { + if (unlikely(!shared_hndl)) { BNXT_DRV_DBG(ERR, "Invalid argument of shared handle\n"); goto parse_error; } @@ -912,19 +912,19 @@ bnxt_ulp_action_handle_destroy(struct rte_eth_dev *dev, memset(¶ms, 0, sizeof(struct ulp_rte_parser_params)); params.ulp_ctx = ulp_ctx; - if (bnxt_ulp_cntxt_app_id_get(ulp_ctx, ¶ms.app_id)) { + if (unlikely(bnxt_ulp_cntxt_app_id_get(ulp_ctx, ¶ms.app_id))) { BNXT_DRV_DBG(ERR, "failed to get the app id\n"); goto parse_error; } /* The template will delete the entry if there are no references */ - if (bnxt_get_action_handle_type(shared_hndl, &shared_action_type)) { + if (unlikely(bnxt_get_action_handle_type(shared_hndl, &shared_action_type))) { BNXT_DRV_DBG(ERR, "Invalid shared handle\n"); goto parse_error; } act_info_entries = 0; act_info = bnxt_ulp_shared_act_info_get(&act_info_entries); - if (shared_action_type >= act_info_entries || !act_info) { + if (unlikely(shared_action_type >= act_info_entries || !act_info)) { BNXT_DRV_DBG(ERR, "Invalid shared handle\n"); goto parse_error; } @@ -934,7 +934,7 @@ bnxt_ulp_action_handle_destroy(struct rte_eth_dev *dev, ULP_BITMAP_SET(params.act_bitmap.bits, BNXT_ULP_ACT_BIT_DELETE); ret = bnxt_get_action_handle_direction(shared_hndl, &dir); - if (ret) { + if (unlikely(ret)) { BNXT_DRV_DBG(ERR, "Invalid shared handle dir\n"); goto parse_error; } @@ -956,14 +956,14 @@ bnxt_ulp_action_handle_destroy(struct rte_eth_dev *dev, &tmp64, BNXT_ULP_ACT_PROP_SZ_SHARED_HANDLE); ret = ulp_matcher_action_match(¶ms, &act_tid); - if (ret != BNXT_TF_RC_SUCCESS) + if (unlikely(ret != BNXT_TF_RC_SUCCESS)) goto parse_error; bnxt_ulp_init_mapper_params(&mparms, ¶ms, BNXT_ULP_FDB_TYPE_REGULAR); mparms.act_tid = act_tid; - if (bnxt_ulp_cntxt_acquire_fdb_lock(ulp_ctx)) { + if (unlikely(bnxt_ulp_cntxt_acquire_fdb_lock(ulp_ctx))) { BNXT_DRV_DBG(ERR, "Flow db lock acquire failed\n"); goto parse_error; } @@ -971,7 +971,7 @@ bnxt_ulp_action_handle_destroy(struct rte_eth_dev *dev, ret = ulp_mapper_flow_create(ulp_ctx, &mparms, (void *)error); bnxt_ulp_cntxt_release_fdb_lock(ulp_ctx); - if (ret) + if (unlikely(ret)) goto parse_error; return 0; @@ -1000,7 +1000,7 @@ bnxt_ulp_tunnel_decap_set(struct rte_eth_dev *eth_dev, int32_t rc = 0; ulp_ctx = bnxt_ulp_eth_dev_ptr2_cntxt_get(eth_dev); - if (ulp_ctx == NULL) { + if (unlikely(ulp_ctx == NULL)) { BNXT_DRV_DBG(ERR, "ULP context is not initialized\n"); rte_flow_error_set(error, EINVAL, RTE_FLOW_ERROR_TYPE_HANDLE, NULL, @@ -1008,7 +1008,7 @@ bnxt_ulp_tunnel_decap_set(struct rte_eth_dev *eth_dev, return -EINVAL; } - if (tunnel == NULL) { + if (unlikely(tunnel == NULL)) { BNXT_DRV_DBG(ERR, "No tunnel specified\n"); rte_flow_error_set(error, EINVAL, RTE_FLOW_ERROR_TYPE_ATTR, NULL, @@ -1016,7 +1016,7 @@ bnxt_ulp_tunnel_decap_set(struct rte_eth_dev *eth_dev, return -EINVAL; } - if (tunnel->type != RTE_FLOW_ITEM_TYPE_VXLAN) { + if (unlikely(tunnel->type != RTE_FLOW_ITEM_TYPE_VXLAN)) { BNXT_DRV_DBG(ERR, "Tunnel type unsupported\n"); rte_flow_error_set(error, EINVAL, RTE_FLOW_ERROR_TYPE_ATTR, NULL, @@ -1025,7 +1025,7 @@ bnxt_ulp_tunnel_decap_set(struct rte_eth_dev *eth_dev, } rc = ulp_app_tun_search_entry(ulp_ctx, tunnel, &tun_entry); - if (rc < 0) { + if (unlikely(rc < 0)) { rte_flow_error_set(error, EINVAL, RTE_FLOW_ERROR_TYPE_ATTR, NULL, "tunnel decap set failed"); @@ -1033,7 +1033,7 @@ bnxt_ulp_tunnel_decap_set(struct rte_eth_dev *eth_dev, } rc = ulp_app_tun_entry_set_decap_action(tun_entry); - if (rc < 0) { + if (unlikely(rc < 0)) { rte_flow_error_set(error, EINVAL, RTE_FLOW_ERROR_TYPE_ATTR, NULL, "tunnel decap set failed"); @@ -1057,7 +1057,7 @@ bnxt_ulp_tunnel_match(struct rte_eth_dev *eth_dev, int32_t rc = 0; ulp_ctx = bnxt_ulp_eth_dev_ptr2_cntxt_get(eth_dev); - if (ulp_ctx == NULL) { + if (unlikely(ulp_ctx == NULL)) { BNXT_DRV_DBG(ERR, "ULP context is not initialized\n"); rte_flow_error_set(error, EINVAL, RTE_FLOW_ERROR_TYPE_HANDLE, NULL, @@ -1065,7 +1065,7 @@ bnxt_ulp_tunnel_match(struct rte_eth_dev *eth_dev, return -EINVAL; } - if (tunnel == NULL) { + if (unlikely(tunnel == NULL)) { BNXT_DRV_DBG(ERR, "No tunnel specified\n"); rte_flow_error_set(error, EINVAL, RTE_FLOW_ERROR_TYPE_HANDLE, NULL, @@ -1073,7 +1073,7 @@ bnxt_ulp_tunnel_match(struct rte_eth_dev *eth_dev, return -EINVAL; } - if (tunnel->type != RTE_FLOW_ITEM_TYPE_VXLAN) { + if (unlikely(tunnel->type != RTE_FLOW_ITEM_TYPE_VXLAN)) { BNXT_DRV_DBG(ERR, "Tunnel type unsupported\n"); rte_flow_error_set(error, EINVAL, RTE_FLOW_ERROR_TYPE_HANDLE, NULL, @@ -1082,7 +1082,7 @@ bnxt_ulp_tunnel_match(struct rte_eth_dev *eth_dev, } rc = ulp_app_tun_search_entry(ulp_ctx, tunnel, &tun_entry); - if (rc < 0) { + if (unlikely(rc < 0)) { rte_flow_error_set(error, EINVAL, RTE_FLOW_ERROR_TYPE_ATTR, NULL, "tunnel match set failed"); @@ -1090,7 +1090,7 @@ bnxt_ulp_tunnel_match(struct rte_eth_dev *eth_dev, } rc = ulp_app_tun_entry_set_decap_item(tun_entry); - if (rc < 0) { + if (unlikely(rc < 0)) { rte_flow_error_set(error, EINVAL, RTE_FLOW_ERROR_TYPE_ATTR, NULL, "tunnel match set failed"); @@ -1113,14 +1113,14 @@ bnxt_ulp_tunnel_decap_release(struct rte_eth_dev *eth_dev, const struct rte_flow_action *action_item = pmd_actions; ulp_ctx = bnxt_ulp_eth_dev_ptr2_cntxt_get(eth_dev); - if (ulp_ctx == NULL) { + if (unlikely(ulp_ctx == NULL)) { BNXT_DRV_DBG(ERR, "ULP context is not initialized\n"); rte_flow_error_set(error, EINVAL, RTE_FLOW_ERROR_TYPE_HANDLE, NULL, "ULP context uninitialized"); return -EINVAL; } - if (num_actions != BNXT_ULP_TUNNEL_OFFLOAD_NUM_ITEMS) { + if (unlikely(num_actions != BNXT_ULP_TUNNEL_OFFLOAD_NUM_ITEMS)) { BNXT_DRV_DBG(ERR, "num actions is invalid\n"); rte_flow_error_set(error, EINVAL, RTE_FLOW_ERROR_TYPE_ATTR, NULL, @@ -1128,8 +1128,8 @@ bnxt_ulp_tunnel_decap_release(struct rte_eth_dev *eth_dev, return -EINVAL; } while (action_item && action_item->type != RTE_FLOW_ACTION_TYPE_END) { - if (action_item->type == (typeof(tun_entry->action.type)) - BNXT_RTE_FLOW_ACTION_TYPE_VXLAN_DECAP) { + if (unlikely(action_item->type == (typeof(tun_entry->action.type)) + BNXT_RTE_FLOW_ACTION_TYPE_VXLAN_DECAP)) { tun_entry = ulp_app_tun_match_entry(ulp_ctx, action_item->conf); ulp_app_tun_entry_delete(tun_entry); @@ -1149,14 +1149,14 @@ bnxt_ulp_tunnel_item_release(struct rte_eth_dev *eth_dev, struct bnxt_flow_app_tun_ent *tun_entry; ulp_ctx = bnxt_ulp_eth_dev_ptr2_cntxt_get(eth_dev); - if (ulp_ctx == NULL) { + if (unlikely(ulp_ctx == NULL)) { BNXT_DRV_DBG(ERR, "ULP context is not initialized\n"); rte_flow_error_set(error, EINVAL, RTE_FLOW_ERROR_TYPE_HANDLE, NULL, "ULP context uninitialized"); return -EINVAL; } - if (num_items != BNXT_ULP_TUNNEL_OFFLOAD_NUM_ITEMS) { + if (unlikely(num_items != BNXT_ULP_TUNNEL_OFFLOAD_NUM_ITEMS)) { BNXT_DRV_DBG(ERR, "num items is invalid\n"); rte_flow_error_set(error, EINVAL, RTE_FLOW_ERROR_TYPE_ATTR, NULL, diff --git a/drivers/net/bnxt/tf_ulp/bnxt_ulp_meter.c b/drivers/net/bnxt/tf_ulp/bnxt_ulp_meter.c index 37b5cc4dfb..591bde96e8 100644 --- a/drivers/net/bnxt/tf_ulp/bnxt_ulp_meter.c +++ b/drivers/net/bnxt/tf_ulp/bnxt_ulp_meter.c @@ -28,6 +28,7 @@ #include "tfp.h" #include "bnxt_tf_common.h" #include "bnxt_ulp_tf.h" +#include "bnxt_ulp_utils.h" #include "ulp_rte_parser.h" #include "ulp_matcher.h" #include "ulp_flow_db.h" diff --git a/drivers/net/bnxt/tf_ulp/bnxt_ulp_tf.c b/drivers/net/bnxt/tf_ulp/bnxt_ulp_tf.c index 5589e36db4..99a6bac0ce 100644 --- a/drivers/net/bnxt/tf_ulp/bnxt_ulp_tf.c +++ b/drivers/net/bnxt/tf_ulp/bnxt_ulp_tf.c @@ -14,6 +14,7 @@ #include "bnxt.h" #include "bnxt_ulp.h" +#include "bnxt_ulp_utils.h" #include "bnxt_ulp_tf.h" #include "bnxt_tf_common.h" #include "hsi_struct_def_dpdk.h" diff --git a/drivers/net/bnxt/tf_ulp/bnxt_ulp_tfc.c b/drivers/net/bnxt/tf_ulp/bnxt_ulp_tfc.c index 45a0778094..d7decacc26 100644 --- a/drivers/net/bnxt/tf_ulp/bnxt_ulp_tfc.c +++ b/drivers/net/bnxt/tf_ulp/bnxt_ulp_tfc.c @@ -14,6 +14,7 @@ #include "bnxt.h" #include "bnxt_ulp.h" +#include "bnxt_ulp_utils.h" #include "bnxt_ulp_tfc.h" #include "bnxt_tf_common.h" #include "hsi_struct_def_dpdk.h" diff --git a/drivers/net/bnxt/tf_ulp/bnxt_ulp_utils.h b/drivers/net/bnxt/tf_ulp/bnxt_ulp_utils.h new file mode 100644 index 0000000000..a1fd00f002 --- /dev/null +++ b/drivers/net/bnxt/tf_ulp/bnxt_ulp_utils.h @@ -0,0 +1,1038 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(c) 2019-2023 Broadcom + * All rights reserved. + */ + +#ifndef _BNXT_ULP_UTILS_H_ +#define _BNXT_ULP_UTILS_H_ + +#include +#include +#include +#include +#include +#include + +#include "bnxt.h" +#include "bnxt_ulp.h" +#include "bnxt_tf_common.h" +#include "bnxt_hwrm.h" +#include "hsi_struct_def_dpdk.h" +#include "tf_core.h" +#include "tf_ext_flow_handle.h" + +#include "ulp_template_db_enum.h" +#include "ulp_template_struct.h" +#include "ulp_mark_mgr.h" +#include "ulp_fc_mgr.h" +#include "ulp_flow_db.h" +#include "ulp_mapper.h" +#include "ulp_matcher.h" +#include "ulp_port_db.h" +#include "ulp_tun.h" +#include "ulp_ha_mgr.h" +#include "bnxt_tf_pmd_shim.h" +#include "ulp_template_db_tbl.h" + +static inline int32_t +bnxt_ulp_devid_get(struct bnxt *bp, + enum bnxt_ulp_device_id *ulp_dev_id) +{ + if (BNXT_CHIP_P7(bp)) { + *ulp_dev_id = BNXT_ULP_DEVICE_ID_THOR2; + return 0; + } + + if (BNXT_CHIP_P5(bp)) { + *ulp_dev_id = BNXT_ULP_DEVICE_ID_THOR; + return 0; + } + + if (BNXT_STINGRAY(bp)) + *ulp_dev_id = BNXT_ULP_DEVICE_ID_STINGRAY; + else + /* Assuming Whitney */ + *ulp_dev_id = BNXT_ULP_DEVICE_ID_WH_PLUS; + + return 0; +} + +static inline struct bnxt_ulp_app_capabilities_info * +bnxt_ulp_app_cap_list_get(uint32_t *num_entries) +{ + if (unlikely(!num_entries)) + return NULL; + *num_entries = BNXT_ULP_APP_CAP_TBL_MAX_SZ; + return ulp_app_cap_info_list; +} + +static inline struct bnxt_ulp_shared_act_info * +bnxt_ulp_shared_act_info_get(uint32_t *num_entries) +{ + if (unlikely(!num_entries)) + return NULL; + + *num_entries = BNXT_ULP_GEN_TBL_MAX_SZ; + + return ulp_shared_act_info; +} + +static inline struct bnxt_ulp_resource_resv_info * +bnxt_ulp_app_resource_resv_list_get(uint32_t *num_entries) +{ + if (unlikely(num_entries == NULL)) + return NULL; + *num_entries = BNXT_ULP_APP_RESOURCE_RESV_LIST_MAX_SZ; + return ulp_app_resource_resv_list; +} + +static inline struct bnxt_ulp_resource_resv_info * +bnxt_ulp_resource_resv_list_get(uint32_t *num_entries) +{ + if (unlikely(!num_entries)) + return NULL; + *num_entries = BNXT_ULP_RESOURCE_RESV_LIST_MAX_SZ; + return ulp_resource_resv_list; +} + +static inline struct bnxt_ulp_glb_resource_info * +bnxt_ulp_app_glb_resource_info_list_get(uint32_t *num_entries) +{ + if (unlikely(!num_entries)) + return NULL; + *num_entries = BNXT_ULP_APP_GLB_RESOURCE_TBL_MAX_SZ; + return ulp_app_glb_resource_tbl; +} + +/* Function to set the number for vxlan_ip (custom vxlan) port into the context */ +static inline int +bnxt_ulp_cntxt_ecpri_udp_port_set(struct bnxt_ulp_context *ulp_ctx, + uint32_t ecpri_udp_port) +{ + if (unlikely(!ulp_ctx || !ulp_ctx->cfg_data)) + return -EINVAL; + + ulp_ctx->cfg_data->ecpri_udp_port = ecpri_udp_port; + + return 0; +} + +/* Function to retrieve the vxlan_ip (custom vxlan) port from the context. */ +static inline unsigned int +bnxt_ulp_cntxt_ecpri_udp_port_get(struct bnxt_ulp_context *ulp_ctx) +{ + if (unlikely(!ulp_ctx || !ulp_ctx->cfg_data)) + return 0; + + return (unsigned int)ulp_ctx->cfg_data->ecpri_udp_port; +} + +/* Function to set the number for vxlan_ip (custom vxlan) port into the context */ +static inline int +bnxt_ulp_cntxt_vxlan_ip_port_set(struct bnxt_ulp_context *ulp_ctx, + uint32_t vxlan_ip_port) +{ + if (unlikely(!ulp_ctx || !ulp_ctx->cfg_data)) + return -EINVAL; + + ulp_ctx->cfg_data->vxlan_ip_port = vxlan_ip_port; + if (vxlan_ip_port) + ulp_ctx->cfg_data->ulp_flags |= BNXT_ULP_STATIC_VXLAN_SUPPORT; + return 0; +} + +/* Function to retrieve the vxlan_ip (custom vxlan) port from the context. */ +static inline unsigned int +bnxt_ulp_cntxt_vxlan_ip_port_get(struct bnxt_ulp_context *ulp_ctx) +{ + if (unlikely(!ulp_ctx || !ulp_ctx->cfg_data)) + return 0; + + return (unsigned int)ulp_ctx->cfg_data->vxlan_ip_port; +} + +/* Function to set the number for vxlan_gpe next_proto into the context */ +static inline uint32_t +bnxt_ulp_vxlan_gpe_next_proto_set(struct bnxt_ulp_context *ulp_ctx, + uint8_t tunnel_next_proto) +{ + if (unlikely(!ulp_ctx || !ulp_ctx->cfg_data)) + return -EINVAL; + + ulp_ctx->cfg_data->tunnel_next_proto = tunnel_next_proto; + + return 0; +} + +/* Function to retrieve the vxlan_gpe next_proto from the context. */ +static inline uint8_t +bnxt_ulp_vxlan_gpe_next_proto_get(struct bnxt_ulp_context *ulp_ctx) +{ + if (unlikely(!ulp_ctx || !ulp_ctx->cfg_data)) + return 0; + + return ulp_ctx->cfg_data->tunnel_next_proto; +} + +/* Function to set the number for vxlan port into the context */ +static inline int +bnxt_ulp_cntxt_vxlan_port_set(struct bnxt_ulp_context *ulp_ctx, + uint32_t vxlan_port) +{ + if (unlikely(!ulp_ctx || !ulp_ctx->cfg_data)) + return -EINVAL; + + ulp_ctx->cfg_data->vxlan_port = vxlan_port; + if (vxlan_port) + ulp_ctx->cfg_data->ulp_flags |= BNXT_ULP_STATIC_VXLAN_SUPPORT; + + return 0; +} + +/* Function to retrieve the vxlan port from the context. */ +static inline unsigned int +bnxt_ulp_cntxt_vxlan_port_get(struct bnxt_ulp_context *ulp_ctx) +{ + if (unlikely(!ulp_ctx || !ulp_ctx->cfg_data)) + return 0; + + return (unsigned int)ulp_ctx->cfg_data->vxlan_port; +} + +static inline int +bnxt_ulp_default_app_priority_set(struct bnxt_ulp_context *ulp_ctx, + uint32_t prio) +{ + if (unlikely(!ulp_ctx || !ulp_ctx->cfg_data)) + return -EINVAL; + + ulp_ctx->cfg_data->default_priority = prio; + return 0; +} + +static inline unsigned int +bnxt_ulp_default_app_priority_get(struct bnxt_ulp_context *ulp_ctx) +{ + if (unlikely(!ulp_ctx || !ulp_ctx->cfg_data)) + return 0; + + return (unsigned int)ulp_ctx->cfg_data->default_priority; +} + +static inline int +bnxt_ulp_max_def_priority_set(struct bnxt_ulp_context *ulp_ctx, + uint32_t prio) +{ + if (unlikely(!ulp_ctx || !ulp_ctx->cfg_data)) + return -EINVAL; + + ulp_ctx->cfg_data->max_def_priority = prio; + return 0; +} + +static inline unsigned int +bnxt_ulp_max_def_priority_get(struct bnxt_ulp_context *ulp_ctx) +{ + if (unlikely(!ulp_ctx || !ulp_ctx->cfg_data)) + return 0; + + return (unsigned int)ulp_ctx->cfg_data->max_def_priority; +} + +static inline int +bnxt_ulp_min_flow_priority_set(struct bnxt_ulp_context *ulp_ctx, uint32_t prio) +{ + if (unlikely(!ulp_ctx || !ulp_ctx->cfg_data)) + return -EINVAL; + + ulp_ctx->cfg_data->min_flow_priority = prio; + return 0; +} + +static inline unsigned int +bnxt_ulp_min_flow_priority_get(struct bnxt_ulp_context *ulp_ctx) +{ + if (unlikely(!ulp_ctx || !ulp_ctx->cfg_data)) + return 0; + + return ulp_ctx->cfg_data->min_flow_priority; +} + +static inline int +bnxt_ulp_max_flow_priority_set(struct bnxt_ulp_context *ulp_ctx, uint32_t prio) +{ + if (unlikely(!ulp_ctx || !ulp_ctx->cfg_data)) + return -EINVAL; + + ulp_ctx->cfg_data->max_flow_priority = prio; + return 0; +} + +static inline unsigned int +bnxt_ulp_max_flow_priority_get(struct bnxt_ulp_context *ulp_ctx) +{ + if (unlikely(!ulp_ctx || !ulp_ctx->cfg_data)) + return 0; + + return ulp_ctx->cfg_data->max_flow_priority; +} + +/* Below are the access functions to access internal data of ulp context. */ +/* Function to set the Mark DB into the context */ +static inline int32_t +bnxt_ulp_cntxt_ptr2_mark_db_set(struct bnxt_ulp_context *ulp_ctx, + struct bnxt_ulp_mark_tbl *mark_tbl) +{ + if (unlikely(!ulp_ctx || !ulp_ctx->cfg_data)) { + BNXT_DRV_DBG(ERR, "Invalid ulp context data\n"); + return -EINVAL; + } + + ulp_ctx->cfg_data->mark_tbl = mark_tbl; + + return 0; +} + +/* Function to retrieve the Mark DB from the context. */ +static inline struct bnxt_ulp_mark_tbl * +bnxt_ulp_cntxt_ptr2_mark_db_get(struct bnxt_ulp_context *ulp_ctx) +{ + if (unlikely(!ulp_ctx || !ulp_ctx->cfg_data)) + return NULL; + + return ulp_ctx->cfg_data->mark_tbl; +} + +static inline bool +bnxt_ulp_cntxt_shared_session_enabled(struct bnxt_ulp_context *ulp_ctx) +{ + return ULP_SHARED_SESSION_IS_ENABLED(ulp_ctx->cfg_data->ulp_flags); +} + +static inline bool +bnxt_ulp_cntxt_multi_shared_session_enabled(struct bnxt_ulp_context *ulp_ctx) +{ + return ULP_MULTI_SHARED_IS_SUPPORTED(ulp_ctx); +} + +static inline int32_t +bnxt_ulp_cntxt_app_id_set(struct bnxt_ulp_context *ulp_ctx, uint8_t app_id) +{ + if (unlikely(!ulp_ctx)) + return -EINVAL; + ulp_ctx->cfg_data->app_id = app_id; + return 0; +} + +static inline int32_t +bnxt_ulp_cntxt_app_id_get(struct bnxt_ulp_context *ulp_ctx, uint8_t *app_id) +{ + /* Default APP id is zero */ + if (unlikely(!ulp_ctx || !app_id)) + return -EINVAL; + *app_id = ulp_ctx->cfg_data->app_id; + return 0; +} + +/* Function to set the device id of the hardware. */ +static inline int32_t +bnxt_ulp_cntxt_dev_id_set(struct bnxt_ulp_context *ulp_ctx, + uint32_t dev_id) +{ + if (likely(ulp_ctx && ulp_ctx->cfg_data)) { + ulp_ctx->cfg_data->dev_id = dev_id; + return 0; + } + + return -EINVAL; +} + +/* Function to get the device id of the hardware. */ +static inline int32_t +bnxt_ulp_cntxt_dev_id_get(struct bnxt_ulp_context *ulp_ctx, + uint32_t *dev_id) +{ + if (likely(ulp_ctx && ulp_ctx->cfg_data)) { + *dev_id = ulp_ctx->cfg_data->dev_id; + return 0; + } + *dev_id = BNXT_ULP_DEVICE_ID_LAST; + BNXT_DRV_DBG(ERR, "Failed to read dev_id from ulp ctxt\n"); + return -EINVAL; +} + +static inline int32_t +bnxt_ulp_cntxt_mem_type_set(struct bnxt_ulp_context *ulp_ctx, + enum bnxt_ulp_flow_mem_type mem_type) +{ + if (likely(ulp_ctx && ulp_ctx->cfg_data)) { + ulp_ctx->cfg_data->mem_type = mem_type; + return 0; + } + BNXT_DRV_DBG(ERR, "Failed to write mem_type in ulp ctxt\n"); + return -EINVAL; +} + +static inline int32_t +bnxt_ulp_cntxt_mem_type_get(struct bnxt_ulp_context *ulp_ctx, + enum bnxt_ulp_flow_mem_type *mem_type) +{ + if (likely(ulp_ctx && ulp_ctx->cfg_data)) { + *mem_type = ulp_ctx->cfg_data->mem_type; + return 0; + } + *mem_type = BNXT_ULP_FLOW_MEM_TYPE_LAST; + BNXT_DRV_DBG(ERR, "Failed to read mem_type in ulp ctxt\n"); + return -EINVAL; +} + +/* Function to get the table scope id of the EEM table. */ +static inline int32_t +bnxt_ulp_cntxt_tbl_scope_id_get(struct bnxt_ulp_context *ulp_ctx, + uint32_t *tbl_scope_id) +{ + if (likely(ulp_ctx && ulp_ctx->cfg_data)) { + *tbl_scope_id = ulp_ctx->cfg_data->tbl_scope_id; + return 0; + } + + return -EINVAL; +} + +/* Function to set the table scope id of the EEM table. */ +static inline int32_t +bnxt_ulp_cntxt_tbl_scope_id_set(struct bnxt_ulp_context *ulp_ctx, + uint32_t tbl_scope_id) +{ + if (likely(ulp_ctx && ulp_ctx->cfg_data)) { + ulp_ctx->cfg_data->tbl_scope_id = tbl_scope_id; + return 0; + } + + return -EINVAL; +} + +/* Function to set the v3 table scope id, only works for tfc objects */ +static inline int32_t +bnxt_ulp_cntxt_tsid_set(struct bnxt_ulp_context *ulp_ctx, uint8_t tsid) +{ + if (likely(ulp_ctx && ulp_ctx->tfo_type == BNXT_ULP_TFO_TYPE_TFC)) { + ulp_ctx->tsid = tsid; + ULP_BITMAP_SET(ulp_ctx->tfo_flags, BNXT_ULP_TFO_TSID_FLAG); + return 0; + } + return -EINVAL; +} + +/* Function to reset the v3 table scope id, only works for tfc objects */ +static inline void +bnxt_ulp_cntxt_tsid_reset(struct bnxt_ulp_context *ulp_ctx) +{ + if (ulp_ctx && ulp_ctx->tfo_type == BNXT_ULP_TFO_TYPE_TFC) + ULP_BITMAP_RESET(ulp_ctx->tfo_flags, BNXT_ULP_TFO_TSID_FLAG); +} + +/* Function to set the v3 table scope id, only works for tfc objects */ +static inline int32_t +bnxt_ulp_cntxt_tsid_get(struct bnxt_ulp_context *ulp_ctx, uint8_t *tsid) +{ + if (likely(ulp_ctx && tsid && + ulp_ctx->tfo_type == BNXT_ULP_TFO_TYPE_TFC && + ULP_BITMAP_ISSET(ulp_ctx->tfo_flags, BNXT_ULP_TFO_TSID_FLAG))) { + *tsid = ulp_ctx->tsid; + return 0; + } + return -EINVAL; +} + +/* Function to set the v3 session id, only works for tfc objects */ +static inline int32_t +bnxt_ulp_cntxt_sid_set(struct bnxt_ulp_context *ulp_ctx, + uint16_t sid) +{ + if (likely(ulp_ctx && ulp_ctx->tfo_type == BNXT_ULP_TFO_TYPE_TFC)) { + ulp_ctx->sid = sid; + ULP_BITMAP_SET(ulp_ctx->tfo_flags, BNXT_ULP_TFO_SID_FLAG); + return 0; + } + return -EINVAL; +} + +/* + * Function to reset the v3 session id, only works for tfc objects + * There isn't a known invalid value for sid, so this is necessary + */ +static inline void +bnxt_ulp_cntxt_sid_reset(struct bnxt_ulp_context *ulp_ctx) +{ + if (ulp_ctx && ulp_ctx->tfo_type == BNXT_ULP_TFO_TYPE_TFC) + ULP_BITMAP_RESET(ulp_ctx->tfo_flags, BNXT_ULP_TFO_SID_FLAG); +} + +/* Function to get the v3 session id, only works for tfc objects */ +static inline int32_t +bnxt_ulp_cntxt_sid_get(struct bnxt_ulp_context *ulp_ctx, + uint16_t *sid) +{ + if (likely(ulp_ctx && sid && + ulp_ctx->tfo_type == BNXT_ULP_TFO_TYPE_TFC && + ULP_BITMAP_ISSET(ulp_ctx->tfo_flags, BNXT_ULP_TFO_SID_FLAG))) { + *sid = ulp_ctx->sid; + return 0; + } + return -EINVAL; +} + +/* Function to get the number of shared clients attached */ +static inline uint8_t +bnxt_ulp_cntxt_num_shared_clients_get(struct bnxt_ulp_context *ulp) +{ + if (likely(ulp == NULL || ulp->cfg_data == NULL)) { + BNXT_DRV_DBG(ERR, "Invalid arguments\n"); + return 0; + } + return ulp->cfg_data->num_shared_clients; +} + +/* Function to set the number of shared clients */ +static inline int +bnxt_ulp_cntxt_num_shared_clients_set(struct bnxt_ulp_context *ulp, bool incr) +{ + if (unlikely(ulp == NULL || ulp->cfg_data == NULL)) { + BNXT_DRV_DBG(ERR, "Invalid arguments\n"); + return 0; + } + if (incr) + ulp->cfg_data->num_shared_clients++; + else if (ulp->cfg_data->num_shared_clients) + ulp->cfg_data->num_shared_clients--; + + BNXT_DRV_DBG(DEBUG, "%d:clients(%d)\n", incr, + ulp->cfg_data->num_shared_clients); + + return 0; +} + +static inline int32_t +bnxt_ulp_cntxt_bp_set(struct bnxt_ulp_context *ulp, struct bnxt *bp) +{ + if (unlikely(ulp == NULL)) { + BNXT_DRV_DBG(ERR, "Invalid arguments\n"); + return -EINVAL; + } + ulp->bp = bp; + return 0; +} + +static inline struct bnxt* +bnxt_ulp_cntxt_bp_get(struct bnxt_ulp_context *ulp) +{ + if (unlikely(ulp == NULL)) { + BNXT_DRV_DBG(ERR, "Invalid arguments\n"); + return NULL; + } + return ulp->bp; +} + +static inline int32_t +bnxt_ulp_cntxt_fid_get(struct bnxt_ulp_context *ulp, uint16_t *fid) +{ + if (unlikely(ulp == NULL || fid == NULL)) + return -EINVAL; + + *fid = ulp->bp->fw_fid; + return 0; +} + +static inline void +bnxt_ulp_cntxt_ptr2_default_class_bits_set(struct bnxt_ulp_context *ulp_ctx, + uint64_t bits) +{ + if (unlikely(!ulp_ctx || !ulp_ctx->cfg_data)) + return; + ulp_ctx->cfg_data->default_class_bits = bits; +} + +static inline uint64_t +bnxt_ulp_cntxt_ptr2_default_class_bits_get(struct bnxt_ulp_context *ulp_ctx) +{ + if (unlikely(!ulp_ctx || !ulp_ctx->cfg_data)) + return 0; + return ulp_ctx->cfg_data->default_class_bits; +} + +static inline void +bnxt_ulp_cntxt_ptr2_default_act_bits_set(struct bnxt_ulp_context *ulp_ctx, + uint64_t bits) +{ + if (unlikely(!ulp_ctx || !ulp_ctx->cfg_data)) + return; + ulp_ctx->cfg_data->default_act_bits = bits; +} + +static inline uint64_t +bnxt_ulp_cntxt_ptr2_default_act_bits_get(struct bnxt_ulp_context *ulp_ctx) +{ + if (unlikely(!ulp_ctx || !ulp_ctx->cfg_data)) + return 0; + return ulp_ctx->cfg_data->default_act_bits; +} + +/* + * Get the device table entry based on the device id. + * + * dev_id [in] The device id of the hardware + * + * Returns the pointer to the device parameters. + */ +static inline struct bnxt_ulp_device_params * +bnxt_ulp_device_params_get(uint32_t dev_id) +{ + if (dev_id < BNXT_ULP_MAX_NUM_DEVICES) + return &ulp_device_params[dev_id]; + return NULL; +} + +/* Function to set the flow database to the ulp context. */ +static inline int32_t +bnxt_ulp_cntxt_ptr2_flow_db_set(struct bnxt_ulp_context *ulp_ctx, + struct bnxt_ulp_flow_db *flow_db) +{ + if (unlikely(!ulp_ctx || !ulp_ctx->cfg_data)) + return -EINVAL; + + ulp_ctx->cfg_data->flow_db = flow_db; + return 0; +} + +/* Function to get the flow database from the ulp context. */ +static inline struct bnxt_ulp_flow_db * +bnxt_ulp_cntxt_ptr2_flow_db_get(struct bnxt_ulp_context *ulp_ctx) +{ + if (unlikely(!ulp_ctx || !ulp_ctx->cfg_data)) + return NULL; + + return ulp_ctx->cfg_data->flow_db; +} + +/* Function to get the tunnel cache table info from the ulp context. */ +static inline struct bnxt_tun_cache_entry * +bnxt_ulp_cntxt_ptr2_tun_tbl_get(struct bnxt_ulp_context *ulp_ctx) +{ + if (unlikely(!ulp_ctx || !ulp_ctx->cfg_data)) + return NULL; + + return ulp_ctx->cfg_data->tun_tbl; +} + +/* Function to get the ulp context from eth device. */ +static inline struct bnxt_ulp_context * +bnxt_ulp_eth_dev_ptr2_cntxt_get(struct rte_eth_dev *dev) +{ + struct bnxt *bp = (struct bnxt *)dev->data->dev_private; + + if (BNXT_ETH_DEV_IS_REPRESENTOR(dev)) { + struct bnxt_representor *vfr = dev->data->dev_private; + + bp = vfr->parent_dev->data->dev_private; + } + + if (unlikely(!bp)) { + BNXT_DRV_DBG(ERR, "Bnxt private data is not initialized\n"); + return NULL; + } + return bp->ulp_ctx; +} + +static inline int32_t +bnxt_ulp_cntxt_ptr2_mapper_data_set(struct bnxt_ulp_context *ulp_ctx, + void *mapper_data) +{ + if (unlikely(!ulp_ctx || !ulp_ctx->cfg_data)) { + BNXT_DRV_DBG(ERR, "Invalid ulp context data\n"); + return -EINVAL; + } + + ulp_ctx->cfg_data->mapper_data = mapper_data; + return 0; +} + +static inline void * +bnxt_ulp_cntxt_ptr2_mapper_data_get(struct bnxt_ulp_context *ulp_ctx) +{ + if (unlikely(!ulp_ctx || !ulp_ctx->cfg_data)) { + BNXT_DRV_DBG(ERR, "Invalid ulp context data\n"); + return NULL; + } + + return ulp_ctx->cfg_data->mapper_data; +} + +static inline int32_t +bnxt_ulp_cntxt_ptr2_matcher_data_set(struct bnxt_ulp_context *ulp_ctx, + void *matcher_data) +{ + if (unlikely(!ulp_ctx || !ulp_ctx->cfg_data)) { + BNXT_DRV_DBG(ERR, "Invalid ulp context data\n"); + return -EINVAL; + } + + ulp_ctx->cfg_data->matcher_data = matcher_data; + return 0; +} + +static inline void * +bnxt_ulp_cntxt_ptr2_matcher_data_get(struct bnxt_ulp_context *ulp_ctx) +{ + if (unlikely(!ulp_ctx || !ulp_ctx->cfg_data)) { + BNXT_DRV_DBG(ERR, "Invalid ulp context data\n"); + return NULL; + } + + return ulp_ctx->cfg_data->matcher_data; +} + +/* Function to set the port database to the ulp context. */ +static inline int32_t +bnxt_ulp_cntxt_ptr2_port_db_set(struct bnxt_ulp_context *ulp_ctx, + struct bnxt_ulp_port_db *port_db) +{ + if (unlikely(!ulp_ctx || !ulp_ctx->cfg_data)) + return -EINVAL; + + ulp_ctx->cfg_data->port_db = port_db; + return 0; +} + +/* Function to get the port database from the ulp context. */ +static inline struct bnxt_ulp_port_db * +bnxt_ulp_cntxt_ptr2_port_db_get(struct bnxt_ulp_context *ulp_ctx) +{ + if (unlikely(!ulp_ctx || !ulp_ctx->cfg_data)) + return NULL; + + return ulp_ctx->cfg_data->port_db; +} + +/* Function to set the flow counter info into the context */ +static inline int32_t +bnxt_ulp_cntxt_ptr2_fc_info_set(struct bnxt_ulp_context *ulp_ctx, + struct bnxt_ulp_fc_info *ulp_fc_info) +{ + if (unlikely(!ulp_ctx || !ulp_ctx->cfg_data)) { + BNXT_DRV_DBG(ERR, "Invalid ulp context data\n"); + return -EINVAL; + } + + ulp_ctx->cfg_data->fc_info = ulp_fc_info; + + return 0; +} + +/* Function to retrieve the flow counter info from the context. */ +static inline struct bnxt_ulp_fc_info * +bnxt_ulp_cntxt_ptr2_fc_info_get(struct bnxt_ulp_context *ulp_ctx) +{ + if (unlikely(!ulp_ctx || !ulp_ctx->cfg_data)) + return NULL; + + return ulp_ctx->cfg_data->fc_info; +} + +/* Function to get the ulp flags from the ulp context. */ +static inline int32_t +bnxt_ulp_cntxt_ptr2_ulp_flags_get(struct bnxt_ulp_context *ulp_ctx, + uint32_t *flags) +{ + if (unlikely(!ulp_ctx || !ulp_ctx->cfg_data)) + return -1; + + *flags = ulp_ctx->cfg_data->ulp_flags; + return 0; +} + +/* Function to get the ulp vfr info from the ulp context. */ +static inline struct bnxt_ulp_vfr_rule_info* +bnxt_ulp_cntxt_ptr2_ulp_vfr_info_get(struct bnxt_ulp_context *ulp_ctx, + uint32_t port_id) +{ + if (unlikely(!ulp_ctx || + !ulp_ctx->cfg_data || + port_id >= RTE_MAX_ETHPORTS)) + return NULL; + + return &ulp_ctx->cfg_data->vfr_rule_info[port_id]; +} + +/* Function to acquire the flow database lock from the ulp context. */ +static inline int32_t +bnxt_ulp_cntxt_acquire_fdb_lock(struct bnxt_ulp_context *ulp_ctx) +{ + if (unlikely(!ulp_ctx || !ulp_ctx->cfg_data)) + return -1; + + if (pthread_mutex_lock(&ulp_ctx->cfg_data->flow_db_lock)) { + BNXT_DRV_DBG(ERR, "unable to acquire fdb lock\n"); + return -1; + } + return 0; +} + +/* Function to release the flow database lock from the ulp context. */ +static inline void +bnxt_ulp_cntxt_release_fdb_lock(struct bnxt_ulp_context *ulp_ctx) +{ + if (unlikely(!ulp_ctx || !ulp_ctx->cfg_data)) + return; + + pthread_mutex_unlock(&ulp_ctx->cfg_data->flow_db_lock); +} + +#if (RTE_VERSION_NUM(21, 05, 0, 0) > RTE_VERSION) + +/* Function to extract the action type from the shared action handle. */ +static inline uint32_t +bnxt_get_shared_action_type(const struct rte_flow_shared_action *handle) +{ + return (uint32_t)(((uint64_t)handle >> 32) & 0xffffffff); +} + +/* Function to extract the direction from the shared action handle. */ +static inline uint32_t +bnxt_get_shared_action_direction(const struct rte_flow_shared_action *handle) +{ + uint32_t shared_type; + + shared_type = bnxt_get_shared_action_type(handle); + return shared_type & 0x1 ? BNXT_ULP_FLOW_ATTR_EGRESS : + BNXT_ULP_FLOW_ATTR_INGRESS; +} + +/* Function to extract the action index from the shared action handle. */ +static inline uint32_t +bnxt_get_shared_action_index(const struct rte_flow_shared_action *handle) +{ + return (uint32_t)((uint64_t)handle & 0xffffffff); +} + +#else /* (RTE_VERSION >= RTE_VERSION_NUM(21,05,0,0)) */ + +/* Function to extract the action type from the shared action handle. */ +static inline int32_t +bnxt_get_action_handle_type(const struct rte_flow_action_handle *handle, + uint32_t *action_handle_type) +{ + if (unlikely(!action_handle_type)) + return -EINVAL; + + *action_handle_type = (uint32_t)(((uint64_t)handle >> 32) & 0xffffffff); + if (*action_handle_type >= BNXT_ULP_GEN_TBL_MAX_SZ) + return -EINVAL; + + return 0; +} + +/* Function to extract the direction from the shared action handle. */ +static inline int32_t +bnxt_get_action_handle_direction(const struct rte_flow_action_handle *handle, + uint32_t *dir) +{ + uint32_t shared_type; + int32_t ret = 0; + + ret = bnxt_get_action_handle_type(handle, &shared_type); + if (unlikely(ret)) + return ret; + + *dir = shared_type & 0x1 ? BNXT_ULP_DIR_EGRESS : BNXT_ULP_DIR_INGRESS; + + return ret; +} + +/* Function to extract the action index from the shared action handle. */ +static inline uint32_t +bnxt_get_action_handle_index(const struct rte_flow_action_handle *handle) +{ + return (uint32_t)((uint64_t)handle & 0xffffffff); +} + +#endif /* RTE_VERSION < RTE_VERSION_NUM(21,05,0,0) */ + +/* Function to set the ha info into the context */ +static inline int32_t +bnxt_ulp_cntxt_ptr2_ha_info_set(struct bnxt_ulp_context *ulp_ctx, + struct bnxt_ulp_ha_mgr_info *ulp_ha_info) +{ + if (unlikely(ulp_ctx == NULL || ulp_ctx->cfg_data == NULL)) { + BNXT_DRV_DBG(ERR, "Invalid ulp context data\n"); + return -EINVAL; + } + ulp_ctx->cfg_data->ha_info = ulp_ha_info; + return 0; +} + +/* Function to retrieve the ha info from the context. */ +static inline struct bnxt_ulp_ha_mgr_info * +bnxt_ulp_cntxt_ptr2_ha_info_get(struct bnxt_ulp_context *ulp_ctx) +{ + if (unlikely(ulp_ctx == NULL || ulp_ctx->cfg_data == NULL)) + return NULL; + return ulp_ctx->cfg_data->ha_info; +} + +static inline bool +bnxt_ulp_cntxt_ha_enabled(struct bnxt_ulp_context *ulp_ctx) +{ + if (unlikely(ulp_ctx == NULL || ulp_ctx->cfg_data == NULL)) + return false; + return !!ULP_HIGH_AVAIL_IS_ENABLED(ulp_ctx->cfg_data->ulp_flags); +} + +/* Function to get the app tunnel details from the ulp context. */ +static inline struct bnxt_flow_app_tun_ent * +bnxt_ulp_cntxt_ptr2_app_tun_list_get(struct bnxt_ulp_context *ulp) +{ + if (unlikely(!ulp || !ulp->cfg_data)) + return NULL; + + return ulp->cfg_data->app_tun; +} + +/* Function to get the truflow app id. This defined in the build file */ +static inline uint32_t +bnxt_ulp_default_app_id_get(void) +{ + return BNXT_TF_APP_ID; +} + +/* Function to convert ulp dev id to regular dev id. */ +static inline uint32_t +bnxt_ulp_cntxt_convert_dev_id(uint32_t ulp_dev_id) +{ + enum tf_device_type type = 0; + + switch (ulp_dev_id) { + case BNXT_ULP_DEVICE_ID_WH_PLUS: + type = TF_DEVICE_TYPE_P4; + break; + case BNXT_ULP_DEVICE_ID_STINGRAY: + type = TF_DEVICE_TYPE_SR; + break; + case BNXT_ULP_DEVICE_ID_THOR: + type = TF_DEVICE_TYPE_P5; + break; + default: + BNXT_DRV_DBG(ERR, "Invalid device id\n"); + break; + } + return type; +} + +/* This function sets the IF table index for the + * Application to poll to get the hot upgrade state and count details from + * the firmware. + */ +static inline int32_t +bnxt_ulp_cntxt_ha_reg_set(struct bnxt_ulp_context *ulp_ctx, + uint8_t state, uint8_t cnt) +{ + if (unlikely(!ulp_ctx || !ulp_ctx->cfg_data)) + return -EINVAL; + + if (ULP_MULTI_SHARED_IS_SUPPORTED(ulp_ctx)) { + ulp_ctx->cfg_data->hu_reg_state = state; + ulp_ctx->cfg_data->hu_reg_cnt = cnt; + } else { + ulp_ctx->cfg_data->hu_reg_state = ULP_HA_IF_TBL_IDX; + ulp_ctx->cfg_data->hu_reg_cnt = ULP_HA_CLIENT_CNT_IF_TBL_IDX; + } + return 0; +} + +/* This function gets the IF table index for the + * Application to poll to get the application hot upgrade state from + * the firmware. + */ +static inline uint32_t +bnxt_ulp_cntxt_ha_reg_state_get(struct bnxt_ulp_context *ulp_ctx) +{ + if (unlikely(!ulp_ctx || !ulp_ctx->cfg_data)) + return 0; + + return (uint32_t)ulp_ctx->cfg_data->hu_reg_state; +} + +/* This function gets the IF table index for the + * Application to poll to get the application count from + * the firmwware. + */ +static inline uint32_t +bnxt_ulp_cntxt_ha_reg_cnt_get(struct bnxt_ulp_context *ulp_ctx) +{ + if (unlikely(!ulp_ctx || !ulp_ctx->cfg_data)) + return 0; + + return (uint32_t)ulp_ctx->cfg_data->hu_reg_cnt; +} + +/* This function sets the number of key recipes supported + * Generally, this should be set to the number of flexible keys + * supported + */ +static inline void +bnxt_ulp_num_key_recipes_set(struct bnxt_ulp_context *ulp_ctx, + uint16_t num_recipes) +{ + if (unlikely(!ulp_ctx || !ulp_ctx->cfg_data)) + return; + ulp_ctx->cfg_data->num_key_recipes_per_dir = num_recipes; +} + +/* This function gets the number of key recipes supported */ +static inline int32_t +bnxt_ulp_num_key_recipes_get(struct bnxt_ulp_context *ulp_ctx) +{ + if (unlikely(!ulp_ctx || !ulp_ctx->cfg_data)) + return 0; + return ulp_ctx->cfg_data->num_key_recipes_per_dir; +} + +/* This function gets the feature bits */ +static inline uint64_t +bnxt_ulp_feature_bits_get(struct bnxt_ulp_context *ulp_ctx) +{ + if (unlikely(!ulp_ctx || !ulp_ctx->cfg_data)) + return 0; + return ulp_ctx->cfg_data->feature_bits; +} + +/* Add the VF Rep endpoint to the session */ +static inline int32_t +bnxt_ulp_vfr_session_fid_add(struct bnxt_ulp_context *ulp_ctx, + uint16_t vfr_fid) +{ + int32_t rc = 0; + + if (unlikely(ulp_ctx == NULL || ulp_ctx->ops == NULL)) + return -EINVAL; + if (ulp_ctx->ops->ulp_vfr_session_fid_add) + rc = ulp_ctx->ops->ulp_vfr_session_fid_add(ulp_ctx, vfr_fid); + + return rc; +} + +/* Remove the VF Rep endpoint from the session */ +static inline int32_t +bnxt_ulp_vfr_session_fid_rem(struct bnxt_ulp_context *ulp_ctx, + uint16_t vfr_fid) +{ + int32_t rc = 0; + + if (unlikely(ulp_ctx == NULL || ulp_ctx->ops == NULL)) + return -EINVAL; + if (ulp_ctx->ops->ulp_vfr_session_fid_rem) + rc = ulp_ctx->ops->ulp_vfr_session_fid_rem(ulp_ctx, vfr_fid); + return rc; +} + +#endif /* _BNXT_ULP_UTILS_H_ */ diff --git a/drivers/net/bnxt/tf_ulp/meson.build b/drivers/net/bnxt/tf_ulp/meson.build index 855380ffac..e19d51ee01 100644 --- a/drivers/net/bnxt/tf_ulp/meson.build +++ b/drivers/net/bnxt/tf_ulp/meson.build @@ -25,7 +25,6 @@ sources += files( 'ulp_rte_handler_tbl.c', 'ulp_rte_parser.c', 'ulp_tun.c', - 'ulp_utils.c', 'ulp_mapper_tf.c', 'ulp_mapper_tfc.c', 'bnxt_ulp_tf.c', diff --git a/drivers/net/bnxt/tf_ulp/ulp_alloc_tbl.c b/drivers/net/bnxt/tf_ulp/ulp_alloc_tbl.c index 5245aa26c5..6401a7a80f 100644 --- a/drivers/net/bnxt/tf_ulp/ulp_alloc_tbl.c +++ b/drivers/net/bnxt/tf_ulp/ulp_alloc_tbl.c @@ -8,6 +8,7 @@ #include "tf_core.h" #include "ulp_mapper.h" #include "ulp_alloc_tbl.h" +#include "bnxt_ulp_utils.h" #ifdef RTE_LIBRTE_BNXT_TRUFLOW_DEBUG #include "ulp_template_debug_proto.h" diff --git a/drivers/net/bnxt/tf_ulp/ulp_def_rules.c b/drivers/net/bnxt/tf_ulp/ulp_def_rules.c index 7a5f90fb09..17d2daeea3 100644 --- a/drivers/net/bnxt/tf_ulp/ulp_def_rules.c +++ b/drivers/net/bnxt/tf_ulp/ulp_def_rules.c @@ -4,6 +4,7 @@ */ #include "bnxt_tf_common.h" +#include "bnxt_ulp_utils.h" #include "ulp_template_struct.h" #include "ulp_template_db_enum.h" #include "ulp_template_db_field.h" diff --git a/drivers/net/bnxt/tf_ulp/ulp_fc_mgr.c b/drivers/net/bnxt/tf_ulp/ulp_fc_mgr.c index eb2d14f3be..67489b76f3 100644 --- a/drivers/net/bnxt/tf_ulp/ulp_fc_mgr.c +++ b/drivers/net/bnxt/tf_ulp/ulp_fc_mgr.c @@ -10,6 +10,7 @@ #include #include "bnxt.h" #include "bnxt_ulp.h" +#include "bnxt_ulp_utils.h" #include "bnxt_ulp_tf.h" #include "bnxt_tf_common.h" #include "ulp_fc_mgr.h" diff --git a/drivers/net/bnxt/tf_ulp/ulp_fc_mgr_tf.c b/drivers/net/bnxt/tf_ulp/ulp_fc_mgr_tf.c index a9b2267881..60a7073514 100644 --- a/drivers/net/bnxt/tf_ulp/ulp_fc_mgr_tf.c +++ b/drivers/net/bnxt/tf_ulp/ulp_fc_mgr_tf.c @@ -10,6 +10,7 @@ #include #include "bnxt.h" #include "bnxt_ulp.h" +#include "bnxt_ulp_utils.h" #include "bnxt_ulp_tf.h" #include "bnxt_tf_common.h" #include "ulp_fc_mgr.h" diff --git a/drivers/net/bnxt/tf_ulp/ulp_fc_mgr_tfc.c b/drivers/net/bnxt/tf_ulp/ulp_fc_mgr_tfc.c index 7c404685df..4a93dd1a33 100644 --- a/drivers/net/bnxt/tf_ulp/ulp_fc_mgr_tfc.c +++ b/drivers/net/bnxt/tf_ulp/ulp_fc_mgr_tfc.c @@ -10,6 +10,7 @@ #include #include "bnxt.h" #include "bnxt_ulp.h" +#include "bnxt_ulp_utils.h" #include "bnxt_ulp_tfc.h" #include "bnxt_tf_common.h" #include "ulp_fc_mgr.h" diff --git a/drivers/net/bnxt/tf_ulp/ulp_flow_db.c b/drivers/net/bnxt/tf_ulp/ulp_flow_db.c index 02a1da24bb..90ee2e8808 100644 --- a/drivers/net/bnxt/tf_ulp/ulp_flow_db.c +++ b/drivers/net/bnxt/tf_ulp/ulp_flow_db.c @@ -8,7 +8,7 @@ #include "bnxt.h" #include "bnxt_tf_common.h" -#include "ulp_utils.h" +#include "bnxt_ulp_utils.h" #include "ulp_template_struct.h" #include "ulp_mapper.h" #include "ulp_flow_db.h" diff --git a/drivers/net/bnxt/tf_ulp/ulp_gen_tbl.c b/drivers/net/bnxt/tf_ulp/ulp_gen_tbl.c index bf92626c50..c2ae3dc3a2 100644 --- a/drivers/net/bnxt/tf_ulp/ulp_gen_tbl.c +++ b/drivers/net/bnxt/tf_ulp/ulp_gen_tbl.c @@ -5,6 +5,7 @@ #include #include +#include "bnxt_ulp_utils.h" #include "tf_core.h" #include "tfp.h" #include "ulp_mapper.h" @@ -625,4 +626,3 @@ ulp_gen_tbl_simple_list_search(struct ulp_mapper_gen_tbl_list *tbl_list, } return rc; } - diff --git a/drivers/net/bnxt/tf_ulp/ulp_ha_mgr.c b/drivers/net/bnxt/tf_ulp/ulp_ha_mgr.c index e53ad4728d..3c75a6b437 100644 --- a/drivers/net/bnxt/tf_ulp/ulp_ha_mgr.c +++ b/drivers/net/bnxt/tf_ulp/ulp_ha_mgr.c @@ -10,6 +10,7 @@ #include #include "bnxt.h" #include "bnxt_ulp.h" +#include "bnxt_ulp_utils.h" #include "bnxt_ulp_tf.h" #include "bnxt_tf_common.h" #include "ulp_ha_mgr.h" diff --git a/drivers/net/bnxt/tf_ulp/ulp_mapper.c b/drivers/net/bnxt/tf_ulp/ulp_mapper.c index dc43d1c810..f0d715d369 100644 --- a/drivers/net/bnxt/tf_ulp/ulp_mapper.c +++ b/drivers/net/bnxt/tf_ulp/ulp_mapper.c @@ -11,6 +11,7 @@ #include "bnxt_tf_common.h" #include "ulp_utils.h" #include "bnxt_ulp.h" +#include "bnxt_ulp_utils.h" #include "tfp.h" #include "tf_ext_flow_handle.h" #include "ulp_mark_mgr.h" @@ -50,7 +51,7 @@ bnxt_ulp_mapper_ops_get(struct bnxt *bp) const struct ulp_mapper_core_ops *func_ops; rc = bnxt_ulp_devid_get(bp, &dev_id); - if (rc) + if (unlikely(rc)) return NULL; switch (dev_id) { @@ -94,7 +95,7 @@ ulp_mapper_tmpl_name_str(enum bnxt_ulp_template_type tmpl_type) static struct bnxt_ulp_glb_resource_info * ulp_mapper_glb_resource_info_list_get(uint32_t *num_entries) { - if (!num_entries) + if (unlikely(!num_entries)) return NULL; *num_entries = BNXT_ULP_GLB_RESOURCE_TBL_MAX_SZ; return ulp_glb_resource_tbl; @@ -114,8 +115,8 @@ ulp_mapper_glb_resource_read(struct bnxt_ulp_mapper_data *mapper_data, uint64_t *regval, bool *shared) { - if (!mapper_data || !regval || !shared || - dir >= TF_DIR_MAX || idx >= BNXT_ULP_GLB_RF_IDX_LAST) + if (unlikely(!mapper_data || !regval || !shared || + dir >= TF_DIR_MAX || idx >= BNXT_ULP_GLB_RF_IDX_LAST)) return -EINVAL; *regval = mapper_data->glb_res_tbl[dir][idx].resource_hndl; @@ -138,8 +139,8 @@ ulp_mapper_glb_resource_write(struct bnxt_ulp_mapper_data *data, struct bnxt_ulp_mapper_glb_resource_entry *ent; /* validate the arguments */ - if (!data || res->direction >= TF_DIR_MAX || - res->glb_regfile_index >= BNXT_ULP_GLB_RF_IDX_LAST) + if (unlikely(!data || res->direction >= TF_DIR_MAX || + res->glb_regfile_index >= BNXT_ULP_GLB_RF_IDX_LAST)) return -EINVAL; /* write to the mapper data */ @@ -178,7 +179,7 @@ ulp_mapper_resource_ident_allocate(struct bnxt_ulp_context *ulp_ctx, glb_res->direction, CFA_TRACK_TYPE_SID, &id); - if (rc) + if (unlikely(rc)) return rc; /* entries are stored as big-endian format */ @@ -189,7 +190,7 @@ ulp_mapper_resource_ident_allocate(struct bnxt_ulp_context *ulp_ctx, * shared flag is always false. */ rc = ulp_mapper_glb_resource_write(mapper_data, glb_res, regval, shared); - if (rc) { + if (unlikely(rc)) { BNXT_DRV_DBG(ERR, "Failed to write to global resource id\n"); /* Free the identifier when update failed */ res.direction = glb_res->direction; @@ -233,7 +234,7 @@ ulp_mapper_resource_index_tbl_alloc(struct bnxt_ulp_context *ulp_ctx, * shared flag is always false. */ rc = ulp_mapper_glb_resource_write(mapper_data, glb_res, regval, shared); - if (rc) { + if (unlikely(rc)) { BNXT_DRV_DBG(ERR, "Failed to write to global resource id\n"); /* Free the index when update failed */ res.direction = glb_res->direction; @@ -252,7 +253,7 @@ ulp_mapper_glb_field_tbl_get(struct bnxt_ulp_mapper_parms *parms, { uint32_t t_idx; - if (operand >= BNXT_ULP_GLB_FIELD_TBL_SIZE) { + if (unlikely(operand >= BNXT_ULP_GLB_FIELD_TBL_SIZE)) { BNXT_DRV_DBG(ERR, "Invalid hdr field index %x:%x\n", parms->class_tid, operand); *val = 0; @@ -274,7 +275,7 @@ ulp_mapper_glb_field_tbl_get(struct bnxt_ulp_mapper_parms *parms, static uint32_t ulp_mapper_act_prop_size_get(uint32_t idx) { - if (idx >= BNXT_ULP_ACT_PROP_IDX_LAST) + if (unlikely(idx >= BNXT_ULP_ACT_PROP_IDX_LAST)) return 0; return ulp_act_prop_map_table[idx]; } @@ -296,7 +297,7 @@ ulp_mapper_cond_oper_list_get(struct bnxt_ulp_mapper_parms *mparms, const struct bnxt_ulp_template_device_tbls *dev_tbls; dev_tbls = &mparms->device_params->dev_tbls[mparms->tmpl_type]; - if (idx >= dev_tbls->cond_oper_list_size) + if (unlikely(idx >= dev_tbls->cond_oper_list_size)) return NULL; return &dev_tbls->cond_oper_list[idx]; } @@ -308,7 +309,7 @@ ulp_mapper_tmpl_cond_list_get(struct bnxt_ulp_mapper_parms *mparms, const struct bnxt_ulp_template_device_tbls *dev_tbls; dev_tbls = &mparms->device_params->dev_tbls[mparms->tmpl_type]; - if (idx >= dev_tbls->cond_list_size) + if (unlikely(idx >= dev_tbls->cond_list_size)) return NULL; return &dev_tbls->cond_list[idx]; } @@ -362,7 +363,7 @@ ulp_mapper_key_fields_get(struct bnxt_ulp_mapper_parms *mparms, const struct bnxt_ulp_template_device_tbls *dev_tbls; dev_tbls = &mparms->device_params->dev_tbls[mparms->tmpl_type]; - if (!dev_tbls->key_info_list) { + if (unlikely(!dev_tbls->key_info_list)) { *num_flds = 0; return NULL; } @@ -417,7 +418,7 @@ ulp_mapper_result_fields_get(struct bnxt_ulp_mapper_parms *mparms, const struct bnxt_ulp_template_device_tbls *dev_tbls; dev_tbls = &mparms->device_params->dev_tbls[mparms->tmpl_type]; - if (!dev_tbls->result_field_list) { + if (unlikely(!dev_tbls->result_field_list)) { *num_flds = 0; *num_encap_flds = 0; return NULL; @@ -448,7 +449,7 @@ ulp_mapper_ident_fields_get(struct bnxt_ulp_mapper_parms *mparms, const struct bnxt_ulp_template_device_tbls *dev_tbls; dev_tbls = &mparms->device_params->dev_tbls[mparms->tmpl_type]; - if (!dev_tbls->ident_list) { + if (unlikely(!dev_tbls->ident_list)) { *num_flds = 0; return NULL; } @@ -466,7 +467,7 @@ ulp_mapper_tmpl_key_ext_list_get(struct bnxt_ulp_mapper_parms *mparms, const struct bnxt_ulp_template_device_tbls *dev_tbls; dev_tbls = &mparms->device_params->dev_tbls[mparms->tmpl_type]; - if (idx >= dev_tbls->key_ext_list_size) + if (unlikely(idx >= dev_tbls->key_ext_list_size)) return NULL; return &dev_tbls->key_ext_list[idx]; } @@ -490,7 +491,7 @@ ulp_mapper_parent_flow_free(struct bnxt_ulp_context *ulp, pc_idx = (uint32_t)res->resource_hndl; /* reset the child flow bitset*/ - if (ulp_flow_db_pc_db_parent_flow_set(ulp, pc_idx, parent_fid, 0)) { + if (unlikely(ulp_flow_db_pc_db_parent_flow_set(ulp, pc_idx, parent_fid, 0))) { BNXT_DRV_DBG(ERR, "error in reset parent flow bitset %x:%x\n", pc_idx, parent_fid); return -EINVAL; @@ -508,7 +509,7 @@ ulp_mapper_child_flow_free(struct bnxt_ulp_context *ulp, pc_idx = (uint32_t)res->resource_hndl; /* reset the child flow bitset*/ - if (ulp_flow_db_pc_db_child_flow_set(ulp, pc_idx, child_fid, 0)) { + if (unlikely(ulp_flow_db_pc_db_child_flow_set(ulp, pc_idx, child_fid, 0))) { BNXT_DRV_DBG(ERR, "error in resetting child flow bitset %x:%x\n", pc_idx, child_fid); @@ -533,7 +534,7 @@ ulp_mapper_fdb_opc_alloc_rid(struct bnxt_ulp_mapper_parms *parms, rc = ulp_flow_db_fid_alloc(parms->ulp_ctx, BNXT_ULP_FDB_TYPE_RID, 0, &rid); - if (rc) { + if (unlikely(rc)) { BNXT_DRV_DBG(ERR, "Unable to allocate flow table entry\n"); return -EINVAL; @@ -542,7 +543,7 @@ ulp_mapper_fdb_opc_alloc_rid(struct bnxt_ulp_mapper_parms *parms, val64 = rid; rc = ulp_regfile_write(parms->regfile, tbl->fdb_operand, tfp_cpu_to_be_64(val64)); - if (rc) { + if (unlikely(rc)) { BNXT_DRV_DBG(ERR, "Write regfile[%d] failed\n", tbl->fdb_operand); ulp_flow_db_fid_free(parms->ulp_ctx, @@ -566,7 +567,7 @@ ulp_mapper_fdb_opc_process(struct bnxt_ulp_mapper_parms *parms, struct ulp_flow_db_res_params *fid_parms) { uint32_t push_fid; - uint64_t val64; + uint64_t val64 = 0; enum bnxt_ulp_fdb_type flow_type; int32_t rc = 0; @@ -579,7 +580,7 @@ ulp_mapper_fdb_opc_process(struct bnxt_ulp_mapper_parms *parms, /* get the fid from the regfile */ rc = ulp_regfile_read(parms->regfile, tbl->fdb_operand, &val64); - if (rc) { + if (unlikely(rc)) { BNXT_DRV_DBG(ERR, "regfile[%d] read oob\n", tbl->fdb_operand); return -EINVAL; @@ -600,7 +601,7 @@ ulp_mapper_fdb_opc_process(struct bnxt_ulp_mapper_parms *parms, /* Add the resource to the flow database */ rc = ulp_flow_db_resource_add(parms->ulp_ctx, flow_type, push_fid, fid_parms); - if (rc) + if (unlikely(rc)) BNXT_DRV_DBG(ERR, "Failed to add res to flow %x rc = %d\n", push_fid, rc); @@ -636,8 +637,8 @@ ulp_mapper_priority_opc_process(struct bnxt_ulp_mapper_parms *parms, *priority = tbl->pri_operand; break; case BNXT_ULP_PRI_OPC_REGFILE: - if (ulp_regfile_read(parms->regfile, tbl->pri_operand, - ®val)) { + if (unlikely(ulp_regfile_read(parms->regfile, tbl->pri_operand, + ®val))) { BNXT_DRV_DBG(ERR, "regfile[%u] read oob\n", tbl->pri_operand); rc = -EINVAL; @@ -645,7 +646,7 @@ ulp_mapper_priority_opc_process(struct bnxt_ulp_mapper_parms *parms, *priority = (uint32_t)tfp_be_to_cpu_64(regval); break; case BNXT_ULP_PRI_OPC_COMP_FIELD: - if (tbl->pri_operand < BNXT_ULP_CF_IDX_LAST) { + if (likely(tbl->pri_operand < BNXT_ULP_CF_IDX_LAST)) { regval = ULP_COMP_FLD_IDX_RD(parms, tbl->pri_operand); *priority = regval; } else { @@ -687,7 +688,7 @@ ulp_mapper_tbl_ident_scan_ext(struct bnxt_ulp_mapper_parms *parms, uint64_t val64; /* validate the null arguments */ - if (!byte_data) { + if (unlikely(!byte_data)) { BNXT_DRV_DBG(ERR, "invalid argument\n"); return -EINVAL; } @@ -697,9 +698,9 @@ ulp_mapper_tbl_ident_scan_ext(struct bnxt_ulp_mapper_parms *parms, for (i = 0; i < num_idents; i++) { /* check the size of the buffer for validation */ - if ((idents[i].ident_bit_pos + idents[i].ident_bit_size) > + if (unlikely((idents[i].ident_bit_pos + idents[i].ident_bit_size) > ULP_BYTE_2_BITS(byte_data_size) || - idents[i].ident_bit_size > ULP_BYTE_2_BITS(sizeof(val64))) { + idents[i].ident_bit_size > ULP_BYTE_2_BITS(sizeof(val64)))) { BNXT_DRV_DBG(ERR, "invalid offset or length %x:%x:%x\n", idents[i].ident_bit_pos, idents[i].ident_bit_size, @@ -718,8 +719,8 @@ ulp_mapper_tbl_ident_scan_ext(struct bnxt_ulp_mapper_parms *parms, idents[i].ident_bit_size); /* Write it to the regfile, val64 is already in big-endian*/ - if (ulp_regfile_write(parms->regfile, - idents[i].regfile_idx, val64)) { + if (unlikely(ulp_regfile_write(parms->regfile, + idents[i].regfile_idx, val64))) { BNXT_DRV_DBG(ERR, "Regfile[%d] write failed.\n", idents[i].regfile_idx); return -EINVAL; @@ -757,14 +758,14 @@ ulp_mapper_ident_process(struct bnxt_ulp_mapper_parms *parms, tbl->direction, tbl->track_type, &id); - if (rc) { + if (unlikely(rc)) { BNXT_DRV_DBG(ERR, "identifier process failed\n"); return rc; } fid_parms.resource_hndl = id; idx = ident->regfile_idx; - if (ulp_regfile_write(parms->regfile, idx, tfp_cpu_to_be_64(id))) { + if (unlikely(ulp_regfile_write(parms->regfile, idx, tfp_cpu_to_be_64(id)))) { BNXT_DRV_DBG(ERR, "Regfile[%d] write failed.\n", idx); rc = -EINVAL; /* Need to free the identifier, so goto error */ @@ -774,7 +775,7 @@ ulp_mapper_ident_process(struct bnxt_ulp_mapper_parms *parms, /* Link the resource to the flow in the flow db */ if (!val) { rc = ulp_mapper_fdb_opc_process(parms, tbl, &fid_parms); - if (rc) { + if (unlikely(rc)) { BNXT_DRV_DBG(ERR, "Failed to link res to flow rc = %d\n", rc); @@ -802,49 +803,49 @@ ulp_mapper_field_port_db_process(struct bnxt_ulp_mapper_parms *parms, switch (port_data) { case BNXT_ULP_PORT_TABLE_DRV_FUNC_PARENT_MAC: - if (ulp_port_db_parent_mac_addr_get(parms->ulp_ctx, port_id, - val)) { + if (unlikely(ulp_port_db_parent_mac_addr_get(parms->ulp_ctx, port_id, + val))) { BNXT_DRV_DBG(ERR, "Invalid port id %u\n", port_id); return -EINVAL; } break; case BNXT_ULP_PORT_TABLE_DRV_FUNC_MAC: - if (ulp_port_db_drv_mac_addr_get(parms->ulp_ctx, port_id, - val)) { + if (unlikely(ulp_port_db_drv_mac_addr_get(parms->ulp_ctx, port_id, + val))) { BNXT_DRV_DBG(ERR, "Invalid port id %u\n", port_id); return -EINVAL; } break; case BNXT_ULP_PORT_TABLE_DRV_FUNC_PARENT_VNIC: - if (ulp_port_db_parent_vnic_get(parms->ulp_ctx, port_id, - val)) { + if (unlikely(ulp_port_db_parent_vnic_get(parms->ulp_ctx, port_id, + val))) { BNXT_DRV_DBG(ERR, "Invalid port id %u\n", port_id); return -EINVAL; } break; case BNXT_ULP_PORT_TABLE_PORT_IS_PF: - if (ulp_port_db_port_is_pf_get(parms->ulp_ctx, port_id, - val)) { + if (unlikely(ulp_port_db_port_is_pf_get(parms->ulp_ctx, port_id, + val))) { BNXT_DRV_DBG(ERR, "Invalid port id %u\n", port_id); return -EINVAL; } break; case BNXT_ULP_PORT_TABLE_VF_FUNC_METADATA: - if (ulp_port_db_port_meta_data_get(parms->ulp_ctx, port_id, - val)) { + if (unlikely(ulp_port_db_port_meta_data_get(parms->ulp_ctx, port_id, + val))) { BNXT_DRV_DBG(ERR, "Invalid port id %u\n", port_id); return -EINVAL; } break; case BNXT_ULP_PORT_TABLE_TABLE_SCOPE: - if (ulp_port_db_port_table_scope_get(parms->ulp_ctx, - port_id, val)) { + if (unlikely(ulp_port_db_port_table_scope_get(parms->ulp_ctx, + port_id, val))) { BNXT_DRV_DBG(ERR, "Invalid port id %u\n", port_id); return -EINVAL; } break; case BNXT_ULP_PORT_TABLE_VF_FUNC_FID: - if (ulp_port_db_port_vf_fid_get(parms->ulp_ctx, port_id, val)) { + if (unlikely(ulp_port_db_port_vf_fid_get(parms->ulp_ctx, port_id, val))) { BNXT_DRV_DBG(ERR, "Invalid port id %u\n", port_id); return -EINVAL; } @@ -894,13 +895,13 @@ ulp_mapper_field_src_process(struct bnxt_ulp_mapper_parms *parms, *value = 1; break; case BNXT_ULP_FIELD_SRC_CF: - if (ulp_operand_read(field_opr, - (uint8_t *)&idx, sizeof(uint16_t))) { + if (unlikely(ulp_operand_read(field_opr, + (uint8_t *)&idx, sizeof(uint16_t)))) { BNXT_DRV_DBG(ERR, "CF operand read failed\n"); return -EINVAL; } idx = tfp_be_to_cpu_16(idx); - if (idx >= BNXT_ULP_CF_IDX_LAST || bytelen > sizeof(uint64_t)) { + if (unlikely(idx >= BNXT_ULP_CF_IDX_LAST || bytelen > sizeof(uint64_t))) { BNXT_DRV_DBG(ERR, "comp field [%d] read oob %d\n", idx, bytelen); return -EINVAL; @@ -910,15 +911,15 @@ ulp_mapper_field_src_process(struct bnxt_ulp_mapper_parms *parms, *value = ULP_COMP_FLD_IDX_RD(parms, idx); break; case BNXT_ULP_FIELD_SRC_RF: - if (ulp_operand_read(field_opr, - (uint8_t *)&idx, sizeof(uint16_t))) { + if (unlikely(ulp_operand_read(field_opr, + (uint8_t *)&idx, sizeof(uint16_t)))) { BNXT_DRV_DBG(ERR, "RF operand read failed\n"); return -EINVAL; } idx = tfp_be_to_cpu_16(idx); /* Uninitialized regfile entries return 0 */ - if (ulp_regfile_read(parms->regfile, idx, &lregval) || + if (unlikely(ulp_regfile_read(parms->regfile, idx, &lregval)) || sizeof(uint64_t) < bytelen) { BNXT_DRV_DBG(ERR, "regfile[%d] read oob %u\n", idx, bytelen); @@ -929,19 +930,19 @@ ulp_mapper_field_src_process(struct bnxt_ulp_mapper_parms *parms, *value = tfp_be_to_cpu_64(lregval); break; case BNXT_ULP_FIELD_SRC_ACT_PROP: - if (ulp_operand_read(field_opr, - (uint8_t *)&idx, sizeof(uint16_t))) { + if (unlikely(ulp_operand_read(field_opr, + (uint8_t *)&idx, sizeof(uint16_t)))) { BNXT_DRV_DBG(ERR, "Action operand read failed\n"); return -EINVAL; } idx = tfp_be_to_cpu_16(idx); - if (idx >= BNXT_ULP_ACT_PROP_IDX_LAST) { + if (unlikely(idx >= BNXT_ULP_ACT_PROP_IDX_LAST)) { BNXT_DRV_DBG(ERR, "act_prop[%d] oob\n", idx); return -EINVAL; } buffer = &parms->act_prop->act_details[idx]; field_size = ulp_mapper_act_prop_size_get(idx); - if (bytelen > field_size) { + if (unlikely(bytelen > field_size)) { BNXT_DRV_DBG(ERR, "act_prop[%d] field size small %u\n", idx, field_size); return -EINVAL; @@ -954,27 +955,27 @@ ulp_mapper_field_src_process(struct bnxt_ulp_mapper_parms *parms, } break; case BNXT_ULP_FIELD_SRC_ACT_PROP_SZ: - if (ulp_operand_read(field_opr, - (uint8_t *)&idx, sizeof(uint16_t))) { + if (unlikely(ulp_operand_read(field_opr, + (uint8_t *)&idx, sizeof(uint16_t)))) { BNXT_DRV_DBG(ERR, "Action sz operand read failed\n"); return -EINVAL; } idx = tfp_be_to_cpu_16(idx); - if (idx >= BNXT_ULP_ACT_PROP_IDX_LAST) { + if (unlikely(idx >= BNXT_ULP_ACT_PROP_IDX_LAST)) { BNXT_DRV_DBG(ERR, "act_prop_sz[%d] oob\n", idx); return -EINVAL; } *val = &parms->act_prop->act_details[idx]; /* get the size index next */ - if (ulp_operand_read(&field_opr[sizeof(uint16_t)], - (uint8_t *)&size_idx, sizeof(uint16_t))) { + if (unlikely(ulp_operand_read(&field_opr[sizeof(uint16_t)], + (uint8_t *)&size_idx, sizeof(uint16_t)))) { BNXT_DRV_DBG(ERR, "Action sz operand read failed\n"); return -EINVAL; } size_idx = tfp_be_to_cpu_16(size_idx); - if (size_idx >= BNXT_ULP_ACT_PROP_IDX_LAST) { + if (unlikely(size_idx >= BNXT_ULP_ACT_PROP_IDX_LAST)) { BNXT_DRV_DBG(ERR, "act_prop[%d] oob\n", size_idx); return -EINVAL; } @@ -984,15 +985,15 @@ ulp_mapper_field_src_process(struct bnxt_ulp_mapper_parms *parms, *val_len = ULP_BYTE_2_BITS(val_size); break; case BNXT_ULP_FIELD_SRC_GLB_RF: - if (ulp_operand_read(field_opr, - (uint8_t *)&idx, sizeof(uint16_t))) { + if (unlikely(ulp_operand_read(field_opr, + (uint8_t *)&idx, sizeof(uint16_t)))) { BNXT_DRV_DBG(ERR, "Global regfile read failed\n"); return -EINVAL; } idx = tfp_be_to_cpu_16(idx); - if (ulp_mapper_glb_resource_read(parms->mapper_data, - dir, idx, &lregval, &shared) || - sizeof(uint64_t) < bytelen) { + if (unlikely(ulp_mapper_glb_resource_read(parms->mapper_data, + dir, idx, &lregval, &shared) || + sizeof(uint64_t) < bytelen)) { BNXT_DRV_DBG(ERR, "Global regfile[%d] read failed %u\n", idx, bytelen); return -EINVAL; @@ -1004,14 +1005,14 @@ ulp_mapper_field_src_process(struct bnxt_ulp_mapper_parms *parms, break; case BNXT_ULP_FIELD_SRC_HF: case BNXT_ULP_FIELD_SRC_SUB_HF: - if (ulp_operand_read(field_opr, - (uint8_t *)&idx, sizeof(uint16_t))) { + if (unlikely(ulp_operand_read(field_opr, + (uint8_t *)&idx, sizeof(uint16_t)))) { BNXT_DRV_DBG(ERR, "Header field read failed\n"); return -EINVAL; } idx = tfp_be_to_cpu_16(idx); /* get the index from the global field list */ - if (ulp_mapper_glb_field_tbl_get(parms, idx, &bit)) { + if (unlikely(ulp_mapper_glb_field_tbl_get(parms, idx, &bit))) { BNXT_DRV_DBG(ERR, "invalid ulp_glb_field_tbl idx %d\n", idx); return -EINVAL; @@ -1026,7 +1027,7 @@ ulp_mapper_field_src_process(struct bnxt_ulp_mapper_parms *parms, /* To support field processing of undefined fields */ *val = mapper_fld_zeros; break; - } else if (bytelen > field_size) { + } else if (unlikely(bytelen > field_size)) { BNXT_DRV_DBG(ERR, "Hdr field[%d] size small %u\n", bit, field_size); return -EINVAL; @@ -1035,15 +1036,15 @@ ulp_mapper_field_src_process(struct bnxt_ulp_mapper_parms *parms, *val = &buffer[field_size - bytelen]; } else { /* get the offset next */ - if (ulp_operand_read(&field_opr[sizeof(uint16_t)], + if (unlikely(ulp_operand_read(&field_opr[sizeof(uint16_t)], (uint8_t *)&offset, - sizeof(uint16_t))) { + sizeof(uint16_t)))) { BNXT_DRV_DBG(ERR, "Hdr fld size read failed\n"); return -EINVAL; } offset = tfp_be_to_cpu_16(offset); offset = ULP_BITS_2_BYTE_NR(offset); - if ((offset + bytelen) > field_size) { + if (unlikely((offset + bytelen) > field_size)) { BNXT_DRV_DBG(ERR, "Hdr field[%d] oob\n", bit); return -EINVAL; } @@ -1051,13 +1052,13 @@ ulp_mapper_field_src_process(struct bnxt_ulp_mapper_parms *parms, } break; case BNXT_ULP_FIELD_SRC_HDR_BIT: - if (ulp_operand_read(field_opr, - (uint8_t *)&lregval, sizeof(uint64_t))) { + if (unlikely(ulp_operand_read(field_opr, + (uint8_t *)&lregval, sizeof(uint64_t)))) { BNXT_DRV_DBG(ERR, "Header bit read failed\n"); return -EINVAL; } lregval = tfp_be_to_cpu_64(lregval); - if (ULP_BITMAP_ISSET(parms->hdr_bitmap->bits, lregval)) { + if (unlikely(ULP_BITMAP_ISSET(parms->hdr_bitmap->bits, lregval))) { *val = mapper_fld_one; *value = 1; } else { @@ -1065,13 +1066,13 @@ ulp_mapper_field_src_process(struct bnxt_ulp_mapper_parms *parms, } break; case BNXT_ULP_FIELD_SRC_ACT_BIT: - if (ulp_operand_read(field_opr, - (uint8_t *)&lregval, sizeof(uint64_t))) { + if (unlikely(ulp_operand_read(field_opr, + (uint8_t *)&lregval, sizeof(uint64_t)))) { BNXT_DRV_DBG(ERR, "Action bit read failed\n"); return -EINVAL; } lregval = tfp_be_to_cpu_64(lregval); - if (ULP_BITMAP_ISSET(parms->act_bitmap->bits, lregval)) { + if (unlikely(ULP_BITMAP_ISSET(parms->act_bitmap->bits, lregval))) { *val = mapper_fld_one; *value = 1; } else { @@ -1079,19 +1080,19 @@ ulp_mapper_field_src_process(struct bnxt_ulp_mapper_parms *parms, } break; case BNXT_ULP_FIELD_SRC_FIELD_BIT: - if (ulp_operand_read(field_opr, - (uint8_t *)&idx, sizeof(uint16_t))) { + if (unlikely(ulp_operand_read(field_opr, + (uint8_t *)&idx, sizeof(uint16_t)))) { BNXT_DRV_DBG(ERR, "Field bit read failed\n"); return -EINVAL; } idx = tfp_be_to_cpu_16(idx); /* get the index from the global field list */ - if (ulp_mapper_glb_field_tbl_get(parms, idx, &bit)) { + if (unlikely(ulp_mapper_glb_field_tbl_get(parms, idx, &bit))) { BNXT_DRV_DBG(ERR, "invalid ulp_glb_field_tbl idx %d\n", idx); return -EINVAL; } - if (ULP_INDEX_BITMAP_GET(parms->fld_bitmap->bits, bit)) { + if (unlikely(ULP_INDEX_BITMAP_GET(parms->fld_bitmap->bits, bit))) { *val = mapper_fld_one; *value = 1; } else { @@ -1099,13 +1100,13 @@ ulp_mapper_field_src_process(struct bnxt_ulp_mapper_parms *parms, } break; case BNXT_ULP_FIELD_SRC_PORT_TABLE: - if (ulp_operand_read(field_opr, - (uint8_t *)&idx, sizeof(uint16_t))) { + if (unlikely(ulp_operand_read(field_opr, + (uint8_t *)&idx, sizeof(uint16_t)))) { BNXT_DRV_DBG(ERR, "CF operand read failed\n"); return -EINVAL; } idx = tfp_be_to_cpu_16(idx); - if (idx >= BNXT_ULP_CF_IDX_LAST || bytelen > sizeof(uint64_t)) { + if (unlikely(idx >= BNXT_ULP_CF_IDX_LAST || bytelen > sizeof(uint64_t))) { BNXT_DRV_DBG(ERR, "comp field [%d] read oob %d\n", idx, bytelen); return -EINVAL; @@ -1114,21 +1115,21 @@ ulp_mapper_field_src_process(struct bnxt_ulp_mapper_parms *parms, /* The port id is present in the comp field list */ port_id = ULP_COMP_FLD_IDX_RD(parms, idx); /* get the port table enum */ - if (ulp_operand_read(field_opr + sizeof(uint16_t), - (uint8_t *)&idx, sizeof(uint16_t))) { + if (unlikely(ulp_operand_read(field_opr + sizeof(uint16_t), + (uint8_t *)&idx, sizeof(uint16_t)))) { BNXT_DRV_DBG(ERR, "Port table enum read failed\n"); return -EINVAL; } idx = tfp_be_to_cpu_16(idx); - if (ulp_mapper_field_port_db_process(parms, port_id, idx, - val)) { + if (unlikely(ulp_mapper_field_port_db_process(parms, port_id, idx, + val))) { BNXT_DRV_DBG(ERR, "field port table failed\n"); return -EINVAL; } break; case BNXT_ULP_FIELD_SRC_ENC_HDR_BIT: - if (ulp_operand_read(field_opr, - (uint8_t *)&lregval, sizeof(uint64_t))) { + if (unlikely(ulp_operand_read(field_opr, + (uint8_t *)&lregval, sizeof(uint64_t)))) { BNXT_DRV_DBG(ERR, "Header bit read failed\n"); return -EINVAL; } @@ -1141,21 +1142,21 @@ ulp_mapper_field_src_process(struct bnxt_ulp_mapper_parms *parms, } break; case BNXT_ULP_FIELD_SRC_ENC_FIELD: - if (ulp_operand_read(field_opr, - (uint8_t *)&idx, sizeof(uint16_t))) { + if (unlikely(ulp_operand_read(field_opr, + (uint8_t *)&idx, sizeof(uint16_t)))) { BNXT_DRV_DBG(ERR, "Header field read failed\n"); return -EINVAL; } idx = tfp_be_to_cpu_16(idx); /* get the index from the global field list */ - if (idx >= BNXT_ULP_ENC_FIELD_LAST) { + if (unlikely(idx >= BNXT_ULP_ENC_FIELD_LAST)) { BNXT_DRV_DBG(ERR, "invalid encap field tbl idx %d\n", idx); return -EINVAL; } buffer = parms->enc_field[idx].spec; field_size = parms->enc_field[idx].size; - if (bytelen > field_size) { + if (unlikely(bytelen > field_size)) { BNXT_DRV_DBG(ERR, "Encap field[%d] size small %u\n", idx, field_size); return -EINVAL; @@ -1172,15 +1173,15 @@ ulp_mapper_field_src_process(struct bnxt_ulp_mapper_parms *parms, case BNXT_ULP_FIELD_SRC_LIST_AND: case BNXT_ULP_FIELD_SRC_LIST_OR: /* read the cond table index and count */ - if (ulp_operand_read(field_opr, - (uint8_t *)&idx, sizeof(uint16_t))) { + if (unlikely(ulp_operand_read(field_opr, + (uint8_t *)&idx, sizeof(uint16_t)))) { BNXT_DRV_DBG(ERR, "Cond idx operand read failed\n"); return -EINVAL; } idx = tfp_be_to_cpu_16(idx); - if (ulp_operand_read(field_opr + sizeof(uint16_t), - (uint8_t *)&size_idx, sizeof(uint16_t))) { + if (unlikely(ulp_operand_read(field_opr + sizeof(uint16_t), + (uint8_t *)&size_idx, sizeof(uint16_t)))) { BNXT_DRV_DBG(ERR, "Cond count operand read failed\n"); return -EINVAL; } @@ -1193,7 +1194,7 @@ ulp_mapper_field_src_process(struct bnxt_ulp_mapper_parms *parms, info.cond_list_opcode = BNXT_ULP_COND_LIST_OPC_OR; info.cond_start_idx = idx; info.cond_nums = size_idx; - if (ulp_mapper_cond_opc_list_process(parms, &info, &cond_res)) { + if (unlikely(ulp_mapper_cond_opc_list_process(parms, &info, &cond_res))) { BNXT_DRV_DBG(ERR, "Cond evaluation failed\n"); return -EINVAL; } @@ -1206,13 +1207,13 @@ ulp_mapper_field_src_process(struct bnxt_ulp_mapper_parms *parms, } break; case BNXT_ULP_FIELD_SRC_CF_BIT: - if (ulp_operand_read(field_opr, - (uint8_t *)&lregval, sizeof(uint64_t))) { + if (unlikely(ulp_operand_read(field_opr, + (uint8_t *)&lregval, sizeof(uint64_t)))) { BNXT_DRV_DBG(ERR, "CF operand read failed\n"); return -EINVAL; } lregval = tfp_be_to_cpu_64(lregval); - if (ULP_BITMAP_ISSET(parms->cf_bitmap, lregval)) { + if (unlikely(ULP_BITMAP_ISSET(parms->cf_bitmap, lregval))) { *val = mapper_fld_one; *value = 1; } else { @@ -1260,19 +1261,19 @@ static int32_t ulp_mapper_field_blob_write(enum bnxt_ulp_field_src fld_src, uint8_t **out_val) { if (fld_src == BNXT_ULP_FIELD_SRC_ZERO) { - if (ulp_blob_pad_push(blob, val_len) < 0) { + if (unlikely(ulp_blob_pad_push(blob, val_len) < 0)) { BNXT_DRV_DBG(ERR, "too large for blob\n"); return -EINVAL; } } else if (fld_src == BNXT_ULP_FIELD_SRC_ACT_PROP_SZ) { - if (ulp_blob_push_encap(blob, val, val_len) < 0) { + if (unlikely(ulp_blob_push_encap(blob, val, val_len) < 0)) { BNXT_DRV_DBG(ERR, "encap blob push failed\n"); return -EINVAL; } } else if (fld_src == BNXT_ULP_FIELD_SRC_SKIP) { /* do nothing */ } else { - if (ulp_blob_push(blob, val, val_len)) { + if (unlikely(ulp_blob_push(blob, val, val_len))) { BNXT_DRV_DBG(ERR, "push of val1 failed\n"); return -EINVAL; } @@ -1293,15 +1294,15 @@ ulp_mapper_field_opc_next(struct bnxt_ulp_mapper_parms *parms, uint16_t idx; /* read the cond table index and count */ - if (ulp_operand_read(field_opr, - (uint8_t *)&idx, sizeof(uint16_t))) { + if (unlikely(ulp_operand_read(field_opr, + (uint8_t *)&idx, sizeof(uint16_t)))) { BNXT_DRV_DBG(ERR, "field idx operand read failed\n"); return -EINVAL; } idx = tfp_be_to_cpu_16(idx); field_info = ulp_mapper_tmpl_key_ext_list_get(parms, idx); - if (field_info == NULL) { + if (unlikely(field_info == NULL)) { BNXT_DRV_DBG(ERR, "Invalid field idx %d\n", idx); return -EINVAL; } @@ -1350,7 +1351,7 @@ ulp_mapper_key_recipe_tbl_init(struct bnxt_ulp_context *ulp_ctx, struct bitalloc *recipe_ba; rc = bnxt_ulp_cntxt_dev_id_get(ulp_ctx, &dev_id); - if (rc) { + if (unlikely(rc)) { BNXT_DRV_DBG(ERR, "Unable to get device id from ulp.\n"); return -EINVAL; } @@ -1374,21 +1375,21 @@ ulp_mapper_key_recipe_tbl_init(struct bnxt_ulp_context *ulp_ctx, for (ftype = 0; ftype < ULP_RECIPE_TYPE_MAX; ftype++) { recipes = rte_zmalloc("key_recipe_list", size_val * num_recipes, 0); - if (!recipes) { + if (unlikely(!recipes)) { BNXT_DRV_DBG(ERR, "Unable to alloc memory\n"); return -ENOMEM; } mdata->key_recipe_info.recipes[dir][ftype] = recipes; recipe_ba = rte_malloc("key_recipe_ba", pool_size, 0); - if (!recipe_ba) { + if (unlikely(!recipe_ba)) { BNXT_DRV_DBG(ERR, "Unable to alloc memory\n"); return -ENOMEM; } mdata->key_recipe_info.recipe_ba[dir][ftype] = recipe_ba; rc = ba_init(recipe_ba, num_recipes, true); - if (rc) { + if (unlikely(rc)) { BNXT_DRV_DBG(ERR, "Unable to alloc recipe ba\n"); return -ENOMEM; @@ -1408,25 +1409,25 @@ ulp_mapper_key_recipe_args_validate(struct bnxt_ulp_context *ulp_ctx, mdata = (struct bnxt_ulp_mapper_data *) bnxt_ulp_cntxt_ptr2_mapper_data_get(ulp_ctx); - if (!mdata) { + if (unlikely(!mdata)) { BNXT_DRV_DBG(ERR, "Unable to get mapper data.\n"); return NULL; } - if (dir >= BNXT_ULP_DIRECTION_LAST) { + if (unlikely(dir >= BNXT_ULP_DIRECTION_LAST)) { BNXT_DRV_DBG(ERR, "Invalid dir (%d) in key recipe\n", dir); return NULL; } - if (mdata->key_recipe_info.num_recipes == 0) { + if (unlikely(mdata->key_recipe_info.num_recipes == 0)) { BNXT_DRV_DBG(ERR, "Recipes are not supported\n"); return NULL; } - if (stype != BNXT_ULP_RESOURCE_SUB_TYPE_KEY_RECIPE_TABLE_WM && - stype != BNXT_ULP_RESOURCE_SUB_TYPE_KEY_RECIPE_TABLE_EM) { + if (unlikely(stype != BNXT_ULP_RESOURCE_SUB_TYPE_KEY_RECIPE_TABLE_WM && + stype != BNXT_ULP_RESOURCE_SUB_TYPE_KEY_RECIPE_TABLE_EM)) { BNXT_DRV_DBG(ERR, "Invalid type (%d) in key recipe\n", stype); return NULL; } - if (recipe_id >= mdata->key_recipe_info.num_recipes || - !mdata->key_recipe_info.num_recipes) { + if (unlikely(recipe_id >= mdata->key_recipe_info.num_recipes || + !mdata->key_recipe_info.num_recipes)) { BNXT_DRV_DBG(ERR, "Key recipe id out of range(%u >= %u)\n", recipe_id, mdata->key_recipe_info.num_recipes); return NULL; @@ -1447,7 +1448,7 @@ ulp_mapper_key_recipe_alloc(struct bnxt_ulp_context *ulp_ctx, mdata = ulp_mapper_key_recipe_args_validate(ulp_ctx, dir, stype, recipe_id); - if (mdata == NULL) + if (unlikely(mdata == NULL)) return NULL; recipes = mdata->key_recipe_info.recipes[dir][stype]; @@ -1488,18 +1489,18 @@ ulp_mapper_key_recipe_free(struct bnxt_ulp_context *ulp_ctx, mdata = ulp_mapper_key_recipe_args_validate(ulp_ctx, dir, stype, index); - if (mdata == NULL) + if (unlikely(mdata == NULL)) return -EINVAL; recipe_ba = mdata->key_recipe_info.recipe_ba[dir][stype]; rc = ba_free(recipe_ba, index); - if (rc < 0) + if (unlikely(rc < 0)) BNXT_DRV_DBG(DEBUG, "Unable to free recipe id[%s][%u] = (%d)\n", (dir == BNXT_ULP_DIRECTION_INGRESS) ? "rx" : "tx", stype, index); recipes = mdata->key_recipe_info.recipes[dir][stype]; - if (recipes[index] == NULL) { + if (unlikely(recipes[index] == NULL)) { BNXT_DRV_DBG(DEBUG, "recipe id[%s][%u] = (%d) already freed\n", (dir == BNXT_ULP_DIRECTION_INGRESS) ? "rx" : "tx", stype, index); @@ -1596,15 +1597,15 @@ ulp_mapper_key_recipe_field_opc_next(struct bnxt_ulp_mapper_parms *parms, uint16_t idx; /* read the cond table index and count */ - if (ulp_operand_read(field_opr, - (uint8_t *)&idx, sizeof(uint16_t))) { + if (unlikely(ulp_operand_read(field_opr, + (uint8_t *)&idx, sizeof(uint16_t)))) { BNXT_DRV_DBG(ERR, "field idx operand read failed\n"); return -EINVAL; } idx = tfp_be_to_cpu_16(idx); field_info = ulp_mapper_tmpl_key_ext_list_get(parms, idx); - if (field_info == NULL) { + if (unlikely(field_info == NULL)) { BNXT_DRV_DBG(ERR, "Invalid field idx %d\n", idx); return -EINVAL; } @@ -1652,10 +1653,10 @@ ulp_mapper_key_recipe_field_opc_process(struct bnxt_ulp_mapper_parms *parms, /* process the src1 opcode */ if (process_src1) { - if (ulp_mapper_field_src_process(parms, fld->field_src1, + if (unlikely(ulp_mapper_field_src_process(parms, fld->field_src1, fld->field_opr1, dir, is_key, fld->field_bit_size, &val1, - &val1_len, &value1)) { + &val1_len, &value1))) { BNXT_DRV_DBG(ERR, "fld src1 process failed\n"); return -EINVAL; } @@ -1677,13 +1678,13 @@ ulp_mapper_key_recipe_field_opc_process(struct bnxt_ulp_mapper_parms *parms, /* check if src2 is next */ if (fld->field_src2 == BNXT_ULP_FIELD_SRC_NEXT) { /* get the next field info */ - if (ulp_mapper_key_recipe_field_opc_next(parms, + if (unlikely(ulp_mapper_key_recipe_field_opc_next(parms, dir, fld->field_opr2, is_key, name, written, - ofld)) { + ofld))) { BNXT_DRV_DBG(ERR, "recipe fld next process fail\n"); return -EINVAL; @@ -1700,13 +1701,13 @@ ulp_mapper_key_recipe_field_opc_process(struct bnxt_ulp_mapper_parms *parms, /* check if src3 is next */ if (fld->field_src3 == BNXT_ULP_FIELD_SRC_NEXT) { /* get the next field info */ - if (ulp_mapper_key_recipe_field_opc_next(parms, + if (unlikely(ulp_mapper_key_recipe_field_opc_next(parms, dir, fld->field_opr3, is_key, name, written, - ofld)) { + ofld))) { BNXT_DRV_DBG(ERR, "recipt fld next process fail\n"); return -EINVAL; @@ -1777,9 +1778,9 @@ ulp_mapper_key_recipe_tbl_process(struct bnxt_ulp_mapper_parms *parms, /* Get the recipe_id from the regfile */ if (!alloc && regfile) { - if (ulp_regfile_read(parms->regfile, - tbl->tbl_operand, - ®val)) { + if (unlikely(ulp_regfile_read(parms->regfile, + tbl->tbl_operand, + ®val))) { BNXT_DRV_DBG(ERR, "Fail to get tbl idx from regfile[%d].\n", tbl->tbl_operand); @@ -1794,16 +1795,16 @@ ulp_mapper_key_recipe_tbl_process(struct bnxt_ulp_mapper_parms *parms, */ mdata = ulp_mapper_key_recipe_args_validate(parms->ulp_ctx, dir, stype, 0); - if (mdata == NULL) + if (unlikely(mdata == NULL)) return -EINVAL; recipe_ba = mdata->key_recipe_info.recipe_ba[dir][stype]; tmp_recipe_id = ba_alloc(recipe_ba); - if (tmp_recipe_id < 0) { + if (unlikely(tmp_recipe_id < 0)) { BNXT_DRV_DBG(ERR, "Failed to allocate a recipe id\n"); return -EINVAL; - } else if ((uint32_t)tmp_recipe_id >= - mdata->key_recipe_info.num_recipes) { + } else if (unlikely((uint32_t)tmp_recipe_id >= + mdata->key_recipe_info.num_recipes)) { /* Shouldn't get here, but could be an issue with the * allocator, so free the recipe_id */ @@ -1824,7 +1825,7 @@ ulp_mapper_key_recipe_tbl_process(struct bnxt_ulp_mapper_parms *parms, regval = rte_cpu_to_be_64(recipe_id); rc = ulp_regfile_write(parms->regfile, tbl->tbl_operand, regval); - if (rc) { + if (unlikely(rc)) { BNXT_DRV_DBG(ERR, "Failed to write regfile[%d] rc=%d\n", tbl->tbl_operand, rc); if (recipe_ba) @@ -1836,7 +1837,7 @@ ulp_mapper_key_recipe_tbl_process(struct bnxt_ulp_mapper_parms *parms, /* allocate or Get the recipe entry based on alloc */ recipe = ulp_mapper_key_recipe_alloc(parms->ulp_ctx, dir, stype, recipe_id, alloc, &max_rflds); - if (!recipe || !max_rflds) { + if (unlikely(!recipe || !max_rflds)) { BNXT_DRV_DBG(ERR, "Failed to get the recipe slot\n"); if (recipe_ba) (void)ba_free(recipe_ba, recipe_id); @@ -1847,7 +1848,7 @@ ulp_mapper_key_recipe_tbl_process(struct bnxt_ulp_mapper_parms *parms, if (write) { /* Get the key fields to process */ kflds = ulp_mapper_key_fields_get(parms, tbl, &num_kflds); - if (!kflds || !num_kflds) { + if (unlikely(!kflds || !num_kflds)) { BNXT_DRV_DBG(ERR, "Failed to get the key fields\n"); rc = -EINVAL; goto error; @@ -1856,7 +1857,7 @@ ulp_mapper_key_recipe_tbl_process(struct bnxt_ulp_mapper_parms *parms, rflds = &recipe->flds[0]; /* iterate over the key fields and write the recipe */ for (i = 0; i < num_kflds; i++) { - if (rnum_flds >= max_rflds) { + if (unlikely(rnum_flds >= max_rflds)) { BNXT_DRV_DBG(ERR, "Max recipe fields exceeded (%d)\n", rnum_flds); @@ -1872,7 +1873,7 @@ ulp_mapper_key_recipe_tbl_process(struct bnxt_ulp_mapper_parms *parms, "KEY", &written, rfld); - if (rc) + if (unlikely(rc)) goto error; if (stype == @@ -1886,7 +1887,7 @@ ulp_mapper_key_recipe_tbl_process(struct bnxt_ulp_mapper_parms *parms, "MASK", &written, rfld); - if (rc) + if (unlikely(rc)) goto error; } if (written) @@ -1904,7 +1905,7 @@ ulp_mapper_key_recipe_tbl_process(struct bnxt_ulp_mapper_parms *parms, fid_parms.critical_resource = tbl->critical_resource; rc = ulp_mapper_fdb_opc_process(parms, tbl, &fid_parms); - if (rc) { + if (unlikely(rc)) { BNXT_DRV_DBG(ERR, "Failed to link resource to flow rc = %d\n", rc); goto error; @@ -1973,16 +1974,16 @@ ulp_mapper_field_opc_process(struct bnxt_ulp_mapper_parms *parms, /* process the src1 opcode */ if (process_src1) { - if (ulp_mapper_field_src_process(parms, fld->field_src1, - fld->field_opr1, dir, is_key, - fld->field_bit_size, &val1, - &val1_len, &value1)) { + if (unlikely(ulp_mapper_field_src_process(parms, fld->field_src1, + fld->field_opr1, dir, is_key, + fld->field_bit_size, &val1, + &val1_len, &value1))) { BNXT_DRV_DBG(ERR, "fld src1 process failed\n"); goto error; } if (eval_src1) { - if (ulp_mapper_field_buffer_eval(val1, val1_len, - &val1_int)) { + if (unlikely(ulp_mapper_field_buffer_eval(val1, val1_len, + &val1_int))) { BNXT_DRV_DBG(ERR, "fld src1 eval failed\n"); goto error; } @@ -2000,10 +2001,10 @@ ulp_mapper_field_opc_process(struct bnxt_ulp_mapper_parms *parms, /* check if src2 is next */ if (fld->field_src2 == BNXT_ULP_FIELD_SRC_NEXT) { /* get the next field info */ - if (ulp_mapper_field_opc_next(parms, dir, - fld->field_opr2, - blob, is_key, - name)) { + if (unlikely(ulp_mapper_field_opc_next(parms, dir, + fld->field_opr2, + blob, is_key, + name))) { BNXT_DRV_DBG(ERR, "fld next process fail\n"); goto error; @@ -2017,10 +2018,10 @@ ulp_mapper_field_opc_process(struct bnxt_ulp_mapper_parms *parms, /* check if src2 is next */ if (fld->field_src3 == BNXT_ULP_FIELD_SRC_NEXT) { /* get the next field info */ - if (ulp_mapper_field_opc_next(parms, dir, - fld->field_opr3, - blob, is_key, - name)) { + if (unlikely(ulp_mapper_field_opc_next(parms, dir, + fld->field_opr3, + blob, is_key, + name))) { BNXT_DRV_DBG(ERR, "fld next process fail\n"); goto error; @@ -2035,16 +2036,16 @@ ulp_mapper_field_opc_process(struct bnxt_ulp_mapper_parms *parms, /* process src2 opcode */ if (process_src2) { - if (ulp_mapper_field_src_process(parms, fld->field_src2, - fld->field_opr2, dir, is_key, - fld->field_bit_size, &val2, - &val2_len, &value2)) { + if (unlikely(ulp_mapper_field_src_process(parms, fld->field_src2, + fld->field_opr2, dir, is_key, + fld->field_bit_size, &val2, + &val2_len, &value2))) { BNXT_DRV_DBG(ERR, "fld src2 process failed\n"); goto error; } if (eval_src2) { - if (ulp_mapper_field_buffer_eval(val2, val2_len, - &val2_int)) { + if (unlikely(ulp_mapper_field_buffer_eval(val2, val2_len, + &val2_int))) { BNXT_DRV_DBG(ERR, "fld src2 eval failed\n"); goto error; } @@ -2053,16 +2054,16 @@ ulp_mapper_field_opc_process(struct bnxt_ulp_mapper_parms *parms, /* process src3 opcode */ if (process_src3) { - if (ulp_mapper_field_src_process(parms, fld->field_src3, - fld->field_opr3, dir, is_key, - fld->field_bit_size, &val3, - &val3_len, &value3)) { + if (unlikely(ulp_mapper_field_src_process(parms, fld->field_src3, + fld->field_opr3, dir, is_key, + fld->field_bit_size, &val3, + &val3_len, &value3))) { BNXT_DRV_DBG(ERR, "fld src3 process failed\n"); goto error; } if (eval_src3) { - if (ulp_mapper_field_buffer_eval(val3, val3_len, - &val3_int)) { + if (unlikely(ulp_mapper_field_buffer_eval(val3, val3_len, + &val3_int))) { BNXT_DRV_DBG(ERR, "fld src3 eval failed\n"); goto error; } @@ -2094,7 +2095,7 @@ ulp_mapper_field_opc_process(struct bnxt_ulp_mapper_parms *parms, val_int = val1_int + val2_int; val_int = tfp_cpu_to_be_64(val_int); val = ulp_blob_push_64(blob, &val_int, fld->field_bit_size); - if (!val) + if (unlikely(!val)) rc = -EINVAL; break; case BNXT_ULP_FIELD_OPC_SRC1_MINUS_SRC2: @@ -2102,35 +2103,35 @@ ulp_mapper_field_opc_process(struct bnxt_ulp_mapper_parms *parms, val_int = val1_int - val2_int; val_int = tfp_cpu_to_be_64(val_int); val = ulp_blob_push_64(blob, &val_int, fld->field_bit_size); - if (!val) + if (unlikely(!val)) rc = -EINVAL; break; case BNXT_ULP_FIELD_OPC_SRC1_OR_SRC2: val_int = val1_int | val2_int; val_int = tfp_cpu_to_be_64(val_int); val = ulp_blob_push_64(blob, &val_int, fld->field_bit_size); - if (!val) + if (unlikely(!val)) rc = -EINVAL; break; case BNXT_ULP_FIELD_OPC_SRC1_OR_SRC2_OR_SRC3: val_int = val1_int | val2_int | val3_int; val_int = tfp_cpu_to_be_64(val_int); val = ulp_blob_push_64(blob, &val_int, fld->field_bit_size); - if (!val) + if (unlikely(!val)) rc = -EINVAL; break; case BNXT_ULP_FIELD_OPC_SRC1_AND_SRC2: val_int = val1_int & val2_int; val_int = tfp_cpu_to_be_64(val_int); val = ulp_blob_push_64(blob, &val_int, fld->field_bit_size); - if (!val) + if (unlikely(!val)) rc = -EINVAL; break; case BNXT_ULP_FIELD_OPC_SRC1_AND_SRC2_OR_SRC3: val_int = val1_int & (val2_int | val3_int); val_int = tfp_cpu_to_be_64(val_int); val = ulp_blob_push_64(blob, &val_int, fld->field_bit_size); - if (!val) + if (unlikely(!val)) rc = -EINVAL; break; case BNXT_ULP_FIELD_OPC_SKIP: @@ -2170,7 +2171,7 @@ ulp_mapper_tbl_result_build(struct bnxt_ulp_mapper_parms *parms, &encap_flds); /* validate the result field list counts */ - if (!dflds || (!num_flds && !encap_flds)) { + if (unlikely(!dflds || (!num_flds && !encap_flds))) { BNXT_DRV_DBG(ERR, "Failed to get data fields %x:%x\n", num_flds, encap_flds); return -EINVAL; @@ -2180,7 +2181,7 @@ ulp_mapper_tbl_result_build(struct bnxt_ulp_mapper_parms *parms, for (i = 0; i < num_flds; i++) { rc = ulp_mapper_field_opc_process(parms, tbl->direction, &dflds[i], data, 0, name); - if (rc) { + if (unlikely(rc)) { BNXT_DRV_DBG(ERR, "result field processing failed\n"); return rc; } @@ -2190,9 +2191,9 @@ ulp_mapper_tbl_result_build(struct bnxt_ulp_mapper_parms *parms, if (encap_flds) { uint32_t pad = 0; /* Initialize the encap blob */ - if (ulp_blob_init(&encap_blob, + if (unlikely(ulp_blob_init(&encap_blob, ULP_BYTE_2_BITS(tbl->record_size), - parms->device_params->encap_byte_order)) { + parms->device_params->encap_byte_order))) { BNXT_DRV_DBG(ERR, "blob inits failed.\n"); return -EINVAL; } @@ -2200,7 +2201,7 @@ ulp_mapper_tbl_result_build(struct bnxt_ulp_mapper_parms *parms, rc = ulp_mapper_field_opc_process(parms, tbl->direction, &dflds[i], &encap_blob, 0, name); - if (rc) { + if (unlikely(rc)) { BNXT_DRV_DBG(ERR, "encap field processing failed\n"); return rc; @@ -2223,7 +2224,7 @@ ulp_mapper_tbl_result_build(struct bnxt_ulp_mapper_parms *parms, pad = ULP_BYTE_2_BITS(tbl->record_size) - ulp_blob_data_len_get(&encap_blob); } - if (ulp_blob_pad_push(&encap_blob, pad) < 0) { + if (unlikely(ulp_blob_pad_push(&encap_blob, pad) < 0)) { BNXT_DRV_DBG(ERR, "encap buffer padding failed\n"); return -EINVAL; } @@ -2232,7 +2233,7 @@ ulp_mapper_tbl_result_build(struct bnxt_ulp_mapper_parms *parms, ulp_blob_perform_64B_byte_swap(&encap_blob); /* Append encap blob to the result blob */ rc = ulp_blob_buffer_copy(data, &encap_blob); - if (rc) { + if (unlikely(rc)) { BNXT_DRV_DBG(ERR, "encap buffer copy failed\n"); return rc; } @@ -2266,7 +2267,7 @@ ulp_mapper_mark_gfid_process(struct bnxt_ulp_mapper_parms *parms, rc = ulp_mark_db_mark_add(parms->ulp_ctx, mark_flag, gfid, mark); - if (rc) { + if (unlikely(rc)) { BNXT_DRV_DBG(ERR, "Failed to add mark to flow\n"); return rc; } @@ -2278,7 +2279,7 @@ ulp_mapper_mark_gfid_process(struct bnxt_ulp_mapper_parms *parms, ulp_flow_db_shared_session_set(&fid_parms, tbl->session_type); rc = ulp_mapper_fdb_opc_process(parms, tbl, &fid_parms); - if (rc) + if (unlikely(rc)) BNXT_DRV_DBG(ERR, "Fail to link res to flow rc = %d\n", rc); return rc; } @@ -2289,7 +2290,7 @@ ulp_mapper_mark_act_ptr_process(struct bnxt_ulp_mapper_parms *parms, { struct ulp_flow_db_res_params fid_parms; uint32_t act_idx, mark, mark_flag; - uint64_t val64; + uint64_t val64 = 0; enum bnxt_ulp_mark_db_opc mark_op = tbl->mark_db_opcode; int32_t rc = 0; @@ -2304,9 +2305,9 @@ ulp_mapper_mark_act_ptr_process(struct bnxt_ulp_mapper_parms *parms, sizeof(mark)); mark = tfp_be_to_cpu_32(mark); - if (ulp_regfile_read(parms->regfile, - BNXT_ULP_RF_IDX_MAIN_ACTION_PTR, - &val64)) { + if (unlikely(ulp_regfile_read(parms->regfile, + BNXT_ULP_RF_IDX_MAIN_ACTION_PTR, + &val64))) { BNXT_DRV_DBG(ERR, "read action ptr main failed\n"); return -EINVAL; } @@ -2314,7 +2315,7 @@ ulp_mapper_mark_act_ptr_process(struct bnxt_ulp_mapper_parms *parms, mark_flag = BNXT_ULP_MARK_LOCAL_HW_FID; rc = ulp_mark_db_mark_add(parms->ulp_ctx, mark_flag, act_idx, mark); - if (rc) { + if (unlikely(rc)) { BNXT_DRV_DBG(ERR, "Failed to add mark to flow\n"); return rc; } @@ -2326,7 +2327,7 @@ ulp_mapper_mark_act_ptr_process(struct bnxt_ulp_mapper_parms *parms, ulp_flow_db_shared_session_set(&fid_parms, tbl->session_type); rc = ulp_mapper_fdb_opc_process(parms, tbl, &fid_parms); - if (rc) + if (unlikely(rc)) BNXT_DRV_DBG(ERR, "Fail to link res to flow rc = %d\n", rc); return rc; } @@ -2337,7 +2338,7 @@ ulp_mapper_mark_vfr_idx_process(struct bnxt_ulp_mapper_parms *parms, { struct ulp_flow_db_res_params fid_parms; uint32_t act_idx, mark, mark_flag; - uint64_t val64; + uint64_t val64 = 0; enum bnxt_ulp_mark_db_opc mark_op = tbl->mark_db_opcode; int32_t rc = 0; @@ -2349,9 +2350,9 @@ ulp_mapper_mark_vfr_idx_process(struct bnxt_ulp_mapper_parms *parms, mark = ULP_COMP_FLD_IDX_RD(parms, BNXT_ULP_CF_IDX_DEV_PORT_ID); /* Get the main action pointer */ - if (ulp_regfile_read(parms->regfile, - BNXT_ULP_RF_IDX_MAIN_ACTION_PTR, - &val64)) { + if (unlikely(ulp_regfile_read(parms->regfile, + BNXT_ULP_RF_IDX_MAIN_ACTION_PTR, + &val64))) { BNXT_DRV_DBG(ERR, "read action ptr main failed\n"); return -EINVAL; } @@ -2362,7 +2363,7 @@ ulp_mapper_mark_vfr_idx_process(struct bnxt_ulp_mapper_parms *parms, rc = ulp_mark_db_mark_add(parms->ulp_ctx, mark_flag, act_idx, mark); - if (rc) { + if (unlikely(rc)) { BNXT_DRV_DBG(ERR, "Failed to add mark to flow\n"); return rc; } @@ -2374,7 +2375,7 @@ ulp_mapper_mark_vfr_idx_process(struct bnxt_ulp_mapper_parms *parms, ulp_flow_db_shared_session_set(&fid_parms, tbl->session_type); rc = ulp_mapper_fdb_opc_process(parms, tbl, &fid_parms); - if (rc) + if (unlikely(rc)) BNXT_DRV_DBG(ERR, "Fail to link res to flow rc = %d\n", rc); return rc; } @@ -2390,8 +2391,8 @@ ulp_mapper_tcam_tbl_ident_alloc(struct bnxt_ulp_mapper_parms *parms, idents = ulp_mapper_ident_fields_get(parms, tbl, &num_idents); for (i = 0; i < num_idents; i++) { - if (ulp_mapper_ident_process(parms, tbl, - &idents[i], NULL)) + if (unlikely(ulp_mapper_ident_process(parms, tbl, + &idents[i], NULL))) return -EINVAL; } return 0; @@ -2438,23 +2439,23 @@ ulp_mapper_wc_tcam_tbl_dyn_post_process(struct bnxt_ulp_device_params *dparms, tlen = tlen << 1; } - if (num_slices > max_slices) { + if (unlikely(num_slices > max_slices)) { BNXT_DRV_DBG(ERR, "Key size (%d) too large for WC\n", blen); return -EINVAL; } /* The key/mask may not be on a natural slice boundary, pad it */ pad = tlen - blen; - if (ulp_blob_pad_push(key, pad) < 0 || - ulp_blob_pad_push(mask, pad) < 0) { + if (unlikely(ulp_blob_pad_push(key, pad) < 0 || + ulp_blob_pad_push(mask, pad) < 0)) { BNXT_DRV_DBG(ERR, "Unable to pad key/mask\n"); return -EINVAL; } /* The new length accounts for the ctrl word length and num slices */ tlen = tlen + clen * num_slices; - if (ulp_blob_init(tkey, tlen, key->byte_order) || - ulp_blob_init(tmask, tlen, mask->byte_order)) { + if (unlikely(ulp_blob_init(tkey, tlen, key->byte_order) || + ulp_blob_init(tmask, tlen, mask->byte_order))) { BNXT_DRV_DBG(ERR, "Unable to post process wc tcam entry\n"); return -EINVAL; } @@ -2465,22 +2466,22 @@ ulp_mapper_wc_tcam_tbl_dyn_post_process(struct bnxt_ulp_device_params *dparms, offset = 0; for (i = 0; i < num_slices; i++) { val = ulp_blob_push_32(tkey, &cword, clen); - if (!val) { + if (unlikely(!val)) { BNXT_DRV_DBG(ERR, "Key ctrl word push failed\n"); return -EINVAL; } val = ulp_blob_push_32(tmask, &cword, clen); - if (!val) { + if (unlikely(!val)) { BNXT_DRV_DBG(ERR, "Mask ctrl word push failed\n"); return -EINVAL; } rc = ulp_blob_append(tkey, key, offset, slice_width); - if (rc) { + if (unlikely(rc)) { BNXT_DRV_DBG(ERR, "Key blob append failed\n"); return rc; } rc = ulp_blob_append(tmask, mask, offset, slice_width); - if (rc) { + if (unlikely(rc)) { BNXT_DRV_DBG(ERR, "Mask blob append failed\n"); return rc; } @@ -2546,7 +2547,7 @@ ulp_mapper_gen_tbl_ref_cnt_process(struct bnxt_ulp_mapper_parms *parms, rc = ulp_regfile_write(parms->regfile, BNXT_ULP_RF_IDX_REF_CNT, val64); - if (rc) { + if (unlikely(rc)) { BNXT_DRV_DBG(ERR, "Failed to write regfile[ref_cnt]\n"); return rc; } @@ -2576,7 +2577,7 @@ ulp_mapper_gen_tbl_process(struct bnxt_ulp_mapper_parms *parms, /* Get the key fields list and build the key. */ kflds = ulp_mapper_key_fields_get(parms, tbl, &num_kflds); - if (!kflds || !num_kflds) { + if (unlikely(!kflds || !num_kflds)) { BNXT_DRV_DBG(ERR, "Failed to get key fields\n"); return -EINVAL; } @@ -2588,9 +2589,9 @@ ulp_mapper_gen_tbl_process(struct bnxt_ulp_mapper_parms *parms, pad = ULP_BYTE_2_BITS(sizeof(uint8_t)) - ULP_BITS_IS_BYTE_NOT_ALIGNED(tbl->key_bit_size); - if (ulp_blob_init(&key, tbl->key_bit_size + pad + + if (unlikely(ulp_blob_init(&key, tbl->key_bit_size + pad + tbl->partial_key_bit_size, - parms->device_params->key_byte_order)) { + parms->device_params->key_byte_order))) { BNXT_DRV_DBG(ERR, "Failed to alloc blob\n"); return -EINVAL; } @@ -2599,7 +2600,7 @@ ulp_mapper_gen_tbl_process(struct bnxt_ulp_mapper_parms *parms, rc = ulp_mapper_field_opc_process(parms, tbl->direction, &kflds[i].field_info_spec, &key, 1, "Gen Tbl Key"); - if (rc) { + if (unlikely(rc)) { BNXT_DRV_DBG(ERR, "Failed to create key for Gen tbl rc=%d\n", rc); @@ -2607,7 +2608,7 @@ ulp_mapper_gen_tbl_process(struct bnxt_ulp_mapper_parms *parms, } /* pad for the alignment between exact key and partial key */ if (num_par_kflds && i == num_kflds - 1) { - if (ulp_blob_pad_push(&key, pad) < 0) { + if (unlikely(ulp_blob_pad_push(&key, pad) < 0)) { BNXT_DRV_DBG(ERR, "key padding failed\n"); return -EINVAL; } @@ -2617,7 +2618,7 @@ ulp_mapper_gen_tbl_process(struct bnxt_ulp_mapper_parms *parms, /* Calculate the table index for the generic table*/ tbl_idx = ulp_mapper_gen_tbl_idx_calculate(tbl->resource_sub_type, tbl->direction); - if (tbl_idx < 0) { + if (unlikely(tbl_idx < 0)) { BNXT_DRV_DBG(ERR, "Invalid table index %x:%x\n", tbl->resource_sub_type, tbl->direction); return -EINVAL; @@ -2630,9 +2631,9 @@ ulp_mapper_gen_tbl_process(struct bnxt_ulp_mapper_parms *parms, gen_tbl_list = &parms->mapper_data->gen_tbl_list[tbl_idx]; /* perform basic validation of generic table */ - if ((gen_tbl_list->tbl_type == BNXT_ULP_GEN_TBL_TYPE_HASH_LIST && - gen_tbl_list->hash_tbl == NULL) || - gen_tbl_list->mem_data == NULL) { + if (unlikely((gen_tbl_list->tbl_type == BNXT_ULP_GEN_TBL_TYPE_HASH_LIST && + gen_tbl_list->hash_tbl == NULL) || + gen_tbl_list->mem_data == NULL)) { BNXT_DRV_DBG(ERR, "Uninitialized gen table index %x:%x\n", tbl->resource_sub_type, tbl->direction); return -EINVAL; @@ -2640,8 +2641,8 @@ ulp_mapper_gen_tbl_process(struct bnxt_ulp_mapper_parms *parms, /* Check if generic hash table */ if (gen_tbl_list->tbl_type == BNXT_ULP_GEN_TBL_TYPE_HASH_LIST) { - if (tbl->gen_tbl_lkup_type != - BNXT_ULP_GENERIC_TBL_LKUP_TYPE_HASH) { + if (unlikely(tbl->gen_tbl_lkup_type != + BNXT_ULP_GENERIC_TBL_LKUP_TYPE_HASH)) { BNXT_DRV_DBG(ERR, "%s: Invalid template lkup type\n", gen_tbl_list->gen_tbl_name); return -EINVAL; @@ -2650,7 +2651,7 @@ ulp_mapper_gen_tbl_process(struct bnxt_ulp_mapper_parms *parms, hash_entry.key_length = ULP_BITS_2_BYTE(keylen); rc = ulp_gen_hash_tbl_list_key_search(gen_tbl_list->hash_tbl, &hash_entry); - if (rc) { + if (unlikely(rc)) { BNXT_DRV_DBG(ERR, "%s: hash tbl search failed\n", gen_tbl_list->gen_tbl_name); return rc; @@ -2658,24 +2659,24 @@ ulp_mapper_gen_tbl_process(struct bnxt_ulp_mapper_parms *parms, if (hash_entry.search_flag == ULP_GEN_HASH_SEARCH_FOUND) { key_index = hash_entry.key_idx; /* Get the generic table entry */ - if (ulp_mapper_gen_tbl_entry_get(gen_tbl_list, - key_index, - &gen_tbl_ent)) + if (unlikely(ulp_mapper_gen_tbl_entry_get(gen_tbl_list, + key_index, + &gen_tbl_ent))) return -EINVAL; /* store the hash index in the fdb */ key_index = hash_entry.hash_index; } } else if (gen_tbl_list->tbl_type == BNXT_ULP_GEN_TBL_TYPE_KEY_LIST) { /* convert key to index directly */ - if (ULP_BITS_2_BYTE(keylen) > (int32_t)sizeof(key_index)) { + if (unlikely(ULP_BITS_2_BYTE(keylen) > (int32_t)sizeof(key_index))) { BNXT_DRV_DBG(ERR, "%s: keysize is bigger then 4 bytes\n", gen_tbl_list->gen_tbl_name); return -EINVAL; } memcpy(&key_index, cache_key, ULP_BITS_2_BYTE(keylen)); /* Get the generic table entry */ - if (ulp_mapper_gen_tbl_entry_get(gen_tbl_list, key_index, - &gen_tbl_ent)) + if (unlikely(ulp_mapper_gen_tbl_entry_get(gen_tbl_list, key_index, + &gen_tbl_ent))) return -EINVAL; } else if (gen_tbl_list->tbl_type == BNXT_ULP_GEN_TBL_TYPE_SIMPLE_LIST) { @@ -2683,9 +2684,9 @@ ulp_mapper_gen_tbl_process(struct bnxt_ulp_mapper_parms *parms, cache_key, &key_index); /* Get the generic table entry */ - if (ulp_mapper_gen_tbl_entry_get(gen_tbl_list, - key_index, - &gen_tbl_ent)) + if (unlikely(ulp_mapper_gen_tbl_entry_get(gen_tbl_list, + key_index, + &gen_tbl_ent))) return -EINVAL; } @@ -2710,7 +2711,7 @@ ulp_mapper_gen_tbl_process(struct bnxt_ulp_mapper_parms *parms, g->byte_data, g->byte_data_size, g->byte_order); - if (rc) { + if (unlikely(rc)) { BNXT_DRV_DBG(ERR, "Failed to scan ident list\n"); return -EINVAL; @@ -2727,13 +2728,13 @@ ulp_mapper_gen_tbl_process(struct bnxt_ulp_mapper_parms *parms, rc = ulp_mapper_gen_tbl_hash_entry_add(gen_tbl_list, &hash_entry, &gen_tbl_ent); - if (rc) + if (unlikely(rc)) return rc; /* store the hash index in the fdb */ key_index = hash_entry.hash_index; } else if (gen_tbl_list->tbl_type == BNXT_ULP_GEN_TBL_TYPE_SIMPLE_LIST) { - if (list_srch == ULP_GEN_LIST_SEARCH_FULL) { + if (unlikely(list_srch == ULP_GEN_LIST_SEARCH_FULL)) { BNXT_DRV_DBG(ERR, "failed to add gen entry\n"); return -ENOMEM; } @@ -2742,16 +2743,16 @@ ulp_mapper_gen_tbl_process(struct bnxt_ulp_mapper_parms *parms, /* check the reference count and ignore ref_cnt if NOP. * NOP allows a write as an update. */ - if (tbl->ref_cnt_opcode != BNXT_ULP_REF_CNT_OPC_NOP && - ULP_GEN_TBL_REF_CNT(&gen_tbl_ent)) { + if (unlikely(tbl->ref_cnt_opcode != BNXT_ULP_REF_CNT_OPC_NOP && + ULP_GEN_TBL_REF_CNT(&gen_tbl_ent))) { /* a hit then error */ BNXT_DRV_DBG(ERR, "generic entry already present\n"); return -EINVAL; /* success */ } /* Initialize the blob data */ - if (ulp_blob_init(&data, tbl->result_bit_size, - gen_tbl_ent.byte_order)) { + if (unlikely(ulp_blob_init(&data, tbl->result_bit_size, + gen_tbl_ent.byte_order))) { BNXT_DRV_DBG(ERR, "Failed initial index table blob\n"); return -EINVAL; } @@ -2759,7 +2760,7 @@ ulp_mapper_gen_tbl_process(struct bnxt_ulp_mapper_parms *parms, /* Get the result fields list */ rc = ulp_mapper_tbl_result_build(parms, tbl, &data, "Gen tbl Result"); - if (rc) { + if (unlikely(rc)) { BNXT_DRV_DBG(ERR, "Failed to build the result blob\n"); return rc; } @@ -2771,7 +2772,7 @@ ulp_mapper_gen_tbl_process(struct bnxt_ulp_mapper_parms *parms, byte_data, ULP_BITS_2_BYTE(datalen) ); - if (rc) { + if (unlikely(rc)) { BNXT_DRV_DBG(ERR, "Failed to write generic table\n"); return -EINVAL; } @@ -2788,7 +2789,7 @@ ulp_mapper_gen_tbl_process(struct bnxt_ulp_mapper_parms *parms, rc = ulp_regfile_write(parms->regfile, BNXT_ULP_RF_IDX_GENERIC_TBL_MISS, tfp_cpu_to_be_64(gen_tbl_miss)); - if (rc) { + if (unlikely(rc)) { BNXT_DRV_DBG(ERR, "Write regfile[%d] failed\n", BNXT_ULP_RF_IDX_GENERIC_TBL_MISS); return -EIO; @@ -2805,7 +2806,7 @@ ulp_mapper_gen_tbl_process(struct bnxt_ulp_mapper_parms *parms, ulp_flow_db_shared_session_set(&fid_parms, tbl->session_type); rc = ulp_mapper_fdb_opc_process(parms, tbl, &fid_parms); - if (rc) { + if (unlikely(rc)) { BNXT_DRV_DBG(ERR, "Fail to add gen ent flowdb %d\n", rc); return rc; @@ -2838,13 +2839,13 @@ ulp_mapper_ctrl_tbl_process(struct bnxt_ulp_mapper_parms *parms, /* process the fdb opcode for alloc push */ if (tbl->fdb_opcode == BNXT_ULP_FDB_OPC_ALLOC_RID_REGFILE) { rc = ulp_mapper_fdb_opc_alloc_rid(parms, tbl); - if (rc) { + if (unlikely(rc)) { BNXT_DRV_DBG(ERR, "Failed to do fdb alloc\n"); return rc; } } else if (tbl->fdb_opcode == BNXT_ULP_FDB_OPC_DELETE_RID_REGFILE) { rc = ulp_regfile_read(parms->regfile, tbl->fdb_operand, &val64); - if (rc) { + if (unlikely(rc)) { BNXT_DRV_DBG(ERR, "Failed to get RID from regfile\n"); return rc; } @@ -2868,23 +2869,23 @@ ulp_mapper_vnic_tbl_process(struct bnxt_ulp_mapper_parms *parms, switch (tbl->resource_sub_type) { case BNXT_ULP_RESOURCE_SUB_TYPE_VNIC_TABLE_RSS: - if (tbl->tbl_opcode != BNXT_ULP_VNIC_TBL_OPC_ALLOC_WR_REGFILE) { + if (unlikely(tbl->tbl_opcode != BNXT_ULP_VNIC_TBL_OPC_ALLOC_WR_REGFILE)) { BNXT_DRV_DBG(ERR, "Invalid vnic table opcode\n"); return -EINVAL; } rc = bnxt_pmd_rss_action_create(parms, &vnic_idx, &vnic_id); - if (rc) { + if (unlikely(rc)) { BNXT_DRV_DBG(ERR, "Failed create rss action\n"); return rc; } break; case BNXT_ULP_RESOURCE_SUB_TYPE_VNIC_TABLE_QUEUE: - if (tbl->tbl_opcode != BNXT_ULP_VNIC_TBL_OPC_ALLOC_WR_REGFILE) { + if (unlikely(tbl->tbl_opcode != BNXT_ULP_VNIC_TBL_OPC_ALLOC_WR_REGFILE)) { BNXT_DRV_DBG(ERR, "Invalid vnic table opcode\n"); return -EINVAL; } rc = bnxt_pmd_queue_action_create(parms, &vnic_idx, &vnic_id); - if (rc) { + if (unlikely(rc)) { BNXT_DRV_DBG(ERR, "Failed create queue action\n"); return rc; } @@ -2903,14 +2904,14 @@ ulp_mapper_vnic_tbl_process(struct bnxt_ulp_mapper_parms *parms, fid_parms.resource_hndl = vnic_idx; fid_parms.critical_resource = tbl->critical_resource; rc = ulp_mapper_fdb_opc_process(parms, tbl, &fid_parms); - if (rc) { + if (unlikely(rc)) { BNXT_DRV_DBG(ERR, "Failed to link resource to flow rc = %d\n", rc); return rc; } rc = ulp_regfile_write(parms->regfile, tbl->tbl_operand, (uint64_t)tfp_cpu_to_be_64(vnic_id)); - if (rc) + if (unlikely(rc)) BNXT_DRV_DBG(ERR, "Failed to write regfile[%d] rc=%d\n", tbl->tbl_operand, rc); @@ -2956,8 +2957,8 @@ ulp_mapper_global_register_tbl_process(struct bnxt_ulp_mapper_parms *parms, int32_t rc = 0, write_reg = 0; /* Initialize the blob data */ - if (ulp_blob_init(&data, tbl->result_bit_size, - BNXT_ULP_BYTE_ORDER_BE)) { + if (unlikely(ulp_blob_init(&data, tbl->result_bit_size, + BNXT_ULP_BYTE_ORDER_BE))) { BNXT_DRV_DBG(ERR, "Failed initial ulp_global table blob\n"); return -EINVAL; } @@ -2965,7 +2966,7 @@ ulp_mapper_global_register_tbl_process(struct bnxt_ulp_mapper_parms *parms, /* read the arguments from the result table */ rc = ulp_mapper_tbl_result_build(parms, tbl, &data, "ULP Global Result"); - if (rc) { + if (unlikely(rc)) { BNXT_DRV_DBG(ERR, "Failed to build the result blob\n"); return rc; } @@ -2989,7 +2990,7 @@ ulp_mapper_global_register_tbl_process(struct bnxt_ulp_mapper_parms *parms, rc = bnxt_pmd_global_tunnel_set(parms->ulp_ctx, parms->port_id, tbl->resource_sub_type, udp_port, &handle); - if (rc) { + if (unlikely(rc)) { BNXT_DRV_DBG(ERR, "Unable to set Type %d port\n", tbl->resource_sub_type); return rc; @@ -3004,7 +3005,7 @@ ulp_mapper_global_register_tbl_process(struct bnxt_ulp_mapper_parms *parms, rc = ulp_mapper_fdb_opc_process(parms, tbl, &fid_parms); - if (rc) + if (unlikely(rc)) return rc; /* write to the regfile if opcode is set */ @@ -3035,14 +3036,14 @@ ulp_mapper_glb_resource_info_init(struct bnxt_ulp_context *ulp_ctx, return 0; rc = bnxt_ulp_cntxt_dev_id_get(ulp_ctx, &dev_id); - if (rc) { + if (unlikely(rc)) { BNXT_DRV_DBG(ERR, "Failed to get device id for glb init (%d)\n", rc); return rc; } rc = bnxt_ulp_cntxt_app_id_get(ulp_ctx, &app_id); - if (rc) { + if (unlikely(rc)) { BNXT_DRV_DBG(ERR, "Failed to get app id for glb init (%d)\n", rc); return rc; @@ -3105,7 +3106,7 @@ ulp_mapper_cond_opc_process(struct bnxt_ulp_mapper_parms *parms, uint32_t field_size = 0; int32_t rc = 0; uint8_t bit, tmp; - uint64_t regval, result = 0; + uint64_t regval = 0, result = 0; switch (opc) { case BNXT_ULP_COND_OPC_CF_IS_SET: @@ -3119,7 +3120,7 @@ ulp_mapper_cond_opc_process(struct bnxt_ulp_mapper_parms *parms, } break; case BNXT_ULP_COND_OPC_CF_NOT_SET: - if (operand < BNXT_ULP_CF_IDX_LAST) { + if (likely(operand < BNXT_ULP_CF_IDX_LAST)) { result = !ULP_COMP_FLD_IDX_RD(parms, operand); } else { BNXT_DRV_DBG(ERR, @@ -3129,7 +3130,7 @@ ulp_mapper_cond_opc_process(struct bnxt_ulp_mapper_parms *parms, } break; case BNXT_ULP_COND_OPC_ACT_BIT_IS_SET: - if (operand < BNXT_ULP_ACT_BIT_LAST) { + if (likely(operand < BNXT_ULP_ACT_BIT_LAST)) { result = ULP_BITMAP_ISSET(parms->act_bitmap->bits, operand); } else { @@ -3140,7 +3141,7 @@ ulp_mapper_cond_opc_process(struct bnxt_ulp_mapper_parms *parms, } break; case BNXT_ULP_COND_OPC_ACT_BIT_NOT_SET: - if (operand < BNXT_ULP_ACT_BIT_LAST) { + if (likely(operand < BNXT_ULP_ACT_BIT_LAST)) { result = !ULP_BITMAP_ISSET(parms->act_bitmap->bits, operand); } else { @@ -3151,7 +3152,7 @@ ulp_mapper_cond_opc_process(struct bnxt_ulp_mapper_parms *parms, } break; case BNXT_ULP_COND_OPC_HDR_BIT_IS_SET: - if (operand < BNXT_ULP_HDR_BIT_LAST) { + if (likely(operand < BNXT_ULP_HDR_BIT_LAST)) { result = ULP_BITMAP_ISSET(parms->hdr_bitmap->bits, operand); } else { @@ -3162,7 +3163,7 @@ ulp_mapper_cond_opc_process(struct bnxt_ulp_mapper_parms *parms, } break; case BNXT_ULP_COND_OPC_HDR_BIT_NOT_SET: - if (operand < BNXT_ULP_HDR_BIT_LAST) { + if (likely(operand < BNXT_ULP_HDR_BIT_LAST)) { result = !ULP_BITMAP_ISSET(parms->hdr_bitmap->bits, operand); } else { @@ -3174,7 +3175,7 @@ ulp_mapper_cond_opc_process(struct bnxt_ulp_mapper_parms *parms, break; case BNXT_ULP_COND_OPC_FIELD_BIT_IS_SET: rc = ulp_mapper_glb_field_tbl_get(parms, operand, &bit); - if (rc) { + if (unlikely(rc)) { BNXT_DRV_DBG(ERR, "invalid ulp_glb_field_tbl idx %" PRIu64 "\n", operand); @@ -3184,7 +3185,7 @@ ulp_mapper_cond_opc_process(struct bnxt_ulp_mapper_parms *parms, break; case BNXT_ULP_COND_OPC_FIELD_BIT_NOT_SET: rc = ulp_mapper_glb_field_tbl_get(parms, operand, &bit); - if (rc) { + if (unlikely(rc)) { BNXT_DRV_DBG(ERR, "invalid ulp_glb_field_tbl idx %" PRIu64 "\n", operand); @@ -3202,7 +3203,7 @@ ulp_mapper_cond_opc_process(struct bnxt_ulp_mapper_parms *parms, result = regval != 0; break; case BNXT_ULP_COND_OPC_RF_NOT_SET: - if (ulp_regfile_read(parms->regfile, operand, ®val)) { + if (unlikely(ulp_regfile_read(parms->regfile, operand, ®val))) { BNXT_DRV_DBG(ERR, "regfile[%" PRIu64 "] read oob\n", operand); return -EINVAL; @@ -3216,21 +3217,21 @@ ulp_mapper_cond_opc_process(struct bnxt_ulp_mapper_parms *parms, result = parms->act_pattern_id == operand; break; case BNXT_ULP_COND_OPC_EXT_MEM_IS_SET: - if (bnxt_ulp_cntxt_mem_type_get(parms->ulp_ctx, &mtype)) { + if (unlikely(bnxt_ulp_cntxt_mem_type_get(parms->ulp_ctx, &mtype))) { BNXT_DRV_DBG(ERR, "Failed to get the mem type\n"); return -EINVAL; } result = (mtype == BNXT_ULP_FLOW_MEM_TYPE_INT) ? 0 : 1; break; case BNXT_ULP_COND_OPC_EXT_MEM_NOT_SET: - if (bnxt_ulp_cntxt_mem_type_get(parms->ulp_ctx, &mtype)) { + if (unlikely(bnxt_ulp_cntxt_mem_type_get(parms->ulp_ctx, &mtype))) { BNXT_DRV_DBG(ERR, "Failed to get the mem type\n"); return -EINVAL; } result = (mtype == BNXT_ULP_FLOW_MEM_TYPE_INT) ? 1 : 0; break; case BNXT_ULP_COND_OPC_ENC_HDR_BIT_IS_SET: - if (operand < BNXT_ULP_HDR_BIT_LAST) { + if (likely(operand < BNXT_ULP_HDR_BIT_LAST)) { result = ULP_BITMAP_ISSET(parms->enc_hdr_bitmap->bits, operand); } else { @@ -3241,7 +3242,7 @@ ulp_mapper_cond_opc_process(struct bnxt_ulp_mapper_parms *parms, } break; case BNXT_ULP_COND_OPC_ENC_HDR_BIT_NOT_SET: - if (operand < BNXT_ULP_HDR_BIT_LAST) { + if (likely(operand < BNXT_ULP_HDR_BIT_LAST)) { result = !ULP_BITMAP_ISSET(parms->enc_hdr_bitmap->bits, operand); } else { @@ -3254,13 +3255,13 @@ ulp_mapper_cond_opc_process(struct bnxt_ulp_mapper_parms *parms, case BNXT_ULP_COND_OPC_ACT_PROP_IS_SET: case BNXT_ULP_COND_OPC_ACT_PROP_NOT_SET: /* only supporting 1-byte action properties for now */ - if (operand >= BNXT_ULP_ACT_PROP_IDX_LAST) { + if (unlikely(operand >= BNXT_ULP_ACT_PROP_IDX_LAST)) { BNXT_DRV_DBG(ERR, "act_prop[%" PRIu64 "] oob\n", operand); return -EINVAL; } field_size = ulp_mapper_act_prop_size_get(operand); - if (sizeof(tmp) != field_size) { + if (unlikely(sizeof(tmp) != field_size)) { BNXT_DRV_DBG(ERR, "act_prop[%" PRIu64 "] field mismatch %u\n", operand, field_size); @@ -3274,7 +3275,7 @@ ulp_mapper_cond_opc_process(struct bnxt_ulp_mapper_parms *parms, break; case BNXT_ULP_COND_OPC_CF_BIT_IS_SET: case BNXT_ULP_COND_OPC_CF_BIT_NOT_SET: - if (operand < BNXT_ULP_CF_BIT_LAST) { + if (likely(operand < BNXT_ULP_CF_BIT_LAST)) { result = ULP_BITMAP_ISSET(parms->cf_bitmap, operand); } else { BNXT_DRV_DBG(ERR, @@ -3288,7 +3289,7 @@ ulp_mapper_cond_opc_process(struct bnxt_ulp_mapper_parms *parms, case BNXT_ULP_COND_OPC_WC_FIELD_BIT_IS_SET: case BNXT_ULP_COND_OPC_WC_FIELD_BIT_NOT_SET: rc = ulp_mapper_glb_field_tbl_get(parms, operand, &bit); - if (rc) { + if (unlikely(rc)) { BNXT_DRV_DBG(ERR, "invalid ulp_glb_field idx %" PRIu64 "\n", operand); @@ -3301,7 +3302,7 @@ ulp_mapper_cond_opc_process(struct bnxt_ulp_mapper_parms *parms, case BNXT_ULP_COND_OPC_EXCLUDE_FIELD_BIT_IS_SET: case BNXT_ULP_COND_OPC_EXCLUDE_FIELD_BIT_NOT_SET: rc = ulp_mapper_glb_field_tbl_get(parms, operand, &bit); - if (rc) { + if (unlikely(rc)) { BNXT_DRV_DBG(ERR, "invalid ulp_glb_field idx %" PRIu64 "\n", operand); @@ -3341,7 +3342,7 @@ ulp_mapper_func_opr_compute(struct bnxt_ulp_mapper_parms *parms, *result = false; switch (func_src) { case BNXT_ULP_FUNC_SRC_COMP_FIELD: - if (func_opr >= BNXT_ULP_CF_IDX_LAST) { + if (unlikely(func_opr >= BNXT_ULP_CF_IDX_LAST)) { BNXT_DRV_DBG(ERR, "invalid index %u\n", (uint32_t)func_opr); return -EINVAL; @@ -3349,7 +3350,7 @@ ulp_mapper_func_opr_compute(struct bnxt_ulp_mapper_parms *parms, *result = ULP_COMP_FLD_IDX_RD(parms, func_opr); break; case BNXT_ULP_FUNC_SRC_REGFILE: - if (ulp_regfile_read(parms->regfile, func_opr, ®val)) { + if (unlikely(ulp_regfile_read(parms->regfile, func_opr, ®val))) { BNXT_DRV_DBG(ERR, "regfile[%d] read oob\n", (uint32_t)func_opr); return -EINVAL; @@ -3357,8 +3358,8 @@ ulp_mapper_func_opr_compute(struct bnxt_ulp_mapper_parms *parms, *result = tfp_be_to_cpu_64(regval); break; case BNXT_ULP_FUNC_SRC_GLB_REGFILE: - if (ulp_mapper_glb_resource_read(parms->mapper_data, dir, - func_opr, ®val, &shared)) { + if (unlikely(ulp_mapper_glb_resource_read(parms->mapper_data, dir, + func_opr, ®val, &shared))) { BNXT_DRV_DBG(ERR, "global regfile[%d] read failed.\n", (uint32_t)func_opr); return -EINVAL; @@ -3395,7 +3396,7 @@ ulp_mapper_vfr_mark_set(struct bnxt_ulp_mapper_parms *parms, rc = ulp_mark_db_mark_add(parms->ulp_ctx, mark_flag, key, port_id); - if (rc) { + if (unlikely(rc)) { BNXT_DRV_DBG(ERR, "Failed to add mark to flow\n"); return rc; } @@ -3442,17 +3443,17 @@ ulp_mapper_func_cond_list_process(struct bnxt_ulp_mapper_parms *parms, /* Get the field info from the key ext list */ fld = ulp_mapper_tmpl_key_ext_list_get(parms, idx); - if (fld == NULL || fld->field_opc != - BNXT_ULP_FIELD_OPC_TERNARY_LIST) { + if (unlikely(fld == NULL || fld->field_opc != + BNXT_ULP_FIELD_OPC_TERNARY_LIST)) { BNXT_DRV_DBG(ERR, "Invalid field idx %d\n", idx); return -EINVAL; } /* process the condition list */ - if (ulp_mapper_field_src_process(parms, fld->field_src1, - fld->field_opr1, dir, - 1, oper_size, &val, - &val_len, &value)) { + if (unlikely(ulp_mapper_field_src_process(parms, fld->field_src1, + fld->field_opr1, dir, + 1, oper_size, &val, + &val_len, &value))) { BNXT_DRV_DBG(ERR, "error processing func opcode %u\n", idx); return -EINVAL; @@ -3460,9 +3461,9 @@ ulp_mapper_func_cond_list_process(struct bnxt_ulp_mapper_parms *parms, if (value) { if (fld->field_src2 == BNXT_ULP_FIELD_SRC_NEXT) { /* read the next key ext table index */ - if (ulp_operand_read(fld->field_opr2, - (uint8_t *)&ext_idx, - sizeof(uint16_t))) { + if (unlikely(ulp_operand_read(fld->field_opr2, + (uint8_t *)&ext_idx, + sizeof(uint16_t)))) { BNXT_DRV_DBG(ERR, "field idx operand read failed\n"); return -EINVAL; @@ -3473,11 +3474,11 @@ ulp_mapper_func_cond_list_process(struct bnxt_ulp_mapper_parms *parms, res, res_size); } else { /* get the value from then part */ - if (ulp_mapper_field_src_process(parms, fld->field_src2, - fld->field_opr2, dir, - 1, oper_size, - &val, &val_len, - &value)) { + if (unlikely(ulp_mapper_field_src_process(parms, fld->field_src2, + fld->field_opr2, dir, + 1, oper_size, + &val, &val_len, + &value))) { BNXT_DRV_DBG(ERR, "error processing func oper %u\n", ext_idx); @@ -3487,9 +3488,9 @@ ulp_mapper_func_cond_list_process(struct bnxt_ulp_mapper_parms *parms, } else { if (fld->field_src3 == BNXT_ULP_FIELD_SRC_NEXT) { /* read the next key ext table index */ - if (ulp_operand_read(fld->field_opr3, - (uint8_t *)&ext_idx, - sizeof(uint16_t))) { + if (unlikely(ulp_operand_read(fld->field_opr3, + (uint8_t *)&ext_idx, + sizeof(uint16_t)))) { BNXT_DRV_DBG(ERR, "field idx operand read failed\n"); return -EINVAL; @@ -3500,11 +3501,11 @@ ulp_mapper_func_cond_list_process(struct bnxt_ulp_mapper_parms *parms, res, res_size); } else { /* get the value from else part */ - if (ulp_mapper_field_src_process(parms, fld->field_src3, - fld->field_opr3, dir, - 1, oper_size, - &val, &val_len, - &value)) { + if (unlikely(ulp_mapper_field_src_process(parms, fld->field_src3, + fld->field_opr3, dir, + 1, oper_size, + &val, &val_len, + &value))) { BNXT_DRV_DBG(ERR, "error processing func oper %u\n", ext_idx); @@ -3651,7 +3652,7 @@ ulp_mapper_func_info_process(struct bnxt_ulp_mapper_parms *parms, return bnxt_rss_config_action_apply(parms); case BNXT_ULP_FUNC_OPC_GET_PARENT_MAC_ADDR: rc = bnxt_pmd_get_parent_mac_addr(parms, (uint8_t *)&res); - if (rc) + if (unlikely(rc)) return -EINVAL; res = tfp_be_to_cpu_64(res); break; @@ -3666,7 +3667,7 @@ ulp_mapper_func_info_process(struct bnxt_ulp_mapper_parms *parms, /* res1 is port_id, res2 is action */ return ulp_mapper_bd_act_set(parms, res1, res2); case BNXT_ULP_FUNC_OPC_COND_LIST: - if (func_info->func_src1 != BNXT_ULP_FUNC_SRC_KEY_EXT_LIST) { + if (unlikely(func_info->func_src1 != BNXT_ULP_FUNC_SRC_KEY_EXT_LIST)) { BNXT_DRV_DBG(ERR, "invalid func source %u\n", func_info->func_opc); return -EINVAL; @@ -3683,8 +3684,8 @@ ulp_mapper_func_info_process(struct bnxt_ulp_mapper_parms *parms, func_info->func_opc); return -EINVAL; } - if (ulp_regfile_write(parms->regfile, func_info->func_dst_opr, - tfp_cpu_to_be_64(res))) { + if (unlikely(ulp_regfile_write(parms->regfile, func_info->func_dst_opr, + tfp_cpu_to_be_64(res)))) { BNXT_DRV_DBG(ERR, "Failed write the func_opc %u\n", func_info->func_dst_opr); return -EINVAL; @@ -3743,7 +3744,7 @@ ulp_mapper_cond_opc_list_process(struct bnxt_ulp_mapper_parms *parms, cond_list[i].cond_opcode, cond_list[i].cond_operand, &trc); - if (rc) + if (unlikely(rc)) return rc; if (info->cond_list_opcode == BNXT_ULP_COND_LIST_OPC_AND) { @@ -3805,7 +3806,7 @@ ulp_mapper_cond_reject_list_process(struct bnxt_ulp_mapper_parms *parms, idx < reject_info->cond_start_idx + reject_info->cond_nums; idx++) { oper = ulp_mapper_cond_oper_list_get(parms, idx); - if (!oper) { + if (unlikely(!oper)) { BNXT_DRV_DBG(ERR, "Invalid cond oper idx %d\n", idx); @@ -3868,7 +3869,7 @@ ulp_mapper_cond_execute_list_process(struct bnxt_ulp_mapper_parms *parms, idx < execute_info->cond_start_idx + execute_info->cond_nums; idx++) { oper = ulp_mapper_cond_oper_list_get(parms, idx); - if (!oper) { + if (unlikely(!oper)) { BNXT_DRV_DBG(ERR, "Invalid cond oper idx %d\n", idx); @@ -3910,7 +3911,7 @@ ulp_mapper_conflict_resolution_process(struct bnxt_ulp_mapper_parms *parms, int32_t *res) { int32_t rc = 0; - uint64_t regval; + uint64_t regval = 0; uint64_t comp_sig; *res = 0; @@ -3923,9 +3924,9 @@ ulp_mapper_conflict_resolution_process(struct bnxt_ulp_mapper_parms *parms, if (tbl->resource_func == BNXT_ULP_RESOURCE_FUNC_GENERIC_TABLE) { /* Perform the check that generic table is hit or not */ - if (ulp_regfile_read(parms->regfile, - BNXT_ULP_RF_IDX_GENERIC_TBL_MISS, - ®val)) { + if (unlikely(ulp_regfile_read(parms->regfile, + BNXT_ULP_RF_IDX_GENERIC_TBL_MISS, + ®val))) { BNXT_DRV_DBG(ERR, "regfile[%d] read oob\n", BNXT_ULP_RF_IDX_GENERIC_TBL_MISS); return -EINVAL; @@ -3937,9 +3938,9 @@ ulp_mapper_conflict_resolution_process(struct bnxt_ulp_mapper_parms *parms, } } /* compare the new flow signature against stored one */ - if (ulp_regfile_read(parms->regfile, - BNXT_ULP_RF_IDX_FLOW_SIG_ID, - ®val)) { + if (unlikely(ulp_regfile_read(parms->regfile, + BNXT_ULP_RF_IDX_FLOW_SIG_ID, + ®val))) { BNXT_DRV_DBG(ERR, "regfile[%d] read oob\n", BNXT_ULP_RF_IDX_FLOW_SIG_ID); return -EINVAL; @@ -3947,7 +3948,7 @@ ulp_mapper_conflict_resolution_process(struct bnxt_ulp_mapper_parms *parms, comp_sig = ULP_COMP_FLD_IDX_RD(parms, BNXT_ULP_CF_IDX_FLOW_SIG_ID); regval = tfp_be_to_cpu_64(regval); - if (comp_sig == regval) + if (likely(comp_sig == regval)) *res = 1; else BNXT_DRV_DBG(ERR, "failed signature match 0x%016" @@ -3977,7 +3978,7 @@ ulp_mapper_allocator_tbl_process(struct bnxt_ulp_mapper_parms *parms, rc = ulp_allocator_tbl_list_alloc(parms->mapper_data, tbl->resource_sub_type, tbl->direction, &alloc_index); - if (rc) { + if (unlikely(rc)) { BNXT_DRV_DBG(ERR, "unable to alloc index %x:%x\n", tbl->resource_sub_type, tbl->direction); return -EINVAL; @@ -3986,7 +3987,7 @@ ulp_mapper_allocator_tbl_process(struct bnxt_ulp_mapper_parms *parms, /* Write to the regfile */ regval = rte_cpu_to_be_64(alloc_index); rc = ulp_regfile_write(parms->regfile, tbl->tbl_operand, regval); - if (rc) { + if (unlikely(rc)) { BNXT_DRV_DBG(ERR, "Failed to write regfile[%d] rc=%d\n", tbl->tbl_operand, rc); return -EINVAL; @@ -4002,7 +4003,7 @@ ulp_mapper_allocator_tbl_process(struct bnxt_ulp_mapper_parms *parms, fid_parms.critical_resource = tbl->critical_resource; rc = ulp_mapper_fdb_opc_process(parms, tbl, &fid_parms); - if (rc) { + if (unlikely(rc)) { BNXT_DRV_DBG(ERR, "Failed to link resource to flow rc = %d\n", rc); goto error; @@ -4035,11 +4036,11 @@ ulp_mapper_tbls_process(struct bnxt_ulp_mapper_parms *parms, void *error) rc = ulp_mapper_cond_reject_list_process(parms, tid, &cond_rc); /* if rc is failure or cond_rc is a reject then exit tbl processing */ - if (rc || cond_rc) + if (unlikely(rc || cond_rc)) return -EINVAL; tbls = ulp_mapper_tbl_list_get(parms, tid, &num_tbls); - if (!tbls || !num_tbls) { + if (unlikely(!tbls || !num_tbls)) { BNXT_DRV_DBG(ERR, "No %s tables for %d:%d\n", ulp_mapper_tmpl_name_str(parms->tmpl_type), parms->dev_id, tid); @@ -4050,7 +4051,7 @@ ulp_mapper_tbls_process(struct bnxt_ulp_mapper_parms *parms, void *error) tbl = &tbls[tbl_idx]; cond_goto = tbl->execute_info.cond_true_goto; /* Process the conditional func code opcodes */ - if (ulp_mapper_func_info_process(parms, tbl)) { + if (unlikely(ulp_mapper_func_info_process(parms, tbl))) { BNXT_DRV_DBG(ERR, "Failed to process cond update\n"); rc = -EINVAL; goto error; @@ -4058,7 +4059,7 @@ ulp_mapper_tbls_process(struct bnxt_ulp_mapper_parms *parms, void *error) /* process the execute info of the table */ rc = ulp_mapper_cond_execute_list_process(parms, tbl, &cond_rc); - if (rc) { + if (unlikely(rc)) { BNXT_DRV_DBG(ERR, "Failed to proc cond opc list (%d)\n", rc); goto error; @@ -4126,14 +4127,14 @@ ulp_mapper_tbls_process(struct bnxt_ulp_mapper_parms *parms, void *error) /* perform the post table process */ rc = ulp_mapper_conflict_resolution_process(parms, tbl, &cond_rc); - if (rc || !cond_rc) { + if (unlikely(rc || !cond_rc)) { BNXT_DRV_DBG(ERR, "Failed due to conflict resolution\n"); rc = -EINVAL; goto error; } next_iteration: if (cond_goto < 0) { - if (((int32_t)tbl_idx + cond_goto) < 0) { + if (unlikely(((int32_t)tbl_idx + cond_goto) < 0)) { BNXT_DRV_DBG(ERR, "invalid conditional goto %d\n", cond_goto); @@ -4146,7 +4147,7 @@ ulp_mapper_tbls_process(struct bnxt_ulp_mapper_parms *parms, void *error) tbl->true_message; BNXT_DRV_DBG(DEBUG, "%s\n", msg); - if (error) + if (unlikely(error)) rte_flow_error_set(error, EINVAL, RTE_FLOW_ERROR_TYPE_ITEM, NULL, msg); @@ -4157,12 +4158,12 @@ ulp_mapper_tbls_process(struct bnxt_ulp_mapper_parms *parms, void *error) goto error; } else if (cond_goto & BNXT_ULP_COND_GOTO_RF) { int32_t rf_idx; - uint64_t regval; + uint64_t regval = 0; /* least significant 16 bits from reg_file index */ rf_idx = (int32_t)(cond_goto & 0xFFFF); - if (ulp_regfile_read(parms->regfile, rf_idx, - ®val)) { + if (unlikely(ulp_regfile_read(parms->regfile, rf_idx, + ®val))) { BNXT_DRV_DBG(ERR, "regfile[%d] read oob\n", rf_idx); rc = -EINVAL; @@ -4191,7 +4192,7 @@ ulp_mapper_resource_free(struct bnxt_ulp_context *ulp, struct bnxt_ulp_mapper_data *mdata; int32_t rc = 0; - if (!res || !ulp) { + if (unlikely(!res || !ulp)) { BNXT_DRV_DBG(ERR, "Unable to free resource\n "); return -EINVAL; } @@ -4239,7 +4240,7 @@ ulp_mapper_resource_free(struct bnxt_ulp_context *ulp, break; case BNXT_ULP_RESOURCE_FUNC_ALLOCATOR_TABLE: mdata = bnxt_ulp_cntxt_ptr2_mapper_data_get(ulp); - if (!mdata) { + if (unlikely(!mdata)) { BNXT_DRV_DBG(ERR, "Unable to get mapper data\n"); return -EINVAL; } @@ -4264,7 +4265,7 @@ ulp_mapper_resources_free(struct bnxt_ulp_context *ulp_ctx, struct ulp_flow_db_res_params res_parms = { 0 }; int32_t rc, trc, frc = 0; - if (!ulp_ctx) { + if (unlikely(!ulp_ctx)) { BNXT_DRV_DBG(ERR, "Invalid parms, unable to free flow\n"); return -EINVAL; } @@ -4368,7 +4369,7 @@ ulp_mapper_flow_destroy(struct bnxt_ulp_context *ulp_ctx, { int32_t rc; - if (!ulp_ctx) { + if (unlikely(!ulp_ctx)) { BNXT_DRV_DBG(ERR, "Invalid parms, unable to free flow\n"); return -EINVAL; } @@ -4388,7 +4389,7 @@ ulp_mapper_flow_create(struct bnxt_ulp_context *ulp_ctx, struct ulp_regfile regfile; int32_t rc = 0, trc; - if (!ulp_ctx || !parms) + if (unlikely(!ulp_ctx || !parms)) return -EINVAL; parms->regfile = ®file; @@ -4397,18 +4398,18 @@ ulp_mapper_flow_create(struct bnxt_ulp_context *ulp_ctx, oper = ulp_mapper_data_oper_get(ulp_ctx); /* Get the device id from the ulp context */ - if (bnxt_ulp_cntxt_dev_id_get(ulp_ctx, &parms->dev_id)) { + if (unlikely(bnxt_ulp_cntxt_dev_id_get(ulp_ctx, &parms->dev_id))) { BNXT_DRV_DBG(ERR, "Invalid ulp context\n"); return -EINVAL; } - if (bnxt_ulp_cntxt_fid_get(ulp_ctx, &parms->fw_fid)) { + if (unlikely(bnxt_ulp_cntxt_fid_get(ulp_ctx, &parms->fw_fid))) { BNXT_DRV_DBG(ERR, "Unable to get the func_id\n"); return -EINVAL; } /* Get the device params, it will be used in later processing */ parms->device_params = bnxt_ulp_device_params_get(parms->dev_id); - if (!parms->device_params) { + if (unlikely(!parms->device_params)) { BNXT_DRV_DBG(ERR, "No device parms for device id %d\n", parms->dev_id); return -EINVAL; @@ -4420,13 +4421,13 @@ ulp_mapper_flow_create(struct bnxt_ulp_context *ulp_ctx, */ parms->mapper_data = (struct bnxt_ulp_mapper_data *) bnxt_ulp_cntxt_ptr2_mapper_data_get(ulp_ctx); - if (!parms->mapper_data) { + if (unlikely(!parms->mapper_data)) { BNXT_DRV_DBG(ERR, "Failed to get the ulp mapper data\n"); return -EINVAL; } /* initialize the registry file for further processing */ - if (ulp_regfile_init(parms->regfile)) { + if (unlikely(ulp_regfile_init(parms->regfile))) { BNXT_DRV_DBG(ERR, "regfile initialization failed.\n"); return -EINVAL; } @@ -4443,7 +4444,7 @@ ulp_mapper_flow_create(struct bnxt_ulp_context *ulp_ctx, parms->tmpl_type = BNXT_ULP_TEMPLATE_TYPE_ACTION; /* Process the action template tables */ rc = ulp_mapper_tbls_process(parms, error); - if (rc) + if (unlikely(rc)) goto batch_error; } @@ -4451,7 +4452,7 @@ ulp_mapper_flow_create(struct bnxt_ulp_context *ulp_ctx, parms->tmpl_type = BNXT_ULP_TEMPLATE_TYPE_CLASS; /* Process the class template tables.*/ rc = ulp_mapper_tbls_process(parms, error); - if (rc) + if (unlikely(rc)) goto batch_error; } @@ -4469,12 +4470,12 @@ ulp_mapper_flow_create(struct bnxt_ulp_context *ulp_ctx, if (parms->parent_flow) { /* create a parent flow details */ rc = ulp_flow_db_parent_flow_create(parms); - if (rc) + if (unlikely(rc)) goto flow_error; } else if (parms->child_flow) { /* create a child flow details */ rc = ulp_flow_db_child_flow_create(parms); - if (rc) + if (unlikely(rc)) goto flow_error; } @@ -4529,20 +4530,20 @@ ulp_mapper_init(struct bnxt_ulp_context *ulp_ctx) data = rte_zmalloc("ulp_mapper_data", sizeof(struct bnxt_ulp_mapper_data), 0); - if (!data) { + if (unlikely(!data)) { BNXT_DRV_DBG(ERR, "Failed to allocate the mapper data\n"); return -ENOMEM; } /* set the mapper operations for the current platform */ data->mapper_oper = bnxt_ulp_mapper_ops_get(ulp_ctx->bp); - if (data->mapper_oper == NULL) { + if (unlikely(data->mapper_oper == NULL)) { rte_free(data); BNXT_DRV_DBG(ERR, "Failed to get mapper ops\n"); return -ENOMEM; } - if (bnxt_ulp_cntxt_ptr2_mapper_data_set(ulp_ctx, data)) { + if (unlikely(bnxt_ulp_cntxt_ptr2_mapper_data_set(ulp_ctx, data))) { BNXT_DRV_DBG(ERR, "Failed to set mapper data in context\n"); /* Don't call deinit since the prof_func wasn't allocated. */ rte_free(data); @@ -4551,7 +4552,7 @@ ulp_mapper_init(struct bnxt_ulp_context *ulp_ctx) /* Allocate the global resource ids */ rc = ulp_mapper_glb_resource_info_init(ulp_ctx, data); - if (rc) { + if (unlikely(rc)) { BNXT_DRV_DBG(ERR, "Failed to initialize global resource ids\n"); goto error; } @@ -4562,7 +4563,7 @@ ulp_mapper_init(struct bnxt_ulp_context *ulp_ctx) */ if (bnxt_ulp_cntxt_shared_session_enabled(ulp_ctx)) { rc = ulp_mapper_app_glb_resource_info_init(ulp_ctx, data); - if (rc) { + if (unlikely(rc)) { BNXT_DRV_DBG(ERR, "Failed to init app glb resources\n"); goto error; } @@ -4570,19 +4571,19 @@ ulp_mapper_init(struct bnxt_ulp_context *ulp_ctx) /* Allocate the generic table list */ rc = ulp_mapper_generic_tbl_list_init(ulp_ctx, data); - if (rc) { + if (unlikely(rc)) { BNXT_DRV_DBG(ERR, "Failed to initialize generic tbl list\n"); goto error; } rc = ulp_mapper_key_recipe_tbl_init(ulp_ctx, data); - if (rc) { + if (unlikely(rc)) { BNXT_DRV_DBG(ERR, "Failed to initialize key_recipe tbl\n"); goto error; } rc = ulp_allocator_tbl_list_init(ulp_ctx, data); - if (rc) { + if (unlikely(rc)) { BNXT_DRV_DBG(ERR, "Failed to initialize allocator tbl\n"); goto error; } @@ -4599,7 +4600,7 @@ ulp_mapper_deinit(struct bnxt_ulp_context *ulp_ctx) { struct bnxt_ulp_mapper_data *data; - if (!ulp_ctx) { + if (unlikely(!ulp_ctx)) { BNXT_DRV_DBG(ERR, "Failed to acquire ulp context, so data may not be released.\n"); return; @@ -4607,7 +4608,7 @@ ulp_mapper_deinit(struct bnxt_ulp_context *ulp_ctx) data = (struct bnxt_ulp_mapper_data *) bnxt_ulp_cntxt_ptr2_mapper_data_get(ulp_ctx); - if (!data) { + if (unlikely(!data)) { /* Go ahead and return since there is no allocated data. */ BNXT_DRV_DBG(ERR, "No data appears to have been allocated.\n"); return; diff --git a/drivers/net/bnxt/tf_ulp/ulp_mapper_tf.c b/drivers/net/bnxt/tf_ulp/ulp_mapper_tf.c index de04deed92..53497c164a 100644 --- a/drivers/net/bnxt/tf_ulp/ulp_mapper_tf.c +++ b/drivers/net/bnxt/tf_ulp/ulp_mapper_tf.c @@ -8,6 +8,7 @@ #include "ulp_ha_mgr.h" #include "tfp.h" #include "tf_util.h" +#include "bnxt_ulp_utils.h" #include "bnxt_ulp_tf.h" #ifdef RTE_LIBRTE_BNXT_TRUFLOW_DEBUG @@ -936,7 +937,7 @@ ulp_mapper_tf_if_tbl_process(struct bnxt_ulp_mapper_parms *parms, struct bnxt_ulp_mapper_tbl_info *tbl) { struct ulp_blob data, res_blob; - uint64_t idx; + uint64_t idx = 0; uint16_t tmplen; int32_t rc = 0; struct tf_set_if_tbl_entry_parms iftbl_params = { 0 }; diff --git a/drivers/net/bnxt/tf_ulp/ulp_mapper_tfc.c b/drivers/net/bnxt/tf_ulp/ulp_mapper_tfc.c index 23383fec43..cbe9aa01c9 100644 --- a/drivers/net/bnxt/tf_ulp/ulp_mapper_tfc.c +++ b/drivers/net/bnxt/tf_ulp/ulp_mapper_tfc.c @@ -9,6 +9,7 @@ #include "cfa_bld.h" #include "tfc_util.h" #include "bnxt_ulp_tfc.h" +#include "bnxt_ulp_utils.h" #include "tfc_action_handle.h" #ifdef RTE_LIBRTE_BNXT_TRUFLOW_DEBUG @@ -34,13 +35,13 @@ ulp_mapper_tfc_tcam_tbl_entry_write(struct bnxt_ulp_mapper_parms *parms, uint16_t fw_fid; tfcp = bnxt_ulp_cntxt_tfcp_get(parms->ulp_ctx); - if (tfcp == NULL) { + if (unlikely(tfcp == NULL)) { BNXT_DRV_DBG(ERR, "Failed to get tfcp pointer\n"); return -EINVAL; } rc = bnxt_ulp_cntxt_fid_get(parms->ulp_ctx, &fw_fid); - if (rc) + if (unlikely(rc)) return rc; tfc_info.dir = tbl->direction; @@ -53,7 +54,7 @@ ulp_mapper_tfc_tcam_tbl_entry_write(struct bnxt_ulp_mapper_parms *parms, remap_size = ULP_BITS_2_BYTE(remap_size); tfc_data.remap_sz_in_bytes = remap_size; - if (tfc_tcam_set(tfcp, fw_fid, &tfc_info, &tfc_data)) { + if (unlikely(tfc_tcam_set(tfcp, fw_fid, &tfc_info, &tfc_data))) { BNXT_DRV_DBG(ERR, "tcam[%s][%s][%x] write failed.\n", tfc_tcam_2_str(tfc_info.rsubtype), tfc_dir_2_str(tfc_info.dir), tfc_info.id); @@ -65,7 +66,7 @@ ulp_mapper_tfc_tcam_tbl_entry_write(struct bnxt_ulp_mapper_parms *parms, /* Mark action */ rc = ulp_mapper_mark_act_ptr_process(parms, tbl); - if (rc) { + if (unlikely(rc)) { BNXT_DRV_DBG(ERR, "failed mark action processing\n"); return rc; } @@ -102,21 +103,21 @@ ulp_mapper_tfc_wc_tcam_post_process(struct bnxt_ulp_device_params *dparms, tlen = tlen << 1; } - if (num_slices > max_slices) { + if (unlikely(num_slices > max_slices)) { BNXT_DRV_DBG(ERR, "Key size (%d) too large for WC\n", blen); return -EINVAL; } /* The key/mask may not be on a natural slice boundary, pad it */ pad = tlen - blen; - if (ulp_blob_pad_push(key, pad) < 0) { + if (unlikely(ulp_blob_pad_push(key, pad) < 0)) { BNXT_DRV_DBG(ERR, "Unable to pad key/mask\n"); return -EINVAL; } /* The new length accounts for the ctrl word length and num slices */ tlen = tlen + (clen + 1) * num_slices; - if (ulp_blob_init(tkey, tlen, key->byte_order)) { + if (unlikely(ulp_blob_init(tkey, tlen, key->byte_order))) { BNXT_DRV_DBG(ERR, "Unable to post process wc tcam entry\n"); return -EINVAL; } @@ -124,7 +125,7 @@ ulp_mapper_tfc_wc_tcam_post_process(struct bnxt_ulp_device_params *dparms, /* pad any remaining bits to do byte alignment */ pad = (slice_width + clen) * num_slices; pad = ULP_BYTE_ROUND_OFF_8(pad) - pad; - if (ulp_blob_pad_push(tkey, pad) < 0) { + if (unlikely(ulp_blob_pad_push(tkey, pad) < 0)) { BNXT_DRV_DBG(ERR, "Unable to pad key/mask\n"); return -EINVAL; } @@ -135,12 +136,12 @@ ulp_mapper_tfc_wc_tcam_post_process(struct bnxt_ulp_device_params *dparms, offset = 0; for (i = 0; i < num_slices; i++) { val = ulp_blob_push_32(tkey, &cword, clen); - if (!val) { + if (unlikely(!val)) { BNXT_DRV_DBG(ERR, "Key ctrl word push failed\n"); return -EINVAL; } rc = ulp_blob_append(tkey, key, offset, slice_width); - if (rc) { + if (unlikely(rc)) { BNXT_DRV_DBG(ERR, "Key blob append failed\n"); return rc; } @@ -195,12 +196,12 @@ ulp_mapper_tfc_tcam_tbl_process(struct bnxt_ulp_mapper_parms *parms, } tfcp = bnxt_ulp_cntxt_tfcp_get(parms->ulp_ctx); - if (tfcp == NULL) { + if (unlikely(tfcp == NULL)) { PMD_DRV_LOG_LINE(ERR, "Failed to get tfcp pointer"); return -EINVAL; } - if (bnxt_ulp_cntxt_fid_get(parms->ulp_ctx, &fw_fid)) { + if (unlikely(bnxt_ulp_cntxt_fid_get(parms->ulp_ctx, &fw_fid))) { BNXT_DRV_DBG(ERR, "Failed to get func_id\n"); return -EINVAL; } @@ -208,14 +209,14 @@ ulp_mapper_tfc_tcam_tbl_process(struct bnxt_ulp_mapper_parms *parms, /* Allocate the identifiers */ if (alloc_ident) { rc = ulp_mapper_tcam_tbl_ident_alloc(parms, tbl); - if (rc) { + if (unlikely(rc)) { BNXT_DRV_DBG(ERR, "Failed to alloc identifier\n"); return rc; } } /* If no allocation or write is needed, then just exit */ - if (!alloc_tcam && !write_tcam) + if (unlikely(!alloc_tcam && !write_tcam)) return rc; /* Initialize the blobs for write */ @@ -225,9 +226,9 @@ ulp_mapper_tfc_tcam_tbl_process(struct bnxt_ulp_mapper_parms *parms, key_byte_order = dparms->key_byte_order; res_byte_order = dparms->result_byte_order; - if (ulp_blob_init(key, tbl->blob_key_bit_size, key_byte_order) || - ulp_blob_init(mask, tbl->blob_key_bit_size, key_byte_order) || - ulp_blob_init(&data, tbl->result_bit_size, res_byte_order)) { + if (unlikely(ulp_blob_init(key, tbl->blob_key_bit_size, key_byte_order) || + ulp_blob_init(mask, tbl->blob_key_bit_size, key_byte_order) || + ulp_blob_init(&data, tbl->result_bit_size, res_byte_order))) { BNXT_DRV_DBG(ERR, "blob inits failed.\n"); return -EINVAL; } @@ -237,7 +238,7 @@ ulp_mapper_tfc_tcam_tbl_process(struct bnxt_ulp_mapper_parms *parms, kflds = ulp_mapper_key_recipe_fields_get(parms, tbl, &num_kflds); else kflds = ulp_mapper_key_fields_get(parms, tbl, &num_kflds); - if (!kflds || !num_kflds) { + if (unlikely(!kflds || !num_kflds)) { BNXT_DRV_DBG(ERR, "Failed to get key fields\n"); return -EINVAL; } @@ -247,7 +248,7 @@ ulp_mapper_tfc_tcam_tbl_process(struct bnxt_ulp_mapper_parms *parms, rc = ulp_mapper_field_opc_process(parms, tbl->direction, &kflds[i].field_info_spec, key, 1, "TCAM Key"); - if (rc) { + if (unlikely(rc)) { BNXT_DRV_DBG(ERR, "Key field set failed %s\n", kflds[i].field_info_spec.description); return rc; @@ -257,7 +258,7 @@ ulp_mapper_tfc_tcam_tbl_process(struct bnxt_ulp_mapper_parms *parms, rc = ulp_mapper_field_opc_process(parms, tbl->direction, &kflds[i].field_info_mask, mask, 0, "TCAM Mask"); - if (rc) { + if (unlikely(rc)) { BNXT_DRV_DBG(ERR, "Mask field set failed %s\n", kflds[i].field_info_mask.description); return rc; @@ -268,14 +269,14 @@ ulp_mapper_tfc_tcam_tbl_process(struct bnxt_ulp_mapper_parms *parms, if (tbl->resource_type == CFA_RSUBTYPE_TCAM_WC) { /* Sets up the slices for writing to the WC TCAM */ rc = ulp_mapper_tfc_wc_tcam_post_process(dparms, key, &tkey); - if (rc) { + if (unlikely(rc)) { BNXT_DRV_DBG(ERR, "Failed to post proc WC key.\n"); return rc; } /* Sets up the slices for writing to the WC TCAM */ rc = ulp_mapper_tfc_wc_tcam_post_process(dparms, mask, &tmask); - if (rc) { + if (unlikely(rc)) { BNXT_DRV_DBG(ERR, "Failed to post proc WC mask.\n"); return rc; @@ -291,14 +292,14 @@ ulp_mapper_tfc_tcam_tbl_process(struct bnxt_ulp_mapper_parms *parms, if (alloc_tcam) { tfcp = bnxt_ulp_cntxt_tfcp_get(parms->ulp_ctx); - if (tfcp == NULL) { + if (unlikely(tfcp == NULL)) { PMD_DRV_LOG_LINE(ERR, "Failed to get tfcp pointer"); return -EINVAL; } /* calculate the entry priority*/ rc = ulp_mapper_priority_opc_process(parms, tbl, &priority); - if (rc) { + if (unlikely(rc)) { BNXT_DRV_DBG(ERR, "entry priority process failed\n"); return rc; } @@ -311,14 +312,14 @@ ulp_mapper_tfc_tcam_tbl_process(struct bnxt_ulp_mapper_parms *parms, rc = tfc_tcam_alloc(tfcp, fw_fid, tt, priority, key_sz_in_words, &tfc_inf); - if (rc) { + if (unlikely(rc)) { BNXT_DRV_DBG(ERR, "TCAM Alloc failed, status:%d\n", rc); return rc; } /* Write the tcam index into the regfile*/ - if (ulp_regfile_write(parms->regfile, tbl->tbl_operand, - (uint64_t)rte_cpu_to_be_64(tfc_inf.id))) { + if (unlikely(ulp_regfile_write(parms->regfile, tbl->tbl_operand, + (uint64_t)rte_cpu_to_be_64(tfc_inf.id)))) { BNXT_DRV_DBG(ERR, "Regfile[%d] write failed.\n", tbl->tbl_operand); /* Need to free the tcam idx, so goto error */ @@ -349,7 +350,7 @@ ulp_mapper_tfc_tcam_tbl_process(struct bnxt_ulp_mapper_parms *parms, ulp_flow_db_shared_session_set(&fid_parms, tbl->session_type); rc = ulp_mapper_fdb_opc_process(parms, tbl, &fid_parms); - if (rc) { + if (unlikely(rc)) { BNXT_DRV_DBG(ERR, "Failed to link resource to flow rc = %d\n", rc); /* Need to free the identifier, so goto error */ @@ -395,14 +396,14 @@ ulp_mapper_blob_block_swap(struct ulp_blob *blob, uint32_t block_sz) int i; /* Shouldn't happen since it is internal function, but check anyway */ - if (blob == NULL || !block_sz) { + if (unlikely(blob == NULL || !block_sz)) { BNXT_DRV_DBG(ERR, "Invalid arguments\n"); return -EINVAL; } pdata = ulp_blob_data_get(blob, &data_sz); data_sz = ULP_BITS_2_BYTE(data_sz); - if (!data_sz || (data_sz % block_sz) != 0) { + if (unlikely(!data_sz || (data_sz % block_sz) != 0)) { BNXT_DRV_DBG(ERR, "length(%d) not a multiple of %d\n", data_sz, block_sz); return -EINVAL; @@ -475,7 +476,7 @@ ulp_mapper_tfc_em_tbl_process(struct bnxt_ulp_mapper_parms *parms, int32_t rc = 0; tfcp = bnxt_ulp_cntxt_tfcp_get(parms->ulp_ctx); - if (tfcp == NULL) { + if (unlikely(tfcp == NULL)) { BNXT_DRV_DBG(ERR, "Failed to get tfcp pointer\n"); return -EINVAL; } @@ -484,15 +485,15 @@ ulp_mapper_tfc_em_tbl_process(struct bnxt_ulp_mapper_parms *parms, kflds = ulp_mapper_key_recipe_fields_get(parms, tbl, &num_kflds); else kflds = ulp_mapper_key_fields_get(parms, tbl, &num_kflds); - if (!kflds || !num_kflds) { + if (unlikely(!kflds || !num_kflds)) { BNXT_DRV_DBG(ERR, "Failed to get key fields\n"); return -EINVAL; } byte_order = dparms->em_byte_order; /* Initialize the key/result blobs */ - if (ulp_blob_init(&key, tbl->blob_key_bit_size, byte_order) || - ulp_blob_init(&data, tbl->result_bit_size, byte_order)) { + if (unlikely(ulp_blob_init(&key, tbl->blob_key_bit_size, byte_order) || + ulp_blob_init(&data, tbl->result_bit_size, byte_order))) { BNXT_DRV_DBG(ERR, "blob inits failed.\n"); return -EINVAL; } @@ -503,7 +504,7 @@ ulp_mapper_tfc_em_tbl_process(struct bnxt_ulp_mapper_parms *parms, rc = ulp_mapper_field_opc_process(parms, tbl->direction, &kflds[i].field_info_spec, &key, 1, "EM Key"); - if (rc) { + if (unlikely(rc)) { BNXT_DRV_DBG(ERR, "Key field set failed.\n"); return rc; } @@ -528,7 +529,7 @@ ulp_mapper_tfc_em_tbl_process(struct bnxt_ulp_mapper_parms *parms, #endif /* Create the result data blob */ rc = ulp_mapper_tbl_result_build(parms, tbl, &data, "EM Result"); - if (rc) { + if (unlikely(rc)) { BNXT_DRV_DBG(ERR, "Failed to build the result blob\n"); return rc; } @@ -542,7 +543,7 @@ ulp_mapper_tfc_em_tbl_process(struct bnxt_ulp_mapper_parms *parms, #endif #endif rc = ulp_blob_append(&key, &data, 0, dparms->em_blk_align_bits); - if (rc) { + if (unlikely(rc)) { BNXT_DRV_DBG(ERR, "EM Failed to append the result to key(%d)", rc); return rc; @@ -553,7 +554,7 @@ ulp_mapper_tfc_em_tbl_process(struct bnxt_ulp_mapper_parms *parms, rc = ulp_mapper_blob_block_swap(&key, ULP_BITS_2_BYTE(dparms->em_blk_size_bits)); /* Error printed within function, just return on error */ - if (rc) + if (unlikely(rc)) return rc; #ifdef RTE_LIBRTE_BNXT_TRUFLOW_DEBUG @@ -570,7 +571,7 @@ ulp_mapper_tfc_em_tbl_process(struct bnxt_ulp_mapper_parms *parms, iparms.batch_info = &parms->batch_info; rc = bnxt_ulp_cntxt_tsid_get(parms->ulp_ctx, &tsid); - if (rc) { + if (unlikely(rc)) { BNXT_DRV_DBG(ERR, "Failed to get the table scope\n"); return rc; } @@ -609,7 +610,7 @@ ulp_mapper_tfc_em_tbl_process(struct bnxt_ulp_mapper_parms *parms, uint64_t val = 0; /* hash collision */ - if (rc == -E2BIG) + if (unlikely(rc == -E2BIG)) BNXT_DRV_DBG(DEBUG, "Duplicate EM entry\n"); /* over max flows */ @@ -641,7 +642,7 @@ ulp_mapper_tfc_em_tbl_process(struct bnxt_ulp_mapper_parms *parms, #endif /* Mark action process */ rc = ulp_mapper_mark_gfid_process(parms, tbl, *iparms.flow_handle); - if (rc) { + if (unlikely(rc)) { BNXT_DRV_DBG(ERR, "Failed to add mark to flow\n"); goto error; } @@ -655,7 +656,7 @@ ulp_mapper_tfc_em_tbl_process(struct bnxt_ulp_mapper_parms *parms, fid_parms.resource_hndl = *iparms.flow_handle; rc = ulp_mapper_fdb_opc_process(parms, tbl, &fid_parms); - if (rc) { + if (unlikely(rc)) { BNXT_DRV_DBG(ERR, "Fail to link res to flow rc = %d\n", rc); /* Need to free the identifier, so goto error */ goto error; @@ -687,13 +688,13 @@ ulp_mapper_tfc_em_entry_free(struct bnxt_ulp_context *ulp, memset(&batch_info, 0, sizeof(batch_info)); - if (bnxt_ulp_cntxt_fid_get(ulp, &fw_fid)) { + if (unlikely(bnxt_ulp_cntxt_fid_get(ulp, &fw_fid))) { BNXT_DRV_DBG(ERR, "Failed to get func_id\n"); return -EINVAL; } tfcp = bnxt_ulp_cntxt_tfcp_get(ulp); - if (tfcp == NULL) { + if (unlikely(tfcp == NULL)) { BNXT_DRV_DBG(ERR, "Failed to get tfcp pointer\n"); return -EINVAL; } @@ -766,12 +767,12 @@ ulp_mapper_tfc_index_tbl_process(struct bnxt_ulp_mapper_parms *parms, uint16_t fw_fid = 0; tfcp = bnxt_ulp_cntxt_tfcp_get(parms->ulp_ctx); - if (tfcp == NULL) { + if (unlikely(tfcp == NULL)) { PMD_DRV_LOG_LINE(ERR, "Failed to get tfcp pointer"); return -EINVAL; } - if (bnxt_ulp_cntxt_fid_get(parms->ulp_ctx, &fw_fid)) { + if (unlikely(bnxt_ulp_cntxt_fid_get(parms->ulp_ctx, &fw_fid))) { BNXT_DRV_DBG(ERR, "Failed to get func id\n"); return -EINVAL; } @@ -780,8 +781,8 @@ ulp_mapper_tfc_index_tbl_process(struct bnxt_ulp_mapper_parms *parms, bit_size = ulp_mapper_tfc_dyn_blob_size_get(parms, tbl); /* Initialize the blob data */ - if (ulp_blob_init(&data, bit_size, - parms->device_params->result_byte_order)) { + if (unlikely(ulp_blob_init(&data, bit_size, + parms->device_params->result_byte_order))) { BNXT_DRV_DBG(ERR, "Failed to initialize index table blob\n"); return -EINVAL; } @@ -818,7 +819,7 @@ ulp_mapper_tfc_index_tbl_process(struct bnxt_ulp_mapper_parms *parms, write = true; break; case BNXT_ULP_INDEX_TBL_OPC_WR_GLB_REGFILE: - if (tbl->fdb_opcode != BNXT_ULP_FDB_OPC_NOP) { + if (unlikely(tbl->fdb_opcode != BNXT_ULP_FDB_OPC_NOP)) { BNXT_DRV_DBG(ERR, "Template error, wrong fdb opcode\n"); return -EINVAL; } @@ -826,10 +827,10 @@ ulp_mapper_tfc_index_tbl_process(struct bnxt_ulp_mapper_parms *parms, * get the index to write to from the global regfile and then * write the table. */ - if (ulp_mapper_glb_resource_read(parms->mapper_data, - tbl->direction, - tbl->tbl_operand, - ®val, &shared)) { + if (unlikely(ulp_mapper_glb_resource_read(parms->mapper_data, + tbl->direction, + tbl->tbl_operand, + ®val, &shared))) { BNXT_DRV_DBG(ERR, "Failed to get tbl idx from Glb RF[%d].\n", tbl->tbl_operand); @@ -846,8 +847,8 @@ ulp_mapper_tfc_index_tbl_process(struct bnxt_ulp_mapper_parms *parms, * with the index from the regfile, scan and store the * identifiers, and return. */ - if (ulp_regfile_read(parms->regfile, - tbl->tbl_operand, ®val)) { + if (unlikely(ulp_regfile_read(parms->regfile, + tbl->tbl_operand, ®val))) { BNXT_DRV_DBG(ERR, "Failed to get tbl idx from regfile[%d]\n", tbl->tbl_operand); @@ -864,7 +865,7 @@ ulp_mapper_tfc_index_tbl_process(struct bnxt_ulp_mapper_parms *parms, rc = tfc_idx_tbl_get(tfcp, fw_fid, &tbl_info, (uint32_t *)data_p, (uint8_t *)&wordlen); - if (rc) { + if (unlikely(rc)) { BNXT_DRV_DBG(ERR, "Failed to read the tbl entry %d:%d\n", tbl->resource_type, index); @@ -874,7 +875,7 @@ ulp_mapper_tfc_index_tbl_process(struct bnxt_ulp_mapper_parms *parms, /* Scan the fields in the entry and push them into the regfile*/ rc = ulp_mapper_tbl_ident_scan_ext(parms, tbl, data_p, wordlen, data.byte_order); - if (rc) { + if (unlikely(rc)) { BNXT_DRV_DBG(ERR, "Failed to get flds on tbl read rc=%d\n", rc); @@ -894,9 +895,9 @@ ulp_mapper_tfc_index_tbl_process(struct bnxt_ulp_mapper_parms *parms, /* read the CMM identifier from the regfile, it is not allocated */ if (!alloc && regfile) { - if (ulp_regfile_read(parms->regfile, - tbl->tbl_operand, - ®val)) { + if (unlikely(ulp_regfile_read(parms->regfile, + tbl->tbl_operand, + ®val))) { BNXT_DRV_DBG(ERR, "Failed to get tbl idx from regfile[%d].\n", tbl->tbl_operand); @@ -910,7 +911,7 @@ ulp_mapper_tfc_index_tbl_process(struct bnxt_ulp_mapper_parms *parms, tbl_info.dir = tbl->direction; tbl_info.rsubtype = tbl->resource_type; rc = tfc_idx_tbl_alloc(tfcp, fw_fid, tt, &tbl_info); - if (rc) { + if (unlikely(rc)) { BNXT_DRV_DBG(ERR, "Alloc table[%s][%s] failed rc=%d\n", tfc_idx_tbl_2_str(tbl_info.rsubtype), tfc_dir_2_str(tbl->direction), rc); @@ -934,7 +935,7 @@ ulp_mapper_tfc_index_tbl_process(struct bnxt_ulp_mapper_parms *parms, rc = ulp_mapper_glb_resource_write(parms->mapper_data, &glb_res, regval, false); - if (rc) { + if (unlikely(rc)) { BNXT_DRV_DBG(ERR, "Failed to write %s regfile[%d] rc=%d\n", (global) ? "global" : "reg", @@ -948,7 +949,7 @@ ulp_mapper_tfc_index_tbl_process(struct bnxt_ulp_mapper_parms *parms, regval = rte_cpu_to_be_64(index); rc = ulp_regfile_write(parms->regfile, tbl->tbl_operand, regval); - if (rc) { + if (unlikely(rc)) { BNXT_DRV_DBG(ERR, "Failed to write %s regfile[%d] rc=%d\n", (global) ? "global" : "reg", @@ -963,7 +964,7 @@ ulp_mapper_tfc_index_tbl_process(struct bnxt_ulp_mapper_parms *parms, tbl, &data, "Indexed Result"); - if (rc) { + if (unlikely(rc)) { BNXT_DRV_DBG(ERR, "Failed to build the result blob\n"); return rc; } @@ -974,7 +975,7 @@ ulp_mapper_tfc_index_tbl_process(struct bnxt_ulp_mapper_parms *parms, wordlen = ULP_BITS_2_BYTE(tmplen); rc = tfc_idx_tbl_set(tfcp, fw_fid, &tbl_info, (uint32_t *)data_p, wordlen); - if (rc) { + if (unlikely(rc)) { BNXT_DRV_DBG(ERR, "Index table[%s][%s][%x] write fail %d\n", tfc_idx_tbl_2_str(tbl_info.rsubtype), @@ -998,7 +999,7 @@ ulp_mapper_tfc_index_tbl_process(struct bnxt_ulp_mapper_parms *parms, ulp_flow_db_shared_session_set(&fid_parms, tbl->session_type); rc = ulp_mapper_fdb_opc_process(parms, tbl, &fid_parms); - if (rc) { + if (unlikely(rc)) { BNXT_DRV_DBG(ERR, "Failed to link resource to flow rc = %d\n", rc); goto error; @@ -1006,7 +1007,7 @@ ulp_mapper_tfc_index_tbl_process(struct bnxt_ulp_mapper_parms *parms, /* Perform the VF rep action */ rc = ulp_mapper_mark_vfr_idx_process(parms, tbl); - if (rc) { + if (unlikely(rc)) { BNXT_DRV_DBG(ERR, "Failed to add vfr mark rc = %d\n", rc); goto error; } @@ -1034,14 +1035,14 @@ ulp_mapper_tfc_index_entry_free(struct bnxt_ulp_context *ulp_ctx, uint16_t fw_fid = 0; int32_t rc; - if (bnxt_ulp_cntxt_fid_get(ulp_ctx, &fw_fid)) { + if (unlikely(bnxt_ulp_cntxt_fid_get(ulp_ctx, &fw_fid))) { BNXT_DRV_DBG(ERR, "Failed to get func_id\n"); return -EINVAL; } #ifndef ULP_MAPPER_TFC_TEST tfcp = bnxt_ulp_cntxt_tfcp_get(ulp_ctx); - if (tfcp == NULL) { + if (unlikely(tfcp == NULL)) { BNXT_DRV_DBG(ERR, "Failed to get tfcp pointer\n"); return -EINVAL; } @@ -1084,7 +1085,7 @@ ulp_mapper_tfc_cmm_tbl_process(struct bnxt_ulp_mapper_parms *parms, int32_t rc = 0; tfcp = bnxt_ulp_cntxt_tfcp_get(parms->ulp_ctx); - if (tfcp == NULL) { + if (unlikely(tfcp == NULL)) { PMD_DRV_LOG_LINE(ERR, "Failed to get tfcp pointer"); return -EINVAL; } @@ -1093,8 +1094,8 @@ ulp_mapper_tfc_cmm_tbl_process(struct bnxt_ulp_mapper_parms *parms, bit_size = ulp_mapper_tfc_dyn_blob_size_get(parms, tbl); /* Initialize the blob data */ - if (ulp_blob_init(&data, bit_size, - parms->device_params->result_byte_order)) { + if (unlikely(ulp_blob_init(&data, bit_size, + parms->device_params->result_byte_order))) { BNXT_DRV_DBG(ERR, "Failed to initialize cmm table blob\n"); return -EINVAL; } @@ -1131,7 +1132,7 @@ ulp_mapper_tfc_cmm_tbl_process(struct bnxt_ulp_mapper_parms *parms, write = true; break; case BNXT_ULP_INDEX_TBL_OPC_WR_GLB_REGFILE: - if (tbl->fdb_opcode != BNXT_ULP_FDB_OPC_NOP) { + if (unlikely(tbl->fdb_opcode != BNXT_ULP_FDB_OPC_NOP)) { BNXT_DRV_DBG(ERR, "Template error, wrong fdb opcode\n"); return -EINVAL; } @@ -1139,10 +1140,10 @@ ulp_mapper_tfc_cmm_tbl_process(struct bnxt_ulp_mapper_parms *parms, * get the index to write to from the global regfile and then * write the table. */ - if (ulp_mapper_glb_resource_read(parms->mapper_data, - tbl->direction, - tbl->tbl_operand, - ®val, &shared)) { + if (unlikely(ulp_mapper_glb_resource_read(parms->mapper_data, + tbl->direction, + tbl->tbl_operand, + ®val, &shared))) { BNXT_DRV_DBG(ERR, "Failed to get tbl idx from Glb RF[%d].\n", tbl->tbl_operand); @@ -1159,8 +1160,8 @@ ulp_mapper_tfc_cmm_tbl_process(struct bnxt_ulp_mapper_parms *parms, * with the index from the regfile, scan and store the * identifiers, and return. */ - if (ulp_regfile_read(parms->regfile, - tbl->tbl_operand, ®val)) { + if (unlikely(ulp_regfile_read(parms->regfile, + tbl->tbl_operand, ®val))) { BNXT_DRV_DBG(ERR, "Failed to get tbl idx from regfile[%d]\n", tbl->tbl_operand); @@ -1180,9 +1181,9 @@ ulp_mapper_tfc_cmm_tbl_process(struct bnxt_ulp_mapper_parms *parms, /* read the CMM handle from the regfile, it is not allocated */ if (!alloc && regfile) { - if (ulp_regfile_read(parms->regfile, + if (unlikely(ulp_regfile_read(parms->regfile, tbl->tbl_operand, - ®val)) { + ®val))) { BNXT_DRV_DBG(ERR, "Failed to get tbl idx from regfile[%d].\n", tbl->tbl_operand); @@ -1196,7 +1197,7 @@ ulp_mapper_tfc_cmm_tbl_process(struct bnxt_ulp_mapper_parms *parms, tbl, &data, "Indexed Result"); - if (rc) { + if (unlikely(rc)) { BNXT_DRV_DBG(ERR, "Failed to build the result blob\n"); return rc; } @@ -1210,13 +1211,13 @@ ulp_mapper_tfc_cmm_tbl_process(struct bnxt_ulp_mapper_parms *parms, act_wordlen = ULP_BITS_TO_32_BYTE_WORD(tmplen); rc = bnxt_ulp_cntxt_tsid_get(parms->ulp_ctx, &tsid); - if (rc) { + if (unlikely(rc)) { BNXT_DRV_DBG(ERR, "Failed to get the table scope\n"); return rc; } /* All failures after the alloc succeeds require a free */ rc = tfc_act_alloc(tfcp, tsid, &cmm_info, act_wordlen); - if (rc) { + if (unlikely(rc)) { BNXT_DRV_DBG(ERR, "Alloc CMM [%d][%s] failed rc=%d\n", cmm_info.rsubtype, tfc_dir_2_str(cmm_info.dir), rc); @@ -1232,7 +1233,7 @@ ulp_mapper_tfc_cmm_tbl_process(struct bnxt_ulp_mapper_parms *parms, &parms->batch_info, &cmm_info, act_data, act_wordlen); - if (rc) { + if (unlikely(rc)) { BNXT_DRV_DBG(ERR, "Stat alloc/clear[%d][%s]" "[0x%" PRIx64 "] failed rc=%d\n", cmm_info.rsubtype, @@ -1267,7 +1268,7 @@ ulp_mapper_tfc_cmm_tbl_process(struct bnxt_ulp_mapper_parms *parms, rc = ulp_mapper_glb_resource_write(parms->mapper_data, &glb_res, regval, false); - if (rc) { + if (unlikely(rc)) { BNXT_DRV_DBG(ERR, "Failed to write %s regfile[%d] rc=%d\n", (global) ? "global" : "reg", @@ -1281,7 +1282,7 @@ ulp_mapper_tfc_cmm_tbl_process(struct bnxt_ulp_mapper_parms *parms, regval = rte_cpu_to_be_64(handle); rc = ulp_regfile_write(parms->regfile, tbl->tbl_operand, regval); - if (rc) { + if (unlikely(rc)) { BNXT_DRV_DBG(ERR, "Failed to write %s regfile[%d] rc=%d\n", (global) ? "global" : "reg", @@ -1297,7 +1298,7 @@ ulp_mapper_tfc_cmm_tbl_process(struct bnxt_ulp_mapper_parms *parms, cmm_info.act_handle = handle; act_wordlen = ULP_BITS_TO_32_BYTE_WORD(tmplen); rc = tfc_act_set(tfcp, &parms->batch_info, &cmm_info, act_data, act_wordlen); - if (rc) { + if (unlikely(rc)) { BNXT_DRV_DBG(ERR, "CMM table[%d][%s][0x%" PRIx64 "] write fail %d\n", @@ -1343,7 +1344,7 @@ ulp_mapper_tfc_cmm_tbl_process(struct bnxt_ulp_mapper_parms *parms, ulp_flow_db_shared_session_set(&fid_parms, tbl->session_type); rc = ulp_mapper_fdb_opc_process(parms, tbl, &fid_parms); - if (rc) { + if (unlikely(rc)) { BNXT_DRV_DBG(ERR, "Failed to link resource to flow rc = %d\n", rc); goto error; @@ -1351,7 +1352,7 @@ ulp_mapper_tfc_cmm_tbl_process(struct bnxt_ulp_mapper_parms *parms, /* Perform the VF rep action */ rc = ulp_mapper_mark_vfr_idx_process(parms, tbl); - if (rc) { + if (unlikely(rc)) { BNXT_DRV_DBG(ERR, "Failed to add vfr mark rc = %d\n", rc); goto error; } @@ -1385,13 +1386,13 @@ ulp_mapper_tfc_cmm_entry_free(struct bnxt_ulp_context *ulp_ctx, if (res->fdb_flags & ULP_FDB_FLAG_SW_ONLY) return 0; - if (bnxt_ulp_cntxt_fid_get(ulp_ctx, &fw_fid)) { + if (unlikely(bnxt_ulp_cntxt_fid_get(ulp_ctx, &fw_fid))) { BNXT_DRV_DBG(ERR, "Failed to get func_id\n"); return -EINVAL; } tfcp = bnxt_ulp_cntxt_tfcp_get(ulp_ctx); - if (tfcp == NULL) { + if (unlikely(tfcp == NULL)) { BNXT_DRV_DBG(ERR, "Failed to get tfcp pointer\n"); return -EINVAL; } @@ -1402,7 +1403,7 @@ ulp_mapper_tfc_cmm_entry_free(struct bnxt_ulp_context *ulp_ctx, /* TBD: check to see if the memory needs to be cleaned as well*/ rc = tfc_act_free(tfcp, &cmm_info); - if (rc) { + if (unlikely(rc)) { BNXT_DRV_DBG(ERR, "Failed to delete CMM entry,res = 0x%" PRIx64 "\n", res->resource_hndl); @@ -1426,7 +1427,7 @@ ulp_mapper_tfc_if_tbl_process(struct bnxt_ulp_mapper_parms *parms, struct bnxt_ulp_mapper_tbl_info *tbl) { struct ulp_blob data, res_blob; - uint64_t idx; + uint64_t idx = 0; int32_t rc = 0; struct tfc *tfcp; enum bnxt_ulp_if_tbl_opc if_opc = tbl->tbl_opcode; @@ -1437,27 +1438,27 @@ ulp_mapper_tfc_if_tbl_process(struct bnxt_ulp_mapper_parms *parms, uint8_t data_size; uint16_t tmplen; - if (bnxt_ulp_cntxt_fid_get(parms->ulp_ctx, &fw_fid)) { + if (unlikely(bnxt_ulp_cntxt_fid_get(parms->ulp_ctx, &fw_fid))) { BNXT_DRV_DBG(ERR, "Failed to get func_id\n"); return -EINVAL; } tfcp = bnxt_ulp_cntxt_tfcp_get(parms->ulp_ctx); - if (tfcp == NULL) { + if (unlikely(tfcp == NULL)) { PMD_DRV_LOG_LINE(ERR, "Failed to get tfcp pointer"); return -EINVAL; } /* Initialize the blob data */ - if (ulp_blob_init(&data, tbl->result_bit_size, - parms->device_params->result_byte_order)) { + if (unlikely(ulp_blob_init(&data, tbl->result_bit_size, + parms->device_params->result_byte_order))) { BNXT_DRV_DBG(ERR, "Failed initial index table blob\n"); return -EINVAL; } /* create the result blob */ rc = ulp_mapper_tbl_result_build(parms, tbl, &data, "IFtable Result"); - if (rc) { + if (unlikely(rc)) { BNXT_DRV_DBG(ERR, "Failed to build the result blob\n"); return rc; } @@ -1468,7 +1469,7 @@ ulp_mapper_tfc_if_tbl_process(struct bnxt_ulp_mapper_parms *parms, idx = ULP_COMP_FLD_IDX_RD(parms, tbl->tbl_operand); break; case BNXT_ULP_IF_TBL_OPC_WR_REGFILE: - if (ulp_regfile_read(parms->regfile, tbl->tbl_operand, &idx)) { + if (unlikely(ulp_regfile_read(parms->regfile, tbl->tbl_operand, &idx))) { BNXT_DRV_DBG(ERR, "regfile[%d] read oob\n", tbl->tbl_operand); return -EINVAL; @@ -1480,8 +1481,8 @@ ulp_mapper_tfc_if_tbl_process(struct bnxt_ulp_mapper_parms *parms, break; case BNXT_ULP_IF_TBL_OPC_RD_COMP_FIELD: /* Initialize the result blob */ - if (ulp_blob_init(&res_blob, tbl->result_bit_size, - parms->device_params->result_byte_order)) { + if (unlikely(ulp_blob_init(&res_blob, tbl->result_bit_size, + parms->device_params->result_byte_order))) { BNXT_DRV_DBG(ERR, "Failed initial result blob\n"); return -EINVAL; } @@ -1493,7 +1494,7 @@ ulp_mapper_tfc_if_tbl_process(struct bnxt_ulp_mapper_parms *parms, res_blob.data, res_size, res_blob.byte_order); - if (rc) + if (unlikely(rc)) BNXT_DRV_DBG(ERR, "Scan and extract failed rc=%d\n", rc); return rc; @@ -1512,7 +1513,7 @@ ulp_mapper_tfc_if_tbl_process(struct bnxt_ulp_mapper_parms *parms, rc = tfc_if_tbl_set(tfcp, fw_fid, &tbl_info, (uint8_t *)data_p, data_size); - if (rc) { + if (unlikely(rc)) { BNXT_DRV_DBG(ERR, "Failed to write the if tbl entry %d:%d\n", tbl->resource_type, (uint32_t)idx); @@ -1535,13 +1536,13 @@ ulp_mapper_tfc_ident_alloc(struct bnxt_ulp_context *ulp_ctx, int32_t rc = 0; uint16_t fw_fid = 0; - if (bnxt_ulp_cntxt_fid_get(ulp_ctx, &fw_fid)) { + if (unlikely(bnxt_ulp_cntxt_fid_get(ulp_ctx, &fw_fid))) { BNXT_DRV_DBG(ERR, "Failed to get func_id\n"); return -EINVAL; } tfcp = bnxt_ulp_cntxt_tfcp_get(ulp_ctx); - if (tfcp == NULL) { + if (unlikely(tfcp == NULL)) { BNXT_DRV_DBG(ERR, "Failed to get tfcp pointer\n"); return -EINVAL; } @@ -1550,7 +1551,7 @@ ulp_mapper_tfc_ident_alloc(struct bnxt_ulp_context *ulp_ctx, ident_info.rsubtype = ident_type; rc = tfc_identifier_alloc(tfcp, fw_fid, tt, &ident_info); - if (rc != 0) { + if (unlikely(rc != 0)) { BNXT_DRV_DBG(ERR, "alloc failed %d\n", rc); return rc; } @@ -1575,13 +1576,13 @@ ulp_mapper_tfc_ident_free(struct bnxt_ulp_context *ulp_ctx, int32_t rc = 0; uint16_t fw_fid = 0; - if (bnxt_ulp_cntxt_fid_get(ulp_ctx, &fw_fid)) { + if (unlikely(bnxt_ulp_cntxt_fid_get(ulp_ctx, &fw_fid))) { BNXT_DRV_DBG(ERR, "Failed to get func_id\n"); return -EINVAL; } tfcp = bnxt_ulp_cntxt_tfcp_get(ulp_ctx); - if (tfcp == NULL) { + if (unlikely(tfcp == NULL)) { BNXT_DRV_DBG(ERR, "Failed to get tfcp pointer\n"); return -EINVAL; } @@ -1591,7 +1592,7 @@ ulp_mapper_tfc_ident_free(struct bnxt_ulp_context *ulp_ctx, ident_info.id = res->resource_hndl; rc = tfc_identifier_free(tfcp, fw_fid, &ident_info); - if (rc != 0) { + if (unlikely(rc != 0)) { BNXT_DRV_DBG(ERR, "free failed %d\n", rc); return rc; } @@ -1614,13 +1615,13 @@ ulp_mapper_tfc_tcam_entry_free(struct bnxt_ulp_context *ulp, struct tfc_tcam_info tcam_info = { 0 }; uint16_t fw_fid = 0; - if (bnxt_ulp_cntxt_fid_get(ulp, &fw_fid)) { + if (unlikely(bnxt_ulp_cntxt_fid_get(ulp, &fw_fid))) { BNXT_DRV_DBG(ERR, "Failed to get func_id\n"); return -EINVAL; } tfcp = bnxt_ulp_cntxt_tfcp_get(ulp); - if (tfcp == NULL) { + if (unlikely(tfcp == NULL)) { PMD_DRV_LOG_LINE(ERR, "Failed to get tfcp pointer"); return -EINVAL; } @@ -1628,7 +1629,7 @@ ulp_mapper_tfc_tcam_entry_free(struct bnxt_ulp_context *ulp, tcam_info.rsubtype = res->resource_type; tcam_info.id = (uint16_t)res->resource_hndl; - if (!tfcp || tfc_tcam_free(tfcp, fw_fid, &tcam_info)) { + if (unlikely(!tfcp || tfc_tcam_free(tfcp, fw_fid, &tcam_info))) { BNXT_DRV_DBG(ERR, "Unable to free tcam resource %u\n", tcam_info.id); return -EINVAL; @@ -1677,12 +1678,12 @@ ulp_mapper_tfc_index_tbl_alloc_process(struct bnxt_ulp_context *ulp, int32_t rc = 0; tfcp = bnxt_ulp_cntxt_tfcp_get(ulp); - if (tfcp == NULL) { + if (unlikely(tfcp == NULL)) { BNXT_DRV_DBG(ERR, "Failed to get tfcp pointer\n"); return -EINVAL; } - if (bnxt_ulp_cntxt_fid_get(ulp, &fw_fid)) { + if (unlikely(bnxt_ulp_cntxt_fid_get(ulp, &fw_fid))) { BNXT_DRV_DBG(ERR, "Failed to get func id\n"); return -EINVAL; } @@ -1690,7 +1691,7 @@ ulp_mapper_tfc_index_tbl_alloc_process(struct bnxt_ulp_context *ulp, tbl_info.rsubtype = table_type; tbl_info.dir = direction; rc = tfc_idx_tbl_alloc(tfcp, fw_fid, tt, &tbl_info); - if (rc) { + if (unlikely(rc)) { BNXT_DRV_DBG(ERR, "Alloc table[%s][%s] failed rc=%d\n", tfc_idx_tbl_2_str(tbl_info.rsubtype), tfc_dir_2_str(direction), rc); diff --git a/drivers/net/bnxt/tf_ulp/ulp_mark_mgr.c b/drivers/net/bnxt/tf_ulp/ulp_mark_mgr.c index 4e892bff46..198a2b95fa 100644 --- a/drivers/net/bnxt/tf_ulp/ulp_mark_mgr.c +++ b/drivers/net/bnxt/tf_ulp/ulp_mark_mgr.c @@ -8,6 +8,7 @@ #include #include "bnxt.h" #include "bnxt_ulp.h" +#include "bnxt_ulp_utils.h" #include "tf_ext_flow_handle.h" #include "ulp_mark_mgr.h" #include "bnxt_tf_common.h" diff --git a/drivers/net/bnxt/tf_ulp/ulp_matcher.c b/drivers/net/bnxt/tf_ulp/ulp_matcher.c index dbb4f640fe..4b2e185278 100644 --- a/drivers/net/bnxt/tf_ulp/ulp_matcher.c +++ b/drivers/net/bnxt/tf_ulp/ulp_matcher.c @@ -7,6 +7,7 @@ #include "ulp_matcher.h" #include "ulp_mapper.h" #include "ulp_utils.h" +#include "bnxt_ulp_utils.h" #ifndef RTE_HASH_BUCKET_ENTRIES /* it is defined in lib/hash/rte_cuckoo_hash.h */ diff --git a/drivers/net/bnxt/tf_ulp/ulp_port_db.c b/drivers/net/bnxt/tf_ulp/ulp_port_db.c index 8471fcc46c..384b89da46 100644 --- a/drivers/net/bnxt/tf_ulp/ulp_port_db.c +++ b/drivers/net/bnxt/tf_ulp/ulp_port_db.c @@ -7,6 +7,7 @@ #include "bnxt.h" #include "bnxt_vnic.h" #include "bnxt_tf_common.h" +#include "bnxt_ulp_utils.h" #include "bnxt_tf_pmd_shim.h" #include "ulp_port_db.h" #include "tfp.h" diff --git a/drivers/net/bnxt/tf_ulp/ulp_rte_parser.c b/drivers/net/bnxt/tf_ulp/ulp_rte_parser.c index 69570c5ef4..72d3514647 100644 --- a/drivers/net/bnxt/tf_ulp/ulp_rte_parser.c +++ b/drivers/net/bnxt/tf_ulp/ulp_rte_parser.c @@ -7,6 +7,7 @@ #include "ulp_template_db_enum.h" #include "ulp_template_struct.h" #include "bnxt_ulp.h" +#include "bnxt_ulp_utils.h" #include "bnxt_tf_common.h" #include "bnxt_tf_pmd_shim.h" #include "ulp_rte_parser.h" @@ -57,7 +58,7 @@ ulp_rte_item_skip_void(const struct rte_flow_item **item, uint32_t increment) } /* Utility function to copy field spec items */ -static struct ulp_rte_hdr_field * +static inline struct ulp_rte_hdr_field * ulp_rte_parser_fld_copy(struct ulp_rte_hdr_field *field, const void *buffer, uint32_t size) @@ -92,7 +93,7 @@ ulp_rte_parser_field_bitmap_update(struct ulp_rte_parser_params *params, #define ulp_deference_struct(x, y) ((x) ? &((x)->y) : NULL) /* Utility function to copy field spec and masks items */ -static void +static inline void ulp_rte_prsr_fld_mask(struct ulp_rte_parser_params *params, uint32_t *idx, uint32_t size, @@ -121,12 +122,12 @@ ulp_rte_prsr_fld_mask(struct ulp_rte_parser_params *params, } /* Utility function to copy field spec and masks items */ -static int32_t +static inline int32_t ulp_rte_prsr_fld_size_validate(struct ulp_rte_parser_params *params, uint32_t *idx, uint32_t size) { - if (params->field_idx + size >= BNXT_ULP_PROTO_HDR_MAX) { + if (unlikely(params->field_idx + size >= BNXT_ULP_PROTO_HDR_MAX)) { BNXT_DRV_DBG(ERR, "OOB for field processing %u\n", *idx); return -EINVAL; } @@ -740,8 +741,8 @@ ulp_rte_eth_hdr_handler(const struct rte_flow_item *item, has_vlan_mask = eth_mask->has_vlan; } - if (ulp_rte_prsr_fld_size_validate(params, &idx, - BNXT_ULP_PROTO_HDR_ETH_NUM)) { + if (unlikely(ulp_rte_prsr_fld_size_validate(params, &idx, + BNXT_ULP_PROTO_HDR_ETH_NUM))) { BNXT_DRV_DBG(ERR, "Error parsing protocol header\n"); return BNXT_TF_RC_ERROR; } @@ -840,8 +841,8 @@ ulp_rte_vlan_hdr_handler(const struct rte_flow_item *item, vlan_tag_mask = htons(vlan_tag_mask); } - if (ulp_rte_prsr_fld_size_validate(params, &idx, - BNXT_ULP_PROTO_HDR_S_VLAN_NUM)) { + if (unlikely(ulp_rte_prsr_fld_size_validate(params, &idx, + BNXT_ULP_PROTO_HDR_S_VLAN_NUM))) { BNXT_DRV_DBG(ERR, "Error parsing protocol header\n"); return BNXT_TF_RC_ERROR; } @@ -1026,8 +1027,8 @@ ulp_rte_ipv4_hdr_handler(const struct rte_flow_item *item, return BNXT_TF_RC_ERROR; } - if (ulp_rte_prsr_fld_size_validate(params, &idx, - BNXT_ULP_PROTO_HDR_IPV4_NUM)) { + if (unlikely(ulp_rte_prsr_fld_size_validate(params, &idx, + BNXT_ULP_PROTO_HDR_IPV4_NUM))) { BNXT_DRV_DBG(ERR, "Error parsing protocol header\n"); return BNXT_TF_RC_ERROR; } @@ -1178,8 +1179,8 @@ ulp_rte_ipv6_hdr_handler(const struct rte_flow_item *item, return BNXT_TF_RC_ERROR; } - if (ulp_rte_prsr_fld_size_validate(params, &idx, - BNXT_ULP_PROTO_HDR_IPV6_NUM)) { + if (unlikely(ulp_rte_prsr_fld_size_validate(params, &idx, + BNXT_ULP_PROTO_HDR_IPV6_NUM))) { BNXT_DRV_DBG(ERR, "Error parsing protocol header\n"); return BNXT_TF_RC_ERROR; } @@ -1421,8 +1422,8 @@ ulp_rte_udp_hdr_handler(const struct rte_flow_item *item, dport_mask = udp_mask->hdr.dst_port; } - if (ulp_rte_prsr_fld_size_validate(params, &idx, - BNXT_ULP_PROTO_HDR_UDP_NUM)) { + if (unlikely(ulp_rte_prsr_fld_size_validate(params, &idx, + BNXT_ULP_PROTO_HDR_UDP_NUM))) { BNXT_DRV_DBG(ERR, "Error parsing protocol header\n"); return BNXT_TF_RC_ERROR; } @@ -1501,8 +1502,8 @@ ulp_rte_tcp_hdr_handler(const struct rte_flow_item *item, dport_mask = tcp_mask->hdr.dst_port; } - if (ulp_rte_prsr_fld_size_validate(params, &idx, - BNXT_ULP_PROTO_HDR_TCP_NUM)) { + if (unlikely(ulp_rte_prsr_fld_size_validate(params, &idx, + BNXT_ULP_PROTO_HDR_TCP_NUM))) { BNXT_DRV_DBG(ERR, "Error parsing protocol header\n"); return BNXT_TF_RC_ERROR; } @@ -1590,8 +1591,8 @@ ulp_rte_vxlan_hdr_handler(const struct rte_flow_item *item, uint16_t dport, stat_port; uint32_t size; - if (ulp_rte_prsr_fld_size_validate(params, &idx, - BNXT_ULP_PROTO_HDR_VXLAN_NUM)) { + if (unlikely(ulp_rte_prsr_fld_size_validate(params, &idx, + BNXT_ULP_PROTO_HDR_VXLAN_NUM))) { BNXT_DRV_DBG(ERR, "Error parsing protocol header\n"); return BNXT_TF_RC_ERROR; } @@ -1700,8 +1701,8 @@ ulp_rte_vxlan_gpe_hdr_handler(const struct rte_flow_item *item, uint16_t dport; uint32_t size; - if (ulp_rte_prsr_fld_size_validate(params, &idx, - BNXT_ULP_PROTO_HDR_VXLAN_GPE_NUM)) { + if (unlikely(ulp_rte_prsr_fld_size_validate(params, &idx, + BNXT_ULP_PROTO_HDR_VXLAN_GPE_NUM))) { BNXT_DRV_DBG(ERR, "Error parsing protocol header\n"); return BNXT_TF_RC_ERROR; } @@ -1780,8 +1781,8 @@ ulp_rte_geneve_hdr_handler(const struct rte_flow_item *item, uint16_t dport; uint32_t size; - if (ulp_rte_prsr_fld_size_validate(params, &idx, - BNXT_ULP_PROTO_HDR_GENEVE_NUM)) { + if (unlikely(ulp_rte_prsr_fld_size_validate(params, &idx, + BNXT_ULP_PROTO_HDR_GENEVE_NUM))) { BNXT_DRV_DBG(ERR, "Error parsing protocol header\n"); return BNXT_TF_RC_ERROR; } @@ -1862,8 +1863,8 @@ ulp_rte_gre_hdr_handler(const struct rte_flow_item *item, uint32_t idx = 0; uint32_t size; - if (ulp_rte_prsr_fld_size_validate(params, &idx, - BNXT_ULP_PROTO_HDR_GRE_NUM)) { + if (unlikely(ulp_rte_prsr_fld_size_validate(params, &idx, + BNXT_ULP_PROTO_HDR_GRE_NUM))) { BNXT_DRV_DBG(ERR, "Error parsing protocol header\n"); return BNXT_TF_RC_ERROR; } @@ -1908,8 +1909,8 @@ ulp_rte_icmp_hdr_handler(const struct rte_flow_item *item, uint32_t idx = 0; uint32_t size; - if (ulp_rte_prsr_fld_size_validate(params, &idx, - BNXT_ULP_PROTO_HDR_ICMP_NUM)) { + if (unlikely(ulp_rte_prsr_fld_size_validate(params, &idx, + BNXT_ULP_PROTO_HDR_ICMP_NUM))) { BNXT_DRV_DBG(ERR, "Error parsing protocol header\n"); return BNXT_TF_RC_ERROR; } @@ -1966,8 +1967,8 @@ ulp_rte_icmp6_hdr_handler(const struct rte_flow_item *item, uint32_t idx = 0; uint32_t size; - if (ulp_rte_prsr_fld_size_validate(params, &idx, - BNXT_ULP_PROTO_HDR_ICMP_NUM)) { + if (unlikely(ulp_rte_prsr_fld_size_validate(params, &idx, + BNXT_ULP_PROTO_HDR_ICMP_NUM))) { BNXT_DRV_DBG(ERR, "Error parsing protocol header\n"); return BNXT_TF_RC_ERROR; } @@ -2020,8 +2021,8 @@ ulp_rte_ecpri_hdr_handler(const struct rte_flow_item *item, uint32_t idx = 0, cnt; uint32_t size; - if (ulp_rte_prsr_fld_size_validate(params, &idx, - BNXT_ULP_PROTO_HDR_ECPRI_NUM)) { + if (unlikely(ulp_rte_prsr_fld_size_validate(params, &idx, + BNXT_ULP_PROTO_HDR_ECPRI_NUM))) { BNXT_DRV_DBG(ERR, "Error parsing protocol header\n"); return BNXT_TF_RC_ERROR; } @@ -3312,18 +3313,18 @@ ulp_rte_action_hdlr_handler(const struct rte_flow_action *action_item, /* direction of shared action must match direction of flow */ ret = bnxt_get_action_handle_direction(handle, &handle_dir); - if (ret || dir != handle_dir) { + if (unlikely(ret || dir != handle_dir)) { BNXT_DRV_DBG(ERR, "Invalid shared handle or direction\n"); return BNXT_TF_RC_ERROR; } - if (bnxt_get_action_handle_type(handle, &shared_action_type)) { + if (unlikely(bnxt_get_action_handle_type(handle, &shared_action_type))) { BNXT_DRV_DBG(ERR, "Invalid shared handle\n"); return BNXT_TF_RC_ERROR; } act_info = bnxt_ulp_shared_act_info_get(&act_info_entries); - if (shared_action_type >= act_info_entries || !act_info) { + if (unlikely(shared_action_type >= act_info_entries || !act_info)) { BNXT_DRV_DBG(ERR, "Invalid shared handle\n"); return BNXT_TF_RC_ERROR; } @@ -3331,7 +3332,7 @@ ulp_rte_action_hdlr_handler(const struct rte_flow_action *action_item, action_bitmask = act_info[shared_action_type].act_bitmask; /* shared actions of the same type cannot be repeated */ - if (params->act_bitmap.bits & action_bitmask) { + if (unlikely(params->act_bitmap.bits & action_bitmask)) { BNXT_DRV_DBG(ERR, "indirect actions cannot be repeated\n"); return BNXT_TF_RC_ERROR; } @@ -3400,7 +3401,7 @@ ulp_rte_meter_act_handler(const struct rte_flow_action *action_item, struct ulp_rte_act_prop *act_prop = ¶ms->act_prop; uint32_t tmp_meter_id; - if (action_item == NULL || action_item->conf == NULL) { + if (unlikely(action_item == NULL || action_item->conf == NULL)) { BNXT_DRV_DBG(ERR, "Parse Err: invalid meter configuration\n"); return BNXT_TF_RC_ERROR; } @@ -3427,7 +3428,7 @@ ulp_rte_set_mac_src_act_handler(const struct rte_flow_action *action_item, struct ulp_rte_act_prop *act = ¶ms->act_prop; set_mac = action_item->conf; - if (set_mac) { + if (likely(set_mac)) { memcpy(&act->act_details[BNXT_ULP_ACT_PROP_IDX_SET_MAC_SRC], set_mac->mac_addr, BNXT_ULP_ACT_PROP_SZ_SET_MAC_SRC); /* Update the hdr_bitmap with set mac src */ @@ -3448,7 +3449,7 @@ ulp_rte_set_mac_dst_act_handler(const struct rte_flow_action *action_item, struct ulp_rte_act_prop *act = ¶ms->act_prop; set_mac = action_item->conf; - if (set_mac) { + if (likely(set_mac)) { memcpy(&act->act_details[BNXT_ULP_ACT_PROP_IDX_SET_MAC_DST], set_mac->mac_addr, BNXT_ULP_ACT_PROP_SZ_SET_MAC_DST); /* Update the hdr_bitmap with set ipv4 dst */ diff --git a/drivers/net/bnxt/tf_ulp/ulp_tun.c b/drivers/net/bnxt/tf_ulp/ulp_tun.c index 345e1d6459..7ebe81c4ea 100644 --- a/drivers/net/bnxt/tf_ulp/ulp_tun.c +++ b/drivers/net/bnxt/tf_ulp/ulp_tun.c @@ -5,6 +5,7 @@ #include "bnxt.h" #include "bnxt_ulp.h" +#include "bnxt_ulp_utils.h" #include "ulp_tun.h" #include "ulp_utils.h" diff --git a/drivers/net/bnxt/tf_ulp/ulp_utils.h b/drivers/net/bnxt/tf_ulp/ulp_utils.h index f0dc6dbfec..7a8145b5df 100644 --- a/drivers/net/bnxt/tf_ulp/ulp_utils.h +++ b/drivers/net/bnxt/tf_ulp/ulp_utils.h @@ -110,22 +110,43 @@ struct ulp_regfile { * * returns zero on success */ -int32_t -ulp_regfile_init(struct ulp_regfile *regfile); +static inline int32_t +ulp_regfile_init(struct ulp_regfile *regfile) +{ + /* validate the arguments */ + if (!regfile) { + BNXT_DRV_DBG(ERR, "invalid argument\n"); + return -EINVAL; + } + memset(regfile, 0, sizeof(struct ulp_regfile)); + return 0; /* Success */ +} /* * Read a value from the regfile * - * regfile [in] The regfile instance. Must be initialized prior to being used + * regfile [in] The regfile instance. Must be initialized prior to being used * * field [in] The field to be read within the regfile. * + * data [in/out] + * * returns zero on success */ -int32_t +static inline int32_t ulp_regfile_read(struct ulp_regfile *regfile, enum bnxt_ulp_rf_idx field, - uint64_t *data); + uint64_t *data) +{ + /* validate the arguments */ + if (unlikely(!regfile || field >= BNXT_ULP_RF_IDX_LAST)) { + BNXT_DRV_DBG(ERR, "invalid argument\n"); + return -EINVAL; + } + + *data = regfile->entry[field].data; + return 0; +} /* * Write a value to the regfile @@ -137,12 +158,115 @@ ulp_regfile_read(struct ulp_regfile *regfile, * data [in] The value is written into this variable. It is going to be in the * same byte order as it was written. * - * returns zero on success + * size [in] The size in bytes of the value being written into this + * variable. + * + * returns 0 on success */ -int32_t +static inline int32_t ulp_regfile_write(struct ulp_regfile *regfile, enum bnxt_ulp_rf_idx field, - uint64_t data); + uint64_t data) +{ + /* validate the arguments */ + if (unlikely(!regfile || field >= BNXT_ULP_RF_IDX_LAST)) { + BNXT_DRV_DBG(ERR, "invalid argument\n"); + return -EINVAL; /* failure */ + } + + regfile->entry[field].data = data; + return 0; /* Success */ +} + + +/* + * Add data to the byte array in Big endian format. + * + * bs [in] The byte array where data is pushed + * + * bitpos [in] The offset where data is pushed + * + * bitlen [in] The number of bits to be added to the data array. + * + * val [in] The data to be added to the data array. + * + */ +static inline void +ulp_bs_put_msb(uint8_t *bs, uint16_t bitpos, uint8_t bitlen, uint8_t val) +{ + uint8_t bitoffs = bitpos % 8; + uint16_t index = bitpos / 8; + uint8_t mask; + uint8_t tmp; + int8_t shift; + + tmp = bs[index]; + mask = ((uint8_t)-1 >> (8 - bitlen)); + shift = 8 - bitoffs - bitlen; + val &= mask; + + if (shift >= 0) { + tmp &= ~(mask << shift); + tmp |= val << shift; + bs[index] = tmp; + } else { + tmp &= ~((uint8_t)-1 >> bitoffs); + tmp |= val >> -shift; + bs[index++] = tmp; + + tmp = bs[index]; + tmp &= ((uint8_t)-1 >> (bitlen - (8 - bitoffs))); + tmp |= val << (8 + shift); + bs[index] = tmp; + } +} + +/* + * Add data to the byte array in Little endian format. + * + * bs [in] The byte array where data is pushed + * + * bitpos [in] The offset where data is pushed + * + * bitlen [in] The number of bits to be added to the data array. + * + * val [in] The data to be added to the data array. + * + */ +static inline void +ulp_bs_put_lsb(uint8_t *bs, uint16_t bitpos, uint8_t bitlen, uint8_t val) +{ + uint8_t bitoffs = bitpos % 8; + uint16_t index = bitpos / 8; + uint8_t mask; + uint8_t tmp; + uint8_t shift; + uint8_t partial; + + tmp = bs[index]; + shift = bitoffs; + + if (bitoffs + bitlen <= 8) { + mask = ((1 << bitlen) - 1) << shift; + tmp &= ~mask; + tmp |= ((val << shift) & mask); + bs[index] = tmp; + } else { + partial = 8 - bitoffs; + mask = ((1 << partial) - 1) << shift; + tmp &= ~mask; + tmp |= ((val << shift) & mask); + bs[index++] = tmp; + + val >>= partial; + partial = bitlen - partial; + mask = ((1 << partial) - 1); + tmp = bs[index]; + tmp &= ~mask; + tmp |= (val & mask); + bs[index] = tmp; + } +} /* * Add data to the byte array in Little endian format. @@ -157,8 +281,27 @@ ulp_regfile_write(struct ulp_regfile *regfile, * * returns the number of bits pushed. */ -uint32_t -ulp_bs_push_lsb(uint8_t *bs, uint16_t pos, uint8_t len, uint8_t *val); +static inline uint32_t +ulp_bs_push_lsb(uint8_t *bs, uint16_t pos, uint8_t len, uint8_t *val) +{ + int i; + int cnt = (len) / 8; + int tlen = len; + + if (cnt > 0 && !(len % 8)) + cnt -= 1; + + for (i = 0; i < cnt; i++) { + ulp_bs_put_lsb(bs, pos, 8, val[cnt - i]); + pos += 8; + tlen -= 8; + } + + /* Handle the remainder bits */ + if (tlen) + ulp_bs_put_lsb(bs, pos, tlen, val[0]); + return len; +} /* * Add data to the byte array in Big endian format. @@ -173,8 +316,29 @@ ulp_bs_push_lsb(uint8_t *bs, uint16_t pos, uint8_t len, uint8_t *val); * * returns the number of bits pushed. */ -uint32_t -ulp_bs_push_msb(uint8_t *bs, uint16_t pos, uint8_t len, uint8_t *val); +static inline uint32_t +ulp_bs_push_msb(uint8_t *bs, uint16_t pos, uint8_t len, uint8_t *val) +{ + int i; + int cnt = (len + 7) / 8; + + /* Handle any remainder bits */ + int tmp = len % 8; + + if (!tmp) + tmp = 8; + + ulp_bs_put_msb(bs, pos, tmp, val[0]); + + pos += tmp; + + for (i = 1; i < cnt; i++) { + ulp_bs_put_msb(bs, pos, 8, val[i]); + pos += 8; + } + + return len; +} /* * Initializes the blob structure for creating binary blob @@ -187,11 +351,27 @@ ulp_bs_push_msb(uint8_t *bs, uint16_t pos, uint8_t len, uint8_t *val); * big endian. All fields are packed with this order. * * returns zero on success + * Notes - If bitlen is zero then set it to max. */ -int32_t +static inline int32_t ulp_blob_init(struct ulp_blob *blob, uint16_t bitlen, - enum bnxt_ulp_byte_order order); + enum bnxt_ulp_byte_order order) +{ + /* validate the arguments */ + if (unlikely(!blob || bitlen > (8 * sizeof(blob->data)))) { + BNXT_DRV_DBG(ERR, "invalid argument\n"); + return -EINVAL; + } + if (bitlen) + blob->bitlen = bitlen; + else + blob->bitlen = BNXT_ULP_FLMP_BLOB_SIZE_IN_BITS; + blob->byte_order = order; + blob->write_idx = 0; + memset(blob->data, 0, sizeof(blob->data)); + return 0; /* Success */ +} /* * Add data to the binary blob at the current offset. @@ -207,10 +387,39 @@ ulp_blob_init(struct ulp_blob *blob, * * returns zero on success */ -int32_t +#define ULP_BLOB_BYTE 8 +#define ULP_BLOB_BYTE_HEX 0xFF +#define BLOB_MASK_CAL(x) ((0xFF << (x)) & 0xFF) +static inline int32_t ulp_blob_push(struct ulp_blob *blob, uint8_t *data, - uint32_t datalen); + uint32_t datalen) +{ + uint32_t rc; + + /* validate the arguments */ + if (unlikely(!blob || datalen > (uint32_t)(blob->bitlen - blob->write_idx))) { + BNXT_DRV_DBG(ERR, "invalid argument\n"); + return -EINVAL; + } + + if (blob->byte_order == BNXT_ULP_BYTE_ORDER_BE) + rc = ulp_bs_push_msb(blob->data, + blob->write_idx, + datalen, + data); + else + rc = ulp_bs_push_lsb(blob->data, + blob->write_idx, + datalen, + data); + if (unlikely(!rc)) { + BNXT_DRV_DBG(ERR, "Failed to write blob\n"); + return -EINVAL; + } + blob->write_idx += datalen; + return 0; +} /* * Insert data into the binary blob at the given offset. @@ -228,9 +437,53 @@ ulp_blob_push(struct ulp_blob *blob, * * returns zero on success */ -int32_t +static inline int32_t ulp_blob_insert(struct ulp_blob *blob, uint32_t offset, - uint8_t *data, uint32_t datalen); + uint8_t *data, uint32_t datalen) +{ + uint32_t rc; + uint8_t local_data[BNXT_ULP_FLMP_BLOB_SIZE]; + uint16_t mov_len; + + /* validate the arguments */ + if (unlikely(!blob || datalen > (uint32_t)(blob->bitlen - blob->write_idx) || + offset > blob->write_idx)) { + BNXT_DRV_DBG(ERR, "invalid argument\n"); + return -EINVAL; + } + + mov_len = blob->write_idx - offset; + /* If offset and data len are not 8 bit aligned then return error */ + if (unlikely(ULP_BITS_IS_BYTE_NOT_ALIGNED(offset) || + ULP_BITS_IS_BYTE_NOT_ALIGNED(datalen))) { + BNXT_DRV_DBG(ERR, "invalid argument, not aligned\n"); + return -EINVAL; + } + + /* copy the data so we can move the data */ + memcpy(local_data, &blob->data[ULP_BITS_2_BYTE_NR(offset)], + ULP_BITS_2_BYTE(mov_len)); + blob->write_idx = offset; + if (blob->byte_order == BNXT_ULP_BYTE_ORDER_BE) + rc = ulp_bs_push_msb(blob->data, + blob->write_idx, + datalen, + data); + else + rc = ulp_bs_push_lsb(blob->data, + blob->write_idx, + datalen, + data); + if (unlikely(!rc)) { + BNXT_DRV_DBG(ERR, "Failed to write blob\n"); + return -EINVAL; + } + /* copy the previously stored data */ + memcpy(&blob->data[ULP_BITS_2_BYTE_NR(offset + datalen)], local_data, + ULP_BITS_2_BYTE(mov_len)); + blob->write_idx += (mov_len + datalen); + return 0; +} /* * Add data to the binary blob at the current offset. @@ -243,12 +496,30 @@ ulp_blob_insert(struct ulp_blob *blob, uint32_t offset, * datalen [in] The number of bits to be added to the blob. * * The offset of the data is updated after each push of data. - * NULL returned on error, ptr to pushed data otherwise + * NULL returned on error, pointer pushed value otherwise. */ -uint8_t * +static inline uint8_t * ulp_blob_push_64(struct ulp_blob *blob, uint64_t *data, - uint32_t datalen); + uint32_t datalen) +{ + uint8_t *val = (uint8_t *)data; + int rc; + + int size = (datalen + 7) / 8; + + if (unlikely(!blob || !data || + datalen > (uint32_t)(blob->bitlen - blob->write_idx))) { + BNXT_DRV_DBG(ERR, "invalid argument\n"); + return NULL; + } + + rc = ulp_blob_push(blob, &val[8 - size], datalen); + if (unlikely(rc)) + return NULL; + + return &val[8 - size]; +} /* * Add data to the binary blob at the current offset. @@ -258,15 +529,31 @@ ulp_blob_push_64(struct ulp_blob *blob, * * data [in] 32-bit value to be added to the blob. * - * datalen [in] The number of bits to be added ot the blob. + * datalen [in] The number of bits to be added to the blob. * * The offset of the data is updated after each push of data. * NULL returned on error, pointer pushed value otherwise. */ -uint8_t * +static inline uint8_t * ulp_blob_push_32(struct ulp_blob *blob, uint32_t *data, - uint32_t datalen); + uint32_t datalen) +{ + uint8_t *val = (uint8_t *)data; + uint32_t rc; + uint32_t size = ULP_BITS_2_BYTE(datalen); + + if (unlikely(!data || size > sizeof(uint32_t))) { + BNXT_DRV_DBG(ERR, "invalid argument\n"); + return NULL; + } + + rc = ulp_blob_push(blob, &val[sizeof(uint32_t) - size], datalen); + if (unlikely(rc)) + return NULL; + + return &val[sizeof(uint32_t) - size]; +} /* * Add encap data to the binary blob at the current offset. @@ -281,34 +568,115 @@ ulp_blob_push_32(struct ulp_blob *blob, * The offset of the data is updated after each push of data. * NULL returned on error, pointer pushed value otherwise. */ -int32_t +static inline int32_t ulp_blob_push_encap(struct ulp_blob *blob, uint8_t *data, - uint32_t datalen); + uint32_t datalen) +{ + uint8_t *val = (uint8_t *)data; + uint32_t initial_size, write_size = datalen; + uint32_t size = 0; + + if (unlikely(!blob || !data || + datalen > (uint32_t)(blob->bitlen - blob->write_idx))) { + BNXT_DRV_DBG(ERR, "invalid argument\n"); + return -1; + } + + initial_size = ULP_BYTE_2_BITS(sizeof(uint64_t)) - + (blob->write_idx % ULP_BYTE_2_BITS(sizeof(uint64_t))); + while (write_size > 0) { + if (initial_size && write_size > initial_size) { + size = initial_size; + initial_size = 0; + } else if (initial_size && write_size <= initial_size) { + size = write_size; + initial_size = 0; + } else if (write_size > ULP_BYTE_2_BITS(sizeof(uint64_t))) { + size = ULP_BYTE_2_BITS(sizeof(uint64_t)); + } else { + size = write_size; + } + if (unlikely(ulp_blob_push(blob, val, size))) { + BNXT_DRV_DBG(ERR, "push field failed\n"); + return -1; + } + val += ULP_BITS_2_BYTE(size); + write_size -= size; + } + return datalen; +} /* - * Get the data portion of the binary blob. + * Adds pad to an initialized blob at the current offset * - * blob [in] The blob's data to be retrieved. The blob must be - * initialized prior to pushing data. + * blob [in] The blob that data is added to. The blob must + * be initialized prior to pushing data. * - * datalen [out] The number of bits to that are filled. + * datalen [in] The number of bits of pad to add * - * returns a byte array of the blob data. Returns NULL on error. + * returns the number of pad bits added, -1 on failure */ -uint8_t * -ulp_blob_data_get(struct ulp_blob *blob, - uint16_t *datalen); +static inline int32_t +ulp_blob_pad_push(struct ulp_blob *blob, + uint32_t datalen) +{ + if (unlikely(datalen > (uint32_t)(blob->bitlen - blob->write_idx))) { + BNXT_DRV_DBG(ERR, "Pad too large for blob\n"); + return -1; + } + + blob->write_idx += datalen; + return datalen; +} /* - * Get the data length of the binary blob. + * Adds pad to an initialized blob at the current offset based on + * the alignment. * - * blob [in] The blob's data len to be retrieved. + * blob [in] The blob that needs to be aligned * - * returns length of the binary blob + * align [in] Alignment in bits. + * + * returns the number of pad bits added, -1 on failure */ -uint16_t -ulp_blob_data_len_get(struct ulp_blob *blob); +static inline int32_t +ulp_blob_pad_align(struct ulp_blob *blob, + uint32_t align) +{ + int32_t pad = 0; + + pad = RTE_ALIGN(blob->write_idx, align) - blob->write_idx; + if (unlikely(pad > (int32_t)(blob->bitlen - blob->write_idx))) { + BNXT_DRV_DBG(ERR, "Pad too large for blob\n"); + return -1; + } + blob->write_idx += pad; + return pad; +} + +/* Get data from src and put into dst using little-endian format */ +static inline void +ulp_bs_get_lsb(uint8_t *src, uint16_t bitpos, uint8_t bitlen, uint8_t *dst) +{ + uint8_t bitoffs = bitpos % ULP_BLOB_BYTE; + uint16_t index = ULP_BITS_2_BYTE_NR(bitpos); + uint8_t mask, partial, shift; + + shift = bitoffs; + partial = ULP_BLOB_BYTE - bitoffs; + if (bitoffs + bitlen <= ULP_BLOB_BYTE) { + mask = ((1 << bitlen) - 1) << shift; + *dst = (src[index] & mask) >> shift; + } else { + mask = ((1 << partial) - 1) << shift; + *dst = (src[index] & mask) >> shift; + index++; + partial = bitlen - partial; + mask = ((1 << partial) - 1); + *dst |= (src[index] & mask) << (ULP_BLOB_BYTE - bitoffs); + } +} /* * Get data from the byte array in Little endian format. @@ -325,9 +693,57 @@ ulp_blob_data_len_get(struct ulp_blob *blob); * * returns None. */ -void +static inline void ulp_bs_pull_lsb(uint8_t *src, uint8_t *dst, uint32_t size, - uint32_t offset, uint32_t len); + uint32_t offset, uint32_t len) +{ + uint32_t idx; + uint32_t cnt = ULP_BITS_2_BYTE_NR(len); + + /* iterate bytewise to get data */ + for (idx = 0; idx < cnt; idx++) { + ulp_bs_get_lsb(src, offset, ULP_BLOB_BYTE, + &dst[size - 1 - idx]); + offset += ULP_BLOB_BYTE; + len -= ULP_BLOB_BYTE; + } + + /* Extract the last reminder data that is not 8 byte boundary */ + if (len) + ulp_bs_get_lsb(src, offset, len, &dst[size - 1 - idx]); +} + +/* + * Get data from the byte array in Big endian format. + * + * src [in] The byte array where data is extracted from + * + * bitpos [in] The offset where data is pulled + * + * bitlen [in] The number of bits to be extracted from the data array + * + * dst [out] The byte array where data is pulled into + * + * returns None. + */ +/* Get data from src and put into dst using big-endian format */ +static inline void +ulp_bs_get_msb(uint8_t *src, uint16_t bitpos, uint8_t bitlen, uint8_t *dst) +{ + uint8_t bitoffs = bitpos % ULP_BLOB_BYTE; + uint16_t index = ULP_BITS_2_BYTE_NR(bitpos); + uint8_t mask; + int32_t shift; + + shift = ULP_BLOB_BYTE - bitoffs - bitlen; + if (shift >= 0) { + mask = 0xFF >> -bitlen; + *dst = (src[index] >> shift) & mask; + } else { + *dst = (src[index] & (0xFF >> bitoffs)) << -shift; + *dst |= src[index + 1] >> -shift; + } +} /* * Get data from the byte array in Big endian format. @@ -342,9 +758,24 @@ ulp_bs_pull_lsb(uint8_t *src, uint8_t *dst, uint32_t size, * * returns None. */ -void +static inline void ulp_bs_pull_msb(uint8_t *src, uint8_t *dst, - uint32_t offset, uint32_t len); + uint32_t offset, uint32_t len) +{ + uint32_t idx; + uint32_t cnt = ULP_BITS_2_BYTE_NR(len); + + /* iterate bytewise to get data */ + for (idx = 0; idx < cnt; idx++) { + ulp_bs_get_msb(src, offset, ULP_BLOB_BYTE, &dst[idx]); + offset += ULP_BLOB_BYTE; + len -= ULP_BLOB_BYTE; + } + + /* Extract the last reminder data that is not 8 byte boundary */ + if (len) + ulp_bs_get_msb(src, offset, len, &dst[idx]); +} /* * Extract data from the binary blob using given offset. @@ -359,48 +790,83 @@ ulp_bs_pull_msb(uint8_t *src, uint8_t *dst, * * Output: zero on success, -1 on failure */ -int32_t +static inline int32_t ulp_blob_pull(struct ulp_blob *blob, uint8_t *data, uint32_t data_size, - uint16_t offset, uint16_t len); + uint16_t offset, uint16_t len) +{ + /* validate the arguments */ + if (unlikely(!blob || (offset + len) > blob->bitlen || + ULP_BYTE_2_BITS(data_size) < len)) { + BNXT_DRV_DBG(ERR, "invalid argument\n"); + return -1; /* failure */ + } + + if (blob->byte_order == BNXT_ULP_BYTE_ORDER_BE) + ulp_bs_pull_msb(blob->data, data, offset, len); + else + ulp_bs_pull_lsb(blob->data, data, data_size, offset, len); + return 0; +} /* - * Adds pad to an initialized blob at the current offset + * Get the data portion of the binary blob. * - * blob [in] The blob that data is added to. The blob must - * be initialized prior to pushing data. + * blob [in] The blob's data to be retrieved. The blob must be + * initialized prior to pushing data. * - * datalen [in] The number of bits of pad to add + * datalen [out] The number of bits that are filled. * - * returns the number of pad bits added, -1 on failure + * Returns a byte array of the blob data or NULL on error. */ -int32_t -ulp_blob_pad_push(struct ulp_blob *blob, - uint32_t datalen); +static inline uint8_t * +ulp_blob_data_get(struct ulp_blob *blob, + uint16_t *datalen) +{ + /* validate the arguments */ + if (unlikely(!blob)) { + BNXT_DRV_DBG(ERR, "invalid argument\n"); + return NULL; /* failure */ + } + *datalen = blob->write_idx; + return blob->data; +} /* - * Adds pad to an initialized blob at the current offset based on - * the alignment. - * - * blob [in] The blob that needs to be aligned + * Get the data length of the binary blob. * - * align [in] Alignment in bits. + * blob [in] The blob's data len to be retrieved. * - * returns the number of pad bits added, -1 on failure + * returns length of the binary blob */ -int32_t -ulp_blob_pad_align(struct ulp_blob *blob, - uint32_t align); +static inline uint16_t +ulp_blob_data_len_get(struct ulp_blob *blob) +{ + /* validate the arguments */ + if (unlikely(!blob)) { + BNXT_DRV_DBG(ERR, "invalid argument\n"); + return 0; /* failure */ + } + return blob->write_idx; +} /* - * Set the 64 bit swap start index of the binary blob. + * Set the encap swap start index of the binary blob. * * blob [in] The blob's data to be retrieved. The blob must be * initialized prior to pushing data. * * returns void. */ -void -ulp_blob_encap_swap_idx_set(struct ulp_blob *blob); +static inline void +ulp_blob_encap_swap_idx_set(struct ulp_blob *blob) +{ + /* validate the arguments */ + if (unlikely(!blob)) { + BNXT_DRV_DBG(ERR, "invalid argument\n"); + return; /* failure */ + } + blob->encap_swap_idx = blob->write_idx; +} /* * Perform the encap buffer swap to 64 bit reversal. @@ -409,8 +875,36 @@ ulp_blob_encap_swap_idx_set(struct ulp_blob *blob); * * returns void. */ -void -ulp_blob_perform_encap_swap(struct ulp_blob *blob); +static inline void +ulp_blob_perform_encap_swap(struct ulp_blob *blob) +{ + uint32_t i, idx = 0, end_idx = 0, roundoff; + uint8_t temp_val_1, temp_val_2; + + /* validate the arguments */ + if (unlikely(!blob)) { + BNXT_DRV_DBG(ERR, "invalid argument\n"); + return; /* failure */ + } + idx = ULP_BITS_2_BYTE_NR(blob->encap_swap_idx); + end_idx = ULP_BITS_2_BYTE(blob->write_idx); + roundoff = ULP_BYTE_2_BITS(ULP_BITS_2_BYTE(end_idx)); + if (roundoff > end_idx) { + blob->write_idx += ULP_BYTE_2_BITS(roundoff - end_idx); + end_idx = roundoff; + } + while (idx <= end_idx) { + for (i = 0; i < 4; i = i + 2) { + temp_val_1 = blob->data[idx + i]; + temp_val_2 = blob->data[idx + i + 1]; + blob->data[idx + i] = blob->data[idx + 6 - i]; + blob->data[idx + i + 1] = blob->data[idx + 7 - i]; + blob->data[idx + 7 - i] = temp_val_2; + blob->data[idx + 6 - i] = temp_val_1; + } + idx += 8; + } +} /* * Perform the blob buffer reversal byte wise. @@ -422,9 +916,31 @@ ulp_blob_perform_encap_swap(struct ulp_blob *blob); * * returns void. */ -void +static inline void ulp_blob_perform_byte_reverse(struct ulp_blob *blob, - uint32_t chunk_size); + uint32_t chunk_size) +{ + uint32_t idx = 0, jdx = 0, num = 0; + uint8_t xchar; + uint8_t *buff; + + /* validate the arguments */ + if (unlikely(!blob)) { + BNXT_DRV_DBG(ERR, "invalid argument\n"); + return; /* failure */ + } + + buff = blob->data; + num = ULP_BITS_2_BYTE(blob->write_idx) / chunk_size; + for (idx = 0; idx < num; idx++) { + for (jdx = 0; jdx < chunk_size / 2; jdx++) { + xchar = buff[jdx]; + buff[jdx] = buff[(chunk_size - 1) - jdx]; + buff[(chunk_size - 1) - jdx] = xchar; + } + buff += chunk_size; + } +} /* * Perform the blob buffer 64 bit word swap. @@ -435,8 +951,27 @@ ulp_blob_perform_byte_reverse(struct ulp_blob *blob, * * returns void. */ -void -ulp_blob_perform_64B_word_swap(struct ulp_blob *blob); +static inline void +ulp_blob_perform_64B_word_swap(struct ulp_blob *blob) +{ + uint32_t i, j, num; + uint8_t xchar; + uint32_t word_size = ULP_64B_IN_BYTES / 2; + + /* validate the arguments */ + if (unlikely(!blob)) { + BNXT_DRV_DBG(ERR, "invalid argument\n"); + return; /* failure */ + } + num = ULP_BITS_2_BYTE(blob->write_idx); + for (i = 0; i < num; i = i + ULP_64B_IN_BYTES) { + for (j = 0; j < word_size; j++) { + xchar = blob->data[i + j]; + blob->data[i + j] = blob->data[i + j + word_size]; + blob->data[i + j + word_size] = xchar; + } + } +} /* * Perform the blob buffer 64 bit byte swap. @@ -447,8 +982,27 @@ ulp_blob_perform_64B_word_swap(struct ulp_blob *blob); * * returns void. */ -void -ulp_blob_perform_64B_byte_swap(struct ulp_blob *blob); +static inline void +ulp_blob_perform_64B_byte_swap(struct ulp_blob *blob) +{ + uint32_t i, j, num; + uint8_t xchar; + uint32_t offset = ULP_64B_IN_BYTES - 1; + + /* validate the arguments */ + if (unlikely(!blob)) { + BNXT_DRV_DBG(ERR, "invalid argument\n"); + return; /* failure */ + } + num = ULP_BITS_2_BYTE(blob->write_idx); + for (i = 0; i < num; i = i + ULP_64B_IN_BYTES) { + for (j = 0; j < (ULP_64B_IN_BYTES / 2); j++) { + xchar = blob->data[i + j]; + blob->data[i + j] = blob->data[i + offset - j]; + blob->data[i + offset - j] = xchar; + } + } +} /* * Perform the blob buffer merge. @@ -457,32 +1011,144 @@ ulp_blob_perform_64B_byte_swap(struct ulp_blob *blob); * * dst [in] The destination blob, the blob to be merged. * src [in] The src blob. - * block_size [in] The size of the block after which padding gets applied. + * block_size [in] The size of the block in bytes after which padding gets + * applied. * pad [in] The size of the pad to be applied. * * returns 0 on success. */ -int32_t -ulp_blob_block_merge(struct ulp_blob *dst, struct ulp_blob *src, - uint32_t block_size, uint32_t pad); +static inline int32_t +ulp_blob_msb_block_merge(struct ulp_blob *dst, struct ulp_blob *src, + uint32_t block_size, uint32_t pad) +{ + uint32_t i, k, write_bytes, remaining; + uint16_t num; + uint8_t *src_buf = ulp_blob_data_get(src, &num); + uint8_t bluff; + + for (i = 0; i < num;) { + if (((dst->write_idx % block_size) + (num - i)) > block_size) + write_bytes = block_size - + (dst->write_idx % block_size); + else + write_bytes = num - i; + for (k = 0; k < ULP_BITS_2_BYTE_NR(write_bytes); k++) { + ulp_bs_put_msb(dst->data, dst->write_idx, ULP_BLOB_BYTE, + *src_buf); + dst->write_idx += ULP_BLOB_BYTE; + src_buf++; + } + remaining = write_bytes % ULP_BLOB_BYTE; + if (remaining) { + bluff = (*src_buf) & ((uint8_t)-1 << + (ULP_BLOB_BYTE - remaining)); + ulp_bs_put_msb(dst->data, dst->write_idx, + ULP_BLOB_BYTE, bluff); + dst->write_idx += remaining; + } + if (write_bytes != (num - i)) { + /* add the padding */ + ulp_blob_pad_push(dst, pad); + if (remaining) { + ulp_bs_put_msb(dst->data, dst->write_idx, + ULP_BLOB_BYTE - remaining, + *src_buf); + dst->write_idx += ULP_BLOB_BYTE - remaining; + src_buf++; + } + } + i += write_bytes; + } + return 0; +} /* - * Append bits from src blob to dst blob. - * Only works on BE blobs - * - * dst [in/out] The destination blob to append to + * Perform the blob buffer merge. + * This api makes the src blob merged to the dst blob. + * The block size and pad size help in padding the dst blob * - * src [in] The src blob to append from + * dst [in] The destination blob, the blob to be merged. + * src [in] The src blob. + * block_size [in] The size of the block in bytes after which padding gets + * applied. + * pad [in] The size of the pad to be applied. * - * src_offset [in] The bit offset from src to start at + * returns 0 on success. + */ +static inline int32_t +ulp_blob_block_merge(struct ulp_blob *dst, struct ulp_blob *src, + uint32_t block_size, uint32_t pad) +{ + if (dst->byte_order == BNXT_ULP_BYTE_ORDER_BE && + src->byte_order == BNXT_ULP_BYTE_ORDER_BE) + return ulp_blob_msb_block_merge(dst, src, block_size, pad); + + BNXT_DRV_DBG(ERR, "block merge not implemented yet\n"); + return -EINVAL; +} + +/* + * Perform the blob buffer append. * - * src_len [in] The number of bits to append to dst + * dst [in] The destination blob, the blob to be merged. + * src [in] The src blob. + * src_offset [in] Offset of src data. + * src_len [in] The size of the src data. * - * returns 0 on success, non-zero on error + * returns 0 on success. */ -int32_t +static inline int32_t ulp_blob_append(struct ulp_blob *dst, struct ulp_blob *src, - uint16_t src_offset, uint16_t src_len); + uint16_t src_offset, uint16_t src_len) +{ + uint32_t k, remaining = 0; + uint16_t num; + uint8_t bluff; + uint8_t *src_buf = ulp_blob_data_get(src, &num); + + if (unlikely((src_offset + src_len) > num)) + return -EINVAL; + + /* Only supporting BE for now */ + if (unlikely(src->byte_order != BNXT_ULP_BYTE_ORDER_BE || + dst->byte_order != BNXT_ULP_BYTE_ORDER_BE)) + return -EINVAL; + + /* Handle if the source offset is not on a byte boundary */ + remaining = src_offset % ULP_BLOB_BYTE; + if (remaining) { + bluff = src_buf[src_offset / ULP_BLOB_BYTE] & ((uint8_t)-1 >> + (ULP_BLOB_BYTE - remaining)); + ulp_bs_put_msb(dst->data, dst->write_idx, remaining, bluff); + dst->write_idx += remaining; + src_offset += remaining; + } + + src_buf += ULP_BITS_2_BYTE_NR(src_offset); + + /* Push the byte aligned pieces */ + for (k = 0; k < ULP_BITS_2_BYTE_NR(src_len); k++) { + ulp_bs_put_msb(dst->data, dst->write_idx, ULP_BLOB_BYTE, + *src_buf); + dst->write_idx += ULP_BLOB_BYTE; + src_buf++; + } + + /* Handle the remaining if length is not a byte boundary */ + if (src_len > remaining) + remaining = (src_len - remaining) % ULP_BLOB_BYTE; + else + remaining = 0; + if (remaining) { + bluff = (*src_buf) & ((uint8_t)-1 << + (ULP_BLOB_BYTE - remaining)); + ulp_bs_put_msb(dst->data, dst->write_idx, + ULP_BLOB_BYTE, bluff); + dst->write_idx += remaining; + } + + return 0; +} /* * Perform the blob buffer copy. @@ -493,8 +1159,23 @@ ulp_blob_append(struct ulp_blob *dst, struct ulp_blob *src, * * returns 0 on success. */ -int32_t -ulp_blob_buffer_copy(struct ulp_blob *dst, struct ulp_blob *src); +static inline int32_t +ulp_blob_buffer_copy(struct ulp_blob *dst, struct ulp_blob *src) +{ + if (unlikely((dst->write_idx + src->write_idx) > dst->bitlen)) { + BNXT_DRV_DBG(ERR, "source buffer too large\n"); + return -EINVAL; + } + if (unlikely(ULP_BITS_IS_BYTE_NOT_ALIGNED(dst->write_idx) || + ULP_BITS_IS_BYTE_NOT_ALIGNED(src->write_idx))) { + BNXT_DRV_DBG(ERR, "source buffer is not aligned\n"); + return -EINVAL; + } + memcpy(&dst->data[ULP_BITS_2_BYTE_NR(dst->write_idx)], + src->data, ULP_BITS_2_BYTE_NR(src->write_idx)); + dst->write_idx += src->write_idx; + return 0; +} /* * Read data from the operand @@ -503,32 +1184,54 @@ ulp_blob_buffer_copy(struct ulp_blob *dst, struct ulp_blob *src); * * val [in/out] The variable to copy the operand to * - * bitlen [in] The number of bits to read into val + * bytes [in] The number of bytes to read into val * * returns zero on success. */ -int32_t +static inline int32_t ulp_operand_read(uint8_t *operand, uint8_t *val, - uint16_t bitlen); - -/* - * Check the buffer is empty - * - * buf [in] The buffer - * size [in] The size of the buffer - */ -int32_t ulp_buffer_is_empty(const uint8_t *buf, uint32_t size); - -/* Function to check if bitmap is zero.Return 1 on success */ -uint32_t ulp_bitmap_is_zero(uint8_t *bitmap, int32_t size); + uint16_t bytes) +{ + /* validate the arguments */ + if (unlikely(!operand || !val)) { + BNXT_DRV_DBG(ERR, "invalid argument\n"); + return -EINVAL; + } + memcpy(val, operand, bytes); + return 0; +} /* Function to check if bitmap is ones. Return 1 on success */ -uint32_t ulp_bitmap_is_ones(uint8_t *bitmap, int32_t size); +static inline uint32_t +ulp_bitmap_is_ones(uint8_t *bitmap, int32_t size) +{ + while (size-- > 0) { + if (*bitmap != 0xFF) + return 0; + bitmap++; + } + return 1; +} /* Function to check if bitmap is not zero. Return 1 on success */ -uint32_t ulp_bitmap_notzero(const uint8_t *bitmap, int32_t size); +static inline uint32_t +ulp_bitmap_notzero(const uint8_t *bitmap, int32_t size) +{ + while (size-- > 0) { + if (*bitmap != 0) + return 1; + bitmap++; + } + return 0; +} /* returns 0 if input is power of 2 */ -int32_t ulp_util_is_power_of_2(uint64_t x); +static inline int32_t +ulp_util_is_power_of_2(uint64_t x) +{ + if (((x - 1) & x)) + return -1; + return 0; +} #endif /* _ULP_UTILS_H_ */ -- 2.39.3