From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-gw2-out.broadcom.com (mail-gw2-out.broadcom.com [216.31.210.63]) by dpdk.org (Postfix) with ESMTP id 34357AD9B for ; Wed, 15 Jun 2016 00:57:13 +0200 (CEST) X-IronPort-AV: E=Sophos;i="5.26,473,1459839600"; d="scan'208";a="97627302" Received: from mail-irv-18.broadcom.com ([10.15.198.37]) by mail-gw2-out.broadcom.com with ESMTP; 14 Jun 2016 16:17:14 -0700 Received: from mail-irva-12.broadcom.com (mail-irva-12.broadcom.com [10.11.16.101]) by mail-irv-18.broadcom.com (Postfix) with ESMTP id 778C58202C; Tue, 14 Jun 2016 15:57:05 -0700 (PDT) Received: from DPDK-C1.broadcom.com (dhcp-10-13-115-104.irv.broadcom.com [10.13.115.104]) by mail-irva-12.broadcom.com (Postfix) with ESMTP id ECBD3A6289; Tue, 14 Jun 2016 15:57:01 -0700 (PDT) From: Stephen Hurd To: dev@dpdk.org, ajit.khaparde@broadcom.com, bruce.richardson@intel.com Date: Tue, 14 Jun 2016 15:55:46 -0700 Message-Id: <1465944971-113413-13-git-send-email-stephen.hurd@broadcom.com> X-Mailer: git-send-email 1.9.1 In-Reply-To: <1465944971-113413-1-git-send-email-stephen.hurd@broadcom.com> References: <1465250923-78695-1-git-send-email-stephen.hurd@broadcom.com> <1465944971-113413-1-git-send-email-stephen.hurd@broadcom.com> Subject: [dpdk-dev] [PATCH v5 13/38] bnxt: add initial Tx code implementation X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: patches and discussions about DPDK List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 14 Jun 2016 22:57:14 -0000 From: Ajit Khaparde Initial implementation of tx_pkt_burst for transmit. bnxt_xmit_pkts() is the top level function that is called during Tx. bnxt_handle_tx_cp() is used to check and process the Tx completions generated for the Tx Buffer Descriptors sent by the hardware. This patch also adds code to allocate rings in the hardware. For each Tx queue allocated in the PMD driver, a corresponding ring in hardware will be created. Every time a Tx request is initiated via the bnxt_xmit_pkts() call, a Buffer Descriptor is created and is sent to the hardware via the associated Tx ring. On completing the Tx operation, the hardware will generates the status in the form of a completion. This completion is processed by the bnxt_handle_tx_cp() function. Functions like bnxt_init_tx_ring_struct() and bnxt_init_one_tx_ring() are used to initialize various members of the structure before starting Tx operations. Signed-off-by: Ajit Khaparde Signed-off-by: Stephen Hurd Reviewed-by: David Christensen -- v4: Address review comments and fix issues pointed out by checkpatch. v5: Expand the patch description Reorder footer --- drivers/net/bnxt/Makefile | 1 + drivers/net/bnxt/bnxt_cpr.h | 4 +- drivers/net/bnxt/bnxt_ethdev.c | 3 +- drivers/net/bnxt/bnxt_ring.c | 145 ++++++++++ drivers/net/bnxt/bnxt_ring.h | 8 + drivers/net/bnxt/bnxt_txq.c | 42 ++- drivers/net/bnxt/bnxt_txr.c | 314 ++++++++++++++++++++ drivers/net/bnxt/bnxt_txr.h | 71 +++++ drivers/net/bnxt/hsi_struct_def_dpdk.h | 512 +++++++++++++++++++++++++++++++++ 9 files changed, 1091 insertions(+), 9 deletions(-) create mode 100644 drivers/net/bnxt/bnxt_txr.c create mode 100644 drivers/net/bnxt/bnxt_txr.h diff --git a/drivers/net/bnxt/Makefile b/drivers/net/bnxt/Makefile index f6a04f8..0785681 100644 --- a/drivers/net/bnxt/Makefile +++ b/drivers/net/bnxt/Makefile @@ -56,6 +56,7 @@ SRCS-$(CONFIG_RTE_LIBRTE_BNXT_PMD) += bnxt_ring.c SRCS-$(CONFIG_RTE_LIBRTE_BNXT_PMD) += bnxt_rxq.c SRCS-$(CONFIG_RTE_LIBRTE_BNXT_PMD) += bnxt_stats.c SRCS-$(CONFIG_RTE_LIBRTE_BNXT_PMD) += bnxt_txq.c +SRCS-$(CONFIG_RTE_LIBRTE_BNXT_PMD) += bnxt_txr.c SRCS-$(CONFIG_RTE_LIBRTE_BNXT_PMD) += bnxt_vnic.c # diff --git a/drivers/net/bnxt/bnxt_cpr.h b/drivers/net/bnxt/bnxt_cpr.h index 2be22de..4bd5606 100644 --- a/drivers/net/bnxt/bnxt_cpr.h +++ b/drivers/net/bnxt/bnxt_cpr.h @@ -51,11 +51,11 @@ #define B_CP_DB_REARM(cpr, raw_cons) \ (*(uint32_t *)((cpr)->cp_doorbell) = (DB_CP_REARM_FLAGS | \ - RING_CMP(&cpr->cp_ring_struct, raw_cons))) + RING_CMP(cpr->cp_ring_struct, raw_cons))) #define B_CP_DIS_DB(cpr, raw_cons) \ (*(uint32_t *)((cpr)->cp_doorbell) = (DB_CP_FLAGS | \ - RING_CMP(&cpr->cp_ring_struct, raw_cons))) + RING_CMP(cpr->cp_ring_struct, raw_cons))) struct bnxt_ring; struct bnxt_cp_ring_info { diff --git a/drivers/net/bnxt/bnxt_ethdev.c b/drivers/net/bnxt/bnxt_ethdev.c index 3453509..4ace543 100644 --- a/drivers/net/bnxt/bnxt_ethdev.c +++ b/drivers/net/bnxt/bnxt_ethdev.c @@ -44,6 +44,7 @@ #include "bnxt_rxq.h" #include "bnxt_stats.h" #include "bnxt_txq.h" +#include "bnxt_txr.h" #define DRV_MODULE_NAME "bnxt" static const char bnxt_version[] = @@ -269,7 +270,7 @@ bnxt_dev_init(struct rte_eth_dev *eth_dev) } eth_dev->dev_ops = &bnxt_dev_ops; /* eth_dev->rx_pkt_burst = &bnxt_recv_pkts; */ - /* eth_dev->tx_pkt_burst = &bnxt_xmit_pkts; */ + eth_dev->tx_pkt_burst = &bnxt_xmit_pkts; rc = bnxt_alloc_hwrm_resources(bp); if (rc) { diff --git a/drivers/net/bnxt/bnxt_ring.c b/drivers/net/bnxt/bnxt_ring.c index b57d3ba..2645dda 100644 --- a/drivers/net/bnxt/bnxt_ring.c +++ b/drivers/net/bnxt/bnxt_ring.c @@ -31,8 +31,14 @@ * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. */ +#include + #include "bnxt.h" +#include "bnxt_cpr.h" #include "bnxt_ring.h" +#include "bnxt_txr.h" + +#include "hsi_struct_def_dpdk.h" /* * Generic ring handling @@ -45,3 +51,142 @@ void bnxt_free_ring(struct bnxt_ring *ring) *ring->vmem = NULL; } } + +/* + * Allocates a completion ring with vmem and stats optionally also allocating + * a TX and/or RX ring. Passing NULL as tx_ring_info and/or rx_ring_info + * to not allocate them. + * + * Order in the allocation is: + * stats - Always non-zero length + * cp vmem - Always zero-length, supported for the bnxt_ring abstraction + * tx vmem - Only non-zero length if tx_ring_info is not NULL + * rx vmem - Only non-zero length if rx_ring_info is not NULL + * cp bd ring - Always non-zero length + * tx bd ring - Only non-zero length if tx_ring_info is not NULL + * rx bd ring - Only non-zero length if rx_ring_info is not NULL + */ +int bnxt_alloc_rings(struct bnxt *bp, uint16_t qidx, + struct bnxt_tx_ring_info *tx_ring_info, + struct bnxt_rx_ring_info *rx_ring_info, + struct bnxt_cp_ring_info *cp_ring_info, + const char *suffix) +{ + struct bnxt_ring *cp_ring = cp_ring_info->cp_ring_struct; + struct bnxt_ring *tx_ring; + /* TODO: RX ring */ + /* struct bnxt_ring *rx_ring; */ + struct rte_pci_device *pdev = bp->pdev; + const struct rte_memzone *mz = NULL; + char mz_name[RTE_MEMZONE_NAMESIZE]; + + int stats_len = (tx_ring_info || rx_ring_info) ? + RTE_CACHE_LINE_ROUNDUP(sizeof(struct ctx_hw_stats64)) : 0; + + int cp_vmem_start = stats_len; + int cp_vmem_len = RTE_CACHE_LINE_ROUNDUP(cp_ring->vmem_size); + + int tx_vmem_start = cp_vmem_start + cp_vmem_len; + int tx_vmem_len = + tx_ring_info ? RTE_CACHE_LINE_ROUNDUP(tx_ring_info-> + tx_ring_struct->vmem_size) : 0; + + int rx_vmem_start = tx_vmem_start + tx_vmem_len; + /* TODO: RX ring */ + int rx_vmem_len = 0; + /* + * rx_ring_info ? RTE_CACHE_LINE_ROUNDUP(rx_ring_info-> + * rx_ring_struct->vmem_size) : 0; + */ + + int cp_ring_start = rx_vmem_start + rx_vmem_len; + int cp_ring_len = RTE_CACHE_LINE_ROUNDUP(cp_ring->ring_size * + sizeof(struct cmpl_base)); + + int tx_ring_start = cp_ring_start + cp_ring_len; + int tx_ring_len = tx_ring_info ? + RTE_CACHE_LINE_ROUNDUP(tx_ring_info->tx_ring_struct->ring_size * + sizeof(struct tx_bd_long)) : 0; + + int rx_ring_start = tx_ring_start + tx_ring_len; + /* TODO: RX ring */ + int rx_ring_len = 0; + /* + * rx_ring_info ? + * RTE_CACHE_LINE_ROUNDUP(rx_ring_info->rx_ring_struct->ring_size * + * sizeof(struct rx_prod_pkt_bd)) : 0; + */ + + int total_alloc_len = rx_ring_start + rx_ring_len; + + snprintf(mz_name, RTE_MEMZONE_NAMESIZE, + "bnxt_%04x:%02x:%02x:%02x-%04x_%s", pdev->addr.domain, + pdev->addr.bus, pdev->addr.devid, pdev->addr.function, qidx, + suffix); + mz_name[RTE_MEMZONE_NAMESIZE - 1] = 0; + mz = rte_memzone_lookup(mz_name); + if (!mz) { + mz = rte_memzone_reserve(mz_name, total_alloc_len, + SOCKET_ID_ANY, + RTE_MEMZONE_2MB | + RTE_MEMZONE_SIZE_HINT_ONLY); + if (mz == NULL) + return -ENOMEM; + } + memset(mz->addr, 0, mz->len); + + if (tx_ring_info) { + tx_ring = tx_ring_info->tx_ring_struct; + + tx_ring->bd = ((char *)mz->addr + tx_ring_start); + tx_ring_info->tx_desc_ring = (struct tx_bd_long *)tx_ring->bd; + tx_ring->bd_dma = mz->phys_addr + tx_ring_start; + tx_ring_info->tx_desc_mapping = tx_ring->bd_dma; + + if (!tx_ring->bd) + return -ENOMEM; + if (tx_ring->vmem_size) { + tx_ring->vmem = + (void **)((char *)mz->addr + tx_vmem_start); + tx_ring_info->tx_buf_ring = + (struct bnxt_sw_tx_bd *)tx_ring->vmem; + } + } + +/* + * if (rx_ring_info) { + * rx_ring = &rx_ring_info->rx_ring_struct; + * + * rx_ring->bd = ((char *)mz->addr + rx_ring_start); + * rx_ring_info->rx_desc_ring = + * (struct rx_prod_pkt_bd *)rx_ring->bd; + * rx_ring->bd_dma = mz->phys_addr + rx_ring_start; + * rx_ring_info->rx_desc_mapping = rx_ring->bd_dma; + * + * if (!rx_ring->bd) + * return -ENOMEM; + * if (rx_ring->vmem_size) { + * rx_ring->vmem = + * (void **)((char *)mz->addr + rx_vmem_start); + * rx_ring_info->rx_buf_ring = + * (struct bnxt_sw_rx_bd *)rx_ring->vmem; + * } + * } + */ + + cp_ring->bd = ((char *)mz->addr + cp_ring_start); + cp_ring->bd_dma = mz->phys_addr + cp_ring_start; + cp_ring_info->cp_desc_ring = cp_ring->bd; + cp_ring_info->cp_desc_mapping = cp_ring->bd_dma; + + if (!cp_ring->bd) + return -ENOMEM; + if (cp_ring->vmem_size) + *cp_ring->vmem = ((char *)mz->addr + stats_len); + if (stats_len) { + cp_ring_info->hw_stats = mz->addr; + cp_ring_info->hw_stats_map = mz->phys_addr; + } + cp_ring_info->hw_stats_ctx_id = HWRM_NA_SIGNATURE; + return 0; +} diff --git a/drivers/net/bnxt/bnxt_ring.h b/drivers/net/bnxt/bnxt_ring.h index 3abfb04..1e9d3be 100644 --- a/drivers/net/bnxt/bnxt_ring.h +++ b/drivers/net/bnxt/bnxt_ring.h @@ -87,6 +87,14 @@ struct bnxt_ring_grp_info { }; struct bnxt; +struct bnxt_tx_ring_info; +struct bnxt_rx_ring_info; +struct bnxt_cp_ring_info; void bnxt_free_ring(struct bnxt_ring *ring); +int bnxt_alloc_rings(struct bnxt *bp, uint16_t qidx, + struct bnxt_tx_ring_info *tx_ring_info, + struct bnxt_rx_ring_info *rx_ring_info, + struct bnxt_cp_ring_info *cp_ring_info, + const char *suffix); #endif diff --git a/drivers/net/bnxt/bnxt_txq.c b/drivers/net/bnxt/bnxt_txq.c index 8793c9d..460f71a 100644 --- a/drivers/net/bnxt/bnxt_txq.c +++ b/drivers/net/bnxt/bnxt_txq.c @@ -39,6 +39,7 @@ #include "bnxt_cpr.h" #include "bnxt_ring.h" #include "bnxt_txq.h" +#include "bnxt_txr.h" /* * TX Queues @@ -54,9 +55,20 @@ void bnxt_free_txq_stats(struct bnxt_tx_queue *txq) cpr->hw_stats = NULL; } -static void bnxt_tx_queue_release_mbufs(struct bnxt_tx_queue *txq __rte_unused) +static void bnxt_tx_queue_release_mbufs(struct bnxt_tx_queue *txq) { - /* TODO: Requires interaction with TX ring */ + struct bnxt_sw_tx_bd *sw_ring; + uint16_t i; + + sw_ring = txq->tx_ring->tx_buf_ring; + if (sw_ring) { + for (i = 0; i < txq->tx_ring->tx_ring_struct->ring_size; i++) { + if (sw_ring[i].mbuf) { + rte_pktmbuf_free(sw_ring[i].mbuf); + sw_ring[i].mbuf = NULL; + } + } + } } void bnxt_free_tx_mbufs(struct bnxt *bp) @@ -75,7 +87,15 @@ void bnxt_tx_queue_release_op(void *tx_queue) struct bnxt_tx_queue *txq = (struct bnxt_tx_queue *)tx_queue; if (txq) { - /* TODO: Free ring and stats here */ + /* Free TX ring hardware descriptors */ + bnxt_tx_queue_release_mbufs(txq); + bnxt_free_ring(txq->tx_ring->tx_ring_struct); + + /* Free TX completion ring hardware descriptors */ + bnxt_free_ring(txq->cp_ring->cp_ring_struct); + + bnxt_free_txq_stats(txq); + rte_free(txq); } } @@ -111,14 +131,24 @@ int bnxt_tx_queue_setup_op(struct rte_eth_dev *eth_dev, txq->nb_tx_desc = nb_desc; txq->tx_free_thresh = tx_conf->tx_free_thresh; - /* TODO: Initialize ring structure */ + bnxt_init_tx_ring_struct(txq); txq->queue_id = queue_idx; txq->port_id = eth_dev->data->port_id; - /* TODO: Allocate TX ring hardware descriptors */ + /* Allocate TX ring hardware descriptors */ + if (bnxt_alloc_rings(bp, queue_idx, txq->tx_ring, NULL, txq->cp_ring, + "txr")) { + RTE_LOG(ERR, PMD, "ring_dma_zone_reserve for tx_ring failed!"); + bnxt_tx_queue_release_op(txq); + return -ENOMEM; + } - /* TODO: Initialize the ring */ + if (bnxt_init_one_tx_ring(txq)) { + RTE_LOG(ERR, PMD, "bnxt_init_one_tx_ring failed!"); + bnxt_tx_queue_release_op(txq); + return -ENOMEM; + } eth_dev->data->tx_queues[queue_idx] = txq; return 0; diff --git a/drivers/net/bnxt/bnxt_txr.c b/drivers/net/bnxt/bnxt_txr.c new file mode 100644 index 0000000..9064e8d --- /dev/null +++ b/drivers/net/bnxt/bnxt_txr.c @@ -0,0 +1,314 @@ +/*- + * BSD LICENSE + * + * Copyright(c) Broadcom Limited. + * All rights reserved. + * + * Redistribution and use in source and binary forms, with or without + * modification, are permitted provided that the following conditions + * are met: + * + * * Redistributions of source code must retain the above copyright + * notice, this list of conditions and the following disclaimer. + * * Redistributions in binary form must reproduce the above copyright + * notice, this list of conditions and the following disclaimer in + * the documentation and/or other materials provided with the + * distribution. + * * Neither the name of Broadcom Corporation nor the names of its + * contributors may be used to endorse or promote products derived + * from this software without specific prior written permission. + * + * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS + * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT + * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR + * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT + * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, + * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT + * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, + * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY + * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT + * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE + * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. + */ + +#include + +#include +#include + +#include "bnxt.h" +#include "bnxt_cpr.h" +#include "bnxt_ring.h" +#include "bnxt_txq.h" +#include "bnxt_txr.h" +#include "hsi_struct_def_dpdk.h" +#include + +/* + * TX Ring handling + */ + +void bnxt_free_tx_rings(struct bnxt *bp) +{ + int i; + + for (i = 0; i < (int)bp->tx_nr_rings; i++) { + struct bnxt_tx_queue *txq = bp->tx_queues[i]; + + if (!txq) + continue; + + bnxt_free_ring(txq->tx_ring->tx_ring_struct); + /* TODO: free() txq->tx_ring and txq->tx_ring->tx_ring_struct */ + bnxt_free_ring(txq->cp_ring->cp_ring_struct); + /* TODO: free() txq->cp_ring and txq->cp_ring->cp_ring_struct */ + + rte_free(txq); + bp->tx_queues[i] = NULL; + } +} + +int bnxt_init_one_tx_ring(struct bnxt_tx_queue *txq) +{ + struct bnxt_tx_ring_info *txr = txq->tx_ring; + struct bnxt_ring *ring = txr->tx_ring_struct; + + txq->tx_wake_thresh = ring->ring_size / 2; + ring->fw_ring_id = INVALID_HW_RING_ID; + + return 0; +} + +void bnxt_init_tx_ring_struct(struct bnxt_tx_queue *txq) +{ + struct bnxt_cp_ring_info *cpr; + struct bnxt_tx_ring_info *txr; + struct bnxt_ring *ring; + + /* TODO: These need to be allocated */ + txr = txq->tx_ring; + ring = txr->tx_ring_struct; + ring->ring_size = rte_align32pow2(txq->nb_tx_desc + 1); + ring->ring_mask = ring->ring_size - 1; + ring->bd = (void *)txr->tx_desc_ring; + ring->bd_dma = txr->tx_desc_mapping; + ring->vmem_size = ring->ring_size * sizeof(struct bnxt_sw_tx_bd); + ring->vmem = (void **)&txr->tx_buf_ring; + + /* TODO: These need to be allocated */ + cpr = txq->cp_ring; + ring = cpr->cp_ring_struct; + ring->ring_size = txr->tx_ring_struct->ring_size; + ring->ring_mask = ring->ring_size - 1; + ring->bd = (void *)cpr->cp_desc_ring; + ring->bd_dma = cpr->cp_desc_mapping; + ring->vmem_size = 0; + ring->vmem = NULL; +} + +static inline uint32_t bnxt_tx_avail(struct bnxt_tx_ring_info *txr) +{ + /* Tell compiler to fetch tx indices from memory. */ + rte_compiler_barrier(); + + return txr->tx_ring_struct->ring_size - + ((txr->tx_prod - txr->tx_cons) & + txr->tx_ring_struct->ring_mask) - 1; +} + +static uint16_t bnxt_start_xmit(struct rte_mbuf *tx_pkt, + struct bnxt_tx_queue *txq) +{ + struct bnxt_tx_ring_info *txr = txq->tx_ring; + struct tx_bd_long *txbd; + struct tx_bd_long_hi *txbd1; + uint32_t vlan_tag_flags, cfa_action; + bool long_bd = false; + uint16_t last_prod = 0; + struct rte_mbuf *m_seg; + struct bnxt_sw_tx_bd *tx_buf; + static const uint32_t lhint_arr[4] = { + TX_BD_LONG_FLAGS_LHINT_LT512, + TX_BD_LONG_FLAGS_LHINT_LT1K, + TX_BD_LONG_FLAGS_LHINT_LT2K, + TX_BD_LONG_FLAGS_LHINT_LT2K + }; + + if (tx_pkt->ol_flags & (PKT_TX_TCP_SEG | PKT_TX_TCP_CKSUM | + PKT_TX_UDP_CKSUM | PKT_TX_IP_CKSUM | + PKT_TX_VLAN_PKT)) + long_bd = true; + + tx_buf = &txr->tx_buf_ring[txr->tx_prod]; + tx_buf->mbuf = tx_pkt; + tx_buf->nr_bds = long_bd + tx_pkt->nb_segs; + last_prod = (txr->tx_prod + tx_buf->nr_bds - 1) & + txr->tx_ring_struct->ring_mask; + + if (unlikely(bnxt_tx_avail(txr) < tx_buf->nr_bds)) + return -ENOMEM; + + txbd = &txr->tx_desc_ring[txr->tx_prod]; + txbd->opaque = txr->tx_prod; + txbd->flags_type = tx_buf->nr_bds << TX_BD_LONG_FLAGS_BD_CNT_SFT; + txbd->len = tx_pkt->data_len; + if (txbd->len >= 2014) + txbd->flags_type |= TX_BD_LONG_FLAGS_LHINT_GTE2K; + else + txbd->flags_type |= lhint_arr[txbd->len >> 9]; + txbd->addr = rte_cpu_to_le_32(RTE_MBUF_DATA_DMA_ADDR(tx_buf->mbuf)); + + if (long_bd) { + txbd->flags_type |= TX_BD_LONG_TYPE_TX_BD_LONG; + vlan_tag_flags = 0; + cfa_action = 0; + if (tx_buf->mbuf->ol_flags & PKT_TX_VLAN_PKT) { + /* shurd: Should this mask at + * TX_BD_LONG_CFA_META_VLAN_VID_MASK? + */ + vlan_tag_flags = TX_BD_LONG_CFA_META_KEY_VLAN_TAG | + tx_buf->mbuf->vlan_tci; + /* Currently supports 8021Q, 8021AD vlan offloads + * QINQ1, QINQ2, QINQ3 vlan headers are deprecated + */ + /* DPDK only supports 802.11q VLAN packets */ + vlan_tag_flags |= + TX_BD_LONG_CFA_META_VLAN_TPID_TPID8100; + } + + txr->tx_prod = RING_NEXT(txr->tx_ring_struct, txr->tx_prod); + + txbd1 = (struct tx_bd_long_hi *) + &txr->tx_desc_ring[txr->tx_prod]; + txbd1->lflags = 0; + txbd1->cfa_meta = vlan_tag_flags; + txbd1->cfa_action = cfa_action; + + if (tx_pkt->ol_flags & PKT_TX_TCP_SEG) { + /* TSO */ + txbd1->lflags = TX_BD_LONG_LFLAGS_LSO; + txbd1->hdr_size = tx_pkt->l2_len + tx_pkt->l3_len + + tx_pkt->l4_len; + txbd1->mss = tx_pkt->tso_segsz; + + } else if (tx_pkt->ol_flags & (PKT_TX_TCP_CKSUM | + PKT_TX_UDP_CKSUM)) { + /* TCP/UDP CSO */ + txbd1->lflags = TX_BD_LONG_LFLAGS_TCP_UDP_CHKSUM; + txbd1->mss = 0; + + } else if (tx_pkt->ol_flags & PKT_TX_IP_CKSUM) { + /* IP CSO */ + txbd1->lflags = TX_BD_LONG_LFLAGS_IP_CHKSUM; + txbd1->mss = 0; + } + } else { + txbd->flags_type |= TX_BD_SHORT_TYPE_TX_BD_SHORT; + } + + m_seg = tx_pkt->next; + /* i is set at the end of the if(long_bd) block */ + while (txr->tx_prod != last_prod) { + txr->tx_prod = RING_NEXT(txr->tx_ring_struct, txr->tx_prod); + tx_buf = &txr->tx_buf_ring[txr->tx_prod]; + + txbd = &txr->tx_desc_ring[txr->tx_prod]; + txbd->addr = rte_cpu_to_le_32(RTE_MBUF_DATA_DMA_ADDR(m_seg)); + txbd->flags_type = TX_BD_SHORT_TYPE_TX_BD_SHORT; + txbd->len = m_seg->data_len; + + m_seg = m_seg->next; + } + + txbd->flags_type |= TX_BD_LONG_FLAGS_PACKET_END; + + txr->tx_prod = RING_NEXT(txr->tx_ring_struct, txr->tx_prod); + + return 0; +} + +static void bnxt_tx_cmp(struct bnxt_tx_queue *txq, int nr_pkts) +{ + struct bnxt_tx_ring_info *txr = txq->tx_ring; + uint16_t cons = txr->tx_cons; + int i, j; + + for (i = 0; i < nr_pkts; i++) { + struct bnxt_sw_tx_bd *tx_buf; + struct rte_mbuf *mbuf; + + tx_buf = &txr->tx_buf_ring[cons]; + cons = RING_NEXT(txr->tx_ring_struct, cons); + mbuf = tx_buf->mbuf; + tx_buf->mbuf = NULL; + + /* EW - no need to unmap DMA memory? */ + + for (j = 1; j < tx_buf->nr_bds; j++) + cons = RING_NEXT(txr->tx_ring_struct, cons); + rte_pktmbuf_free(mbuf); + } + + txr->tx_cons = cons; +} + +static int bnxt_handle_tx_cp(struct bnxt_tx_queue *txq) +{ + struct bnxt_cp_ring_info *cpr = txq->cp_ring; + uint32_t raw_cons = cpr->cp_raw_cons; + uint32_t cons; + int nb_tx_pkts = 0; + struct tx_cmpl *txcmp; + + if ((txq->tx_ring->tx_ring_struct->ring_size - + (bnxt_tx_avail(txq->tx_ring))) > + txq->tx_free_thresh) { + while (1) { + cons = RING_CMP(cpr->cp_ring_struct, raw_cons); + txcmp = (struct tx_cmpl *)&cpr->cp_desc_ring[cons]; + + if (!CMP_VALID(txcmp, raw_cons, cpr->cp_ring_struct)) + break; + + if (CMP_TYPE(txcmp) == TX_CMPL_TYPE_TX_L2) + nb_tx_pkts++; + else + RTE_LOG(DEBUG, PMD, + "Unhandled CMP type %02x\n", + CMP_TYPE(txcmp)); + raw_cons = NEXT_RAW_CMP(raw_cons); + } + if (nb_tx_pkts) + bnxt_tx_cmp(txq, nb_tx_pkts); + cpr->cp_raw_cons = raw_cons; + B_CP_DIS_DB(cpr, cpr->cp_raw_cons); + } + return nb_tx_pkts; +} + +uint16_t bnxt_xmit_pkts(void *tx_queue, struct rte_mbuf **tx_pkts, + uint16_t nb_pkts) +{ + struct bnxt_tx_queue *txq = tx_queue; + uint16_t nb_tx_pkts = 0; + uint16_t db_mask = txq->tx_ring->tx_ring_struct->ring_size >> 2; + uint16_t last_db_mask = 0; + + /* Handle TX completions */ + bnxt_handle_tx_cp(txq); + + /* Handle TX burst request */ + for (nb_tx_pkts = 0; nb_tx_pkts < nb_pkts; nb_tx_pkts++) { + if (bnxt_start_xmit(tx_pkts[nb_tx_pkts], txq)) { + break; + } else if ((nb_tx_pkts & db_mask) != last_db_mask) { + B_TX_DB(txq->tx_ring->tx_doorbell, + txq->tx_ring->tx_prod); + last_db_mask = nb_tx_pkts & db_mask; + } + } + if (nb_tx_pkts) + B_TX_DB(txq->tx_ring->tx_doorbell, txq->tx_ring->tx_prod); + + return nb_tx_pkts; +} diff --git a/drivers/net/bnxt/bnxt_txr.h b/drivers/net/bnxt/bnxt_txr.h new file mode 100644 index 0000000..712ce65 --- /dev/null +++ b/drivers/net/bnxt/bnxt_txr.h @@ -0,0 +1,71 @@ +/*- + * BSD LICENSE + * + * Copyright(c) Broadcom Limited. + * All rights reserved. + * + * Redistribution and use in source and binary forms, with or without + * modification, are permitted provided that the following conditions + * are met: + * + * * Redistributions of source code must retain the above copyright + * notice, this list of conditions and the following disclaimer. + * * Redistributions in binary form must reproduce the above copyright + * notice, this list of conditions and the following disclaimer in + * the documentation and/or other materials provided with the + * distribution. + * * Neither the name of Broadcom Corporation nor the names of its + * contributors may be used to endorse or promote products derived + * from this software without specific prior written permission. + * + * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS + * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT + * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR + * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT + * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, + * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT + * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, + * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY + * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT + * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE + * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. + */ + +#ifndef _BNXT_TXR_H_ +#define _BNXT_TXR_H_ + +#define MAX_TX_RINGS 16 +#define BNXT_TX_PUSH_THRESH 92 + +#define B_TX_DB(db, prod) \ + (*(uint32_t *)db = (DB_KEY_TX | prod)) + +struct bnxt_tx_ring_info { + uint16_t tx_prod; + uint16_t tx_cons; + void *tx_doorbell; + + struct tx_bd_long *tx_desc_ring; + struct bnxt_sw_tx_bd *tx_buf_ring; + + phys_addr_t tx_desc_mapping; + +#define BNXT_DEV_STATE_CLOSING 0x1 + uint32_t dev_state; + + struct bnxt_ring *tx_ring_struct; +}; + +struct bnxt_sw_tx_bd { + struct rte_mbuf *mbuf; /* mbuf associated with TX descriptor */ + uint8_t is_gso; + unsigned short nr_bds; +}; + +void bnxt_free_tx_rings(struct bnxt *bp); +int bnxt_init_one_tx_ring(struct bnxt_tx_queue *txq); +void bnxt_init_tx_ring_struct(struct bnxt_tx_queue *txq); +uint16_t bnxt_xmit_pkts(void *tx_queue, struct rte_mbuf **tx_pkts, + uint16_t nb_pkts); + +#endif diff --git a/drivers/net/bnxt/hsi_struct_def_dpdk.h b/drivers/net/bnxt/hsi_struct_def_dpdk.h index 91a83d1..1543f20 100644 --- a/drivers/net/bnxt/hsi_struct_def_dpdk.h +++ b/drivers/net/bnxt/hsi_struct_def_dpdk.h @@ -99,6 +99,518 @@ struct ctx_hw_stats64 { #define HWRM_ERR_CODE_INVALID_PARAMS (UINT32_C(0x2)) #define HWRM_ERR_CODE_RESOURCE_ACCESS_DENIED (UINT32_C(0x3)) +/* Short TX BD (16 bytes) */ +struct tx_bd_short { + /* + * All bits in this field must be valid on the first BD of a packet. + * Only the packet_end bit must be valid for the remaining BDs of a + * packet. + */ + /* This value identifies the type of buffer descriptor. */ + #define TX_BD_SHORT_TYPE_MASK UINT32_C(0x3f) + #define TX_BD_SHORT_TYPE_SFT 0 + /* + * Indicates that this BD is 16B long and is used for normal L2 + * packet transmission. + */ + #define TX_BD_SHORT_TYPE_TX_BD_SHORT (UINT32_C(0x0) << 0) + /* + * If set to 1, the packet ends with the data in the buffer pointed to + * by this descriptor. This flag must be valid on every BD. + */ + #define TX_BD_SHORT_FLAGS_PACKET_END UINT32_C(0x40) + /* + * If set to 1, the device will not generate a completion for this + * transmit packet unless there is an error in it's processing. If this + * bit is set to 0, then the packet will be completed normally. This bit + * must be valid only on the first BD of a packet. + */ + #define TX_BD_SHORT_FLAGS_NO_CMPL UINT32_C(0x80) + /* + * This value indicates how many 16B BD locations are consumed in the + * ring by this packet. A value of 1 indicates that this BD is the only + * BD (and that the it is a short BD). A value of 3 indicates either 3 + * short BDs or 1 long BD and one short BD in the packet. A value of 0 + * indicates that there are 32 BD locations in the packet (the maximum). + * This field is valid only on the first BD of a packet. + */ + #define TX_BD_SHORT_FLAGS_BD_CNT_MASK UINT32_C(0x1f00) + #define TX_BD_SHORT_FLAGS_BD_CNT_SFT 8 + /* + * This value is a hint for the length of the entire packet. It is used + * by the chip to optimize internal processing. The packet will be + * dropped if the hint is too short. This field is valid only on the + * first BD of a packet. + */ + #define TX_BD_SHORT_FLAGS_LHINT_MASK UINT32_C(0x6000) + #define TX_BD_SHORT_FLAGS_LHINT_SFT 13 + /* indicates packet length < 512B */ + #define TX_BD_SHORT_FLAGS_LHINT_LT512 (UINT32_C(0x0) << 13) + /* indicates 512 <= packet length < 1KB */ + #define TX_BD_SHORT_FLAGS_LHINT_LT1K (UINT32_C(0x1) << 13) + /* indicates 1KB <= packet length < 2KB */ + #define TX_BD_SHORT_FLAGS_LHINT_LT2K (UINT32_C(0x2) << 13) + /* indicates packet length >= 2KB */ + #define TX_BD_SHORT_FLAGS_LHINT_GTE2K (UINT32_C(0x3) << 13) + #define TX_BD_SHORT_FLAGS_LHINT_LAST TX_BD_SHORT_FLAGS_LHINT_GTE2K + /* + * If set to 1, the device immediately updates the Send Consumer Index + * after the buffer associated with this descriptor has been transferred + * via DMA to NIC memory from host memory. An interrupt may or may not + * be generated according to the state of the interrupt avoidance + * mechanisms. If this bit is set to 0, then the Consumer Index is only + * updated as soon as one of the host interrupt coalescing conditions + * has been met. This bit must be valid on the first BD of a packet. + */ + #define TX_BD_SHORT_FLAGS_COAL_NOW UINT32_C(0x8000) + /* + * All bits in this field must be valid on the first BD of a packet. + * Only the packet_end bit must be valid for the remaining BDs of a + * packet. + */ + #define TX_BD_SHORT_FLAGS_MASK UINT32_C(0xffc0) + #define TX_BD_SHORT_FLAGS_SFT 6 + uint16_t flags_type; + + /* + * This is the length of the host physical buffer this BD describes in + * bytes. This field must be valid on all BDs of a packet. + */ + uint16_t len; + /* + * The opaque data field is pass through to the completion and can be + * used for any data that the driver wants to associate with the + * transmit BD. This field must be valid on the first BD of a packet. + */ + uint32_t opaque; + + /* + * This is the host physical address for the portion of the packet + * described by this TX BD. This value must be valid on all BDs of a + * packet. + */ + uint64_t addr; +} __attribute__((packed)); + +/* Long TX BD (32 bytes split to 2 16-byte struct) */ +struct tx_bd_long { + /* + * All bits in this field must be valid on the first BD of a packet. + * Only the packet_end bit must be valid for the remaining BDs of a + * packet. + */ + /* This value identifies the type of buffer descriptor. */ + #define TX_BD_LONG_TYPE_MASK UINT32_C(0x3f) + #define TX_BD_LONG_TYPE_SFT 0 + /* + * Indicates that this BD is 32B long and is used for normal L2 + * packet transmission. + */ + #define TX_BD_LONG_TYPE_TX_BD_LONG (UINT32_C(0x10) << 0) + /* + * If set to 1, the packet ends with the data in the buffer pointed to + * by this descriptor. This flag must be valid on every BD. + */ + #define TX_BD_LONG_FLAGS_PACKET_END UINT32_C(0x40) + /* + * If set to 1, the device will not generate a completion for this + * transmit packet unless there is an error in it's processing. If this + * bit is set to 0, then the packet will be completed normally. This bit + * must be valid only on the first BD of a packet. + */ + #define TX_BD_LONG_FLAGS_NO_CMPL UINT32_C(0x80) + /* + * This value indicates how many 16B BD locations are consumed in the + * ring by this packet. A value of 1 indicates that this BD is the only + * BD (and that the it is a short BD). A value of 3 indicates either 3 + * short BDs or 1 long BD and one short BD in the packet. A value of 0 + * indicates that there are 32 BD locations in the packet (the maximum). + * This field is valid only on the first BD of a packet. + */ + #define TX_BD_LONG_FLAGS_BD_CNT_MASK UINT32_C(0x1f00) + #define TX_BD_LONG_FLAGS_BD_CNT_SFT 8 + /* + * This value is a hint for the length of the entire packet. It is used + * by the chip to optimize internal processing. The packet will be + * dropped if the hint is too short. This field is valid only on the + * first BD of a packet. + */ + #define TX_BD_LONG_FLAGS_LHINT_MASK UINT32_C(0x6000) + #define TX_BD_LONG_FLAGS_LHINT_SFT 13 + /* indicates packet length < 512B */ + #define TX_BD_LONG_FLAGS_LHINT_LT512 (UINT32_C(0x0) << 13) + /* indicates 512 <= packet length < 1KB */ + #define TX_BD_LONG_FLAGS_LHINT_LT1K (UINT32_C(0x1) << 13) + /* indicates 1KB <= packet length < 2KB */ + #define TX_BD_LONG_FLAGS_LHINT_LT2K (UINT32_C(0x2) << 13) + /* indicates packet length >= 2KB */ + #define TX_BD_LONG_FLAGS_LHINT_GTE2K (UINT32_C(0x3) << 13) + #define TX_BD_LONG_FLAGS_LHINT_LAST TX_BD_LONG_FLAGS_LHINT_GTE2K + /* + * If set to 1, the device immediately updates the Send Consumer Index + * after the buffer associated with this descriptor has been transferred + * via DMA to NIC memory from host memory. An interrupt may or may not + * be generated according to the state of the interrupt avoidance + * mechanisms. If this bit is set to 0, then the Consumer Index is only + * updated as soon as one of the host interrupt coalescing conditions + * has been met. This bit must be valid on the first BD of a packet. + */ + #define TX_BD_LONG_FLAGS_COAL_NOW UINT32_C(0x8000) + /* + * All bits in this field must be valid on the first BD of a packet. + * Only the packet_end bit must be valid for the remaining BDs of a + * packet. + */ + #define TX_BD_LONG_FLAGS_MASK UINT32_C(0xffc0) + #define TX_BD_LONG_FLAGS_SFT 6 + uint16_t flags_type; + + /* + * This is the length of the host physical buffer this BD describes in + * bytes. This field must be valid on all BDs of a packet. + */ + uint16_t len; + + /* + * The opaque data field is pass through to the completion and can be + * used for any data that the driver wants to associate with the + * transmit BD. This field must be valid on the first BD of a packet. + */ + uint32_t opaque; + + /* + * This is the host physical address for the portion of the packet + * described by this TX BD. This value must be valid on all BDs of a + * packet. + */ + uint64_t addr; +} __attribute__((packed)); + +/* last 16 bytes of Long TX BD */ + +struct tx_bd_long_hi { + /* + * All bits in this field must be valid on the first BD of a packet. + * Their value on other BDs of the packet will be ignored. + */ + /* + * If set to 1, the controller replaces the TCP/UPD checksum fields of + * normal TCP/UPD checksum, or the inner TCP/UDP checksum field of the + * encapsulated TCP/UDP packets with the hardware calculated TCP/UDP + * checksum for the packet associated with this descriptor. This bit + * must be valid on the first BD of a packet. + */ + #define TX_BD_LONG_LFLAGS_TCP_UDP_CHKSUM UINT32_C(0x1) + /* + * If set to 1, the controller replaces the IP checksum of the normal + * packets, or the inner IP checksum of the encapsulated packets with + * the hardware calculated IP checksum for the packet associated with + * this descriptor. This bit must be valid on the first BD of a packet. + */ + #define TX_BD_LONG_LFLAGS_IP_CHKSUM UINT32_C(0x2) + /* + * If set to 1, the controller will not append an Ethernet CRC to the + * end of the frame. This bit must be valid on the first BD of a packet. + * Packet must be 64B or longer when this flag is set. It is not useful + * to use this bit with any form of TX offload such as CSO or LSO. The + * intent is that the packet from the host already has a valid Ethernet + * CRC on the packet. + */ + #define TX_BD_LONG_LFLAGS_NOCRC UINT32_C(0x4) + /* + * If set to 1, the device will record the time at which the packet was + * actually transmitted at the TX MAC. This bit must be valid on the + * first BD of a packet. + */ + #define TX_BD_LONG_LFLAGS_STAMP UINT32_C(0x8) + /* + * If set to 1, The controller replaces the tunnel IP checksum field + * with hardware calculated IP checksum for the IP header of the packet + * associated with this descriptor. In case of VXLAN, the controller + * also replaces the outer header UDP checksum with hardware calculated + * UDP checksum for the packet associated with this descriptor. + */ + #define TX_BD_LONG_LFLAGS_T_IP_CHKSUM UINT32_C(0x10) + /* + * If set to 1, the device will treat this packet with LSO(Large Send + * Offload) processing for both normal or encapsulated packets, which is + * a form of TCP segmentation. When this bit is 1, the hdr_size and mss + * fields must be valid. The driver doesn't need to set t_ip_chksum, + * ip_chksum, and tcp_udp_chksum flags since the controller will replace + * the appropriate checksum fields for segmented packets. When this bit + * is 1, the hdr_size and mss fields must be valid. + */ + #define TX_BD_LONG_LFLAGS_LSO UINT32_C(0x20) + /* + * If set to zero when LSO is '1', then the IPID will be treated as a + * 16b number and will be wrapped if it exceeds a value of 0xffff. If + * set to one when LSO is '1', then the IPID will be treated as a 15b + * number and will be wrapped if it exceeds a value 0f 0x7fff. + */ + #define TX_BD_LONG_LFLAGS_IPID_FMT UINT32_C(0x40) + /* + * If set to zero when LSO is '1', then the IPID of the tunnel IP header + * will not be modified during LSO operations. If set to one when LSO is + * '1', then the IPID of the tunnel IP header will be incremented for + * each subsequent segment of an LSO operation. + */ + #define TX_BD_LONG_LFLAGS_T_IPID UINT32_C(0x80) + /* + * If set to '1', then the RoCE ICRC will be appended to the packet. + * Packet must be a valid RoCE format packet. + */ + #define TX_BD_LONG_LFLAGS_ROCE_CRC UINT32_C(0x100) + /* + * If set to '1', then the FCoE CRC will be appended to the packet. + * Packet must be a valid FCoE format packet. + */ + #define TX_BD_LONG_LFLAGS_FCOE_CRC UINT32_C(0x200) + uint16_t lflags; + + /* + * When LSO is '1', this field must contain the offset of the TCP + * payload from the beginning of the packet in as 16b words. In case of + * encapsulated/tunneling packet, this field contains the offset of the + * inner TCP payload from beginning of the packet as 16-bit words. This + * value must be valid on the first BD of a packet. + */ + #define TX_BD_LONG_HDR_SIZE_MASK UINT32_C(0x1ff) + #define TX_BD_LONG_HDR_SIZE_SFT 0 + uint16_t hdr_size; + + /* + * This is the MSS value that will be used to do the LSO processing. The + * value is the length in bytes of the TCP payload for each segment + * generated by the LSO operation. This value must be valid on the first + * BD of a packet. + */ + #define TX_BD_LONG_MSS_MASK UINT32_C(0x7fff) + #define TX_BD_LONG_MSS_SFT 0 + uint32_t mss; + + uint16_t unused_2; + + /* + * This value selects a CFA action to perform on the packet. Set this + * value to zero if no CFA action is desired. This value must be valid + * on the first BD of a packet. + */ + uint16_t cfa_action; + + /* + * This value is action meta-data that defines CFA edit operations that + * are done in addition to any action editing. + */ + /* When key=1, This is the VLAN tag VID value. */ + #define TX_BD_LONG_CFA_META_VLAN_VID_MASK UINT32_C(0xfff) + #define TX_BD_LONG_CFA_META_VLAN_VID_SFT 0 + /* When key=1, This is the VLAN tag DE value. */ + #define TX_BD_LONG_CFA_META_VLAN_DE UINT32_C(0x1000) + /* When key=1, This is the VLAN tag PRI value. */ + #define TX_BD_LONG_CFA_META_VLAN_PRI_MASK UINT32_C(0xe000) + #define TX_BD_LONG_CFA_META_VLAN_PRI_SFT 13 + /* When key=1, This is the VLAN tag TPID select value. */ + #define TX_BD_LONG_CFA_META_VLAN_TPID_MASK UINT32_C(0x70000) + #define TX_BD_LONG_CFA_META_VLAN_TPID_SFT 16 + /* 0x88a8 */ + #define TX_BD_LONG_CFA_META_VLAN_TPID_TPID88A8 (UINT32_C(0x0) << 16) + /* 0x8100 */ + #define TX_BD_LONG_CFA_META_VLAN_TPID_TPID8100 (UINT32_C(0x1) << 16) + /* 0x9100 */ + #define TX_BD_LONG_CFA_META_VLAN_TPID_TPID9100 (UINT32_C(0x2) << 16) + /* 0x9200 */ + #define TX_BD_LONG_CFA_META_VLAN_TPID_TPID9200 (UINT32_C(0x3) << 16) + /* 0x9300 */ + #define TX_BD_LONG_CFA_META_VLAN_TPID_TPID9300 (UINT32_C(0x4) << 16) + /* Value programmed in CFA VLANTPID register. */ + #define TX_BD_LONG_CFA_META_VLAN_TPID_TPIDCFG (UINT32_C(0x5) << 16) + #define TX_BD_LONG_CFA_META_VLAN_TPID_LAST \ + TX_BD_LONG_CFA_META_VLAN_TPID_TPIDCFG + /* When key=1, This is the VLAN tag TPID select value. */ + #define TX_BD_LONG_CFA_META_VLAN_RESERVED_MASK UINT32_C(0xff80000) + #define TX_BD_LONG_CFA_META_VLAN_RESERVED_SFT 19 + /* + * This field identifies the type of edit to be performed on the packet. + * This value must be valid on the first BD of a packet. + */ + #define TX_BD_LONG_CFA_META_KEY_MASK UINT32_C(0xf0000000) + #define TX_BD_LONG_CFA_META_KEY_SFT 28 + /* No editing */ + #define TX_BD_LONG_CFA_META_KEY_NONE (UINT32_C(0x0) << 28) + /* + * - meta[17:16] - TPID select value (0 = 0x8100). - meta[15:12] + * - PRI/DE value. - meta[11:0] - VID value. + */ + #define TX_BD_LONG_CFA_META_KEY_VLAN_TAG (UINT32_C(0x1) << 28) + #define TX_BD_LONG_CFA_META_KEY_LAST TX_BD_LONG_CFA_META_KEY_VLAN_TAG + uint32_t cfa_meta; +} __attribute__((packed)); + +/* Completion Ring Structures */ +/* Note: This structure is used by the HWRM to communicate HWRM Error. */ +/* Base Completion Record (16 bytes) */ +struct cmpl_base { + /* unused is 10 b */ + /* + * This field indicates the exact type of the completion. By convention, + * the LSB identifies the length of the record in 16B units. Even values + * indicate 16B records. Odd values indicate 32B records. + */ + #define CMPL_BASE_TYPE_MASK UINT32_C(0x3f) + #define CMPL_BASE_TYPE_SFT 0 + /* TX L2 completion: Completion of TX packet. Length = 16B */ + #define CMPL_BASE_TYPE_TX_L2 (UINT32_C(0x0) << 0) + /* + * RX L2 completion: Completion of and L2 RX packet. + * Length = 32B + */ + #define CMPL_BASE_TYPE_RX_L2 (UINT32_C(0x11) << 0) + /* + * RX Aggregation Buffer completion : Completion of an L2 + * aggregation buffer in support of TPA, HDS, or Jumbo packet + * completion. Length = 16B + */ + #define CMPL_BASE_TYPE_RX_AGG (UINT32_C(0x12) << 0) + /* + * RX L2 TPA Start Completion: Completion at the beginning of a + * TPA operation. Length = 32B + */ + #define CMPL_BASE_TYPE_RX_TPA_START (UINT32_C(0x13) << 0) + /* + * RX L2 TPA End Completion: Completion at the end of a TPA + * operation. Length = 32B + */ + #define CMPL_BASE_TYPE_RX_TPA_END (UINT32_C(0x15) << 0) + /* + * Statistics Ejection Completion: Completion of statistics data + * ejection buffer. Length = 16B + */ + #define CMPL_BASE_TYPE_STAT_EJECT (UINT32_C(0x1a) << 0) + /* HWRM Command Completion: Completion of an HWRM command. */ + #define CMPL_BASE_TYPE_HWRM_DONE (UINT32_C(0x20) << 0) + /* Forwarded HWRM Request */ + #define CMPL_BASE_TYPE_HWRM_FWD_REQ (UINT32_C(0x22) << 0) + /* Forwarded HWRM Response */ + #define CMPL_BASE_TYPE_HWRM_FWD_RESP (UINT32_C(0x24) << 0) + /* HWRM Asynchronous Event Information */ + #define CMPL_BASE_TYPE_HWRM_ASYNC_EVENT (UINT32_C(0x2e) << 0) + /* CQ Notification */ + #define CMPL_BASE_TYPE_CQ_NOTIFICATION (UINT32_C(0x30) << 0) + /* SRQ Threshold Event */ + #define CMPL_BASE_TYPE_SRQ_EVENT (UINT32_C(0x32) << 0) + /* DBQ Threshold Event */ + #define CMPL_BASE_TYPE_DBQ_EVENT (UINT32_C(0x34) << 0) + /* QP Async Notification */ + #define CMPL_BASE_TYPE_QP_EVENT (UINT32_C(0x38) << 0) + /* Function Async Notification */ + #define CMPL_BASE_TYPE_FUNC_EVENT (UINT32_C(0x3a) << 0) + uint16_t type; + + uint16_t info1; + uint32_t info2; + + /* + * This value is written by the NIC such that it will be different for + * each pass through the completion queue. The even passes will write 1. + * The odd passes will write 0. + */ + #define CMPL_BASE_V UINT32_C(0x1) + /* info3 is 31 b */ + #define CMPL_BASE_INFO3_MASK UINT32_C(0xfffffffe) + #define CMPL_BASE_INFO3_SFT 1 + uint32_t info3_v; + + uint32_t info4; +} __attribute__((packed)); + +/* TX Completion Record (16 bytes) */ +struct tx_cmpl { + /* + * This field indicates the exact type of the completion. By convention, + * the LSB identifies the length of the record in 16B units. Even values + * indicate 16B records. Odd values indicate 32B records. + */ + #define TX_CMPL_TYPE_MASK UINT32_C(0x3f) + #define TX_CMPL_TYPE_SFT 0 + /* TX L2 completion: Completion of TX packet. Length = 16B */ + #define TX_CMPL_TYPE_TX_L2 (UINT32_C(0x0) << 0) + /* + * When this bit is '1', it indicates a packet that has an error of some + * type. Type of error is indicated in error_flags. + */ + #define TX_CMPL_FLAGS_ERROR UINT32_C(0x40) + /* + * When this bit is '1', it indicates that the packet completed was + * transmitted using the push acceleration data provided by the driver. + * When this bit is '0', it indicates that the packet had not push + * acceleration data written or was executed as a normal packet even + * though push data was provided. + */ + #define TX_CMPL_FLAGS_PUSH UINT32_C(0x80) + #define TX_CMPL_FLAGS_MASK UINT32_C(0xffc0) + #define TX_CMPL_FLAGS_SFT 6 + uint16_t flags_type; + + uint16_t unused_0; + + /* + * This is a copy of the opaque field from the first TX BD of this + * transmitted packet. + */ + uint32_t opaque; + + /* + * This value is written by the NIC such that it will be different for + * each pass through the completion queue. The even passes will write 1. + * The odd passes will write 0. + */ + #define TX_CMPL_V UINT32_C(0x1) + /* + * This error indicates that there was some sort of problem with the BDs + * for the packet. + */ + #define TX_CMPL_ERRORS_BUFFER_ERROR_MASK UINT32_C(0xe) + #define TX_CMPL_ERRORS_BUFFER_ERROR_SFT 1 + /* No error */ + #define TX_CMPL_ERRORS_BUFFER_ERROR_NO_ERROR (UINT32_C(0x0) << 1) + /* Bad Format: BDs were not formatted correctly. */ + #define TX_CMPL_ERRORS_BUFFER_ERROR_BAD_FMT (UINT32_C(0x2) << 1) + #define TX_CMPL_ERRORS_BUFFER_ERROR_LAST \ + TX_CMPL_ERRORS_BUFFER_ERROR_BAD_FMT + /* + * When this bit is '1', it indicates that the length of the packet was + * zero. No packet was transmitted. + */ + #define TX_CMPL_ERRORS_ZERO_LENGTH_PKT UINT32_C(0x10) + /* + * When this bit is '1', it indicates that the packet was longer than + * the programmed limit in TDI. No packet was transmitted. + */ + #define TX_CMPL_ERRORS_EXCESSIVE_BD_LENGTH UINT32_C(0x20) + /* + * When this bit is '1', it indicates that one or more of the BDs + * associated with this packet generated a PCI error. This probably + * means the address was not valid. + */ + #define TX_CMPL_ERRORS_DMA_ERROR UINT32_C(0x40) + /* + * When this bit is '1', it indicates that the packet was longer than + * indicated by the hint. No packet was transmitted. + */ + #define TX_CMPL_ERRORS_HINT_TOO_SHORT UINT32_C(0x80) + /* + * When this bit is '1', it indicates that the packet was dropped due to + * Poison TLP error on one or more of the TLPs in the PXP completion. + */ + #define TX_CMPL_ERRORS_POISON_TLP_ERROR UINT32_C(0x100) + #define TX_CMPL_ERRORS_MASK UINT32_C(0xfffe) + #define TX_CMPL_ERRORS_SFT 1 + uint16_t errors_v; + + uint16_t unused_1; + uint32_t unused_2; +} __attribute__((packed)) tx_cmpl_t, *ptx_cmpl_t; + /* HWRM Forwarded Request (16 bytes) */ struct hwrm_fwd_req_cmpl { /* Length of forwarded request in bytes. */ -- 1.9.1