From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id C21D1A04BB; Tue, 6 Oct 2020 19:38:03 +0200 (CEST) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 25102F64; Tue, 6 Oct 2020 19:38:02 +0200 (CEST) Received: from mail-pl1-f196.google.com (mail-pl1-f196.google.com [209.85.214.196]) by dpdk.org (Postfix) with ESMTP id D1363F3E for ; Tue, 6 Oct 2020 19:38:00 +0200 (CEST) Received: by mail-pl1-f196.google.com with SMTP id bb1so1558423plb.2 for ; Tue, 06 Oct 2020 10:38:00 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=broadcom.com; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version; bh=pzgdqTSfqY/VZDvAVgWrY6Eb2R19/HvMInygp0a4jjk=; b=OQEdrtazZQ64sc473tNZG6Ue7Gwa/bMOUWD3YiMNKux0wp9C4LcTOlQdK75pIBjhZr i259IlQ3IWdpyn/HpwBIVFHUdDyAsVHyvXy4BlNoWZopuXmUcxTGpypbijfsxe8zVSDb UYgBWRyz5wZfAdUEFibtNQqwLhePFAqv6og6w= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version; bh=pzgdqTSfqY/VZDvAVgWrY6Eb2R19/HvMInygp0a4jjk=; b=JuEoOkcA4RW9sEiPWchkn0A3ymhQyqDVNnCOb+Vtoo7i358xDd2O4sZBZl4NdfKkOe tHjG4k35buiTLsfK7pwhUFRa3XQsavFXpA+bYXUb0Wc1CcFuO11LhzIMq1xkw9DCTSB6 U72L3EW1lZ37ahpUMBIMOUM2ZffAtIrzYKLHp4LOSt/h4BR0r4xVoeSo0lCfswbchdLI +IJLIRATN1YLbt8HUbFMSbgVw7Ou5yw+YTTLk1rZPzzgshi4ScpAVz1+YMW2Ku2dLiHG i/kPjBB3SZmeDbvc4nubJlpXZ6UaBGuWwHsZhWTgXQds3BNeTWr176P3v7udH3CbaYDT L8+Q== X-Gm-Message-State: AOAM53362OQ8eqVkO5OfJCOy2ofDFvVOoa5tCDL0CpfYE7/jdo6/0oAO tGGOB235ViK47c6mlNyprl/52A== X-Google-Smtp-Source: ABdhPJyspbOuc9LUtarjTcEYbJiCX1SwRx/47a6+FCawMHtlvuNcAPwYOhiOi+zOXRANQyYc9520jw== X-Received: by 2002:a17:90a:9b0f:: with SMTP id f15mr5267507pjp.89.1602005878680; Tue, 06 Oct 2020 10:37:58 -0700 (PDT) Received: from localhost.localdomain ([192.19.231.250]) by smtp.gmail.com with ESMTPSA id q65sm3738238pga.88.2020.10.06.10.37.57 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 06 Oct 2020 10:37:57 -0700 (PDT) From: Lance Richardson To: Ajit Khaparde , Somnath Kotur Cc: dev@dpdk.org Date: Tue, 6 Oct 2020 13:37:52 -0400 Message-Id: <20201006173752.104002-1-lance.richardson@broadcom.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20201002134002.83845-1-lance.richardson@broadcom.com> References: <20201002134002.83845-1-lance.richardson@broadcom.com> MIME-Version: 1.0 Content-Type: text/plain Content-Transfer-Encoding: 8bit X-Content-Filtered-By: Mailman/MimeDel 2.1.15 Subject: [dpdk-dev] [PATCH v3] net/bnxt: support fast mbuf free X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" Add support for DEV_TX_OFFLOAD_MBUF_FAST_FREE to bnxt vector mode transmit. This offload may be enabled only when multi-segment transmit is not needed, all transmitted mbufs for a given queue will be allocated from the same pool, and all transmitted mbufs will have a reference count of 1. Signed-off-by: Lance Richardson --- - v2: Fixed mangling of advertised tx offload capabilities. - v3: Reduced duplicated code. Enable for non-vector transmit path as well. doc/guides/nics/features/bnxt.ini | 1 + drivers/net/bnxt/bnxt_ethdev.c | 11 ++++--- drivers/net/bnxt/bnxt_rxtx_vec_common.h | 32 ++++++++++++++++++++ drivers/net/bnxt/bnxt_rxtx_vec_neon.c | 38 ++++-------------------- drivers/net/bnxt/bnxt_rxtx_vec_sse.c | 39 ++++--------------------- drivers/net/bnxt/bnxt_txq.c | 2 ++ drivers/net/bnxt/bnxt_txq.h | 1 + drivers/net/bnxt/bnxt_txr.c | 5 +++- drivers/net/bnxt/bnxt_txr.h | 27 +++++++++++++++++ 9 files changed, 84 insertions(+), 72 deletions(-) diff --git a/doc/guides/nics/features/bnxt.ini b/doc/guides/nics/features/bnxt.ini index f1f300c8b..e75cfc44d 100644 --- a/doc/guides/nics/features/bnxt.ini +++ b/doc/guides/nics/features/bnxt.ini @@ -8,6 +8,7 @@ Speed capabilities = Y Link status = Y Link status event = Y Rx interrupt = Y +Fast mbuf free = Y Queue start/stop = Y Burst mode info = Y MTU update = Y diff --git a/drivers/net/bnxt/bnxt_ethdev.c b/drivers/net/bnxt/bnxt_ethdev.c index 624cb2031..c2a9f80e2 100644 --- a/drivers/net/bnxt/bnxt_ethdev.c +++ b/drivers/net/bnxt/bnxt_ethdev.c @@ -920,7 +920,9 @@ static int bnxt_dev_info_get_op(struct rte_eth_dev *eth_dev, dev_info->rx_offload_capa = BNXT_DEV_RX_OFFLOAD_SUPPORT; if (bp->flags & BNXT_FLAG_PTP_SUPPORTED) dev_info->rx_offload_capa |= DEV_RX_OFFLOAD_TIMESTAMP; - dev_info->tx_offload_capa = BNXT_DEV_TX_OFFLOAD_SUPPORT; + dev_info->tx_queue_offload_capa = DEV_TX_OFFLOAD_MBUF_FAST_FREE; + dev_info->tx_offload_capa = BNXT_DEV_TX_OFFLOAD_SUPPORT | + dev_info->tx_queue_offload_capa; dev_info->flow_type_rss_offloads = BNXT_ETH_RSS_SUPPORT; dev_info->speed_capa = bnxt_get_speed_capabilities(bp); @@ -1191,6 +1193,7 @@ bnxt_transmit_function(__rte_unused struct rte_eth_dev *eth_dev) { #if defined(RTE_ARCH_X86) || defined(RTE_ARCH_ARM64) #ifndef RTE_LIBRTE_IEEE1588 + uint64_t offloads = eth_dev->data->dev_conf.txmode.offloads; struct bnxt *bp = eth_dev->data->dev_private; /* @@ -1198,7 +1201,7 @@ bnxt_transmit_function(__rte_unused struct rte_eth_dev *eth_dev) * or tx offloads. */ if (!eth_dev->data->scattered_rx && - !eth_dev->data->dev_conf.txmode.offloads && + !(offloads & ~DEV_TX_OFFLOAD_MBUF_FAST_FREE) && !BNXT_TRUFLOW_EN(bp)) { PMD_DRV_LOG(INFO, "Using vector mode transmit for port %d\n", eth_dev->data->port_id); @@ -1210,7 +1213,7 @@ bnxt_transmit_function(__rte_unused struct rte_eth_dev *eth_dev) "Port %d scatter: %d tx offload: %" PRIX64 "\n", eth_dev->data->port_id, eth_dev->data->scattered_rx, - eth_dev->data->dev_conf.txmode.offloads); + offloads); #endif #endif return bnxt_xmit_pkts; @@ -2685,7 +2688,7 @@ bnxt_txq_info_get_op(struct rte_eth_dev *dev, uint16_t queue_id, qinfo->conf.tx_free_thresh = txq->tx_free_thresh; qinfo->conf.tx_rs_thresh = 0; qinfo->conf.tx_deferred_start = txq->tx_deferred_start; - qinfo->conf.offloads = dev->data->dev_conf.txmode.offloads; + qinfo->conf.offloads = txq->offloads; } static const struct { diff --git a/drivers/net/bnxt/bnxt_rxtx_vec_common.h b/drivers/net/bnxt/bnxt_rxtx_vec_common.h index 4a48152fc..33ac53568 100644 --- a/drivers/net/bnxt/bnxt_rxtx_vec_common.h +++ b/drivers/net/bnxt/bnxt_rxtx_vec_common.h @@ -94,4 +94,36 @@ bnxt_rxq_rearm(struct bnxt_rx_queue *rxq, struct bnxt_rx_ring_info *rxr) rxq->rxrearm_nb -= nb; } + +static inline void +bnxt_tx_cmp_vec(struct bnxt_tx_queue *txq, int nr_pkts) +{ + struct bnxt_tx_ring_info *txr = txq->tx_ring; + struct rte_mbuf **free = txq->free; + uint16_t cons = txr->tx_cons; + unsigned int blk = 0; + uint32_t ring_mask = txr->tx_ring_struct->ring_mask; + + while (nr_pkts--) { + struct bnxt_sw_tx_bd *tx_buf; + struct rte_mbuf *mbuf; + + tx_buf = &txr->tx_buf_ring[cons]; + cons = (cons + 1) & ring_mask; + mbuf = rte_pktmbuf_prefree_seg(tx_buf->mbuf); + if (unlikely(mbuf == NULL)) + continue; + tx_buf->mbuf = NULL; + + if (blk && mbuf->pool != free[0]->pool) { + rte_mempool_put_bulk(free[0]->pool, (void **)free, blk); + blk = 0; + } + free[blk++] = mbuf; + } + if (blk) + rte_mempool_put_bulk(free[0]->pool, (void **)free, blk); + + txr->tx_cons = cons; +} #endif /* _BNXT_RXTX_VEC_COMMON_H_ */ diff --git a/drivers/net/bnxt/bnxt_rxtx_vec_neon.c b/drivers/net/bnxt/bnxt_rxtx_vec_neon.c index 299b6b86d..4c04cc43a 100644 --- a/drivers/net/bnxt/bnxt_rxtx_vec_neon.c +++ b/drivers/net/bnxt/bnxt_rxtx_vec_neon.c @@ -13,10 +13,10 @@ #include "bnxt.h" #include "bnxt_cpr.h" #include "bnxt_ring.h" -#include "bnxt_rxtx_vec_common.h" #include "bnxt_txq.h" #include "bnxt_txr.h" +#include "bnxt_rxtx_vec_common.h" /* * RX Ring handling @@ -338,37 +338,6 @@ bnxt_recv_pkts_vec(void *rx_queue, struct rte_mbuf **rx_pkts, return nb_rx_pkts; } -static void -bnxt_tx_cmp_vec(struct bnxt_tx_queue *txq, int nr_pkts) -{ - struct bnxt_tx_ring_info *txr = txq->tx_ring; - struct rte_mbuf **free = txq->free; - uint16_t cons = txr->tx_cons; - unsigned int blk = 0; - - while (nr_pkts--) { - struct bnxt_sw_tx_bd *tx_buf; - struct rte_mbuf *mbuf; - - tx_buf = &txr->tx_buf_ring[cons]; - cons = RING_NEXT(txr->tx_ring_struct, cons); - mbuf = rte_pktmbuf_prefree_seg(tx_buf->mbuf); - if (unlikely(mbuf == NULL)) - continue; - tx_buf->mbuf = NULL; - - if (blk && mbuf->pool != free[0]->pool) { - rte_mempool_put_bulk(free[0]->pool, (void **)free, blk); - blk = 0; - } - free[blk++] = mbuf; - } - if (blk) - rte_mempool_put_bulk(free[0]->pool, (void **)free, blk); - - txr->tx_cons = cons; -} - static void bnxt_handle_tx_cp_vec(struct bnxt_tx_queue *txq) { @@ -399,7 +368,10 @@ bnxt_handle_tx_cp_vec(struct bnxt_tx_queue *txq) cpr->valid = !!(raw_cons & cp_ring_struct->ring_size); if (nb_tx_pkts) { - bnxt_tx_cmp_vec(txq, nb_tx_pkts); + if (txq->offloads & DEV_TX_OFFLOAD_MBUF_FAST_FREE) + bnxt_tx_cmp_fast(txq, nb_tx_pkts); + else + bnxt_tx_cmp_vec(txq, nb_tx_pkts); cpr->cp_raw_cons = raw_cons; bnxt_db_cq(cpr); } diff --git a/drivers/net/bnxt/bnxt_rxtx_vec_sse.c b/drivers/net/bnxt/bnxt_rxtx_vec_sse.c index 271e7908e..f71f46985 100644 --- a/drivers/net/bnxt/bnxt_rxtx_vec_sse.c +++ b/drivers/net/bnxt/bnxt_rxtx_vec_sse.c @@ -13,10 +13,10 @@ #include "bnxt.h" #include "bnxt_cpr.h" #include "bnxt_ring.h" -#include "bnxt_rxtx_vec_common.h" #include "bnxt_txq.h" #include "bnxt_txr.h" +#include "bnxt_rxtx_vec_common.h" /* * RX Ring handling @@ -309,38 +309,6 @@ bnxt_recv_pkts_vec(void *rx_queue, struct rte_mbuf **rx_pkts, return nb_rx_pkts; } -static void -bnxt_tx_cmp_vec(struct bnxt_tx_queue *txq, int nr_pkts) -{ - struct bnxt_tx_ring_info *txr = txq->tx_ring; - struct rte_mbuf **free = txq->free; - uint16_t cons = txr->tx_cons; - unsigned int blk = 0; - uint32_t ring_mask = txr->tx_ring_struct->ring_mask; - - while (nr_pkts--) { - struct bnxt_sw_tx_bd *tx_buf; - struct rte_mbuf *mbuf; - - tx_buf = &txr->tx_buf_ring[cons]; - cons = (cons + 1) & ring_mask; - mbuf = rte_pktmbuf_prefree_seg(tx_buf->mbuf); - if (unlikely(mbuf == NULL)) - continue; - tx_buf->mbuf = NULL; - - if (blk && mbuf->pool != free[0]->pool) { - rte_mempool_put_bulk(free[0]->pool, (void **)free, blk); - blk = 0; - } - free[blk++] = mbuf; - } - if (blk) - rte_mempool_put_bulk(free[0]->pool, (void **)free, blk); - - txr->tx_cons = cons; -} - static void bnxt_handle_tx_cp_vec(struct bnxt_tx_queue *txq) { @@ -371,7 +339,10 @@ bnxt_handle_tx_cp_vec(struct bnxt_tx_queue *txq) cpr->valid = !!(raw_cons & cp_ring_struct->ring_size); if (nb_tx_pkts) { - bnxt_tx_cmp_vec(txq, nb_tx_pkts); + if (txq->offloads & DEV_TX_OFFLOAD_MBUF_FAST_FREE) + bnxt_tx_cmp_fast(txq, nb_tx_pkts); + else + bnxt_tx_cmp_vec(txq, nb_tx_pkts); cpr->cp_raw_cons = raw_cons; bnxt_db_cq(cpr); } diff --git a/drivers/net/bnxt/bnxt_txq.c b/drivers/net/bnxt/bnxt_txq.c index bdc7ffaaa..c8d75ac95 100644 --- a/drivers/net/bnxt/bnxt_txq.c +++ b/drivers/net/bnxt/bnxt_txq.c @@ -131,6 +131,8 @@ int bnxt_tx_queue_setup_op(struct rte_eth_dev *eth_dev, txq->nb_tx_desc = nb_desc; txq->tx_free_thresh = RTE_MIN(rte_align32pow2(nb_desc) / 4, RTE_BNXT_MAX_TX_BURST); + txq->offloads = eth_dev->data->dev_conf.txmode.offloads | + tx_conf->offloads; txq->tx_deferred_start = tx_conf->tx_deferred_start; diff --git a/drivers/net/bnxt/bnxt_txq.h b/drivers/net/bnxt/bnxt_txq.h index 9f849fc71..8033e241e 100644 --- a/drivers/net/bnxt/bnxt_txq.h +++ b/drivers/net/bnxt/bnxt_txq.h @@ -39,6 +39,7 @@ struct bnxt_tx_queue { struct bnxt_cp_ring_info *cp_ring; const struct rte_memzone *mz; struct rte_mbuf **free; + uint64_t offloads; }; void bnxt_free_txq_stats(struct bnxt_tx_queue *txq); diff --git a/drivers/net/bnxt/bnxt_txr.c b/drivers/net/bnxt/bnxt_txr.c index c55497960..125ac8341 100644 --- a/drivers/net/bnxt/bnxt_txr.c +++ b/drivers/net/bnxt/bnxt_txr.c @@ -428,7 +428,10 @@ static int bnxt_handle_tx_cp(struct bnxt_tx_queue *txq) } while (nb_tx_pkts < ring_mask); if (nb_tx_pkts) { - bnxt_tx_cmp(txq, nb_tx_pkts); + if (txq->offloads & DEV_TX_OFFLOAD_MBUF_FAST_FREE) + bnxt_tx_cmp_fast(txq, nb_tx_pkts); + else + bnxt_tx_cmp(txq, nb_tx_pkts); cpr->cp_raw_cons = raw_cons; bnxt_db_cq(cpr); } diff --git a/drivers/net/bnxt/bnxt_txr.h b/drivers/net/bnxt/bnxt_txr.h index 7715c11b8..026a65a72 100644 --- a/drivers/net/bnxt/bnxt_txr.h +++ b/drivers/net/bnxt/bnxt_txr.h @@ -52,6 +52,33 @@ static inline uint32_t bnxt_tx_avail(struct bnxt_tx_queue *txq) bnxt_tx_bds_in_hw(txq)) - 1); } +/* + * Transmit completion function for use when DEV_TX_OFFLOAD_MBUF_FAST_FREE + * is enabled. + */ +static inline void +bnxt_tx_cmp_fast(struct bnxt_tx_queue *txq, int nr_pkts) +{ + struct bnxt_tx_ring_info *txr = txq->tx_ring; + uint32_t ring_mask = txr->tx_ring_struct->ring_mask; + struct rte_mbuf **free = txq->free; + uint16_t cons = txr->tx_cons; + unsigned int blk = 0; + + while (nr_pkts--) { + struct bnxt_sw_tx_bd *tx_buf; + + tx_buf = &txr->tx_buf_ring[cons]; + cons = (cons + 1) & ring_mask; + free[blk++] = tx_buf->mbuf; + tx_buf->mbuf = NULL; + } + if (blk) + rte_mempool_put_bulk(free[0]->pool, (void **)free, blk); + + txr->tx_cons = cons; +} + void bnxt_free_tx_rings(struct bnxt *bp); int bnxt_init_one_tx_ring(struct bnxt_tx_queue *txq); int bnxt_init_tx_ring_struct(struct bnxt_tx_queue *txq, unsigned int socket_id); -- 2.25.1