From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id DA8DE4886F; Tue, 30 Sep 2025 12:01:32 +0200 (CEST) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id A354D40E1E; Tue, 30 Sep 2025 12:00:49 +0200 (CEST) Received: from smtpbgau2.qq.com (smtpbgau2.qq.com [54.206.34.216]) by mails.dpdk.org (Postfix) with ESMTP id DA48840A79 for ; Tue, 30 Sep 2025 12:00:47 +0200 (CEST) X-QQ-mid: esmtpgz13t1759226442t4091c2b7 X-QQ-Originating-IP: ywn53P/0Fk2zJ0KOSu2ARDFluyv50oV0h+6/T5A9Lqg= Received: from DSK-zaiyuwang.trustnetic.com ( [115.220.225.164]) by bizesmtp.qq.com (ESMTP) with id ; Tue, 30 Sep 2025 18:00:41 +0800 (CST) X-QQ-SSF: 0000000000000000000000000000000 X-QQ-GoodBg: 0 X-BIZMAIL-ID: 16171448251923589421 EX-QQ-RecipientCnt: 4 From: Zaiyu Wang To: dev@dpdk.org Cc: Zaiyu Wang , Jiawen Wu , Jian Wang Subject: [PATCH v4 10/20] net/txgbe: add TX head Write-Back mode for Amber-Lite NICs Date: Tue, 30 Sep 2025 17:59:42 +0800 Message-Id: <20250930095953.18508-11-zaiyuwang@trustnetic.com> X-Mailer: git-send-email 2.21.0.windows.1 In-Reply-To: <20250930095953.18508-1-zaiyuwang@trustnetic.com> References: <20250418094131.24136-1-zaiyuwang@trustnetic.com> <20250930095953.18508-1-zaiyuwang@trustnetic.com> MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-QQ-SENDSIZE: 520 Feedback-ID: esmtpgz:trustnetic.com:qybglogicsvrgz:qybglogicsvrgz5a-1 X-QQ-XMAILINFO: NoaupJ8ahyDyD9tuI22GVQdIJNylObIWsoyzRMRJf7kHOu/F0Nc1UbTW F3qPxjvDBamHG0ua0ll01N3Y9Q7K2FzRRJrQtc47LNLtMrPRpqi3AjTYKnfvw0SaaQK834m ixzjaB1ljt8vaqjqkn3ULDaeqQb+VbkKetIa5k0rbNC7oO1v2oAM5N0cmLLMCCIgKtd53pI vXTiV6HRgltMfNQxwnm1lTfnOU0rD3SsmTZUROVR5uII8VBmnauxkldq4w5SP5x56J2kTTu bbRm6mmxn+yJpZ9fdT54jLI9oAa4rpw2/FjsbbMLPjVnx0dI9d1Ad0zMe1Bn0fwKdbaA624 ug/Z1qYV5OHh2RN3Mel3OdLOTNZNKldf7TDEEEEJ1o8f29Q9KEX1ilXv+Sb1cIlkfK6w69f 1wvTYtJCtYsscT4uYRW6hLVC/Oq51WKq4GaBX9JmL7GUjN+N1IGsnnKxL7ovaL0v1ucemUu KhT9ltdP4YSeDbqszUuMHlFaQBTcAV+hm71m0cpn1THyt7qmqFBNLbBaXNCwaaZRzR/0rtw NaG9ryuzrOD2C4rMd8ErQmCb3CZidMUd1j9CA4rQ4rjiukMALrOf7Xkb2K++q+dViTEgXRz yzzlKvB/lykQOlQNrRdTnYF7HGhz7EObbUDuJYA81lnHHFTp59hPybGQpDYCICf73FAFBbs Js0VNE5x4EbofRft/d+P4ZpfVs2pBoMKsaOdCS3FD74AReRitJrhLVdU6MFeo42ek5z699N 8b0ZJWwTahd7I1PaoreiMNnZz1J8ecPEeZOga6ZDxryxEnWAg0VTx9poM5h9PeDDFROqC50 dJiSX5PYptmAfTXRf/St7FLIRSQY/dSXpWsAzKUoKEaAiCLO4R4bgSsz6tujo1rfZ0ruKmV qxkJ/s+g6/sUgxjzO3A6vs8s2VQWOdqaud9jUhQrbw4BGjyMsaUBP05WpGRl0wuPAyctoT4 oMpJYA8WYrIItLU+pR5PlmdgL0hoZhf8d2CJnkABir7askmRaP7wIRQ27Q8rtPZcbIXzSch Jyl7RSdHK5+f07f69WFOh8ZGqhexn8GCqUhbUbZiHTeYntjXtcamWfR8cxJFK2bftwDJWk+ q23W19nZs4q X-QQ-XMRINFO: NyFYKkN4Ny6FSmKK/uo/jdU= X-QQ-RECHKSPAM: 0 X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Add TX head Write-Back mode for Amber-Lite NICs. When enabled, the hardware no longer individually rewrites descriptors but intermittently notifies the driver of processed descriptor indices. This feature significantly improves performance and is enabled by default in the driver. Users can configure it via tx_headwb and tx_headwb_size in devargs. Signed-off-by: Zaiyu Wang --- drivers/net/txgbe/base/txgbe_regs.h | 7 ++ drivers/net/txgbe/base/txgbe_type.h | 6 ++ drivers/net/txgbe/txgbe_ethdev.c | 13 ++- drivers/net/txgbe/txgbe_rxtx.c | 124 ++++++++++++++++++---- drivers/net/txgbe/txgbe_rxtx.h | 3 + drivers/net/txgbe/txgbe_rxtx_vec_common.h | 27 +++-- 6 files changed, 153 insertions(+), 27 deletions(-) diff --git a/drivers/net/txgbe/base/txgbe_regs.h b/drivers/net/txgbe/base/txgbe_regs.h index 23d39857de..1a544bcd57 100644 --- a/drivers/net/txgbe/base/txgbe_regs.h +++ b/drivers/net/txgbe/base/txgbe_regs.h @@ -1407,6 +1407,13 @@ enum txgbe_5tuple_protocol { #define TXGBE_TXCFG_WTHRESH(v) LS(v, 16, 0x7F) #define TXGBE_TXCFG_WTHRESH_AML(v) LS(v, 16, 0x1FF) #define TXGBE_TXCFG_FLUSH MS(26, 0x1) +#define TXGBE_PX_TR_CFG_HEAD_WB MS(27, 0x1) /* amlite head wb */ +#define TXGBE_PX_TR_CFG_HEAD_WB_64BYTE MS(28, 0x1) /* amlite head wb 64byte */ +#define TXGBE_PX_TR_CFG_HEAD_WB_MASK MS(27, 0x3) + +/* amlite: tx head wb */ +#define TXGBE_PX_TR_HEAD_ADDRL(_i) (0x03028 + ((_i) * 0x40)) +#define TXGBE_PX_TR_HEAD_ADDRH(_i) (0x0302C + ((_i) * 0x40)) #define TXGBE_TDM_DESC_CHK(i) (0x0180B0 + (i) * 4) /*0-3*/ #define TXGBE_TDM_DESC_NONFATAL(i) (0x0180C0 + (i) * 4) /*0-3*/ diff --git a/drivers/net/txgbe/base/txgbe_type.h b/drivers/net/txgbe/base/txgbe_type.h index 55123d0b6c..844a2827bc 100644 --- a/drivers/net/txgbe/base/txgbe_type.h +++ b/drivers/net/txgbe/base/txgbe_type.h @@ -716,6 +716,8 @@ struct txgbe_phy_info { #define TXGBE_DEVARG_FFE_MAIN "ffe_main" #define TXGBE_DEVARG_FFE_PRE "ffe_pre" #define TXGBE_DEVARG_FFE_POST "ffe_post" +#define TXGBE_DEVARG_TX_HEAD_WB "tx_headwb" +#define TXGBE_DEVARG_TX_HEAD_WB_SIZE "tx_headwb_size" static const char * const txgbe_valid_arguments[] = { TXGBE_DEVARG_BP_AUTO, @@ -726,6 +728,8 @@ static const char * const txgbe_valid_arguments[] = { TXGBE_DEVARG_FFE_MAIN, TXGBE_DEVARG_FFE_PRE, TXGBE_DEVARG_FFE_POST, + TXGBE_DEVARG_TX_HEAD_WB, + TXGBE_DEVARG_TX_HEAD_WB_SIZE, NULL }; @@ -776,6 +780,8 @@ struct txgbe_devargs { u16 poll; u16 present; u16 sgmii; + u16 tx_headwb; + u16 tx_headwb_size; }; struct txgbe_hw { diff --git a/drivers/net/txgbe/txgbe_ethdev.c b/drivers/net/txgbe/txgbe_ethdev.c index 10e088ee95..9c14e4b8ed 100644 --- a/drivers/net/txgbe/txgbe_ethdev.c +++ b/drivers/net/txgbe/txgbe_ethdev.c @@ -513,6 +513,9 @@ txgbe_parse_devargs(struct txgbe_hw *hw, struct rte_devargs *devargs) u16 ffe_main = 27; u16 ffe_pre = 8; u16 ffe_post = 44; + /* New devargs for amberlite config */ + u16 tx_headwb = 1; + u16 tx_headwb_size = 16; if (devargs == NULL) goto null; @@ -537,6 +540,10 @@ txgbe_parse_devargs(struct txgbe_hw *hw, struct rte_devargs *devargs) &txgbe_handle_devarg, &ffe_pre); rte_kvargs_process(kvlist, TXGBE_DEVARG_FFE_POST, &txgbe_handle_devarg, &ffe_post); + rte_kvargs_process(kvlist, TXGBE_DEVARG_TX_HEAD_WB, + &txgbe_handle_devarg, &tx_headwb); + rte_kvargs_process(kvlist, TXGBE_DEVARG_TX_HEAD_WB_SIZE, + &txgbe_handle_devarg, &tx_headwb_size); rte_kvargs_free(kvlist); null: @@ -544,6 +551,8 @@ txgbe_parse_devargs(struct txgbe_hw *hw, struct rte_devargs *devargs) hw->devarg.poll = poll; hw->devarg.present = present; hw->devarg.sgmii = sgmii; + hw->devarg.tx_headwb = tx_headwb; + hw->devarg.tx_headwb_size = tx_headwb_size; hw->phy.ffe_set = ffe_set; hw->phy.ffe_main = ffe_main; hw->phy.ffe_pre = ffe_pre; @@ -5718,7 +5727,9 @@ RTE_PMD_REGISTER_PARAM_STRING(net_txgbe, TXGBE_DEVARG_FFE_SET "=<0-4>" TXGBE_DEVARG_FFE_MAIN "=" TXGBE_DEVARG_FFE_PRE "=" - TXGBE_DEVARG_FFE_POST "="); + TXGBE_DEVARG_FFE_POST "=" + TXGBE_DEVARG_TX_HEAD_WB "=<0|1>" + TXGBE_DEVARG_TX_HEAD_WB_SIZE "=<1|16>"); RTE_LOG_REGISTER_SUFFIX(txgbe_logtype_init, init, NOTICE); RTE_LOG_REGISTER_SUFFIX(txgbe_logtype_driver, driver, NOTICE); diff --git a/drivers/net/txgbe/txgbe_rxtx.c b/drivers/net/txgbe/txgbe_rxtx.c index 3f157527c6..b15b6788e6 100644 --- a/drivers/net/txgbe/txgbe_rxtx.c +++ b/drivers/net/txgbe/txgbe_rxtx.c @@ -92,14 +92,29 @@ txgbe_tx_free_bufs(struct txgbe_tx_queue *txq) int i, nb_free = 0; struct rte_mbuf *m, *free[RTE_TXGBE_TX_MAX_FREE_BUF_SZ]; - /* check DD bit on threshold descriptor */ - status = txq->tx_ring[txq->tx_next_dd].dw3; - if (!(status & rte_cpu_to_le_32(TXGBE_TXD_DD))) { - if (txq->nb_tx_free >> 1 < txq->tx_free_thresh) - txgbe_set32_masked(txq->tdc_reg_addr, - TXGBE_TXCFG_FLUSH, TXGBE_TXCFG_FLUSH); - return 0; + if (txq->headwb_mem) { + uint16_t tx_last_dd = txq->nb_tx_desc + + txq->tx_next_dd - txq->tx_free_thresh; + if (tx_last_dd >= txq->nb_tx_desc) + tx_last_dd -= txq->nb_tx_desc; + + volatile uint16_t head = (uint16_t)*txq->headwb_mem; + + if (txq->tx_next_dd > head && head > tx_last_dd) + return 0; + else if (tx_last_dd > txq->tx_next_dd && + (head > tx_last_dd || head < txq->tx_next_dd)) + return 0; + } else { + /* check DD bit on threshold descriptor */ + status = txq->tx_ring[txq->tx_next_dd].dw3; + if (!(status & rte_cpu_to_le_32(TXGBE_TXD_DD))) { + if (txq->nb_tx_free >> 1 < txq->tx_free_thresh) + txgbe_set32_masked(txq->tdc_reg_addr, + TXGBE_TXCFG_FLUSH, TXGBE_TXCFG_FLUSH); + return 0; } +} /* * first buffer to free from S/W ring is at index @@ -628,17 +643,28 @@ txgbe_xmit_cleanup(struct txgbe_tx_queue *txq) /* Check to make sure the last descriptor to clean is done */ desc_to_clean_to = sw_ring[desc_to_clean_to].last_id; status = txr[desc_to_clean_to].dw3; - if (!(status & rte_cpu_to_le_32(TXGBE_TXD_DD))) { - PMD_TX_FREE_LOG(DEBUG, - "TX descriptor %4u is not done" - "(port=%d queue=%d)", - desc_to_clean_to, - txq->port_id, txq->queue_id); - if (txq->nb_tx_free >> 1 < txq->tx_free_thresh) - txgbe_set32_masked(txq->tdc_reg_addr, - TXGBE_TXCFG_FLUSH, TXGBE_TXCFG_FLUSH); - /* Failed to clean any descriptors, better luck next time */ - return -(1); + + if (txq->headwb_mem) { + u32 head = *txq->headwb_mem; + + PMD_TX_FREE_LOG(DEBUG, "queue[%02d]: headwb_mem = %03d, desc_to_clean_to = %03d", + txq->reg_idx, head, desc_to_clean_to); + /* we have caught up to head, no work left to do */ + if (desc_to_clean_to == head) + return -(1); + } else { + if (!(status & rte_cpu_to_le_32(TXGBE_TXD_DD))) { + PMD_TX_FREE_LOG(DEBUG, + "TX descriptor %4u is not done" + "(port=%d queue=%d)", + desc_to_clean_to, + txq->port_id, txq->queue_id); + if (txq->nb_tx_free >> 1 < txq->tx_free_thresh) + txgbe_set32_masked(txq->tdc_reg_addr, + TXGBE_TXCFG_FLUSH, TXGBE_TXCFG_FLUSH); + /* Failed to clean any descriptors, better luck next time */ + return -(1); + } } /* Figure out how many descriptors will be cleaned */ @@ -2246,6 +2272,8 @@ txgbe_tx_queue_release(struct txgbe_tx_queue *txq) txq->ops->release_mbufs(txq); txq->ops->free_swring(txq); rte_memzone_free(txq->mz); + if (txq->headwb_mem) + rte_memzone_free(txq->headwb); rte_free(txq); } } @@ -2382,6 +2410,43 @@ txgbe_get_tx_port_offloads(struct rte_eth_dev *dev) return tx_offload_capa; } +static int +txgbe_setup_headwb_resources(struct rte_eth_dev *dev, + void *tx_queue, + unsigned int socket_id) +{ + struct txgbe_hw *hw = TXGBE_DEV_HW(dev); + const struct rte_memzone *headwb; + struct txgbe_tx_queue *txq = tx_queue; + u8 i, headwb_size = 0; + + if (hw->mac.type != txgbe_mac_aml && hw->mac.type != txgbe_mac_aml40) { + txq->headwb_mem = NULL; + return 0; + } + + headwb_size = hw->devarg.tx_headwb_size; + headwb = rte_eth_dma_zone_reserve(dev, "tx_headwb_mem", txq->queue_id, + sizeof(u32) * headwb_size, + TXGBE_ALIGN, socket_id); + + if (headwb == NULL) { + DEBUGOUT("Fail to setup headwb resources: no mem"); + txgbe_tx_queue_release(txq); + return -ENOMEM; + } + + txq->headwb = headwb; + txq->headwb_dma = TMZ_PADDR(headwb); + txq->headwb_mem = (uint32_t *)TMZ_VADDR(headwb); + + /* Zero out headwb_mem memory */ + for (i = 0; i < headwb_size; i++) + txq->headwb_mem[i] = 0; + + return 0; +} + int __rte_cold txgbe_dev_tx_queue_setup(struct rte_eth_dev *dev, uint16_t queue_idx, @@ -2394,6 +2459,7 @@ txgbe_dev_tx_queue_setup(struct rte_eth_dev *dev, struct txgbe_hw *hw; uint16_t tx_free_thresh; uint64_t offloads; + s32 err = 0; PMD_INIT_FUNC_TRACE(); hw = TXGBE_DEV_HW(dev); @@ -2513,12 +2579,15 @@ txgbe_dev_tx_queue_setup(struct rte_eth_dev *dev, /* set up scalar TX function as appropriate */ txgbe_set_tx_function(dev, txq); + if (hw->devarg.tx_headwb) + err = txgbe_setup_headwb_resources(dev, txq, socket_id); + txq->ops->reset(txq); txq->desc_error = 0; dev->data->tx_queues[queue_idx] = txq; - return 0; + return err; } /** @@ -4676,6 +4745,23 @@ txgbe_dev_tx_init(struct rte_eth_dev *dev) /* Setup the HW Tx Head and TX Tail descriptor pointers */ wr32(hw, TXGBE_TXRP(txq->reg_idx), 0); wr32(hw, TXGBE_TXWP(txq->reg_idx), 0); + + if ((hw->mac.type == txgbe_mac_aml || hw->mac.type == txgbe_mac_aml40) && + hw->devarg.tx_headwb) { + uint32_t txdctl; + + wr32(hw, TXGBE_PX_TR_HEAD_ADDRL(txq->reg_idx), + (uint32_t)(txq->headwb_dma & BIT_MASK32)); + wr32(hw, TXGBE_PX_TR_HEAD_ADDRH(txq->reg_idx), + (uint32_t)(txq->headwb_dma >> 32)); + if (hw->devarg.tx_headwb_size == 16) + txdctl = TXGBE_PX_TR_CFG_HEAD_WB | + TXGBE_PX_TR_CFG_HEAD_WB_64BYTE; + else + txdctl = TXGBE_PX_TR_CFG_HEAD_WB; + wr32m(hw, TXGBE_TXCFG(txq->reg_idx), + TXGBE_PX_TR_CFG_HEAD_WB_MASK, txdctl); + } } #ifndef RTE_LIB_SECURITY diff --git a/drivers/net/txgbe/txgbe_rxtx.h b/drivers/net/txgbe/txgbe_rxtx.h index 622a0d3981..b1ac03576f 100644 --- a/drivers/net/txgbe/txgbe_rxtx.h +++ b/drivers/net/txgbe/txgbe_rxtx.h @@ -414,6 +414,9 @@ struct txgbe_tx_queue { const struct rte_memzone *mz; uint64_t desc_error; bool resetting; + const struct rte_memzone *headwb; + uint64_t headwb_dma; + volatile uint32_t *headwb_mem; }; struct txgbe_txq_ops { diff --git a/drivers/net/txgbe/txgbe_rxtx_vec_common.h b/drivers/net/txgbe/txgbe_rxtx_vec_common.h index cf67df66d8..00847d087b 100644 --- a/drivers/net/txgbe/txgbe_rxtx_vec_common.h +++ b/drivers/net/txgbe/txgbe_rxtx_vec_common.h @@ -89,13 +89,26 @@ txgbe_tx_free_bufs(struct txgbe_tx_queue *txq) int nb_free = 0; struct rte_mbuf *m, *free[RTE_TXGBE_TX_MAX_FREE_BUF_SZ]; - /* check DD bit on threshold descriptor */ - status = txq->tx_ring[txq->tx_next_dd].dw3; - if (!(status & TXGBE_TXD_DD)) { - if (txq->nb_tx_free >> 1 < txq->tx_free_thresh) - txgbe_set32_masked(txq->tdc_reg_addr, - TXGBE_TXCFG_FLUSH, TXGBE_TXCFG_FLUSH); - return 0; + if (txq->headwb_mem) { + uint16_t tx_last_dd = txq->nb_tx_desc + + txq->tx_next_dd - txq->tx_free_thresh; + if (tx_last_dd >= txq->nb_tx_desc) + tx_last_dd -= txq->nb_tx_desc; + volatile uint16_t head = (uint16_t)*txq->headwb_mem; + if (txq->tx_next_dd > head && head > tx_last_dd) + return 0; + else if (tx_last_dd > txq->tx_next_dd && + (head > tx_last_dd || head < txq->tx_next_dd)) + return 0; + } else { + /* check DD bit on threshold descriptor */ + status = txq->tx_ring[txq->tx_next_dd].dw3; + if (!(status & TXGBE_TXD_DD)) { + if (txq->nb_tx_free >> 1 < txq->tx_free_thresh) + txgbe_set32_masked(txq->tdc_reg_addr, + TXGBE_TXCFG_FLUSH, TXGBE_TXCFG_FLUSH); + return 0; + } } n = txq->tx_free_thresh; -- 2.21.0.windows.1