From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 84EB745CB1; Thu, 7 Nov 2024 10:26:58 +0100 (CET) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 6248842E86; Thu, 7 Nov 2024 10:26:58 +0100 (CET) Received: from dkmailrelay1.smartsharesystems.com (smartserver.smartsharesystems.com [77.243.40.215]) by mails.dpdk.org (Postfix) with ESMTP id 7B7E041611 for ; Thu, 7 Nov 2024 10:26:56 +0100 (CET) Received: from smartserver.smartsharesystems.com (smartserver.smartsharesys.local [192.168.4.10]) by dkmailrelay1.smartsharesystems.com (Postfix) with ESMTP id 2E6F321D0E for ; Thu, 7 Nov 2024 10:26:56 +0100 (CET) Received: from dkrd4.smartsharesys.local ([192.168.4.26]) by smartserver.smartsharesystems.com with Microsoft SMTPSVC(6.0.3790.4675); Thu, 7 Nov 2024 10:26:55 +0100 From: =?UTF-8?q?Morten=20Br=C3=B8rup?= To: dev@dpdk.org Cc: =?UTF-8?q?Morten=20Br=C3=B8rup?= Subject: [TEST v11] ethdev: test single queue per port Date: Thu, 7 Nov 2024 09:26:49 +0000 Message-ID: <20241107092649.1507375-1-mb@smartsharesystems.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20241024102253.785037-1-mb@smartsharesystems.com> References: <20241024102253.785037-1-mb@smartsharesystems.com> MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-OriginalArrivalTime: 07 Nov 2024 09:26:55.0731 (UTC) FILETIME=[2FF6A430:01DB30F7] X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Configuring one queue per port (#define RTE_MAX_QUEUES_PER_PORT 1) fails compilation with e.g.: ../drivers/net/bnxt/bnxt_rxq.c: In function 'bnxt_rx_queue_stop': ../drivers/net/bnxt/bnxt_rxq.c:587:34: error: array subscript 1 is above array bounds of 'uint8_t[1]' {aka 'unsigned char[1]'} [-Werror=array-bounds=] 587 | dev->data->rx_queue_state[q_id] = RTE_ETH_QUEUE_STATE_STOPPED; | ~~~~~~~~~~~~~~~~~~~~~~~~~^~~~~~ In file included from ../drivers/net/bnxt/bnxt.h:16, from ../drivers/net/bnxt/bnxt_rxq.c:10: ../lib/ethdev/ethdev_driver.h:168:17: note: while referencing 'rx_queue_state' 168 | uint8_t rx_queue_state[RTE_MAX_QUEUES_PER_PORT]; | ^~~~~~~~~~~~~~ Use the CI to test my ideas to fix this. Signed-off-by: Morten Brørup --- v11: * Fix offset-by-one error in comparison outside loop. * Moved __rte_assume() outside loop in all drivers, to test if it works. (Bruce, Thomas) v10: * Rebased to main. v9: * Moved __rte_assume() in mana driver, to test if it works outside loop. (Bruce, Thomas) v8: * Added __rte_unreachable() macro. * GCC version 13 is required for __attribute__((assume())). Provide alternative implementation, using __rte_unreachable(), for older GCC versions. * Added MSVC implementations of the two new macros. v7: * Introduce __rte_assume() in rte_common.h, and use this instead. v6: * Use __attribute__((assume(i < RTE_MAX_QUEUES_PER_PORT))) where the indexing warning occurs, instead of pragma GCC optimize("no-peel-loops") for the entire function. v5: * Wrap GCC optimizer pragmas in if defined(RTE_TOOLCHAIN_GCC). v4: * Workaound GCC optimizer incorrectly throwing a warning in these network drivers: * bnxt * e1000 * failsafe * hns3 v3: * Fix net/ixgbe driver. v2: * Fix net/vmxnet3 driver. --- config/rte_config.h | 4 ++-- drivers/net/bnxt/bnxt_ethdev.c | 4 ++++ drivers/net/bnxt/bnxt_rxq.c | 1 + drivers/net/e1000/igb_rxtx.c | 4 ++++ drivers/net/failsafe/failsafe_ops.c | 14 ++++++++++-- drivers/net/hns3/hns3_rxtx.c | 4 ++++ drivers/net/mana/tx.c | 2 ++ drivers/net/vmxnet3/vmxnet3_ethdev.c | 34 +++++++++++++++++----------- drivers/net/vmxnet3/vmxnet3_ethdev.h | 4 ++-- lib/eal/include/rte_common.h | 27 ++++++++++++++++++++++ 10 files changed, 79 insertions(+), 19 deletions(-) diff --git a/config/rte_config.h b/config/rte_config.h index fd6f8a2f1a..924192c71c 100644 --- a/config/rte_config.h +++ b/config/rte_config.h @@ -65,8 +65,8 @@ #define RTE_MBUF_DEFAULT_MEMPOOL_OPS "ring_mp_mc" /* ether defines */ -#define RTE_MAX_QUEUES_PER_PORT 1024 -#define RTE_ETHDEV_QUEUE_STAT_CNTRS 16 /* max 256 */ +#define RTE_MAX_QUEUES_PER_PORT 1 /* default 1024 */ +#define RTE_ETHDEV_QUEUE_STAT_CNTRS 1 /* max 256, default 16 */ #define RTE_ETHDEV_RXTX_CALLBACKS 1 #define RTE_MAX_MULTI_HOST_CTRLS 4 diff --git a/drivers/net/bnxt/bnxt_ethdev.c b/drivers/net/bnxt/bnxt_ethdev.c index 1f7c0d77d5..e26c8a1bbc 100644 --- a/drivers/net/bnxt/bnxt_ethdev.c +++ b/drivers/net/bnxt/bnxt_ethdev.c @@ -906,10 +906,12 @@ static int bnxt_start_nic(struct bnxt *bp) goto err_out; } + __rte_assume(bp->rx_nr_rings <= RTE_MAX_QUEUES_PER_PORT); for (j = 0; j < bp->rx_nr_rings; j++) { struct bnxt_rx_queue *rxq = bp->rx_queues[j]; if (!rxq->rx_deferred_start) { + /* __rte_assume(j < RTE_MAX_QUEUES_PER_PORT); */ bp->eth_dev->data->rx_queue_state[j] = RTE_ETH_QUEUE_STATE_STARTED; rxq->rx_started = true; @@ -926,10 +928,12 @@ static int bnxt_start_nic(struct bnxt *bp) goto err_out; } + __rte_assume(bp->tx_nr_rings <= RTE_MAX_QUEUES_PER_PORT); for (j = 0; j < bp->tx_nr_rings; j++) { struct bnxt_tx_queue *txq = bp->tx_queues[j]; if (!txq->tx_deferred_start) { + /* __rte_assume(j < RTE_MAX_QUEUES_PER_PORT); */ bp->eth_dev->data->tx_queue_state[j] = RTE_ETH_QUEUE_STATE_STARTED; txq->tx_started = true; diff --git a/drivers/net/bnxt/bnxt_rxq.c b/drivers/net/bnxt/bnxt_rxq.c index 1c25c57ca6..1651c26545 100644 --- a/drivers/net/bnxt/bnxt_rxq.c +++ b/drivers/net/bnxt/bnxt_rxq.c @@ -584,6 +584,7 @@ int bnxt_rx_queue_stop(struct rte_eth_dev *dev, uint16_t rx_queue_id) return -EINVAL; } + __rte_assume(q_id < RTE_MAX_QUEUES_PER_PORT); dev->data->rx_queue_state[q_id] = RTE_ETH_QUEUE_STATE_STOPPED; rxq->rx_started = false; PMD_DRV_LOG_LINE(DEBUG, "Rx queue stopped"); diff --git a/drivers/net/e1000/igb_rxtx.c b/drivers/net/e1000/igb_rxtx.c index d61eaad2de..1b6a96465c 100644 --- a/drivers/net/e1000/igb_rxtx.c +++ b/drivers/net/e1000/igb_rxtx.c @@ -1867,7 +1867,9 @@ igb_dev_clear_queues(struct rte_eth_dev *dev) struct igb_tx_queue *txq; struct igb_rx_queue *rxq; + __rte_assume(dev->data->nb_tx_queues <= RTE_MAX_QUEUES_PER_PORT); for (i = 0; i < dev->data->nb_tx_queues; i++) { + /* __rte_assume(i < RTE_MAX_QUEUES_PER_PORT); */ txq = dev->data->tx_queues[i]; if (txq != NULL) { igb_tx_queue_release_mbufs(txq); @@ -1876,7 +1878,9 @@ igb_dev_clear_queues(struct rte_eth_dev *dev) } } + __rte_assume(dev->data->nb_rx_queues <= RTE_MAX_QUEUES_PER_PORT); for (i = 0; i < dev->data->nb_rx_queues; i++) { + /* __rte_assume(i < RTE_MAX_QUEUES_PER_PORT); */ rxq = dev->data->rx_queues[i]; if (rxq != NULL) { igb_rx_queue_release_mbufs(rxq); diff --git a/drivers/net/failsafe/failsafe_ops.c b/drivers/net/failsafe/failsafe_ops.c index 9c013e0419..8b3977301e 100644 --- a/drivers/net/failsafe/failsafe_ops.c +++ b/drivers/net/failsafe/failsafe_ops.c @@ -110,13 +110,17 @@ fs_set_queues_state_start(struct rte_eth_dev *dev) struct txq *txq; uint16_t i; + __rte_assume(dev->data->nb_rx_queues <= RTE_MAX_QUEUES_PER_PORT); for (i = 0; i < dev->data->nb_rx_queues; i++) { + /* __rte_assume(i < RTE_MAX_QUEUES_PER_PORT); */ rxq = dev->data->rx_queues[i]; if (rxq != NULL && !rxq->info.conf.rx_deferred_start) dev->data->rx_queue_state[i] = RTE_ETH_QUEUE_STATE_STARTED; } + __rte_assume(dev->data->nb_tx_queues <= RTE_MAX_QUEUES_PER_PORT); for (i = 0; i < dev->data->nb_tx_queues; i++) { + /* __rte_assume(i < RTE_MAX_QUEUES_PER_PORT); */ txq = dev->data->tx_queues[i]; if (txq != NULL && !txq->info.conf.tx_deferred_start) dev->data->tx_queue_state[i] = @@ -176,14 +180,20 @@ fs_set_queues_state_stop(struct rte_eth_dev *dev) { uint16_t i; - for (i = 0; i < dev->data->nb_rx_queues; i++) + __rte_assume(dev->data->nb_rx_queues <= RTE_MAX_QUEUES_PER_PORT); + for (i = 0; i < dev->data->nb_rx_queues; i++) { + /* __rte_assume(i < RTE_MAX_QUEUES_PER_PORT); */ if (dev->data->rx_queues[i] != NULL) dev->data->rx_queue_state[i] = RTE_ETH_QUEUE_STATE_STOPPED; - for (i = 0; i < dev->data->nb_tx_queues; i++) + } + __rte_assume(dev->data->nb_tx_queues <= RTE_MAX_QUEUES_PER_PORT); + for (i = 0; i < dev->data->nb_tx_queues; i++) { + /* __rte_assume(i < RTE_MAX_QUEUES_PER_PORT); */ if (dev->data->tx_queues[i] != NULL) dev->data->tx_queue_state[i] = RTE_ETH_QUEUE_STATE_STOPPED; + } } static int diff --git a/drivers/net/hns3/hns3_rxtx.c b/drivers/net/hns3/hns3_rxtx.c index 5941b966e0..f77daa4862 100644 --- a/drivers/net/hns3/hns3_rxtx.c +++ b/drivers/net/hns3/hns3_rxtx.c @@ -1308,14 +1308,18 @@ hns3_start_tqps(struct hns3_hw *hw) hns3_enable_all_queues(hw, true); + __rte_assume(hw->data->nb_tx_queues <= RTE_MAX_QUEUES_PER_PORT); for (i = 0; i < hw->data->nb_tx_queues; i++) { + /* __rte_assume(i < RTE_MAX_QUEUES_PER_PORT); */ txq = hw->data->tx_queues[i]; if (txq->enabled) hw->data->tx_queue_state[i] = RTE_ETH_QUEUE_STATE_STARTED; } + __rte_assume(hw->data->nb_rx_queues <= RTE_MAX_QUEUES_PER_PORT); for (i = 0; i < hw->data->nb_rx_queues; i++) { + /* __rte_assume(i < RTE_MAX_QUEUES_PER_PORT); */ rxq = hw->data->rx_queues[i]; if (rxq->enabled) hw->data->rx_queue_state[i] = diff --git a/drivers/net/mana/tx.c b/drivers/net/mana/tx.c index 272a28bcba..70bb2cd517 100644 --- a/drivers/net/mana/tx.c +++ b/drivers/net/mana/tx.c @@ -74,6 +74,7 @@ mana_start_tx_queues(struct rte_eth_dev *dev) if (dev->data->tx_queue_state[i] == RTE_ETH_QUEUE_STATE_STARTED) return -EINVAL; + __rte_assume(priv->num_queues <= RTE_MAX_QUEUES_PER_PORT); for (i = 0; i < priv->num_queues; i++) { struct mana_txq *txq; struct ibv_qp_init_attr qp_attr = { 0 }; @@ -154,6 +155,7 @@ mana_start_tx_queues(struct rte_eth_dev *dev) txq->gdma_cq.count, txq->gdma_cq.size, txq->gdma_cq.head); + /* __rte_assume(i < RTE_MAX_QUEUES_PER_PORT); */ dev->data->tx_queue_state[i] = RTE_ETH_QUEUE_STATE_STARTED; } diff --git a/drivers/net/vmxnet3/vmxnet3_ethdev.c b/drivers/net/vmxnet3/vmxnet3_ethdev.c index 78fac63ab6..1752c58069 100644 --- a/drivers/net/vmxnet3/vmxnet3_ethdev.c +++ b/drivers/net/vmxnet3/vmxnet3_ethdev.c @@ -1470,42 +1470,52 @@ vmxnet3_dev_stats_get(struct rte_eth_dev *dev, struct rte_eth_stats *stats) struct vmxnet3_hw *hw = dev->data->dev_private; struct UPT1_TxStats txStats; struct UPT1_RxStats rxStats; + uint64_t packets, bytes; VMXNET3_WRITE_BAR1_REG(hw, VMXNET3_REG_CMD, VMXNET3_CMD_GET_STATS); for (i = 0; i < hw->num_tx_queues; i++) { vmxnet3_tx_stats_get(hw, i, &txStats); - stats->q_opackets[i] = txStats.ucastPktsTxOK + + packets = txStats.ucastPktsTxOK + txStats.mcastPktsTxOK + txStats.bcastPktsTxOK; - stats->q_obytes[i] = txStats.ucastBytesTxOK + + bytes = txStats.ucastBytesTxOK + txStats.mcastBytesTxOK + txStats.bcastBytesTxOK; - stats->opackets += stats->q_opackets[i]; - stats->obytes += stats->q_obytes[i]; + stats->opackets += packets; + stats->obytes += bytes; stats->oerrors += txStats.pktsTxError + txStats.pktsTxDiscard; + + if (i < RTE_ETHDEV_QUEUE_STAT_CNTRS) { + stats->q_opackets[i] = packets; + stats->q_obytes[i] = bytes; + } } for (i = 0; i < hw->num_rx_queues; i++) { vmxnet3_rx_stats_get(hw, i, &rxStats); - stats->q_ipackets[i] = rxStats.ucastPktsRxOK + + packets = rxStats.ucastPktsRxOK + rxStats.mcastPktsRxOK + rxStats.bcastPktsRxOK; - stats->q_ibytes[i] = rxStats.ucastBytesRxOK + + bytes = rxStats.ucastBytesRxOK + rxStats.mcastBytesRxOK + rxStats.bcastBytesRxOK; - stats->ipackets += stats->q_ipackets[i]; - stats->ibytes += stats->q_ibytes[i]; - - stats->q_errors[i] = rxStats.pktsRxError; + stats->ipackets += packets; + stats->ibytes += bytes; stats->ierrors += rxStats.pktsRxError; stats->imissed += rxStats.pktsRxOutOfBuf; + + if (i < RTE_ETHDEV_QUEUE_STAT_CNTRS) { + stats->q_ipackets[i] = packets; + stats->q_ibytes[i] = bytes; + stats->q_errors[i] = rxStats.pktsRxError; + } } return 0; @@ -1521,8 +1531,6 @@ vmxnet3_dev_stats_reset(struct rte_eth_dev *dev) VMXNET3_WRITE_BAR1_REG(hw, VMXNET3_REG_CMD, VMXNET3_CMD_GET_STATS); - RTE_BUILD_BUG_ON(RTE_ETHDEV_QUEUE_STAT_CNTRS < VMXNET3_MAX_TX_QUEUES); - for (i = 0; i < hw->num_tx_queues; i++) { vmxnet3_hw_tx_stats_get(hw, i, &txStats); memcpy(&hw->snapshot_tx_stats[i], &txStats, @@ -1566,7 +1574,7 @@ vmxnet3_dev_info_get(struct rte_eth_dev *dev, dev_info->min_rx_bufsize = 1518 + RTE_PKTMBUF_HEADROOM; dev_info->max_rx_pktlen = 16384; /* includes CRC, cf MAXFRS register */ dev_info->min_mtu = VMXNET3_MIN_MTU; - dev_info->max_mtu = VMXNET3_MAX_MTU; + dev_info->max_mtu = VMXNET3_VERSION_GE_6(hw) ? VMXNET3_V6_MAX_MTU : VMXNET3_MAX_MTU; dev_info->speed_capa = RTE_ETH_LINK_SPEED_10G; dev_info->max_mac_addrs = VMXNET3_MAX_MAC_ADDRS; diff --git a/drivers/net/vmxnet3/vmxnet3_ethdev.h b/drivers/net/vmxnet3/vmxnet3_ethdev.h index 2b3e2c4caa..e9ded6663d 100644 --- a/drivers/net/vmxnet3/vmxnet3_ethdev.h +++ b/drivers/net/vmxnet3/vmxnet3_ethdev.h @@ -121,8 +121,8 @@ struct vmxnet3_hw { #define VMXNET3_VFT_TABLE_SIZE (VMXNET3_VFT_SIZE * sizeof(uint32_t)) UPT1_TxStats saved_tx_stats[VMXNET3_EXT_MAX_TX_QUEUES]; UPT1_RxStats saved_rx_stats[VMXNET3_EXT_MAX_RX_QUEUES]; - UPT1_TxStats snapshot_tx_stats[VMXNET3_MAX_TX_QUEUES]; - UPT1_RxStats snapshot_rx_stats[VMXNET3_MAX_RX_QUEUES]; + UPT1_TxStats snapshot_tx_stats[VMXNET3_EXT_MAX_TX_QUEUES]; + UPT1_RxStats snapshot_rx_stats[VMXNET3_EXT_MAX_RX_QUEUES]; uint16_t tx_prod_offset; uint16_t rx_prod_offset[2]; /* device capability bit map */ diff --git a/lib/eal/include/rte_common.h b/lib/eal/include/rte_common.h index c79f9ed319..2f143c87e4 100644 --- a/lib/eal/include/rte_common.h +++ b/lib/eal/include/rte_common.h @@ -366,6 +366,16 @@ static void __attribute__((destructor(RTE_PRIO(prio)), used)) func(void) #define __rte_noreturn __attribute__((noreturn)) #endif +/** + * Hint point in program never reached + */ +#if defined(RTE_TOOLCHAIN_GCC) || defined(RTE_TOOLCHAIN_CLANG) +#define __rte_unreachable() __extension__(__builtin_unreachable()) +#else +/* MSVC or ICC */ +#define __rte_unreachable() __assume(0) +#endif + /** * Issue a warning in case the function's return value is ignored. * @@ -423,6 +433,23 @@ static void __attribute__((destructor(RTE_PRIO(prio)), used)) func(void) #define __rte_cold __attribute__((cold)) #endif +/** + * Hint precondition + * + * @warning Depending on the compiler, any code in ``condition`` might be executed. + * This currently only occurs with GCC prior to version 13. + */ +#if defined(RTE_TOOLCHAIN_GCC) && (GCC_VERSION >= 130000) +#define __rte_assume(condition) __attribute__((assume(condition))) +#elif defined(RTE_TOOLCHAIN_GCC) +#define __rte_assume(condition) do { if (!(condition)) __rte_unreachable(); } while (0) +#elif defined(RTE_TOOLCHAIN_CLANG) +#define __rte_assume(condition) __extension__(__builtin_assume(condition)) +#else +/* MSVC or ICC */ +#define __rte_assume(condition) __assume(condition) +#endif + /** * Disable AddressSanitizer on some code */ -- 2.43.0