From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id C379E45D6F; Fri, 22 Nov 2024 13:57:26 +0100 (CET) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 5ED4643432; Fri, 22 Nov 2024 13:55:16 +0100 (CET) Received: from mgamail.intel.com (mgamail.intel.com [192.198.163.9]) by mails.dpdk.org (Postfix) with ESMTP id A65D343406 for ; Fri, 22 Nov 2024 13:55:12 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1732280113; x=1763816113; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=ezU77FVrP8S3SwTu7/MyGKItHUWTQq2jrjX6nsgCzMU=; b=B6xhoh7QzOuG5mYVieoeAk79JkzZVsP734CQJ31v2CAw7Ka8s8of+uat 5sVZD9IODuEzZ8MfjW+BrTvnwqHZAmcBhQkCwnP+9ctF2GTRCicZVnc7Z TZ3Ntp+Xe9aSDhjnHWtFHotLLSFM7np6Srr7uo219QLhv9vHppnWztA5Z t+Mc6IUioDleMK83jk48yshpFiu/j5jNlkwvLdu0VjsGjiSoAqvyT62ki 3eCASUdiG925N33N9pxKriFhL60W+OSF4XoiE80US/l5p/HP2UK4xLdbU f4TvPyRQV2Ux58Wl4FlQJoXW9Mr1T2eUcy20ZeiRH8jNCbG9J5Vwe9BOM Q==; X-CSE-ConnectionGUID: y2u5Ya5bQ+S5/OuFSA6O+w== X-CSE-MsgGUID: 1YRVKUmUSdCaX9FcuZZ0hQ== X-IronPort-AV: E=McAfee;i="6700,10204,11263"; a="43085401" X-IronPort-AV: E=Sophos;i="6.12,175,1728975600"; d="scan'208";a="43085401" Received: from fmviesa007.fm.intel.com ([10.60.135.147]) by fmvoesa103.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 22 Nov 2024 04:55:12 -0800 X-CSE-ConnectionGUID: mqz1IBMiROqO6vZgTAO3kQ== X-CSE-MsgGUID: D80AJMGySLO6wKdBwyK69Q== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.12,175,1728975600"; d="scan'208";a="90373304" Received: from unknown (HELO silpixa00401385.ir.intel.com) ([10.237.214.25]) by fmviesa007.fm.intel.com with ESMTP; 22 Nov 2024 04:55:10 -0800 From: Bruce Richardson To: dev@dpdk.org Cc: Bruce Richardson , Vladimir Medvedkin , Ian Stokes , Konstantin Ananyev Subject: [RFC PATCH 20/21] net/iavf: use vector SW ring for all vector paths Date: Fri, 22 Nov 2024 12:54:13 +0000 Message-ID: <20241122125418.2857301-21-bruce.richardson@intel.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20241122125418.2857301-1-bruce.richardson@intel.com> References: <20241122125418.2857301-1-bruce.richardson@intel.com> MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org The AVX-512 code path used a smaller SW ring structure only containing the mbuf pointer, but no other fields. The other fields are only used in the scalar code path, so update all vector driver code paths (AVX2, SSE) to use the smaller, faster structure. Signed-off-by: Bruce Richardson --- drivers/net/iavf/iavf_rxtx.c | 7 ------- drivers/net/iavf/iavf_rxtx_vec_avx2.c | 12 ++++++------ drivers/net/iavf/iavf_rxtx_vec_avx512.c | 8 -------- drivers/net/iavf/iavf_rxtx_vec_common.h | 6 ------ drivers/net/iavf/iavf_rxtx_vec_sse.c | 14 +++++++------- 5 files changed, 13 insertions(+), 34 deletions(-) diff --git a/drivers/net/iavf/iavf_rxtx.c b/drivers/net/iavf/iavf_rxtx.c index c574b23f34..869fce00eb 100644 --- a/drivers/net/iavf/iavf_rxtx.c +++ b/drivers/net/iavf/iavf_rxtx.c @@ -4193,14 +4193,7 @@ iavf_set_tx_function(struct rte_eth_dev *dev) txq = dev->data->tx_queues[i]; if (!txq) continue; -#ifdef CC_AVX512_SUPPORT - if (use_avx512) - iavf_txq_vec_setup_avx512(txq); - else - iavf_txq_vec_setup(txq); -#else iavf_txq_vec_setup(txq); -#endif } if (no_poll_on_link_down) { diff --git a/drivers/net/iavf/iavf_rxtx_vec_avx2.c b/drivers/net/iavf/iavf_rxtx_vec_avx2.c index 25dc339303..e0c7146c9b 100644 --- a/drivers/net/iavf/iavf_rxtx_vec_avx2.c +++ b/drivers/net/iavf/iavf_rxtx_vec_avx2.c @@ -1736,14 +1736,14 @@ iavf_xmit_fixed_burst_vec_avx2(void *tx_queue, struct rte_mbuf **tx_pkts, { struct ieth_tx_queue *txq = (struct ieth_tx_queue *)tx_queue; volatile struct iavf_tx_desc *txdp; - struct ieth_tx_entry *txep; + struct ieth_vec_tx_entry *txep; uint16_t n, nb_commit, tx_id; /* bit2 is reserved and must be set to 1 according to Spec */ uint64_t flags = IAVF_TX_DESC_CMD_EOP | IAVF_TX_DESC_CMD_ICRC; uint64_t rs = IAVF_TX_DESC_CMD_RS | flags; if (txq->nb_tx_free < txq->tx_free_thresh) - iavf_tx_free_bufs(txq); + ieth_tx_free_bufs_vector(txq, iavf_tx_desc_done, false); nb_commit = nb_pkts = (uint16_t)RTE_MIN(txq->nb_tx_free, nb_pkts); if (unlikely(nb_pkts == 0)) @@ -1751,13 +1751,13 @@ iavf_xmit_fixed_burst_vec_avx2(void *tx_queue, struct rte_mbuf **tx_pkts, tx_id = txq->tx_tail; txdp = &txq->iavf_tx_ring[tx_id]; - txep = &txq->sw_ring[tx_id]; + txep = &txq->sw_ring_v[tx_id]; txq->nb_tx_free = (uint16_t)(txq->nb_tx_free - nb_pkts); n = (uint16_t)(txq->nb_tx_desc - tx_id); if (nb_commit >= n) { - ieth_tx_backlog_entry(txep, tx_pkts, n); + ieth_tx_backlog_entry_vec(txep, tx_pkts, n); iavf_vtx(txdp, tx_pkts, n - 1, flags, offload); tx_pkts += (n - 1); @@ -1772,10 +1772,10 @@ iavf_xmit_fixed_burst_vec_avx2(void *tx_queue, struct rte_mbuf **tx_pkts, /* avoid reach the end of ring */ txdp = &txq->iavf_tx_ring[tx_id]; - txep = &txq->sw_ring[tx_id]; + txep = &txq->sw_ring_v[tx_id]; } - ieth_tx_backlog_entry(txep, tx_pkts, nb_commit); + ieth_tx_backlog_entry_vec(txep, tx_pkts, nb_commit); iavf_vtx(txdp, tx_pkts, nb_commit, flags, offload); diff --git a/drivers/net/iavf/iavf_rxtx_vec_avx512.c b/drivers/net/iavf/iavf_rxtx_vec_avx512.c index 16cfd6a5b3..bda5fb3b22 100644 --- a/drivers/net/iavf/iavf_rxtx_vec_avx512.c +++ b/drivers/net/iavf/iavf_rxtx_vec_avx512.c @@ -2356,14 +2356,6 @@ iavf_xmit_pkts_vec_avx512(void *tx_queue, struct rte_mbuf **tx_pkts, return iavf_xmit_pkts_vec_avx512_cmn(tx_queue, tx_pkts, nb_pkts, false); } -int __rte_cold -iavf_txq_vec_setup_avx512(struct ieth_tx_queue *txq) -{ - txq->vector_tx = true; - txq->vector_sw_ring = true; - return 0; -} - uint16_t iavf_xmit_pkts_vec_avx512_offload(void *tx_queue, struct rte_mbuf **tx_pkts, uint16_t nb_pkts) diff --git a/drivers/net/iavf/iavf_rxtx_vec_common.h b/drivers/net/iavf/iavf_rxtx_vec_common.h index 20d8262e7f..14569e9e3b 100644 --- a/drivers/net/iavf/iavf_rxtx_vec_common.h +++ b/drivers/net/iavf/iavf_rxtx_vec_common.h @@ -24,12 +24,6 @@ iavf_tx_desc_done(struct ieth_tx_queue *txq, uint16_t idx) rte_cpu_to_le_64(IAVF_TX_DESC_DTYPE_DESC_DONE); } -static __rte_always_inline int -iavf_tx_free_bufs(struct ieth_tx_queue *txq) -{ - return ieth_tx_free_bufs(txq, iavf_tx_desc_done); -} - static inline void _iavf_rx_queue_release_mbufs_vec(struct iavf_rx_queue *rxq) { diff --git a/drivers/net/iavf/iavf_rxtx_vec_sse.c b/drivers/net/iavf/iavf_rxtx_vec_sse.c index 21ad685ff1..89f4a22271 100644 --- a/drivers/net/iavf/iavf_rxtx_vec_sse.c +++ b/drivers/net/iavf/iavf_rxtx_vec_sse.c @@ -1368,14 +1368,14 @@ iavf_xmit_fixed_burst_vec(void *tx_queue, struct rte_mbuf **tx_pkts, { struct ieth_tx_queue *txq = (struct ieth_tx_queue *)tx_queue; volatile struct iavf_tx_desc *txdp; - struct ieth_tx_entry *txep; + struct ieth_vec_tx_entry *txep; uint16_t n, nb_commit, tx_id; uint64_t flags = IAVF_TX_DESC_CMD_EOP | 0x04; /* bit 2 must be set */ uint64_t rs = IAVF_TX_DESC_CMD_RS | flags; int i; if (txq->nb_tx_free < txq->tx_free_thresh) - iavf_tx_free_bufs(txq); + ieth_tx_free_bufs_vector(txq, iavf_tx_desc_done, false); nb_pkts = (uint16_t)RTE_MIN(txq->nb_tx_free, nb_pkts); if (unlikely(nb_pkts == 0)) @@ -1384,13 +1384,13 @@ iavf_xmit_fixed_burst_vec(void *tx_queue, struct rte_mbuf **tx_pkts, tx_id = txq->tx_tail; txdp = &txq->iavf_tx_ring[tx_id]; - txep = &txq->sw_ring[tx_id]; + txep = &txq->sw_ring_v[tx_id]; txq->nb_tx_free = (uint16_t)(txq->nb_tx_free - nb_pkts); n = (uint16_t)(txq->nb_tx_desc - tx_id); if (nb_commit >= n) { - ieth_tx_backlog_entry(txep, tx_pkts, n); + ieth_tx_backlog_entry_vec(txep, tx_pkts, n); for (i = 0; i < n - 1; ++i, ++tx_pkts, ++txdp) vtx1(txdp, *tx_pkts, flags); @@ -1404,10 +1404,10 @@ iavf_xmit_fixed_burst_vec(void *tx_queue, struct rte_mbuf **tx_pkts, /* avoid reach the end of ring */ txdp = &txq->iavf_tx_ring[tx_id]; - txep = &txq->sw_ring[tx_id]; + txep = &txq->sw_ring_v[tx_id]; } - ieth_tx_backlog_entry(txep, tx_pkts, nb_commit); + ieth_tx_backlog_entry_vec(txep, tx_pkts, nb_commit); iavf_vtx(txdp, tx_pkts, nb_commit, flags); @@ -1462,7 +1462,7 @@ int __rte_cold iavf_txq_vec_setup(struct ieth_tx_queue *txq) { txq->vector_tx = true; - txq->vector_sw_ring = false; + txq->vector_sw_ring = txq->vector_tx; return 0; } -- 2.43.0