From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 9B625460C2; Mon, 20 Jan 2025 13:00:51 +0100 (CET) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 73BC640E2A; Mon, 20 Jan 2025 13:00:42 +0100 (CET) Received: from mgamail.intel.com (mgamail.intel.com [192.198.163.19]) by mails.dpdk.org (Postfix) with ESMTP id D9D8240BA4 for ; Mon, 20 Jan 2025 13:00:39 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1737374440; x=1768910440; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=fCAHgk4PgDGui90EngPH5Re8H7rgO4kzukrlmga0XUg=; b=J6XkBfx4XHpJ8w0BmadD5QGIPcilL8h+IBst/Qeid2g1jQKcFOdCoVYF WQ6G+rOCOFxMx4Te9p3WPoXDdCQJjfvcNoWFY+dUI/BQmQcPga0JpkJ0+ 9awELjvaRYfqNbzNj4jJOFx5hEWXJNJIai+d8ncmcfN0zn3qhzjlsJ1E4 UzD3iAK6DaZ4eslzE2GrTd2NRP+EXFmPnDREnAUrHquGj8PZ74zZWR1w8 jy3cyzs5o/QEUVUZu7BuGR5TIE/L/Z0MYiYlzl5aRL0j9mvsmvIrEujme pmkDdsucffC0jguxLG5Ef1/i/0P/EPKfV/JwiUbdXIbPXKX/wP/lC0GMd g==; X-CSE-ConnectionGUID: DUckurF0QcCkDvWiV41pnQ== X-CSE-MsgGUID: PVXxiR5JTjyX++/u8LGGmg== X-IronPort-AV: E=McAfee;i="6700,10204,11320"; a="36979005" X-IronPort-AV: E=Sophos;i="6.13,219,1732608000"; d="scan'208";a="36979005" Received: from orviesa001.jf.intel.com ([10.64.159.141]) by fmvoesa113.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 20 Jan 2025 04:00:39 -0800 X-CSE-ConnectionGUID: nnfKg66/RLOYao2T49RIIA== X-CSE-MsgGUID: 7ELOB4mhTxOpd8ae1/Ytjw== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.12,224,1728975600"; d="scan'208";a="143766757" Received: from silpixa00401197coob.ir.intel.com (HELO silpixa00401385.ir.intel.com) ([10.237.214.45]) by orviesa001.jf.intel.com with ESMTP; 20 Jan 2025 04:00:38 -0800 From: Bruce Richardson To: dev@dpdk.org Cc: david.marchand@redhat.com, Bruce Richardson , Ian Stokes , Vladimir Medvedkin , Anatoly Burakov Subject: [PATCH v5 02/25] net/intel/common: add pkt reassembly fn for intel drivers Date: Mon, 20 Jan 2025 11:59:44 +0000 Message-ID: <20250120120016.1530274-3-bruce.richardson@intel.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20250120120016.1530274-1-bruce.richardson@intel.com> References: <20241122125418.2857301-1-bruce.richardson@intel.com> <20250120120016.1530274-1-bruce.richardson@intel.com> MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org The code for reassembling a single, multi-mbuf packet from multiple buffers received from the NIC is duplicated across many drivers. Rather than having multiple copies of this function, we can create an "intel/common" directory to hold such functions and consolidate multiple functions down to a single one for easier maintenance. Signed-off-by: Bruce Richardson --- drivers/net/intel/common/rx.h | 79 +++++++++++++++++++ .../net/intel/i40e/i40e_rxtx_vec_altivec.c | 4 +- drivers/net/intel/i40e/i40e_rxtx_vec_avx2.c | 4 +- drivers/net/intel/i40e/i40e_rxtx_vec_avx512.c | 4 +- drivers/net/intel/i40e/i40e_rxtx_vec_common.h | 64 +-------------- drivers/net/intel/i40e/i40e_rxtx_vec_neon.c | 4 +- drivers/net/intel/i40e/i40e_rxtx_vec_sse.c | 4 +- drivers/net/intel/iavf/iavf_rxtx_vec_avx2.c | 8 +- drivers/net/intel/iavf/iavf_rxtx_vec_avx512.c | 8 +- drivers/net/intel/iavf/iavf_rxtx_vec_common.h | 65 +-------------- drivers/net/intel/iavf/iavf_rxtx_vec_sse.c | 8 +- drivers/net/intel/ice/ice_rxtx_vec_avx2.c | 4 +- drivers/net/intel/ice/ice_rxtx_vec_avx512.c | 8 +- drivers/net/intel/ice/ice_rxtx_vec_common.h | 66 +--------------- drivers/net/intel/ice/ice_rxtx_vec_sse.c | 4 +- .../net/intel/ixgbe/ixgbe_rxtx_vec_common.h | 63 +-------------- drivers/net/intel/ixgbe/ixgbe_rxtx_vec_neon.c | 4 +- drivers/net/intel/ixgbe/ixgbe_rxtx_vec_sse.c | 4 +- 18 files changed, 117 insertions(+), 288 deletions(-) create mode 100644 drivers/net/intel/common/rx.h diff --git a/drivers/net/intel/common/rx.h b/drivers/net/intel/common/rx.h new file mode 100644 index 0000000000..5bd2fea7e3 --- /dev/null +++ b/drivers/net/intel/common/rx.h @@ -0,0 +1,79 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(c) 2024 Intel Corporation + */ + +#ifndef _COMMON_INTEL_RX_H_ +#define _COMMON_INTEL_RX_H_ + +#include +#include +#include + +#define CI_RX_BURST 32 + +static inline uint16_t +ci_rx_reassemble_packets(struct rte_mbuf **rx_bufs, uint16_t nb_bufs, uint8_t *split_flags, + struct rte_mbuf **pkt_first_seg, struct rte_mbuf **pkt_last_seg, + const uint8_t crc_len) +{ + struct rte_mbuf *pkts[CI_RX_BURST] = {0}; /*finished pkts*/ + struct rte_mbuf *start = *pkt_first_seg; + struct rte_mbuf *end = *pkt_last_seg; + unsigned int pkt_idx, buf_idx; + + for (buf_idx = 0, pkt_idx = 0; buf_idx < nb_bufs; buf_idx++) { + if (end) { + /* processing a split packet */ + end->next = rx_bufs[buf_idx]; + rx_bufs[buf_idx]->data_len += crc_len; + + start->nb_segs++; + start->pkt_len += rx_bufs[buf_idx]->data_len; + end = end->next; + + if (!split_flags[buf_idx]) { + /* it's the last packet of the set */ + start->hash = end->hash; + start->vlan_tci = end->vlan_tci; + start->ol_flags = end->ol_flags; + /* we need to strip crc for the whole packet */ + start->pkt_len -= crc_len; + if (end->data_len > crc_len) { + end->data_len -= crc_len; + } else { + /* free up last mbuf */ + struct rte_mbuf *secondlast = start; + + start->nb_segs--; + while (secondlast->next != end) + secondlast = secondlast->next; + secondlast->data_len -= (crc_len - end->data_len); + secondlast->next = NULL; + rte_pktmbuf_free_seg(end); + } + pkts[pkt_idx++] = start; + start = NULL; + end = NULL; + } + } else { + /* not processing a split packet */ + if (!split_flags[buf_idx]) { + /* not a split packet, save and skip */ + pkts[pkt_idx++] = rx_bufs[buf_idx]; + continue; + } + start = rx_bufs[buf_idx]; + end = start; + rx_bufs[buf_idx]->data_len += crc_len; + rx_bufs[buf_idx]->pkt_len += crc_len; + } + } + + /* save the partial packet for next time */ + *pkt_first_seg = start; + *pkt_last_seg = end; + memcpy(rx_bufs, pkts, pkt_idx * (sizeof(*pkts))); + return pkt_idx; +} + +#endif /* _COMMON_INTEL_RX_H_ */ diff --git a/drivers/net/intel/i40e/i40e_rxtx_vec_altivec.c b/drivers/net/intel/i40e/i40e_rxtx_vec_altivec.c index b6b0d38ec1..95829f65d5 100644 --- a/drivers/net/intel/i40e/i40e_rxtx_vec_altivec.c +++ b/drivers/net/intel/i40e/i40e_rxtx_vec_altivec.c @@ -494,8 +494,8 @@ i40e_recv_scattered_burst_vec(void *rx_queue, struct rte_mbuf **rx_pkts, if (i == nb_bufs) return nb_bufs; } - return i + reassemble_packets(rxq, &rx_pkts[i], nb_bufs - i, - &split_flags[i]); + return i + ci_rx_reassemble_packets(&rx_pkts[i], nb_bufs - i, &split_flags[i], + &rxq->pkt_first_seg, &rxq->pkt_last_seg, rxq->crc_len); } /** diff --git a/drivers/net/intel/i40e/i40e_rxtx_vec_avx2.c b/drivers/net/intel/i40e/i40e_rxtx_vec_avx2.c index 19cf0ac718..6dd6e55d9c 100644 --- a/drivers/net/intel/i40e/i40e_rxtx_vec_avx2.c +++ b/drivers/net/intel/i40e/i40e_rxtx_vec_avx2.c @@ -657,8 +657,8 @@ i40e_recv_scattered_burst_vec_avx2(void *rx_queue, struct rte_mbuf **rx_pkts, return nb_bufs; rxq->pkt_first_seg = rx_pkts[i]; } - return i + reassemble_packets(rxq, &rx_pkts[i], nb_bufs - i, - &split_flags[i]); + return i + ci_rx_reassemble_packets(&rx_pkts[i], nb_bufs - i, &split_flags[i], + &rxq->pkt_first_seg, &rxq->pkt_last_seg, rxq->crc_len); } /* diff --git a/drivers/net/intel/i40e/i40e_rxtx_vec_avx512.c b/drivers/net/intel/i40e/i40e_rxtx_vec_avx512.c index 3b2750221b..506f1b5878 100644 --- a/drivers/net/intel/i40e/i40e_rxtx_vec_avx512.c +++ b/drivers/net/intel/i40e/i40e_rxtx_vec_avx512.c @@ -725,8 +725,8 @@ i40e_recv_scattered_burst_vec_avx512(void *rx_queue, return nb_bufs; rxq->pkt_first_seg = rx_pkts[i]; } - return i + reassemble_packets(rxq, &rx_pkts[i], nb_bufs - i, - &split_flags[i]); + return i + ci_rx_reassemble_packets(&rx_pkts[i], nb_bufs - i, &split_flags[i], + &rxq->pkt_first_seg, &rxq->pkt_last_seg, rxq->crc_len); } /** diff --git a/drivers/net/intel/i40e/i40e_rxtx_vec_common.h b/drivers/net/intel/i40e/i40e_rxtx_vec_common.h index 8b745630e4..a529cdbddf 100644 --- a/drivers/net/intel/i40e/i40e_rxtx_vec_common.h +++ b/drivers/net/intel/i40e/i40e_rxtx_vec_common.h @@ -8,6 +8,7 @@ #include #include +#include "../common/rx.h" #include "i40e_ethdev.h" #include "i40e_rxtx.h" @@ -15,69 +16,6 @@ #pragma GCC diagnostic ignored "-Wcast-qual" #endif -static inline uint16_t -reassemble_packets(struct i40e_rx_queue *rxq, struct rte_mbuf **rx_bufs, - uint16_t nb_bufs, uint8_t *split_flags) -{ - struct rte_mbuf *pkts[RTE_I40E_VPMD_RX_BURST]; /*finished pkts*/ - struct rte_mbuf *start = rxq->pkt_first_seg; - struct rte_mbuf *end = rxq->pkt_last_seg; - unsigned pkt_idx, buf_idx; - - for (buf_idx = 0, pkt_idx = 0; buf_idx < nb_bufs; buf_idx++) { - if (end != NULL) { - /* processing a split packet */ - end->next = rx_bufs[buf_idx]; - rx_bufs[buf_idx]->data_len += rxq->crc_len; - - start->nb_segs++; - start->pkt_len += rx_bufs[buf_idx]->data_len; - end = end->next; - - if (!split_flags[buf_idx]) { - /* it's the last packet of the set */ - start->hash = end->hash; - start->vlan_tci = end->vlan_tci; - start->ol_flags = end->ol_flags; - /* we need to strip crc for the whole packet */ - start->pkt_len -= rxq->crc_len; - if (end->data_len > rxq->crc_len) - end->data_len -= rxq->crc_len; - else { - /* free up last mbuf */ - struct rte_mbuf *secondlast = start; - - start->nb_segs--; - while (secondlast->next != end) - secondlast = secondlast->next; - secondlast->data_len -= (rxq->crc_len - - end->data_len); - secondlast->next = NULL; - rte_pktmbuf_free_seg(end); - } - pkts[pkt_idx++] = start; - start = end = NULL; - } - } else { - /* not processing a split packet */ - if (!split_flags[buf_idx]) { - /* not a split packet, save and skip */ - pkts[pkt_idx++] = rx_bufs[buf_idx]; - continue; - } - end = start = rx_bufs[buf_idx]; - rx_bufs[buf_idx]->data_len += rxq->crc_len; - rx_bufs[buf_idx]->pkt_len += rxq->crc_len; - } - } - - /* save the partial packet for next time */ - rxq->pkt_first_seg = start; - rxq->pkt_last_seg = end; - memcpy(rx_bufs, pkts, pkt_idx * (sizeof(*pkts))); - return pkt_idx; -} - static __rte_always_inline int i40e_tx_free_bufs(struct i40e_tx_queue *txq) { diff --git a/drivers/net/intel/i40e/i40e_rxtx_vec_neon.c b/drivers/net/intel/i40e/i40e_rxtx_vec_neon.c index e1c5c7041b..159d971796 100644 --- a/drivers/net/intel/i40e/i40e_rxtx_vec_neon.c +++ b/drivers/net/intel/i40e/i40e_rxtx_vec_neon.c @@ -623,8 +623,8 @@ i40e_recv_scattered_burst_vec(void *rx_queue, struct rte_mbuf **rx_pkts, return nb_bufs; rxq->pkt_first_seg = rx_pkts[i]; } - return i + reassemble_packets(rxq, &rx_pkts[i], nb_bufs - i, - &split_flags[i]); + return i + ci_rx_reassemble_packets(&rx_pkts[i], nb_bufs - i, &split_flags[i], + &rxq->pkt_first_seg, &rxq->pkt_last_seg, rxq->crc_len); } /** diff --git a/drivers/net/intel/i40e/i40e_rxtx_vec_sse.c b/drivers/net/intel/i40e/i40e_rxtx_vec_sse.c index ad560d2b6b..3a8128e014 100644 --- a/drivers/net/intel/i40e/i40e_rxtx_vec_sse.c +++ b/drivers/net/intel/i40e/i40e_rxtx_vec_sse.c @@ -641,8 +641,8 @@ i40e_recv_scattered_burst_vec(void *rx_queue, struct rte_mbuf **rx_pkts, return nb_bufs; rxq->pkt_first_seg = rx_pkts[i]; } - return i + reassemble_packets(rxq, &rx_pkts[i], nb_bufs - i, - &split_flags[i]); + return i + ci_rx_reassemble_packets(&rx_pkts[i], nb_bufs - i, &split_flags[i], + &rxq->pkt_first_seg, &rxq->pkt_last_seg, rxq->crc_len); } /** diff --git a/drivers/net/intel/iavf/iavf_rxtx_vec_avx2.c b/drivers/net/intel/iavf/iavf_rxtx_vec_avx2.c index 49d41af953..0baf5045c8 100644 --- a/drivers/net/intel/iavf/iavf_rxtx_vec_avx2.c +++ b/drivers/net/intel/iavf/iavf_rxtx_vec_avx2.c @@ -1508,8 +1508,8 @@ iavf_recv_scattered_burst_vec_avx2(void *rx_queue, struct rte_mbuf **rx_pkts, return nb_bufs; rxq->pkt_first_seg = rx_pkts[i]; } - return i + reassemble_packets(rxq, &rx_pkts[i], nb_bufs - i, - &split_flags[i]); + return i + ci_rx_reassemble_packets(&rx_pkts[i], nb_bufs - i, &split_flags[i], + &rxq->pkt_first_seg, &rxq->pkt_last_seg, rxq->crc_len); } /** @@ -1597,8 +1597,8 @@ iavf_recv_scattered_burst_vec_avx2_flex_rxd(void *rx_queue, return nb_bufs; rxq->pkt_first_seg = rx_pkts[i]; } - return i + reassemble_packets(rxq, &rx_pkts[i], nb_bufs - i, - &split_flags[i]); + return i + ci_rx_reassemble_packets(&rx_pkts[i], nb_bufs - i, &split_flags[i], + &rxq->pkt_first_seg, &rxq->pkt_last_seg, rxq->crc_len); } /** diff --git a/drivers/net/intel/iavf/iavf_rxtx_vec_avx512.c b/drivers/net/intel/iavf/iavf_rxtx_vec_avx512.c index d6a861bf80..5a88007096 100644 --- a/drivers/net/intel/iavf/iavf_rxtx_vec_avx512.c +++ b/drivers/net/intel/iavf/iavf_rxtx_vec_avx512.c @@ -1685,8 +1685,8 @@ iavf_recv_scattered_burst_vec_avx512(void *rx_queue, struct rte_mbuf **rx_pkts, return nb_bufs; rxq->pkt_first_seg = rx_pkts[i]; } - return i + reassemble_packets(rxq, &rx_pkts[i], nb_bufs - i, - &split_flags[i]); + return i + ci_rx_reassemble_packets(&rx_pkts[i], nb_bufs - i, &split_flags[i], + &rxq->pkt_first_seg, &rxq->pkt_last_seg, rxq->crc_len); } /** @@ -1761,8 +1761,8 @@ iavf_recv_scattered_burst_vec_avx512_flex_rxd(void *rx_queue, return nb_bufs; rxq->pkt_first_seg = rx_pkts[i]; } - return i + reassemble_packets(rxq, &rx_pkts[i], nb_bufs - i, - &split_flags[i]); + return i + ci_rx_reassemble_packets(&rx_pkts[i], nb_bufs - i, &split_flags[i], + &rxq->pkt_first_seg, &rxq->pkt_last_seg, rxq->crc_len); } /** diff --git a/drivers/net/intel/iavf/iavf_rxtx_vec_common.h b/drivers/net/intel/iavf/iavf_rxtx_vec_common.h index 5c5220048d..69d24ff011 100644 --- a/drivers/net/intel/iavf/iavf_rxtx_vec_common.h +++ b/drivers/net/intel/iavf/iavf_rxtx_vec_common.h @@ -8,6 +8,7 @@ #include #include +#include "../common/rx.h" #include "iavf.h" #include "iavf_rxtx.h" @@ -15,70 +16,6 @@ #pragma GCC diagnostic ignored "-Wcast-qual" #endif -static __rte_always_inline uint16_t -reassemble_packets(struct iavf_rx_queue *rxq, struct rte_mbuf **rx_bufs, - uint16_t nb_bufs, uint8_t *split_flags) -{ - struct rte_mbuf *pkts[IAVF_VPMD_RX_MAX_BURST]; - struct rte_mbuf *start = rxq->pkt_first_seg; - struct rte_mbuf *end = rxq->pkt_last_seg; - unsigned int pkt_idx, buf_idx; - - for (buf_idx = 0, pkt_idx = 0; buf_idx < nb_bufs; buf_idx++) { - if (end) { - /* processing a split packet */ - end->next = rx_bufs[buf_idx]; - rx_bufs[buf_idx]->data_len += rxq->crc_len; - - start->nb_segs++; - start->pkt_len += rx_bufs[buf_idx]->data_len; - end = end->next; - - if (!split_flags[buf_idx]) { - /* it's the last packet of the set */ - start->hash = end->hash; - start->vlan_tci = end->vlan_tci; - start->ol_flags = end->ol_flags; - /* we need to strip crc for the whole packet */ - start->pkt_len -= rxq->crc_len; - if (end->data_len > rxq->crc_len) { - end->data_len -= rxq->crc_len; - } else { - /* free up last mbuf */ - struct rte_mbuf *secondlast = start; - - start->nb_segs--; - while (secondlast->next != end) - secondlast = secondlast->next; - secondlast->data_len -= (rxq->crc_len - - end->data_len); - secondlast->next = NULL; - rte_pktmbuf_free_seg(end); - } - pkts[pkt_idx++] = start; - start = NULL; - end = NULL; - } - } else { - /* not processing a split packet */ - if (!split_flags[buf_idx]) { - /* not a split packet, save and skip */ - pkts[pkt_idx++] = rx_bufs[buf_idx]; - continue; - } - end = start = rx_bufs[buf_idx]; - rx_bufs[buf_idx]->data_len += rxq->crc_len; - rx_bufs[buf_idx]->pkt_len += rxq->crc_len; - } - } - - /* save the partial packet for next time */ - rxq->pkt_first_seg = start; - rxq->pkt_last_seg = end; - memcpy(rx_bufs, pkts, pkt_idx * (sizeof(*pkts))); - return pkt_idx; -} - static __rte_always_inline int iavf_tx_free_bufs(struct iavf_tx_queue *txq) { diff --git a/drivers/net/intel/iavf/iavf_rxtx_vec_sse.c b/drivers/net/intel/iavf/iavf_rxtx_vec_sse.c index 0db6fa8bd4..48b01462ea 100644 --- a/drivers/net/intel/iavf/iavf_rxtx_vec_sse.c +++ b/drivers/net/intel/iavf/iavf_rxtx_vec_sse.c @@ -1238,8 +1238,8 @@ iavf_recv_scattered_burst_vec(void *rx_queue, struct rte_mbuf **rx_pkts, return nb_bufs; rxq->pkt_first_seg = rx_pkts[i]; } - return i + reassemble_packets(rxq, &rx_pkts[i], nb_bufs - i, - &split_flags[i]); + return i + ci_rx_reassemble_packets(&rx_pkts[i], nb_bufs - i, &split_flags[i], + &rxq->pkt_first_seg, &rxq->pkt_last_seg, rxq->crc_len); } /** @@ -1307,8 +1307,8 @@ iavf_recv_scattered_burst_vec_flex_rxd(void *rx_queue, return nb_bufs; rxq->pkt_first_seg = rx_pkts[i]; } - return i + reassemble_packets(rxq, &rx_pkts[i], nb_bufs - i, - &split_flags[i]); + return i + ci_rx_reassemble_packets(&rx_pkts[i], nb_bufs - i, &split_flags[i], + &rxq->pkt_first_seg, &rxq->pkt_last_seg, rxq->crc_len); } /** diff --git a/drivers/net/intel/ice/ice_rxtx_vec_avx2.c b/drivers/net/intel/ice/ice_rxtx_vec_avx2.c index d6e88dbb29..ca247b155c 100644 --- a/drivers/net/intel/ice/ice_rxtx_vec_avx2.c +++ b/drivers/net/intel/ice/ice_rxtx_vec_avx2.c @@ -726,8 +726,8 @@ ice_recv_scattered_burst_vec_avx2(void *rx_queue, struct rte_mbuf **rx_pkts, return nb_bufs; rxq->pkt_first_seg = rx_pkts[i]; } - return i + ice_rx_reassemble_packets(rxq, &rx_pkts[i], nb_bufs - i, - &split_flags[i]); + return i + ci_rx_reassemble_packets(&rx_pkts[i], nb_bufs - i, &split_flags[i], + &rxq->pkt_first_seg, &rxq->pkt_last_seg, rxq->crc_len); } /** diff --git a/drivers/net/intel/ice/ice_rxtx_vec_avx512.c b/drivers/net/intel/ice/ice_rxtx_vec_avx512.c index add095ef06..1e603d5d8f 100644 --- a/drivers/net/intel/ice/ice_rxtx_vec_avx512.c +++ b/drivers/net/intel/ice/ice_rxtx_vec_avx512.c @@ -763,8 +763,8 @@ ice_recv_scattered_burst_vec_avx512(void *rx_queue, struct rte_mbuf **rx_pkts, return nb_bufs; rxq->pkt_first_seg = rx_pkts[i]; } - return i + ice_rx_reassemble_packets(rxq, &rx_pkts[i], nb_bufs - i, - &split_flags[i]); + return i + ci_rx_reassemble_packets(&rx_pkts[i], nb_bufs - i, &split_flags[i], + &rxq->pkt_first_seg, &rxq->pkt_last_seg, rxq->crc_len); } /** @@ -805,8 +805,8 @@ ice_recv_scattered_burst_vec_avx512_offload(void *rx_queue, return nb_bufs; rxq->pkt_first_seg = rx_pkts[i]; } - return i + ice_rx_reassemble_packets(rxq, &rx_pkts[i], nb_bufs - i, - &split_flags[i]); + return i + ci_rx_reassemble_packets(&rx_pkts[i], nb_bufs - i, &split_flags[i], + &rxq->pkt_first_seg, &rxq->pkt_last_seg, rxq->crc_len); } /** diff --git a/drivers/net/intel/ice/ice_rxtx_vec_common.h b/drivers/net/intel/ice/ice_rxtx_vec_common.h index 4b73465af5..cdae6efea4 100644 --- a/drivers/net/intel/ice/ice_rxtx_vec_common.h +++ b/drivers/net/intel/ice/ice_rxtx_vec_common.h @@ -5,77 +5,13 @@ #ifndef _ICE_RXTX_VEC_COMMON_H_ #define _ICE_RXTX_VEC_COMMON_H_ +#include "../common/rx.h" #include "ice_rxtx.h" #ifndef __INTEL_COMPILER #pragma GCC diagnostic ignored "-Wcast-qual" #endif -static inline uint16_t -ice_rx_reassemble_packets(struct ice_rx_queue *rxq, struct rte_mbuf **rx_bufs, - uint16_t nb_bufs, uint8_t *split_flags) -{ - struct rte_mbuf *pkts[ICE_VPMD_RX_BURST] = {0}; /*finished pkts*/ - struct rte_mbuf *start = rxq->pkt_first_seg; - struct rte_mbuf *end = rxq->pkt_last_seg; - unsigned int pkt_idx, buf_idx; - - for (buf_idx = 0, pkt_idx = 0; buf_idx < nb_bufs; buf_idx++) { - if (end) { - /* processing a split packet */ - end->next = rx_bufs[buf_idx]; - rx_bufs[buf_idx]->data_len += rxq->crc_len; - - start->nb_segs++; - start->pkt_len += rx_bufs[buf_idx]->data_len; - end = end->next; - - if (!split_flags[buf_idx]) { - /* it's the last packet of the set */ - start->hash = end->hash; - start->vlan_tci = end->vlan_tci; - start->ol_flags = end->ol_flags; - /* we need to strip crc for the whole packet */ - start->pkt_len -= rxq->crc_len; - if (end->data_len > rxq->crc_len) { - end->data_len -= rxq->crc_len; - } else { - /* free up last mbuf */ - struct rte_mbuf *secondlast = start; - - start->nb_segs--; - while (secondlast->next != end) - secondlast = secondlast->next; - secondlast->data_len -= (rxq->crc_len - - end->data_len); - secondlast->next = NULL; - rte_pktmbuf_free_seg(end); - } - pkts[pkt_idx++] = start; - start = NULL; - end = NULL; - } - } else { - /* not processing a split packet */ - if (!split_flags[buf_idx]) { - /* not a split packet, save and skip */ - pkts[pkt_idx++] = rx_bufs[buf_idx]; - continue; - } - start = rx_bufs[buf_idx]; - end = start; - rx_bufs[buf_idx]->data_len += rxq->crc_len; - rx_bufs[buf_idx]->pkt_len += rxq->crc_len; - } - } - - /* save the partial packet for next time */ - rxq->pkt_first_seg = start; - rxq->pkt_last_seg = end; - memcpy(rx_bufs, pkts, pkt_idx * (sizeof(*pkts))); - return pkt_idx; -} - static __rte_always_inline int ice_tx_free_bufs_vec(struct ice_tx_queue *txq) { diff --git a/drivers/net/intel/ice/ice_rxtx_vec_sse.c b/drivers/net/intel/ice/ice_rxtx_vec_sse.c index c01d8ede29..01533454ba 100644 --- a/drivers/net/intel/ice/ice_rxtx_vec_sse.c +++ b/drivers/net/intel/ice/ice_rxtx_vec_sse.c @@ -640,8 +640,8 @@ ice_recv_scattered_burst_vec(void *rx_queue, struct rte_mbuf **rx_pkts, return nb_bufs; rxq->pkt_first_seg = rx_pkts[i]; } - return i + ice_rx_reassemble_packets(rxq, &rx_pkts[i], nb_bufs - i, - &split_flags[i]); + return i + ci_rx_reassemble_packets(&rx_pkts[i], nb_bufs - i, &split_flags[i], + &rxq->pkt_first_seg, &rxq->pkt_last_seg, rxq->crc_len); } /** diff --git a/drivers/net/intel/ixgbe/ixgbe_rxtx_vec_common.h b/drivers/net/intel/ixgbe/ixgbe_rxtx_vec_common.h index a4d9ec9b08..fbb80f7fb5 100644 --- a/drivers/net/intel/ixgbe/ixgbe_rxtx_vec_common.h +++ b/drivers/net/intel/ixgbe/ixgbe_rxtx_vec_common.h @@ -7,71 +7,10 @@ #include #include +#include "../common/rx.h" #include "ixgbe_ethdev.h" #include "ixgbe_rxtx.h" -static inline uint16_t -reassemble_packets(struct ixgbe_rx_queue *rxq, struct rte_mbuf **rx_bufs, - uint16_t nb_bufs, uint8_t *split_flags) -{ - struct rte_mbuf *pkts[nb_bufs]; /*finished pkts*/ - struct rte_mbuf *start = rxq->pkt_first_seg; - struct rte_mbuf *end = rxq->pkt_last_seg; - unsigned int pkt_idx, buf_idx; - - for (buf_idx = 0, pkt_idx = 0; buf_idx < nb_bufs; buf_idx++) { - if (end != NULL) { - /* processing a split packet */ - end->next = rx_bufs[buf_idx]; - rx_bufs[buf_idx]->data_len += rxq->crc_len; - - start->nb_segs++; - start->pkt_len += rx_bufs[buf_idx]->data_len; - end = end->next; - - if (!split_flags[buf_idx]) { - /* it's the last packet of the set */ - start->hash = end->hash; - start->ol_flags = end->ol_flags; - /* we need to strip crc for the whole packet */ - start->pkt_len -= rxq->crc_len; - if (end->data_len > rxq->crc_len) - end->data_len -= rxq->crc_len; - else { - /* free up last mbuf */ - struct rte_mbuf *secondlast = start; - - start->nb_segs--; - while (secondlast->next != end) - secondlast = secondlast->next; - secondlast->data_len -= (rxq->crc_len - - end->data_len); - secondlast->next = NULL; - rte_pktmbuf_free_seg(end); - } - pkts[pkt_idx++] = start; - start = end = NULL; - } - } else { - /* not processing a split packet */ - if (!split_flags[buf_idx]) { - /* not a split packet, save and skip */ - pkts[pkt_idx++] = rx_bufs[buf_idx]; - continue; - } - end = start = rx_bufs[buf_idx]; - rx_bufs[buf_idx]->data_len += rxq->crc_len; - rx_bufs[buf_idx]->pkt_len += rxq->crc_len; - } - } - - /* save the partial packet for next time */ - rxq->pkt_first_seg = start; - rxq->pkt_last_seg = end; - memcpy(rx_bufs, pkts, pkt_idx * (sizeof(*pkts))); - return pkt_idx; -} - static __rte_always_inline int ixgbe_tx_free_bufs(struct ixgbe_tx_queue *txq) { diff --git a/drivers/net/intel/ixgbe/ixgbe_rxtx_vec_neon.c b/drivers/net/intel/ixgbe/ixgbe_rxtx_vec_neon.c index 952b032eb6..7b35093075 100644 --- a/drivers/net/intel/ixgbe/ixgbe_rxtx_vec_neon.c +++ b/drivers/net/intel/ixgbe/ixgbe_rxtx_vec_neon.c @@ -516,8 +516,8 @@ ixgbe_recv_scattered_burst_vec(void *rx_queue, struct rte_mbuf **rx_pkts, return nb_bufs; rxq->pkt_first_seg = rx_pkts[i]; } - return i + reassemble_packets(rxq, &rx_pkts[i], nb_bufs - i, - &split_flags[i]); + return i + ci_rx_reassemble_packets(&rx_pkts[i], nb_bufs - i, &split_flags[i], + &rxq->pkt_first_seg, &rxq->pkt_last_seg, rxq->crc_len); } /** diff --git a/drivers/net/intel/ixgbe/ixgbe_rxtx_vec_sse.c b/drivers/net/intel/ixgbe/ixgbe_rxtx_vec_sse.c index a77370cdb7..a709bf8c7f 100644 --- a/drivers/net/intel/ixgbe/ixgbe_rxtx_vec_sse.c +++ b/drivers/net/intel/ixgbe/ixgbe_rxtx_vec_sse.c @@ -639,8 +639,8 @@ ixgbe_recv_scattered_burst_vec(void *rx_queue, struct rte_mbuf **rx_pkts, return nb_bufs; rxq->pkt_first_seg = rx_pkts[i]; } - return i + reassemble_packets(rxq, &rx_pkts[i], nb_bufs - i, - &split_flags[i]); + return i + ci_rx_reassemble_packets(&rx_pkts[i], nb_bufs - i, &split_flags[i], + &rxq->pkt_first_seg, &rxq->pkt_last_seg, rxq->crc_len); } /** -- 2.43.0