From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id D0F3245EFA; Fri, 20 Dec 2024 15:39:45 +0100 (CET) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id E9116402B5; Fri, 20 Dec 2024 15:39:42 +0100 (CET) Received: from mgamail.intel.com (mgamail.intel.com [198.175.65.14]) by mails.dpdk.org (Postfix) with ESMTP id 3D021402D1 for ; Fri, 20 Dec 2024 15:39:40 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1734705581; x=1766241581; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=oS4EPbtHaXIkOj73yqGTTHn65RNakdPRiSss+wq/9a0=; b=EFImtg7DESMIQSkgrZZ68VnccH//jtpzE2IfWzb8wCH4a8fVNyp1E7tr Kqj6Jx8nv/SQrqtOMVXAbGuBBrO6ZqS69jwwjWLOrhtJ+sCuy8Yibu1BI GqGUpBmTsi3mUy4Jo9Ruf2JBl6aA9vmWZDhA1x5/JuKQSc9ZFQLYg5dec Hf00ZH19d9K9xuwSi0hz6lzt4fFH5hmbKQcGv+s6YmFSMCTbbPo3xJ1B3 lyvsOCRA/L+fNaAFAxkvvOqmjtAfCHKYr0ts1BpgilhEX/Zk0KTvfSTkJ AZL49llJyTa/FQbCwP5b7G8olIylApKS2o67abPEuPVdNynfOFAGLFkDo Q==; X-CSE-ConnectionGUID: 2IEK7j89QpWwXN5swf81Eg== X-CSE-MsgGUID: oPT9LTGoQnGqPoLzcbCqMQ== X-IronPort-AV: E=McAfee;i="6700,10204,11292"; a="39040098" X-IronPort-AV: E=Sophos;i="6.12,251,1728975600"; d="scan'208";a="39040098" Received: from fmviesa006.fm.intel.com ([10.60.135.146]) by orvoesa106.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 20 Dec 2024 06:39:40 -0800 X-CSE-ConnectionGUID: 0+TZL03tRUusNOmeySYudg== X-CSE-MsgGUID: 5yG9OJexR1yIplj13euOaQ== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.12,251,1728975600"; d="scan'208";a="98310524" Received: from silpixa00401197coob.ir.intel.com (HELO silpixa00401385.ir.intel.com) ([10.237.214.45]) by fmviesa006.fm.intel.com with ESMTP; 20 Dec 2024 06:39:36 -0800 From: Bruce Richardson To: dev@dpdk.org Cc: Bruce Richardson , David Christensen , Ian Stokes , Konstantin Ananyev , Wathsala Vithanage , Vladimir Medvedkin , Anatoly Burakov Subject: [PATCH v4 01/24] net/_common_intel: add pkt reassembly fn for intel drivers Date: Fri, 20 Dec 2024 14:38:58 +0000 Message-ID: <20241220143925.609044-2-bruce.richardson@intel.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20241220143925.609044-1-bruce.richardson@intel.com> References: <20241122125418.2857301-1-bruce.richardson@intel.com> <20241220143925.609044-1-bruce.richardson@intel.com> MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org The code for reassembling a single, multi-mbuf packet from multiple buffers received from the NIC is duplicated across many drivers. Rather than having multiple copies of this function, we can create an "_common_intel" directory to hold such functions and consolidate multiple functions down to a single one for easier maintenance. Signed-off-by: Bruce Richardson --- drivers/net/_common_intel/rx.h | 79 +++++++++++++++++++++++ drivers/net/i40e/i40e_rxtx_vec_altivec.c | 4 +- drivers/net/i40e/i40e_rxtx_vec_avx2.c | 4 +- drivers/net/i40e/i40e_rxtx_vec_avx512.c | 4 +- drivers/net/i40e/i40e_rxtx_vec_common.h | 64 +----------------- drivers/net/i40e/i40e_rxtx_vec_neon.c | 4 +- drivers/net/i40e/i40e_rxtx_vec_sse.c | 4 +- drivers/net/i40e/meson.build | 2 +- drivers/net/iavf/iavf_rxtx_vec_avx2.c | 8 +-- drivers/net/iavf/iavf_rxtx_vec_avx512.c | 8 +-- drivers/net/iavf/iavf_rxtx_vec_common.h | 65 +------------------ drivers/net/iavf/iavf_rxtx_vec_sse.c | 8 +-- drivers/net/iavf/meson.build | 2 +- drivers/net/ice/ice_rxtx_vec_avx2.c | 4 +- drivers/net/ice/ice_rxtx_vec_avx512.c | 8 +-- drivers/net/ice/ice_rxtx_vec_common.h | 66 +------------------ drivers/net/ice/ice_rxtx_vec_sse.c | 4 +- drivers/net/ice/meson.build | 2 +- drivers/net/ixgbe/ixgbe_rxtx_vec_common.h | 63 +----------------- drivers/net/ixgbe/ixgbe_rxtx_vec_neon.c | 4 +- drivers/net/ixgbe/ixgbe_rxtx_vec_sse.c | 4 +- drivers/net/ixgbe/meson.build | 2 +- 22 files changed, 121 insertions(+), 292 deletions(-) create mode 100644 drivers/net/_common_intel/rx.h diff --git a/drivers/net/_common_intel/rx.h b/drivers/net/_common_intel/rx.h new file mode 100644 index 0000000000..5bd2fea7e3 --- /dev/null +++ b/drivers/net/_common_intel/rx.h @@ -0,0 +1,79 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(c) 2024 Intel Corporation + */ + +#ifndef _COMMON_INTEL_RX_H_ +#define _COMMON_INTEL_RX_H_ + +#include +#include +#include + +#define CI_RX_BURST 32 + +static inline uint16_t +ci_rx_reassemble_packets(struct rte_mbuf **rx_bufs, uint16_t nb_bufs, uint8_t *split_flags, + struct rte_mbuf **pkt_first_seg, struct rte_mbuf **pkt_last_seg, + const uint8_t crc_len) +{ + struct rte_mbuf *pkts[CI_RX_BURST] = {0}; /*finished pkts*/ + struct rte_mbuf *start = *pkt_first_seg; + struct rte_mbuf *end = *pkt_last_seg; + unsigned int pkt_idx, buf_idx; + + for (buf_idx = 0, pkt_idx = 0; buf_idx < nb_bufs; buf_idx++) { + if (end) { + /* processing a split packet */ + end->next = rx_bufs[buf_idx]; + rx_bufs[buf_idx]->data_len += crc_len; + + start->nb_segs++; + start->pkt_len += rx_bufs[buf_idx]->data_len; + end = end->next; + + if (!split_flags[buf_idx]) { + /* it's the last packet of the set */ + start->hash = end->hash; + start->vlan_tci = end->vlan_tci; + start->ol_flags = end->ol_flags; + /* we need to strip crc for the whole packet */ + start->pkt_len -= crc_len; + if (end->data_len > crc_len) { + end->data_len -= crc_len; + } else { + /* free up last mbuf */ + struct rte_mbuf *secondlast = start; + + start->nb_segs--; + while (secondlast->next != end) + secondlast = secondlast->next; + secondlast->data_len -= (crc_len - end->data_len); + secondlast->next = NULL; + rte_pktmbuf_free_seg(end); + } + pkts[pkt_idx++] = start; + start = NULL; + end = NULL; + } + } else { + /* not processing a split packet */ + if (!split_flags[buf_idx]) { + /* not a split packet, save and skip */ + pkts[pkt_idx++] = rx_bufs[buf_idx]; + continue; + } + start = rx_bufs[buf_idx]; + end = start; + rx_bufs[buf_idx]->data_len += crc_len; + rx_bufs[buf_idx]->pkt_len += crc_len; + } + } + + /* save the partial packet for next time */ + *pkt_first_seg = start; + *pkt_last_seg = end; + memcpy(rx_bufs, pkts, pkt_idx * (sizeof(*pkts))); + return pkt_idx; +} + +#endif /* _COMMON_INTEL_RX_H_ */ diff --git a/drivers/net/i40e/i40e_rxtx_vec_altivec.c b/drivers/net/i40e/i40e_rxtx_vec_altivec.c index b6b0d38ec1..95829f65d5 100644 --- a/drivers/net/i40e/i40e_rxtx_vec_altivec.c +++ b/drivers/net/i40e/i40e_rxtx_vec_altivec.c @@ -494,8 +494,8 @@ i40e_recv_scattered_burst_vec(void *rx_queue, struct rte_mbuf **rx_pkts, if (i == nb_bufs) return nb_bufs; } - return i + reassemble_packets(rxq, &rx_pkts[i], nb_bufs - i, - &split_flags[i]); + return i + ci_rx_reassemble_packets(&rx_pkts[i], nb_bufs - i, &split_flags[i], + &rxq->pkt_first_seg, &rxq->pkt_last_seg, rxq->crc_len); } /** diff --git a/drivers/net/i40e/i40e_rxtx_vec_avx2.c b/drivers/net/i40e/i40e_rxtx_vec_avx2.c index 19cf0ac718..6dd6e55d9c 100644 --- a/drivers/net/i40e/i40e_rxtx_vec_avx2.c +++ b/drivers/net/i40e/i40e_rxtx_vec_avx2.c @@ -657,8 +657,8 @@ i40e_recv_scattered_burst_vec_avx2(void *rx_queue, struct rte_mbuf **rx_pkts, return nb_bufs; rxq->pkt_first_seg = rx_pkts[i]; } - return i + reassemble_packets(rxq, &rx_pkts[i], nb_bufs - i, - &split_flags[i]); + return i + ci_rx_reassemble_packets(&rx_pkts[i], nb_bufs - i, &split_flags[i], + &rxq->pkt_first_seg, &rxq->pkt_last_seg, rxq->crc_len); } /* diff --git a/drivers/net/i40e/i40e_rxtx_vec_avx512.c b/drivers/net/i40e/i40e_rxtx_vec_avx512.c index 3b2750221b..506f1b5878 100644 --- a/drivers/net/i40e/i40e_rxtx_vec_avx512.c +++ b/drivers/net/i40e/i40e_rxtx_vec_avx512.c @@ -725,8 +725,8 @@ i40e_recv_scattered_burst_vec_avx512(void *rx_queue, return nb_bufs; rxq->pkt_first_seg = rx_pkts[i]; } - return i + reassemble_packets(rxq, &rx_pkts[i], nb_bufs - i, - &split_flags[i]); + return i + ci_rx_reassemble_packets(&rx_pkts[i], nb_bufs - i, &split_flags[i], + &rxq->pkt_first_seg, &rxq->pkt_last_seg, rxq->crc_len); } /** diff --git a/drivers/net/i40e/i40e_rxtx_vec_common.h b/drivers/net/i40e/i40e_rxtx_vec_common.h index 8b745630e4..1248cecacd 100644 --- a/drivers/net/i40e/i40e_rxtx_vec_common.h +++ b/drivers/net/i40e/i40e_rxtx_vec_common.h @@ -8,6 +8,7 @@ #include #include +#include <_common_intel/rx.h> #include "i40e_ethdev.h" #include "i40e_rxtx.h" @@ -15,69 +16,6 @@ #pragma GCC diagnostic ignored "-Wcast-qual" #endif -static inline uint16_t -reassemble_packets(struct i40e_rx_queue *rxq, struct rte_mbuf **rx_bufs, - uint16_t nb_bufs, uint8_t *split_flags) -{ - struct rte_mbuf *pkts[RTE_I40E_VPMD_RX_BURST]; /*finished pkts*/ - struct rte_mbuf *start = rxq->pkt_first_seg; - struct rte_mbuf *end = rxq->pkt_last_seg; - unsigned pkt_idx, buf_idx; - - for (buf_idx = 0, pkt_idx = 0; buf_idx < nb_bufs; buf_idx++) { - if (end != NULL) { - /* processing a split packet */ - end->next = rx_bufs[buf_idx]; - rx_bufs[buf_idx]->data_len += rxq->crc_len; - - start->nb_segs++; - start->pkt_len += rx_bufs[buf_idx]->data_len; - end = end->next; - - if (!split_flags[buf_idx]) { - /* it's the last packet of the set */ - start->hash = end->hash; - start->vlan_tci = end->vlan_tci; - start->ol_flags = end->ol_flags; - /* we need to strip crc for the whole packet */ - start->pkt_len -= rxq->crc_len; - if (end->data_len > rxq->crc_len) - end->data_len -= rxq->crc_len; - else { - /* free up last mbuf */ - struct rte_mbuf *secondlast = start; - - start->nb_segs--; - while (secondlast->next != end) - secondlast = secondlast->next; - secondlast->data_len -= (rxq->crc_len - - end->data_len); - secondlast->next = NULL; - rte_pktmbuf_free_seg(end); - } - pkts[pkt_idx++] = start; - start = end = NULL; - } - } else { - /* not processing a split packet */ - if (!split_flags[buf_idx]) { - /* not a split packet, save and skip */ - pkts[pkt_idx++] = rx_bufs[buf_idx]; - continue; - } - end = start = rx_bufs[buf_idx]; - rx_bufs[buf_idx]->data_len += rxq->crc_len; - rx_bufs[buf_idx]->pkt_len += rxq->crc_len; - } - } - - /* save the partial packet for next time */ - rxq->pkt_first_seg = start; - rxq->pkt_last_seg = end; - memcpy(rx_bufs, pkts, pkt_idx * (sizeof(*pkts))); - return pkt_idx; -} - static __rte_always_inline int i40e_tx_free_bufs(struct i40e_tx_queue *txq) { diff --git a/drivers/net/i40e/i40e_rxtx_vec_neon.c b/drivers/net/i40e/i40e_rxtx_vec_neon.c index e1c5c7041b..159d971796 100644 --- a/drivers/net/i40e/i40e_rxtx_vec_neon.c +++ b/drivers/net/i40e/i40e_rxtx_vec_neon.c @@ -623,8 +623,8 @@ i40e_recv_scattered_burst_vec(void *rx_queue, struct rte_mbuf **rx_pkts, return nb_bufs; rxq->pkt_first_seg = rx_pkts[i]; } - return i + reassemble_packets(rxq, &rx_pkts[i], nb_bufs - i, - &split_flags[i]); + return i + ci_rx_reassemble_packets(&rx_pkts[i], nb_bufs - i, &split_flags[i], + &rxq->pkt_first_seg, &rxq->pkt_last_seg, rxq->crc_len); } /** diff --git a/drivers/net/i40e/i40e_rxtx_vec_sse.c b/drivers/net/i40e/i40e_rxtx_vec_sse.c index ad560d2b6b..3a8128e014 100644 --- a/drivers/net/i40e/i40e_rxtx_vec_sse.c +++ b/drivers/net/i40e/i40e_rxtx_vec_sse.c @@ -641,8 +641,8 @@ i40e_recv_scattered_burst_vec(void *rx_queue, struct rte_mbuf **rx_pkts, return nb_bufs; rxq->pkt_first_seg = rx_pkts[i]; } - return i + reassemble_packets(rxq, &rx_pkts[i], nb_bufs - i, - &split_flags[i]); + return i + ci_rx_reassemble_packets(&rx_pkts[i], nb_bufs - i, &split_flags[i], + &rxq->pkt_first_seg, &rxq->pkt_last_seg, rxq->crc_len); } /** diff --git a/drivers/net/i40e/meson.build b/drivers/net/i40e/meson.build index 5c93493124..0e0b416b8f 100644 --- a/drivers/net/i40e/meson.build +++ b/drivers/net/i40e/meson.build @@ -36,7 +36,7 @@ sources = files( testpmd_sources = files('i40e_testpmd.c') deps += ['hash'] -includes += include_directories('base') +includes += include_directories('base', '..') if arch_subdir == 'x86' sources += files('i40e_rxtx_vec_sse.c') diff --git a/drivers/net/iavf/iavf_rxtx_vec_avx2.c b/drivers/net/iavf/iavf_rxtx_vec_avx2.c index 49d41af953..0baf5045c8 100644 --- a/drivers/net/iavf/iavf_rxtx_vec_avx2.c +++ b/drivers/net/iavf/iavf_rxtx_vec_avx2.c @@ -1508,8 +1508,8 @@ iavf_recv_scattered_burst_vec_avx2(void *rx_queue, struct rte_mbuf **rx_pkts, return nb_bufs; rxq->pkt_first_seg = rx_pkts[i]; } - return i + reassemble_packets(rxq, &rx_pkts[i], nb_bufs - i, - &split_flags[i]); + return i + ci_rx_reassemble_packets(&rx_pkts[i], nb_bufs - i, &split_flags[i], + &rxq->pkt_first_seg, &rxq->pkt_last_seg, rxq->crc_len); } /** @@ -1597,8 +1597,8 @@ iavf_recv_scattered_burst_vec_avx2_flex_rxd(void *rx_queue, return nb_bufs; rxq->pkt_first_seg = rx_pkts[i]; } - return i + reassemble_packets(rxq, &rx_pkts[i], nb_bufs - i, - &split_flags[i]); + return i + ci_rx_reassemble_packets(&rx_pkts[i], nb_bufs - i, &split_flags[i], + &rxq->pkt_first_seg, &rxq->pkt_last_seg, rxq->crc_len); } /** diff --git a/drivers/net/iavf/iavf_rxtx_vec_avx512.c b/drivers/net/iavf/iavf_rxtx_vec_avx512.c index d6a861bf80..5a88007096 100644 --- a/drivers/net/iavf/iavf_rxtx_vec_avx512.c +++ b/drivers/net/iavf/iavf_rxtx_vec_avx512.c @@ -1685,8 +1685,8 @@ iavf_recv_scattered_burst_vec_avx512(void *rx_queue, struct rte_mbuf **rx_pkts, return nb_bufs; rxq->pkt_first_seg = rx_pkts[i]; } - return i + reassemble_packets(rxq, &rx_pkts[i], nb_bufs - i, - &split_flags[i]); + return i + ci_rx_reassemble_packets(&rx_pkts[i], nb_bufs - i, &split_flags[i], + &rxq->pkt_first_seg, &rxq->pkt_last_seg, rxq->crc_len); } /** @@ -1761,8 +1761,8 @@ iavf_recv_scattered_burst_vec_avx512_flex_rxd(void *rx_queue, return nb_bufs; rxq->pkt_first_seg = rx_pkts[i]; } - return i + reassemble_packets(rxq, &rx_pkts[i], nb_bufs - i, - &split_flags[i]); + return i + ci_rx_reassemble_packets(&rx_pkts[i], nb_bufs - i, &split_flags[i], + &rxq->pkt_first_seg, &rxq->pkt_last_seg, rxq->crc_len); } /** diff --git a/drivers/net/iavf/iavf_rxtx_vec_common.h b/drivers/net/iavf/iavf_rxtx_vec_common.h index 5c5220048d..26b6f07614 100644 --- a/drivers/net/iavf/iavf_rxtx_vec_common.h +++ b/drivers/net/iavf/iavf_rxtx_vec_common.h @@ -8,6 +8,7 @@ #include #include +#include <_common_intel/rx.h> #include "iavf.h" #include "iavf_rxtx.h" @@ -15,70 +16,6 @@ #pragma GCC diagnostic ignored "-Wcast-qual" #endif -static __rte_always_inline uint16_t -reassemble_packets(struct iavf_rx_queue *rxq, struct rte_mbuf **rx_bufs, - uint16_t nb_bufs, uint8_t *split_flags) -{ - struct rte_mbuf *pkts[IAVF_VPMD_RX_MAX_BURST]; - struct rte_mbuf *start = rxq->pkt_first_seg; - struct rte_mbuf *end = rxq->pkt_last_seg; - unsigned int pkt_idx, buf_idx; - - for (buf_idx = 0, pkt_idx = 0; buf_idx < nb_bufs; buf_idx++) { - if (end) { - /* processing a split packet */ - end->next = rx_bufs[buf_idx]; - rx_bufs[buf_idx]->data_len += rxq->crc_len; - - start->nb_segs++; - start->pkt_len += rx_bufs[buf_idx]->data_len; - end = end->next; - - if (!split_flags[buf_idx]) { - /* it's the last packet of the set */ - start->hash = end->hash; - start->vlan_tci = end->vlan_tci; - start->ol_flags = end->ol_flags; - /* we need to strip crc for the whole packet */ - start->pkt_len -= rxq->crc_len; - if (end->data_len > rxq->crc_len) { - end->data_len -= rxq->crc_len; - } else { - /* free up last mbuf */ - struct rte_mbuf *secondlast = start; - - start->nb_segs--; - while (secondlast->next != end) - secondlast = secondlast->next; - secondlast->data_len -= (rxq->crc_len - - end->data_len); - secondlast->next = NULL; - rte_pktmbuf_free_seg(end); - } - pkts[pkt_idx++] = start; - start = NULL; - end = NULL; - } - } else { - /* not processing a split packet */ - if (!split_flags[buf_idx]) { - /* not a split packet, save and skip */ - pkts[pkt_idx++] = rx_bufs[buf_idx]; - continue; - } - end = start = rx_bufs[buf_idx]; - rx_bufs[buf_idx]->data_len += rxq->crc_len; - rx_bufs[buf_idx]->pkt_len += rxq->crc_len; - } - } - - /* save the partial packet for next time */ - rxq->pkt_first_seg = start; - rxq->pkt_last_seg = end; - memcpy(rx_bufs, pkts, pkt_idx * (sizeof(*pkts))); - return pkt_idx; -} - static __rte_always_inline int iavf_tx_free_bufs(struct iavf_tx_queue *txq) { diff --git a/drivers/net/iavf/iavf_rxtx_vec_sse.c b/drivers/net/iavf/iavf_rxtx_vec_sse.c index 0db6fa8bd4..48b01462ea 100644 --- a/drivers/net/iavf/iavf_rxtx_vec_sse.c +++ b/drivers/net/iavf/iavf_rxtx_vec_sse.c @@ -1238,8 +1238,8 @@ iavf_recv_scattered_burst_vec(void *rx_queue, struct rte_mbuf **rx_pkts, return nb_bufs; rxq->pkt_first_seg = rx_pkts[i]; } - return i + reassemble_packets(rxq, &rx_pkts[i], nb_bufs - i, - &split_flags[i]); + return i + ci_rx_reassemble_packets(&rx_pkts[i], nb_bufs - i, &split_flags[i], + &rxq->pkt_first_seg, &rxq->pkt_last_seg, rxq->crc_len); } /** @@ -1307,8 +1307,8 @@ iavf_recv_scattered_burst_vec_flex_rxd(void *rx_queue, return nb_bufs; rxq->pkt_first_seg = rx_pkts[i]; } - return i + reassemble_packets(rxq, &rx_pkts[i], nb_bufs - i, - &split_flags[i]); + return i + ci_rx_reassemble_packets(&rx_pkts[i], nb_bufs - i, &split_flags[i], + &rxq->pkt_first_seg, &rxq->pkt_last_seg, rxq->crc_len); } /** diff --git a/drivers/net/iavf/meson.build b/drivers/net/iavf/meson.build index b48bb83438..9106e016ef 100644 --- a/drivers/net/iavf/meson.build +++ b/drivers/net/iavf/meson.build @@ -5,7 +5,7 @@ if dpdk_conf.get('RTE_IOVA_IN_MBUF') == 0 subdir_done() endif -includes += include_directories('../../common/iavf') +includes += include_directories('../../common/iavf', '..') testpmd_sources = files('iavf_testpmd.c') diff --git a/drivers/net/ice/ice_rxtx_vec_avx2.c b/drivers/net/ice/ice_rxtx_vec_avx2.c index d6e88dbb29..ca247b155c 100644 --- a/drivers/net/ice/ice_rxtx_vec_avx2.c +++ b/drivers/net/ice/ice_rxtx_vec_avx2.c @@ -726,8 +726,8 @@ ice_recv_scattered_burst_vec_avx2(void *rx_queue, struct rte_mbuf **rx_pkts, return nb_bufs; rxq->pkt_first_seg = rx_pkts[i]; } - return i + ice_rx_reassemble_packets(rxq, &rx_pkts[i], nb_bufs - i, - &split_flags[i]); + return i + ci_rx_reassemble_packets(&rx_pkts[i], nb_bufs - i, &split_flags[i], + &rxq->pkt_first_seg, &rxq->pkt_last_seg, rxq->crc_len); } /** diff --git a/drivers/net/ice/ice_rxtx_vec_avx512.c b/drivers/net/ice/ice_rxtx_vec_avx512.c index add095ef06..1e603d5d8f 100644 --- a/drivers/net/ice/ice_rxtx_vec_avx512.c +++ b/drivers/net/ice/ice_rxtx_vec_avx512.c @@ -763,8 +763,8 @@ ice_recv_scattered_burst_vec_avx512(void *rx_queue, struct rte_mbuf **rx_pkts, return nb_bufs; rxq->pkt_first_seg = rx_pkts[i]; } - return i + ice_rx_reassemble_packets(rxq, &rx_pkts[i], nb_bufs - i, - &split_flags[i]); + return i + ci_rx_reassemble_packets(&rx_pkts[i], nb_bufs - i, &split_flags[i], + &rxq->pkt_first_seg, &rxq->pkt_last_seg, rxq->crc_len); } /** @@ -805,8 +805,8 @@ ice_recv_scattered_burst_vec_avx512_offload(void *rx_queue, return nb_bufs; rxq->pkt_first_seg = rx_pkts[i]; } - return i + ice_rx_reassemble_packets(rxq, &rx_pkts[i], nb_bufs - i, - &split_flags[i]); + return i + ci_rx_reassemble_packets(&rx_pkts[i], nb_bufs - i, &split_flags[i], + &rxq->pkt_first_seg, &rxq->pkt_last_seg, rxq->crc_len); } /** diff --git a/drivers/net/ice/ice_rxtx_vec_common.h b/drivers/net/ice/ice_rxtx_vec_common.h index 4b73465af5..dd7da4761f 100644 --- a/drivers/net/ice/ice_rxtx_vec_common.h +++ b/drivers/net/ice/ice_rxtx_vec_common.h @@ -5,77 +5,13 @@ #ifndef _ICE_RXTX_VEC_COMMON_H_ #define _ICE_RXTX_VEC_COMMON_H_ +#include <_common_intel/rx.h> #include "ice_rxtx.h" #ifndef __INTEL_COMPILER #pragma GCC diagnostic ignored "-Wcast-qual" #endif -static inline uint16_t -ice_rx_reassemble_packets(struct ice_rx_queue *rxq, struct rte_mbuf **rx_bufs, - uint16_t nb_bufs, uint8_t *split_flags) -{ - struct rte_mbuf *pkts[ICE_VPMD_RX_BURST] = {0}; /*finished pkts*/ - struct rte_mbuf *start = rxq->pkt_first_seg; - struct rte_mbuf *end = rxq->pkt_last_seg; - unsigned int pkt_idx, buf_idx; - - for (buf_idx = 0, pkt_idx = 0; buf_idx < nb_bufs; buf_idx++) { - if (end) { - /* processing a split packet */ - end->next = rx_bufs[buf_idx]; - rx_bufs[buf_idx]->data_len += rxq->crc_len; - - start->nb_segs++; - start->pkt_len += rx_bufs[buf_idx]->data_len; - end = end->next; - - if (!split_flags[buf_idx]) { - /* it's the last packet of the set */ - start->hash = end->hash; - start->vlan_tci = end->vlan_tci; - start->ol_flags = end->ol_flags; - /* we need to strip crc for the whole packet */ - start->pkt_len -= rxq->crc_len; - if (end->data_len > rxq->crc_len) { - end->data_len -= rxq->crc_len; - } else { - /* free up last mbuf */ - struct rte_mbuf *secondlast = start; - - start->nb_segs--; - while (secondlast->next != end) - secondlast = secondlast->next; - secondlast->data_len -= (rxq->crc_len - - end->data_len); - secondlast->next = NULL; - rte_pktmbuf_free_seg(end); - } - pkts[pkt_idx++] = start; - start = NULL; - end = NULL; - } - } else { - /* not processing a split packet */ - if (!split_flags[buf_idx]) { - /* not a split packet, save and skip */ - pkts[pkt_idx++] = rx_bufs[buf_idx]; - continue; - } - start = rx_bufs[buf_idx]; - end = start; - rx_bufs[buf_idx]->data_len += rxq->crc_len; - rx_bufs[buf_idx]->pkt_len += rxq->crc_len; - } - } - - /* save the partial packet for next time */ - rxq->pkt_first_seg = start; - rxq->pkt_last_seg = end; - memcpy(rx_bufs, pkts, pkt_idx * (sizeof(*pkts))); - return pkt_idx; -} - static __rte_always_inline int ice_tx_free_bufs_vec(struct ice_tx_queue *txq) { diff --git a/drivers/net/ice/ice_rxtx_vec_sse.c b/drivers/net/ice/ice_rxtx_vec_sse.c index c01d8ede29..01533454ba 100644 --- a/drivers/net/ice/ice_rxtx_vec_sse.c +++ b/drivers/net/ice/ice_rxtx_vec_sse.c @@ -640,8 +640,8 @@ ice_recv_scattered_burst_vec(void *rx_queue, struct rte_mbuf **rx_pkts, return nb_bufs; rxq->pkt_first_seg = rx_pkts[i]; } - return i + ice_rx_reassemble_packets(rxq, &rx_pkts[i], nb_bufs - i, - &split_flags[i]); + return i + ci_rx_reassemble_packets(&rx_pkts[i], nb_bufs - i, &split_flags[i], + &rxq->pkt_first_seg, &rxq->pkt_last_seg, rxq->crc_len); } /** diff --git a/drivers/net/ice/meson.build b/drivers/net/ice/meson.build index 1c9dc0cc6d..02c028db73 100644 --- a/drivers/net/ice/meson.build +++ b/drivers/net/ice/meson.build @@ -19,7 +19,7 @@ sources = files( testpmd_sources = files('ice_testpmd.c') deps += ['hash', 'net', 'common_iavf'] -includes += include_directories('base', '../../common/iavf') +includes += include_directories('base', '..') if arch_subdir == 'x86' sources += files('ice_rxtx_vec_sse.c') diff --git a/drivers/net/ixgbe/ixgbe_rxtx_vec_common.h b/drivers/net/ixgbe/ixgbe_rxtx_vec_common.h index a4d9ec9b08..2bab17c934 100644 --- a/drivers/net/ixgbe/ixgbe_rxtx_vec_common.h +++ b/drivers/net/ixgbe/ixgbe_rxtx_vec_common.h @@ -7,71 +7,10 @@ #include #include +#include <_common_intel/rx.h> #include "ixgbe_ethdev.h" #include "ixgbe_rxtx.h" -static inline uint16_t -reassemble_packets(struct ixgbe_rx_queue *rxq, struct rte_mbuf **rx_bufs, - uint16_t nb_bufs, uint8_t *split_flags) -{ - struct rte_mbuf *pkts[nb_bufs]; /*finished pkts*/ - struct rte_mbuf *start = rxq->pkt_first_seg; - struct rte_mbuf *end = rxq->pkt_last_seg; - unsigned int pkt_idx, buf_idx; - - for (buf_idx = 0, pkt_idx = 0; buf_idx < nb_bufs; buf_idx++) { - if (end != NULL) { - /* processing a split packet */ - end->next = rx_bufs[buf_idx]; - rx_bufs[buf_idx]->data_len += rxq->crc_len; - - start->nb_segs++; - start->pkt_len += rx_bufs[buf_idx]->data_len; - end = end->next; - - if (!split_flags[buf_idx]) { - /* it's the last packet of the set */ - start->hash = end->hash; - start->ol_flags = end->ol_flags; - /* we need to strip crc for the whole packet */ - start->pkt_len -= rxq->crc_len; - if (end->data_len > rxq->crc_len) - end->data_len -= rxq->crc_len; - else { - /* free up last mbuf */ - struct rte_mbuf *secondlast = start; - - start->nb_segs--; - while (secondlast->next != end) - secondlast = secondlast->next; - secondlast->data_len -= (rxq->crc_len - - end->data_len); - secondlast->next = NULL; - rte_pktmbuf_free_seg(end); - } - pkts[pkt_idx++] = start; - start = end = NULL; - } - } else { - /* not processing a split packet */ - if (!split_flags[buf_idx]) { - /* not a split packet, save and skip */ - pkts[pkt_idx++] = rx_bufs[buf_idx]; - continue; - } - end = start = rx_bufs[buf_idx]; - rx_bufs[buf_idx]->data_len += rxq->crc_len; - rx_bufs[buf_idx]->pkt_len += rxq->crc_len; - } - } - - /* save the partial packet for next time */ - rxq->pkt_first_seg = start; - rxq->pkt_last_seg = end; - memcpy(rx_bufs, pkts, pkt_idx * (sizeof(*pkts))); - return pkt_idx; -} - static __rte_always_inline int ixgbe_tx_free_bufs(struct ixgbe_tx_queue *txq) { diff --git a/drivers/net/ixgbe/ixgbe_rxtx_vec_neon.c b/drivers/net/ixgbe/ixgbe_rxtx_vec_neon.c index 952b032eb6..7b35093075 100644 --- a/drivers/net/ixgbe/ixgbe_rxtx_vec_neon.c +++ b/drivers/net/ixgbe/ixgbe_rxtx_vec_neon.c @@ -516,8 +516,8 @@ ixgbe_recv_scattered_burst_vec(void *rx_queue, struct rte_mbuf **rx_pkts, return nb_bufs; rxq->pkt_first_seg = rx_pkts[i]; } - return i + reassemble_packets(rxq, &rx_pkts[i], nb_bufs - i, - &split_flags[i]); + return i + ci_rx_reassemble_packets(&rx_pkts[i], nb_bufs - i, &split_flags[i], + &rxq->pkt_first_seg, &rxq->pkt_last_seg, rxq->crc_len); } /** diff --git a/drivers/net/ixgbe/ixgbe_rxtx_vec_sse.c b/drivers/net/ixgbe/ixgbe_rxtx_vec_sse.c index a77370cdb7..a709bf8c7f 100644 --- a/drivers/net/ixgbe/ixgbe_rxtx_vec_sse.c +++ b/drivers/net/ixgbe/ixgbe_rxtx_vec_sse.c @@ -639,8 +639,8 @@ ixgbe_recv_scattered_burst_vec(void *rx_queue, struct rte_mbuf **rx_pkts, return nb_bufs; rxq->pkt_first_seg = rx_pkts[i]; } - return i + reassemble_packets(rxq, &rx_pkts[i], nb_bufs - i, - &split_flags[i]); + return i + ci_rx_reassemble_packets(&rx_pkts[i], nb_bufs - i, &split_flags[i], + &rxq->pkt_first_seg, &rxq->pkt_last_seg, rxq->crc_len); } /** diff --git a/drivers/net/ixgbe/meson.build b/drivers/net/ixgbe/meson.build index 0ae12dd5ff..a65ff51379 100644 --- a/drivers/net/ixgbe/meson.build +++ b/drivers/net/ixgbe/meson.build @@ -35,6 +35,6 @@ elif arch_subdir == 'arm' sources += files('ixgbe_recycle_mbufs_vec_common.c') endif -includes += include_directories('base') +includes += include_directories('base', '..') headers = files('rte_pmd_ixgbe.h') -- 2.43.0