From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id DFBB34240B; Wed, 18 Jan 2023 09:10:41 +0100 (CET) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id D997B42D4A; Wed, 18 Jan 2023 09:10:28 +0100 (CET) Received: from mga05.intel.com (mga05.intel.com [192.55.52.43]) by mails.dpdk.org (Postfix) with ESMTP id 73B0942C4D for ; Wed, 18 Jan 2023 09:10:25 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1674029425; x=1705565425; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=6yi6ECni9tlBuICXgya15Wrl2cpxwVJ5D4JEUrjp4Ms=; b=Y3z1U7uQaAZ7ttSpqnck2quZrXWb97Tf/LOrbDpH2Wc6Ygv8YhYPS/JV Tq69ZhwnKs3BAvmf4P/QoQDey9dYfJ458Mq+8BiiZ0c4sjGqx+MnOZxgE XmiwYRx7I/NgAohh8oV/HRHLUHvDLZfkQdXJa9WhAHQu1n7ScUXcBHwGq /ElwjVSPXwheGhZbhzCizCbLFte/178NM1GBjJu5/HUkjDPtrQlqqabKe 7m0nKfwwjbfkRajG1G2XS7B7/958q+AfLHHFtL7Lka9UDbfQkj0nrbGnb J6MmBPIuoDayV1CvXBItSnStuZWDdeeKTjJxfOzAvJ5xacnZOqe5tCs0W w==; X-IronPort-AV: E=McAfee;i="6500,9779,10593"; a="411166162" X-IronPort-AV: E=Sophos;i="5.97,224,1669104000"; d="scan'208";a="411166162" Received: from orsmga004.jf.intel.com ([10.7.209.38]) by fmsmga105.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 18 Jan 2023 00:10:25 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6500,9779,10593"; a="783577528" X-IronPort-AV: E=Sophos;i="5.97,224,1669104000"; d="scan'208";a="783577528" Received: from dpdk-mingxial-01.sh.intel.com ([10.67.119.167]) by orsmga004.jf.intel.com with ESMTP; 18 Jan 2023 00:10:23 -0800 From: Mingxia Liu To: dev@dpdk.org Cc: jingjing.wu@intel.com, beilei.xing@intel.com, Mingxia Liu , Wenjun Wu Subject: [PATCH v3 3/6] common/idpf: support single q scatter RX datapath Date: Wed, 18 Jan 2023 07:14:37 +0000 Message-Id: <20230118071440.902155-4-mingxia.liu@intel.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20230118071440.902155-1-mingxia.liu@intel.com> References: <20230111071545.504706-1-mingxia.liu@intel.com> <20230118071440.902155-1-mingxia.liu@intel.com> MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org This patch add single q recv scatter rx function. Signed-off-by: Mingxia Liu Signed-off-by: Wenjun Wu --- drivers/common/idpf/idpf_common_rxtx.c | 134 +++++++++++++++++++++++++ drivers/common/idpf/idpf_common_rxtx.h | 3 + drivers/common/idpf/version.map | 1 + drivers/net/idpf/idpf_ethdev.c | 3 +- drivers/net/idpf/idpf_rxtx.c | 26 ++++- drivers/net/idpf/idpf_rxtx.h | 2 + 6 files changed, 166 insertions(+), 3 deletions(-) diff --git a/drivers/common/idpf/idpf_common_rxtx.c b/drivers/common/idpf/idpf_common_rxtx.c index 7a5dc3f04c..9dbf0f4764 100644 --- a/drivers/common/idpf/idpf_common_rxtx.c +++ b/drivers/common/idpf/idpf_common_rxtx.c @@ -1146,6 +1146,140 @@ idpf_singleq_recv_pkts(void *rx_queue, struct rte_mbuf **rx_pkts, return nb_rx; } +uint16_t +idpf_singleq_recv_scatter_pkts(void *rx_queue, struct rte_mbuf **rx_pkts, + uint16_t nb_pkts) +{ + struct idpf_rx_queue *rxq = rx_queue; + volatile union virtchnl2_rx_desc *rx_ring = rxq->rx_ring; + volatile union virtchnl2_rx_desc *rxdp; + union virtchnl2_rx_desc rxd; + struct idpf_adapter *ad; + struct rte_mbuf *first_seg = rxq->pkt_first_seg; + struct rte_mbuf *last_seg = rxq->pkt_last_seg; + struct rte_mbuf *rxm; + struct rte_mbuf *nmb; + struct rte_eth_dev *dev; + const uint32_t *ptype_tbl = rxq->adapter->ptype_tbl; + uint16_t nb_hold = 0, nb_rx = 0; + uint16_t rx_id = rxq->rx_tail; + uint16_t rx_packet_len; + uint16_t rx_status0; + uint64_t pkt_flags; + uint64_t dma_addr; + uint64_t ts_ns; + + ad = rxq->adapter; + + if (unlikely(!rxq) || unlikely(!rxq->q_started)) + return nb_rx; + + while (nb_rx < nb_pkts) { + rxdp = &rx_ring[rx_id]; + rx_status0 = rte_le_to_cpu_16(rxdp->flex_nic_wb.status_error0); + + /* Check the DD bit first */ + if (!(rx_status0 & (1 << VIRTCHNL2_RX_FLEX_DESC_STATUS0_DD_S))) + break; + + nmb = rte_mbuf_raw_alloc(rxq->mp); + if (unlikely(!nmb)) { + rte_atomic64_inc(&(rxq->rx_stats.mbuf_alloc_failed)); + RX_LOG(DEBUG, "RX mbuf alloc failed port_id=%u " + "queue_id=%u", rxq->port_id, rxq->queue_id); + break; + } + + rxd = *rxdp; + + nb_hold++; + rxm = rxq->sw_ring[rx_id]; + rxq->sw_ring[rx_id] = nmb; + rx_id++; + if (unlikely(rx_id == rxq->nb_rx_desc)) + rx_id = 0; + + /* Prefetch next mbuf */ + rte_prefetch0(rxq->sw_ring[rx_id]); + + /* When next RX descriptor is on a cache line boundary, + * prefetch the next 4 RX descriptors and next 8 pointers + * to mbufs. + */ + if ((rx_id & 0x3) == 0) { + rte_prefetch0(&rx_ring[rx_id]); + rte_prefetch0(rxq->sw_ring[rx_id]); + } + dma_addr = + rte_cpu_to_le_64(rte_mbuf_data_iova_default(nmb)); + rxdp->read.hdr_addr = 0; + rxdp->read.pkt_addr = dma_addr; + rx_packet_len = (rte_cpu_to_le_16(rxd.flex_nic_wb.pkt_len) & + VIRTCHNL2_RX_FLEX_DESC_PKT_LEN_M); + rxm->data_len = rx_packet_len; + rxm->data_off = RTE_PKTMBUF_HEADROOM; + + /** + * If this is the first buffer of the received packet, set the + * pointer to the first mbuf of the packet and initialize its + * context. Otherwise, update the total length and the number + * of segments of the current scattered packet, and update the + * pointer to the last mbuf of the current packet. + */ + if (!first_seg) { + first_seg = rxm; + first_seg->nb_segs = 1; + first_seg->pkt_len = rx_packet_len; + } else { + first_seg->pkt_len = + (uint16_t)(first_seg->pkt_len + + rx_packet_len); + first_seg->nb_segs++; + last_seg->next = rxm; + } + + if (!(rx_status0 & (1 << VIRTCHNL2_RX_FLEX_DESC_STATUS0_EOF_S))) { + last_seg = rxm; + continue; + } + + rxm->next = NULL; + + first_seg->port = rxq->port_id; + first_seg->ol_flags = 0; + pkt_flags = idpf_rxd_to_pkt_flags(rx_status0); + first_seg->packet_type = + ptype_tbl[(uint8_t)(rte_cpu_to_le_16(rxd.flex_nic_wb.ptype_flex_flags0) & + VIRTCHNL2_RX_FLEX_DESC_PTYPE_M)]; + + if (idpf_timestamp_dynflag > 0 && + (rxq->offloads & IDPF_RX_OFFLOAD_TIMESTAMP) != 0) { + /* timestamp */ + ts_ns = idpf_tstamp_convert_32b_64b(ad, + rxq->hw_register_set, + rte_le_to_cpu_32(rxd.flex_nic_wb.flex_ts.ts_high)); + rxq->hw_register_set = 0; + *RTE_MBUF_DYNFIELD(rxm, + idpf_timestamp_dynfield_offset, + rte_mbuf_timestamp_t *) = ts_ns; + first_seg->ol_flags |= idpf_timestamp_dynflag; + } + + first_seg->ol_flags |= pkt_flags; + rte_prefetch0(RTE_PTR_ADD(first_seg->buf_addr, + first_seg->data_off)); + rx_pkts[nb_rx++] = first_seg; + first_seg = NULL; + } + rxq->rx_tail = rx_id; + rxq->pkt_first_seg = first_seg; + rxq->pkt_last_seg = last_seg; + + idpf_update_rx_tail(rxq, nb_hold, rx_id); + + return nb_rx; +} + static inline int idpf_xmit_cleanup(struct idpf_tx_queue *txq) { diff --git a/drivers/common/idpf/idpf_common_rxtx.h b/drivers/common/idpf/idpf_common_rxtx.h index 98f492a8c1..aac61ea2cb 100644 --- a/drivers/common/idpf/idpf_common_rxtx.h +++ b/drivers/common/idpf/idpf_common_rxtx.h @@ -260,6 +260,9 @@ __rte_internal uint16_t idpf_splitq_recv_pkts(void *rx_queue, struct rte_mbuf **rx_pkts, uint16_t nb_pkts); __rte_internal +uint16_t idpf_singleq_recv_scatter_pkts(void *rx_queue, struct rte_mbuf **rx_pkts, + uint16_t nb_pkts); +__rte_internal uint16_t idpf_splitq_xmit_pkts(void *tx_queue, struct rte_mbuf **tx_pkts, uint16_t nb_pkts); __rte_internal diff --git a/drivers/common/idpf/version.map b/drivers/common/idpf/version.map index 36a3a90d39..591af6b046 100644 --- a/drivers/common/idpf/version.map +++ b/drivers/common/idpf/version.map @@ -32,6 +32,7 @@ INTERNAL { idpf_reset_split_tx_descq; idpf_rx_queue_release; idpf_singleq_recv_pkts; + idpf_singleq_recv_scatter_pkts; idpf_singleq_recv_pkts_avx512; idpf_singleq_rx_vec_setup; idpf_splitq_rx_vec_setup; diff --git a/drivers/net/idpf/idpf_ethdev.c b/drivers/net/idpf/idpf_ethdev.c index 037cabb04e..2ab31792ba 100644 --- a/drivers/net/idpf/idpf_ethdev.c +++ b/drivers/net/idpf/idpf_ethdev.c @@ -119,7 +119,8 @@ idpf_dev_info_get(struct rte_eth_dev *dev, struct rte_eth_dev_info *dev_info) RTE_ETH_RX_OFFLOAD_UDP_CKSUM | RTE_ETH_RX_OFFLOAD_TCP_CKSUM | RTE_ETH_RX_OFFLOAD_OUTER_IPV4_CKSUM | - RTE_ETH_RX_OFFLOAD_TIMESTAMP; + RTE_ETH_RX_OFFLOAD_TIMESTAMP | + RTE_ETH_RX_OFFLOAD_SCATTER; dev_info->tx_offload_capa = RTE_ETH_TX_OFFLOAD_IPV4_CKSUM | diff --git a/drivers/net/idpf/idpf_rxtx.c b/drivers/net/idpf/idpf_rxtx.c index 6eeaab41cc..a865d14fea 100644 --- a/drivers/net/idpf/idpf_rxtx.c +++ b/drivers/net/idpf/idpf_rxtx.c @@ -503,6 +503,8 @@ int idpf_rx_queue_init(struct rte_eth_dev *dev, uint16_t rx_queue_id) { struct idpf_rx_queue *rxq; + uint16_t max_pkt_len; + uint32_t frame_size; int err; if (rx_queue_id >= dev->data->nb_rx_queues) @@ -516,6 +518,17 @@ idpf_rx_queue_init(struct rte_eth_dev *dev, uint16_t rx_queue_id) return -EINVAL; } + frame_size = dev->data->mtu + IDPF_ETH_OVERHEAD; + + max_pkt_len = + RTE_MIN((uint32_t)IDPF_SUPPORT_CHAIN_NUM * rxq->rx_buf_len, + frame_size); + + rxq->max_pkt_len = max_pkt_len; + if ((dev->data->dev_conf.rxmode.offloads & RTE_ETH_RX_OFFLOAD_SCATTER) || + frame_size > rxq->rx_buf_len) + dev->data->scattered_rx = 1; + err = idpf_register_ts_mbuf(rxq); if (err != 0) { PMD_DRV_LOG(ERR, "fail to regidter timestamp mbuf %u", @@ -801,13 +814,22 @@ idpf_set_rx_function(struct rte_eth_dev *dev) #endif /* CC_AVX512_SUPPORT */ } + if (dev->data->scattered_rx) { + dev->rx_pkt_burst = idpf_singleq_recv_scatter_pkts; + return; + } dev->rx_pkt_burst = idpf_singleq_recv_pkts; } #else - if (vport->rxq_model == VIRTCHNL2_QUEUE_MODEL_SPLIT) + if (vport->rxq_model == VIRTCHNL2_QUEUE_MODEL_SPLIT) { dev->rx_pkt_burst = idpf_splitq_recv_pkts; - else + } else { + if (dev->data->scattered_rx) { + dev->rx_pkt_burst = idpf_singleq_recv_scatter_pkts; + return; + } dev->rx_pkt_burst = idpf_singleq_recv_pkts; + } #endif /* RTE_ARCH_X86 */ } diff --git a/drivers/net/idpf/idpf_rxtx.h b/drivers/net/idpf/idpf_rxtx.h index 3a5084dfd6..41a7495083 100644 --- a/drivers/net/idpf/idpf_rxtx.h +++ b/drivers/net/idpf/idpf_rxtx.h @@ -23,6 +23,8 @@ #define IDPF_DEFAULT_TX_RS_THRESH 32 #define IDPF_DEFAULT_TX_FREE_THRESH 32 +#define IDPF_SUPPORT_CHAIN_NUM 5 + int idpf_rx_queue_setup(struct rte_eth_dev *dev, uint16_t queue_idx, uint16_t nb_desc, unsigned int socket_id, const struct rte_eth_rxconf *rx_conf, -- 2.25.1