From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id C109843A1B; Wed, 7 Feb 2024 03:20:18 +0100 (CET) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 4AE7242D90; Wed, 7 Feb 2024 03:19:51 +0100 (CET) Received: from NAM04-BN8-obe.outbound.protection.outlook.com (mail-bn8nam04on2074.outbound.protection.outlook.com [40.107.100.74]) by mails.dpdk.org (Postfix) with ESMTP id BE1E742DA1 for ; Wed, 7 Feb 2024 03:19:49 +0100 (CET) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=l/WoIU0Aep/ADwWxhGWF3BZ/813lMrcFQzVCqWDYnjV0HITYQS3eceIPdzk4TYLo5VUTcfM0SJb4A9nCXfRPaBQgpZlyDSbjQm88ExZgvpkdNaZP2hqll2UY3uuE9mS0n89GALi0XDsF/OpdywwmwApfT+w8WCrrZIf2PCnfDEs+YI6j8uU4feCSmKTaF9rYNvgvMNwMJDvvzjSfdwx79MH86rIm0G40upYbgRyR5qTM4VhC3FVx/qNYG4PsyjmTiJe/kCaJBUauS9GQbklqFyCjjkYwYG1pyMAu/s6OLW1M0QOemHgex6Lf3X3ywhxywrft6rmqmscLR2tjpFtyDQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=Piv+91DX9Ew8AtbrutD+yjnuUa3Llhmpuv88ba/t/Rs=; b=A3zJBjDZFXD9Zai8+CpLZeoic4zCghpAueNiUkFMlROYxp0UlPILYJVaRxmxLoL6u2CLAnazR8HZobkyv0b9Grkiaq6H2XW+qRBziHPu1eW3D2THJo0jmztzT3ai1FReIOgjsWegwsJQxAaT/t0bIDtNHCdiV+CdsYACrFMk6qknIMftTCTK2vcgNZvkvX+G0Ba+1dMDmLZQxAUWDe1tP5U6LhWAs+xJ/CvmHXeqVArRC2N2swHCDHQRrGfdSrkqvUPsBFDKHklBtpv6VXUkFRLYhGTJr74E+JDtqtWPZt4e0tgT+4VoDdiNAMTCXFN4g4bwpbHWXtFyFa0pYEQFyg== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 165.204.84.17) smtp.rcpttodomain=dpdk.org smtp.mailfrom=amd.com; dmarc=pass (p=quarantine sp=quarantine pct=100) action=none header.from=amd.com; dkim=none (message not signed); arc=none (0) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=Piv+91DX9Ew8AtbrutD+yjnuUa3Llhmpuv88ba/t/Rs=; b=j2wEEcgifgoTubCupxcC+NRLYqk2BVvqFTQGbLj6nPNp8/9cuwhE1pu0R3bIZOCazUg5x+9X2neJfmxTvF0/Y2dmBy1QEMxu58BmcLur1H7OBrdsQsEzqaN/gPWSVSf5iDG+CiqwA+wmXZDwq1MjoBLiU3LjZ7GHJ1Ak0eU/ud0= Received: from PR3PR09CA0011.eurprd09.prod.outlook.com (2603:10a6:102:b7::16) by SN7PR12MB7912.namprd12.prod.outlook.com (2603:10b6:806:341::9) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7270.17; Wed, 7 Feb 2024 02:19:47 +0000 Received: from SA2PEPF000015CC.namprd03.prod.outlook.com (2603:10a6:102:b7:cafe::a6) by PR3PR09CA0011.outlook.office365.com (2603:10a6:102:b7::16) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7249.36 via Frontend Transport; Wed, 7 Feb 2024 02:19:46 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 165.204.84.17) smtp.mailfrom=amd.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=amd.com; Received-SPF: Pass (protection.outlook.com: domain of amd.com designates 165.204.84.17 as permitted sender) receiver=protection.outlook.com; client-ip=165.204.84.17; helo=SATLEXMB04.amd.com; pr=C Received: from SATLEXMB04.amd.com (165.204.84.17) by SA2PEPF000015CC.mail.protection.outlook.com (10.167.241.202) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.20.7249.19 via Frontend Transport; Wed, 7 Feb 2024 02:19:45 +0000 Received: from driver-dev1.pensando.io (10.180.168.240) by SATLEXMB04.amd.com (10.181.40.145) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.34; Tue, 6 Feb 2024 20:19:42 -0600 From: Andrew Boyer To: CC: Neel Patel , Andrew Boyer Subject: [PATCH v2 06/13] net/ionic: memcpy descriptors when using Q-in-CMB Date: Tue, 6 Feb 2024 18:18:42 -0800 Message-ID: <20240207021849.52988-7-andrew.boyer@amd.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20240202193238.62669-1-andrew.boyer@amd.com> References: <20240202193238.62669-1-andrew.boyer@amd.com> MIME-Version: 1.0 Content-Type: text/plain X-Originating-IP: [10.180.168.240] X-ClientProxiedBy: SATLEXMB03.amd.com (10.181.40.144) To SATLEXMB04.amd.com (10.181.40.145) X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: SA2PEPF000015CC:EE_|SN7PR12MB7912:EE_ X-MS-Office365-Filtering-Correlation-Id: 0a821d82-17b0-4783-6147-08dc278340ac X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: zfryJ3pqTG7tfplnm4eS/qPW54waG6I3qAJ3B2IZyaYdZphQL8jrt0cqhDCrrWXY2Z9MXnvQ8HfEvbz8Mr53REg2jJel9cTMRs8bv89vDdYvoLs8+h8WS5JyfSVRYmNeodfXpW2tWQCGxxhGUe6m4xyLdOPXLboo6MKf6+pWYqR92GpDJrSq3wFfCa6UxmX+obSya/wnzdsBJJhXA5hdM4imuQKBFGzl9CDPeJNKJGlssqL7RyTfCsKNISA8JayvjbxwWgsMtXgXTbgzuyvXstaIB8vug4WSB7QBKhsc8R+/+zaBKIHjR1+r57kG9dndhGWZnZr0gqBpt3vF2er8HfhQbnbHOKWBdeCFmo7iHOmXNGZAYRLk57BjcNRDQIi80/7Yb2ExkAiIpdZfCkdnfGatFu/fNMah/tAblanJ4sgu9PvH9TqCPPt1Hv7E4cRnHD7kct4Gd1lHp+Gj/bNim/dpBSiBzaavc6Z4CtvXfQ8CNgfI4upw6NMWlj3PP+9+r6/0dp2PqU5hgmikqlmrdQl1cHg+MvPYxivjylM8J4CnSqBm2rgoJOldAkyWKGoIRIeQoO25mjohgIzYkWBlTsdBLoMpzjDfhhQpbmUP3rABagE59DPDkH/Vwbohi1IRYNOK8iWf4/Ou3Uef2KmyWNSX9cfj4abD2wE9EYGAWgpzfeIPyfuC6kp1e93GZkCrxwDyypsA3OSvkPDFIOY0qDjrUUKZfCA4yNhN/lNyLitHgM9hVOGOKjkKjhghxScte3LOajITnIQbBIRuOAKlZw== X-Forefront-Antispam-Report: CIP:165.204.84.17; CTRY:US; LANG:en; SCL:1; SRV:; IPV:CAL; SFV:NSPM; H:SATLEXMB04.amd.com; PTR:InfoDomainNonexistent; CAT:NONE; SFS:(13230031)(4636009)(136003)(396003)(39860400002)(346002)(376002)(230922051799003)(451199024)(82310400011)(186009)(64100799003)(1800799012)(46966006)(40470700004)(36840700001)(41300700001)(26005)(2906002)(47076005)(70206006)(426003)(8936002)(4326008)(44832011)(30864003)(5660300002)(2616005)(336012)(478600001)(16526019)(83380400001)(82740400003)(6916009)(54906003)(316002)(356005)(1076003)(86362001)(8676002)(81166007)(36860700001)(70586007)(36756003)(40480700001)(40460700003)(36900700001); DIR:OUT; SFP:1101; X-OriginatorOrg: amd.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 07 Feb 2024 02:19:45.8068 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 0a821d82-17b0-4783-6147-08dc278340ac X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d; Ip=[165.204.84.17]; Helo=[SATLEXMB04.amd.com] X-MS-Exchange-CrossTenant-AuthSource: SA2PEPF000015CC.namprd03.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: SN7PR12MB7912 X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org From: Neel Patel They can be batched together this way, reducing the number of PCIe transactions. This improves transmit PPS by up to 50% in some configurations. Signed-off-by: Andrew Boyer Signed-off-by: Neel Patel --- drivers/net/ionic/ionic_dev.c | 9 +++-- drivers/net/ionic/ionic_dev.h | 6 ++- drivers/net/ionic/ionic_lif.c | 26 +++++++++---- drivers/net/ionic/ionic_rxtx.h | 56 +++++++++++++++++++++++++++ drivers/net/ionic/ionic_rxtx_sg.c | 18 ++++----- drivers/net/ionic/ionic_rxtx_simple.c | 18 ++++----- 6 files changed, 101 insertions(+), 32 deletions(-) diff --git a/drivers/net/ionic/ionic_dev.c b/drivers/net/ionic/ionic_dev.c index 70c14882ed..7f15914f74 100644 --- a/drivers/net/ionic/ionic_dev.c +++ b/drivers/net/ionic/ionic_dev.c @@ -369,17 +369,19 @@ ionic_q_init(struct ionic_queue *q, uint32_t index, uint16_t num_descs) q->index = index; q->num_descs = num_descs; q->size_mask = num_descs - 1; - q->head_idx = 0; - q->tail_idx = 0; + ionic_q_reset(q); return 0; } void -ionic_q_map(struct ionic_queue *q, void *base, rte_iova_t base_pa) +ionic_q_map(struct ionic_queue *q, void *base, rte_iova_t base_pa, + void *cmb_base, rte_iova_t cmb_base_pa) { q->base = base; q->base_pa = base_pa; + q->cmb_base = cmb_base; + q->cmb_base_pa = cmb_base_pa; } void @@ -393,5 +395,6 @@ void ionic_q_reset(struct ionic_queue *q) { q->head_idx = 0; + q->cmb_head_idx = 0; q->tail_idx = 0; } diff --git a/drivers/net/ionic/ionic_dev.h b/drivers/net/ionic/ionic_dev.h index 971c261b27..3a366247f1 100644 --- a/drivers/net/ionic/ionic_dev.h +++ b/drivers/net/ionic/ionic_dev.h @@ -145,11 +145,13 @@ struct ionic_queue { uint16_t num_descs; uint16_t num_segs; uint16_t head_idx; + uint16_t cmb_head_idx; uint16_t tail_idx; uint16_t size_mask; uint8_t type; uint8_t hw_type; void *base; + void *cmb_base; void *sg_base; struct ionic_doorbell __iomem *db; void **info; @@ -158,6 +160,7 @@ struct ionic_queue { uint32_t hw_index; rte_iova_t base_pa; rte_iova_t sg_base_pa; + rte_iova_t cmb_base_pa; }; #define IONIC_INTR_NONE (-1) @@ -244,7 +247,8 @@ uint32_t ionic_cq_service(struct ionic_cq *cq, uint32_t work_to_do, int ionic_q_init(struct ionic_queue *q, uint32_t index, uint16_t num_descs); void ionic_q_reset(struct ionic_queue *q); -void ionic_q_map(struct ionic_queue *q, void *base, rte_iova_t base_pa); +void ionic_q_map(struct ionic_queue *q, void *base, rte_iova_t base_pa, + void *cmb_base, rte_iova_t cmb_base_pa); void ionic_q_sg_map(struct ionic_queue *q, void *base, rte_iova_t base_pa); static inline uint16_t diff --git a/drivers/net/ionic/ionic_lif.c b/drivers/net/ionic/ionic_lif.c index fe2112c057..2713f8aa24 100644 --- a/drivers/net/ionic/ionic_lif.c +++ b/drivers/net/ionic/ionic_lif.c @@ -572,10 +572,11 @@ ionic_qcq_alloc(struct ionic_lif *lif, { struct ionic_qcq *new; uint32_t q_size, cq_size, sg_size, total_size; - void *q_base, *cq_base, *sg_base; + void *q_base, *cmb_q_base, *cq_base, *sg_base; rte_iova_t q_base_pa = 0; rte_iova_t cq_base_pa = 0; rte_iova_t sg_base_pa = 0; + rte_iova_t cmb_q_base_pa = 0; size_t page_size = rte_mem_page_size(); int err; @@ -666,19 +667,22 @@ ionic_qcq_alloc(struct ionic_lif *lif, IONIC_PRINT(ERR, "Cannot reserve queue from NIC mem"); return -ENOMEM; } - q_base = (void *) + cmb_q_base = (void *) ((uintptr_t)lif->adapter->bars.bar[2].vaddr + (uintptr_t)lif->adapter->cmb_offset); /* CMB PA is a relative address */ - q_base_pa = lif->adapter->cmb_offset; + cmb_q_base_pa = lif->adapter->cmb_offset; lif->adapter->cmb_offset += q_size; + } else { + cmb_q_base = NULL; + cmb_q_base_pa = 0; } IONIC_PRINT(DEBUG, "Q-Base-PA = %#jx CQ-Base-PA = %#jx " "SG-base-PA = %#jx", q_base_pa, cq_base_pa, sg_base_pa); - ionic_q_map(&new->q, q_base, q_base_pa); + ionic_q_map(&new->q, q_base, q_base_pa, cmb_q_base, cmb_q_base_pa); ionic_cq_map(&new->cq, cq_base, cq_base_pa); *qcq = new; @@ -1583,7 +1587,6 @@ ionic_lif_txq_init(struct ionic_tx_qcq *txq) .flags = rte_cpu_to_le_16(IONIC_QINIT_F_ENA), .intr_index = rte_cpu_to_le_16(IONIC_INTR_NONE), .ring_size = rte_log2_u32(q->num_descs), - .ring_base = rte_cpu_to_le_64(q->base_pa), .cq_ring_base = rte_cpu_to_le_64(cq->base_pa), .sg_ring_base = rte_cpu_to_le_64(q->sg_base_pa), }, @@ -1592,8 +1595,12 @@ ionic_lif_txq_init(struct ionic_tx_qcq *txq) if (txq->flags & IONIC_QCQ_F_SG) ctx.cmd.q_init.flags |= rte_cpu_to_le_16(IONIC_QINIT_F_SG); - if (txq->flags & IONIC_QCQ_F_CMB) + if (txq->flags & IONIC_QCQ_F_CMB) { ctx.cmd.q_init.flags |= rte_cpu_to_le_16(IONIC_QINIT_F_CMB); + ctx.cmd.q_init.ring_base = rte_cpu_to_le_64(q->cmb_base_pa); + } else { + ctx.cmd.q_init.ring_base = rte_cpu_to_le_64(q->base_pa); + } IONIC_PRINT(DEBUG, "txq_init.index %d", q->index); IONIC_PRINT(DEBUG, "txq_init.ring_base 0x%" PRIx64 "", q->base_pa); @@ -1638,7 +1645,6 @@ ionic_lif_rxq_init(struct ionic_rx_qcq *rxq) .flags = rte_cpu_to_le_16(IONIC_QINIT_F_ENA), .intr_index = rte_cpu_to_le_16(IONIC_INTR_NONE), .ring_size = rte_log2_u32(q->num_descs), - .ring_base = rte_cpu_to_le_64(q->base_pa), .cq_ring_base = rte_cpu_to_le_64(cq->base_pa), .sg_ring_base = rte_cpu_to_le_64(q->sg_base_pa), }, @@ -1647,8 +1653,12 @@ ionic_lif_rxq_init(struct ionic_rx_qcq *rxq) if (rxq->flags & IONIC_QCQ_F_SG) ctx.cmd.q_init.flags |= rte_cpu_to_le_16(IONIC_QINIT_F_SG); - if (rxq->flags & IONIC_QCQ_F_CMB) + if (rxq->flags & IONIC_QCQ_F_CMB) { ctx.cmd.q_init.flags |= rte_cpu_to_le_16(IONIC_QINIT_F_CMB); + ctx.cmd.q_init.ring_base = rte_cpu_to_le_64(q->cmb_base_pa); + } else { + ctx.cmd.q_init.ring_base = rte_cpu_to_le_64(q->base_pa); + } IONIC_PRINT(DEBUG, "rxq_init.index %d", q->index); IONIC_PRINT(DEBUG, "rxq_init.ring_base 0x%" PRIx64 "", q->base_pa); diff --git a/drivers/net/ionic/ionic_rxtx.h b/drivers/net/ionic/ionic_rxtx.h index 8537141597..5348395956 100644 --- a/drivers/net/ionic/ionic_rxtx.h +++ b/drivers/net/ionic/ionic_rxtx.h @@ -77,4 +77,60 @@ uint16_t ionic_xmit_pkts_sg(void *tx_queue, struct rte_mbuf **tx_pkts, int ionic_rx_fill_sg(struct ionic_rx_qcq *rxq); +static inline void +ionic_rxq_flush(struct ionic_queue *q) +{ + struct ionic_rxq_desc *desc_base = q->base; + struct ionic_rxq_desc *cmb_desc_base = q->cmb_base; + + if (q->cmb_base) { + if (q->head_idx < q->cmb_head_idx) { + /* copy [cmb_head, num_descs) */ + rte_memcpy((void *)&cmb_desc_base[q->cmb_head_idx], + (void *)&desc_base[q->cmb_head_idx], + (q->num_descs - q->cmb_head_idx) * sizeof(*desc_base)); + /* copy [0, head) */ + rte_memcpy((void *)&cmb_desc_base[0], + (void *)&desc_base[0], + q->head_idx * sizeof(*desc_base)); + } else { + /* copy [cmb_head, head) */ + rte_memcpy((void *)&cmb_desc_base[q->cmb_head_idx], + (void *)&desc_base[q->cmb_head_idx], + (q->head_idx - q->cmb_head_idx) * sizeof(*desc_base)); + } + q->cmb_head_idx = q->head_idx; + } + + ionic_q_flush(q); +} + +static inline void +ionic_txq_flush(struct ionic_queue *q) +{ + struct ionic_txq_desc *desc_base = q->base; + struct ionic_txq_desc *cmb_desc_base = q->cmb_base; + + if (q->cmb_base) { + if (q->head_idx < q->cmb_head_idx) { + /* copy [cmb_head, num_descs) */ + rte_memcpy((void *)&cmb_desc_base[q->cmb_head_idx], + (void *)&desc_base[q->cmb_head_idx], + (q->num_descs - q->cmb_head_idx) * sizeof(*desc_base)); + /* copy [0, head) */ + rte_memcpy((void *)&cmb_desc_base[0], + (void *)&desc_base[0], + q->head_idx * sizeof(*desc_base)); + } else { + /* copy [cmb_head, head) */ + rte_memcpy((void *)&cmb_desc_base[q->cmb_head_idx], + (void *)&desc_base[q->cmb_head_idx], + (q->head_idx - q->cmb_head_idx) * sizeof(*desc_base)); + } + q->cmb_head_idx = q->head_idx; + } + + ionic_q_flush(q); +} + #endif /* _IONIC_RXTX_H_ */ diff --git a/drivers/net/ionic/ionic_rxtx_sg.c b/drivers/net/ionic/ionic_rxtx_sg.c index 1392342463..92e1d6e259 100644 --- a/drivers/net/ionic/ionic_rxtx_sg.c +++ b/drivers/net/ionic/ionic_rxtx_sg.c @@ -166,6 +166,7 @@ ionic_xmit_pkts_sg(void *tx_queue, struct rte_mbuf **tx_pkts, { struct ionic_tx_qcq *txq = tx_queue; struct ionic_queue *q = &txq->qcq.q; + struct ionic_txq_desc *desc_base = q->base; struct ionic_tx_stats *stats = &txq->stats; struct rte_mbuf *mbuf; uint32_t bytes_tx = 0; @@ -173,9 +174,7 @@ ionic_xmit_pkts_sg(void *tx_queue, struct rte_mbuf **tx_pkts, uint64_t then, now, hz, delta; int err; - struct ionic_txq_desc *desc_base = q->base; - if (!(txq->flags & IONIC_QCQ_F_CMB)) - rte_prefetch0(&desc_base[q->head_idx]); + rte_prefetch0(&desc_base[q->head_idx]); rte_prefetch0(IONIC_INFO_PTR(q, q->head_idx)); if (nb_pkts) { @@ -196,8 +195,7 @@ ionic_xmit_pkts_sg(void *tx_queue, struct rte_mbuf **tx_pkts, while (nb_tx < nb_pkts) { uint16_t next_idx = Q_NEXT_TO_POST(q, 1); - if (!(txq->flags & IONIC_QCQ_F_CMB)) - rte_prefetch0(&desc_base[next_idx]); + rte_prefetch0(&desc_base[next_idx]); rte_prefetch0(IONIC_INFO_PTR(q, next_idx)); if (nb_tx + 1 < nb_pkts) { @@ -222,7 +220,7 @@ ionic_xmit_pkts_sg(void *tx_queue, struct rte_mbuf **tx_pkts, if (nb_tx > 0) { rte_wmb(); - ionic_q_flush(q); + ionic_txq_flush(q); txq->last_wdog_cycles = rte_get_timer_cycles(); @@ -458,8 +456,7 @@ ionic_rxq_service_sg(struct ionic_rx_qcq *rxq, uint32_t work_to_do, /* Prefetch 4 x 16B comp */ rte_prefetch0(&cq_desc_base[Q_NEXT_TO_SRVC(cq, 4)]); /* Prefetch 4 x 16B descriptors */ - if (!(rxq->flags & IONIC_QCQ_F_CMB)) - rte_prefetch0(&q_desc_base[Q_NEXT_TO_POST(q, 4)]); + rte_prefetch0(&q_desc_base[Q_NEXT_TO_POST(q, 4)]); /* Clean one descriptor */ ionic_rx_clean_one_sg(rxq, cq_desc, rx_svc); @@ -478,7 +475,8 @@ ionic_rxq_service_sg(struct ionic_rx_qcq *rxq, uint32_t work_to_do, /* Update the queue indices and ring the doorbell */ if (work_done) { - ionic_q_flush(q); + ionic_rxq_flush(q); + rxq->last_wdog_cycles = rte_get_timer_cycles(); rxq->wdog_ms = IONIC_Q_WDOG_MS; } else { @@ -542,7 +540,7 @@ ionic_rx_fill_sg(struct ionic_rx_qcq *rxq) q->head_idx = Q_NEXT_TO_POST(q, 1); } - ionic_q_flush(q); + ionic_rxq_flush(q); return err; } diff --git a/drivers/net/ionic/ionic_rxtx_simple.c b/drivers/net/ionic/ionic_rxtx_simple.c index 00152c885a..f12f66f40c 100644 --- a/drivers/net/ionic/ionic_rxtx_simple.c +++ b/drivers/net/ionic/ionic_rxtx_simple.c @@ -139,6 +139,7 @@ ionic_xmit_pkts(void *tx_queue, struct rte_mbuf **tx_pkts, { struct ionic_tx_qcq *txq = tx_queue; struct ionic_queue *q = &txq->qcq.q; + struct ionic_txq_desc *desc_base = q->base; struct ionic_tx_stats *stats = &txq->stats; struct rte_mbuf *mbuf; uint32_t bytes_tx = 0; @@ -146,9 +147,7 @@ ionic_xmit_pkts(void *tx_queue, struct rte_mbuf **tx_pkts, uint64_t then, now, hz, delta; int err; - struct ionic_txq_desc *desc_base = q->base; - if (!(txq->flags & IONIC_QCQ_F_CMB)) - rte_prefetch0(&desc_base[q->head_idx]); + rte_prefetch0(&desc_base[q->head_idx]); rte_prefetch0(&q->info[q->head_idx]); if (nb_pkts) { @@ -169,8 +168,7 @@ ionic_xmit_pkts(void *tx_queue, struct rte_mbuf **tx_pkts, while (nb_tx < nb_pkts) { uint16_t next_idx = Q_NEXT_TO_POST(q, 1); - if (!(txq->flags & IONIC_QCQ_F_CMB)) - rte_prefetch0(&desc_base[next_idx]); + rte_prefetch0(&desc_base[next_idx]); rte_prefetch0(&q->info[next_idx]); if (nb_tx + 1 < nb_pkts) { @@ -195,7 +193,7 @@ ionic_xmit_pkts(void *tx_queue, struct rte_mbuf **tx_pkts, if (nb_tx > 0) { rte_wmb(); - ionic_q_flush(q); + ionic_txq_flush(q); txq->last_wdog_cycles = rte_get_timer_cycles(); @@ -379,8 +377,7 @@ ionic_rxq_service(struct ionic_rx_qcq *rxq, uint32_t work_to_do, /* Prefetch 4 x 16B comp */ rte_prefetch0(&cq_desc_base[Q_NEXT_TO_SRVC(cq, 4)]); /* Prefetch 4 x 16B descriptors */ - if (!(rxq->flags & IONIC_QCQ_F_CMB)) - rte_prefetch0(&q_desc_base[Q_NEXT_TO_POST(q, 4)]); + rte_prefetch0(&q_desc_base[Q_NEXT_TO_POST(q, 4)]); /* Clean one descriptor */ ionic_rx_clean_one(rxq, cq_desc, rx_svc); @@ -399,7 +396,8 @@ ionic_rxq_service(struct ionic_rx_qcq *rxq, uint32_t work_to_do, /* Update the queue indices and ring the doorbell */ if (work_done) { - ionic_q_flush(q); + ionic_rxq_flush(q); + rxq->last_wdog_cycles = rte_get_timer_cycles(); rxq->wdog_ms = IONIC_Q_WDOG_MS; } else { @@ -463,7 +461,7 @@ ionic_rx_fill(struct ionic_rx_qcq *rxq) q->head_idx = Q_NEXT_TO_POST(q, 1); } - ionic_q_flush(q); + ionic_rxq_flush(q); return err; } -- 2.17.1