From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 7524B43A53; Fri, 2 Feb 2024 20:34:20 +0100 (CET) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id CE00742EAE; Fri, 2 Feb 2024 20:33:29 +0100 (CET) Received: from NAM12-MW2-obe.outbound.protection.outlook.com (mail-mw2nam12on2069.outbound.protection.outlook.com [40.107.244.69]) by mails.dpdk.org (Postfix) with ESMTP id 6D94942E98 for ; Fri, 2 Feb 2024 20:33:26 +0100 (CET) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=VNVySuKwZpxeo8fkm98j1Y3caCyvaSEHuFBrU5Pmes8Wa4G2UWtYlU03FDTPi1qHDAqe/tYi+l+oARfQpdIIuhkqIf+HT99Y3fIgwyFldfSvom5t/Vp/DWtFpOtgGg7yywNPKF7z+1bwzzNbwWkgl8jlDDdWcRybr6QbTFcJcvl2M9pS8o6/y0kmYzV7AaF3TOHnmPxrd+LXFmpykE/lEyKGjDiGNHS+U7xwPb0WmNItMtgbvOWL6Lnbx6FGkaTS6DF0YhKyLUNmGjqmQrKxEiy1TV2hoeNrIFGhzjygRy6KE47O6Qd38tZLrxFBb8pu20bsTVKE84W5ZtTGkZyuAw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=Piv+91DX9Ew8AtbrutD+yjnuUa3Llhmpuv88ba/t/Rs=; b=Ssa0Ehr2I8OdZwV4cZvPInyVhhojztPfVLlfsF6XLS4o04xA1pEH033u/cj8lp16un+p/2Nm1OTVUd1pZL5p0jzcveqJGdxff8FtxTiaLcJyWe9UQeXnxWV2cF84VzWHlBdLNJ19P0AcP67VKRWJrKFbiBslQ403hrNAaqi1qGsZnh4s0GjdOw0obOu2JSdP113ozHPfy/sZsBzhogKmp2J+rqzhdYtiRx6jeg6lYnHfJAuCF5Ud1iM+AQzQ3I06csGnOCpwBYuG0+b84FlgvM9obGJGGOM2eZGsCmRhjO9WtqfrwZi9AF7Rxtj+h/7+UIRZZRGUPIt4CU9J4zV7rQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 165.204.84.17) smtp.rcpttodomain=dpdk.org smtp.mailfrom=amd.com; dmarc=pass (p=quarantine sp=quarantine pct=100) action=none header.from=amd.com; dkim=none (message not signed); arc=none (0) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=Piv+91DX9Ew8AtbrutD+yjnuUa3Llhmpuv88ba/t/Rs=; b=ashO9GbynUYNWlgRWKirMYV/dQgWL4fVnOTPl26T2lRfUssp5eGmikok15TpMeUoHju4e+gG0AO+W3DSLzaHTd3kcyu/QPupUTyu9AdUj8viVwH/J2FOrFfDZTNRIiEti3jT7uwSpQcSH0RVnSWx9FQ4hEuYX0A9RqS8AuPrb4A= Received: from DM6PR10CA0033.namprd10.prod.outlook.com (2603:10b6:5:60::46) by PH7PR12MB8038.namprd12.prod.outlook.com (2603:10b6:510:27c::9) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7228.27; Fri, 2 Feb 2024 19:33:19 +0000 Received: from DS1PEPF0001708F.namprd03.prod.outlook.com (2603:10b6:5:60:cafe::27) by DM6PR10CA0033.outlook.office365.com (2603:10b6:5:60::46) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7249.22 via Frontend Transport; Fri, 2 Feb 2024 19:33:18 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 165.204.84.17) smtp.mailfrom=amd.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=amd.com; Received-SPF: Pass (protection.outlook.com: domain of amd.com designates 165.204.84.17 as permitted sender) receiver=protection.outlook.com; client-ip=165.204.84.17; helo=SATLEXMB04.amd.com; pr=C Received: from SATLEXMB04.amd.com (165.204.84.17) by DS1PEPF0001708F.mail.protection.outlook.com (10.167.17.139) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.20.7249.19 via Frontend Transport; Fri, 2 Feb 2024 19:33:18 +0000 Received: from driver-dev1.pensando.io (10.180.168.240) by SATLEXMB04.amd.com (10.181.40.145) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.34; Fri, 2 Feb 2024 13:33:17 -0600 From: Andrew Boyer To: CC: Neel Patel , Andrew Boyer Subject: [PATCH 06/13] net/ionic: memcpy descriptors when using Q-in-CMB Date: Fri, 2 Feb 2024 11:32:31 -0800 Message-ID: <20240202193238.62669-7-andrew.boyer@amd.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20240202193238.62669-1-andrew.boyer@amd.com> References: <20240202193238.62669-1-andrew.boyer@amd.com> MIME-Version: 1.0 Content-Type: text/plain X-Originating-IP: [10.180.168.240] X-ClientProxiedBy: SATLEXMB04.amd.com (10.181.40.145) To SATLEXMB04.amd.com (10.181.40.145) X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: DS1PEPF0001708F:EE_|PH7PR12MB8038:EE_ X-MS-Office365-Filtering-Correlation-Id: 282d84a0-36c1-4e9b-0744-08dc2425cf06 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: +Svv/WrGXz6mJ3rxVTg41AhMEAF/XY/fz4sMrnm3vCITdp3aDqjs0hSKfAfFw1PW2c4xWu1SL2OVKL+/rP7Q7WSEEQSd8cCL+tGwGUs9ObC8zN2+ERgkBuUotx/A0oQ+QhBha1hA2OQZJ6C/oIb8SOKiF4DdQ98TdNNT7tGI4JDzO9DtvgaOp0TmKrZ997W/uhs7ZwHW8Tqy6GNzZ1WF/DgtIKV1DQuD5uI1X7kf4wOPBMXNyDsn3R1NiQjLCNbZQFzweppUUYJwI6WlcMRQbWhKeFxGfNtGM8HeNIoNTOBwA1jpD2NapDwF/Wvs+iTv9SFvjx6dwl2GR1AUTYxU1rBM3GIiaMaatZp7hfLObKsR36GWapl8XrNmc/MBV7TR0c7bLX0vJjmA4yAgLxpS3PGElIeE6N1ropyV49B5AMPArvKQKF/IS3KanlXEVHdUMZappLICB3EMrGMYnMYyuXSLwY8gLCnjhP5qm5xASFXb0ZFO8SAEH42ml3MOd13TXN289eNyujp2s+iqvn25mS9TCCpsIaEPu//vyxAiy5BjB/UqbUWD10vfMEEB65qIfDKGrX1OoeklOlNtYO1xaMzgU9vUur5Ya2Nn7HaGwWJ2XJzfzDpJj5+Qp4Il3Zru5XZ5L0T6aYgw8d0/K4sNgzSDczjn6KLgcOISqFmKnYoqiZlgPmeJnnRM+HVUpBxJW8VEREjKeM7L6pfOznBEOLLlknXLYT+mAGMZkU2Z7Rej7f7iNVMe/ZlVjxbvwpSS9Hz59ue0K/bi6pSqbWzmUg== X-Forefront-Antispam-Report: CIP:165.204.84.17; CTRY:US; LANG:en; SCL:1; SRV:; IPV:CAL; SFV:NSPM; H:SATLEXMB04.amd.com; PTR:InfoDomainNonexistent; CAT:NONE; SFS:(13230031)(4636009)(39860400002)(346002)(136003)(376002)(396003)(230922051799003)(186009)(1800799012)(64100799003)(451199024)(82310400011)(46966006)(36840700001)(40470700004)(83380400001)(40460700003)(40480700001)(356005)(86362001)(36756003)(36860700001)(82740400003)(426003)(1076003)(41300700001)(26005)(81166007)(2616005)(47076005)(5660300002)(16526019)(336012)(6666004)(8676002)(316002)(70206006)(6916009)(478600001)(54906003)(8936002)(4326008)(2906002)(70586007)(30864003)(44832011)(36900700001); DIR:OUT; SFP:1101; X-OriginatorOrg: amd.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 02 Feb 2024 19:33:18.4655 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 282d84a0-36c1-4e9b-0744-08dc2425cf06 X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d; Ip=[165.204.84.17]; Helo=[SATLEXMB04.amd.com] X-MS-Exchange-CrossTenant-AuthSource: DS1PEPF0001708F.namprd03.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: PH7PR12MB8038 X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org From: Neel Patel They can be batched together this way, reducing the number of PCIe transactions. This improves transmit PPS by up to 50% in some configurations. Signed-off-by: Andrew Boyer Signed-off-by: Neel Patel --- drivers/net/ionic/ionic_dev.c | 9 +++-- drivers/net/ionic/ionic_dev.h | 6 ++- drivers/net/ionic/ionic_lif.c | 26 +++++++++---- drivers/net/ionic/ionic_rxtx.h | 56 +++++++++++++++++++++++++++ drivers/net/ionic/ionic_rxtx_sg.c | 18 ++++----- drivers/net/ionic/ionic_rxtx_simple.c | 18 ++++----- 6 files changed, 101 insertions(+), 32 deletions(-) diff --git a/drivers/net/ionic/ionic_dev.c b/drivers/net/ionic/ionic_dev.c index 70c14882ed..7f15914f74 100644 --- a/drivers/net/ionic/ionic_dev.c +++ b/drivers/net/ionic/ionic_dev.c @@ -369,17 +369,19 @@ ionic_q_init(struct ionic_queue *q, uint32_t index, uint16_t num_descs) q->index = index; q->num_descs = num_descs; q->size_mask = num_descs - 1; - q->head_idx = 0; - q->tail_idx = 0; + ionic_q_reset(q); return 0; } void -ionic_q_map(struct ionic_queue *q, void *base, rte_iova_t base_pa) +ionic_q_map(struct ionic_queue *q, void *base, rte_iova_t base_pa, + void *cmb_base, rte_iova_t cmb_base_pa) { q->base = base; q->base_pa = base_pa; + q->cmb_base = cmb_base; + q->cmb_base_pa = cmb_base_pa; } void @@ -393,5 +395,6 @@ void ionic_q_reset(struct ionic_queue *q) { q->head_idx = 0; + q->cmb_head_idx = 0; q->tail_idx = 0; } diff --git a/drivers/net/ionic/ionic_dev.h b/drivers/net/ionic/ionic_dev.h index 971c261b27..3a366247f1 100644 --- a/drivers/net/ionic/ionic_dev.h +++ b/drivers/net/ionic/ionic_dev.h @@ -145,11 +145,13 @@ struct ionic_queue { uint16_t num_descs; uint16_t num_segs; uint16_t head_idx; + uint16_t cmb_head_idx; uint16_t tail_idx; uint16_t size_mask; uint8_t type; uint8_t hw_type; void *base; + void *cmb_base; void *sg_base; struct ionic_doorbell __iomem *db; void **info; @@ -158,6 +160,7 @@ struct ionic_queue { uint32_t hw_index; rte_iova_t base_pa; rte_iova_t sg_base_pa; + rte_iova_t cmb_base_pa; }; #define IONIC_INTR_NONE (-1) @@ -244,7 +247,8 @@ uint32_t ionic_cq_service(struct ionic_cq *cq, uint32_t work_to_do, int ionic_q_init(struct ionic_queue *q, uint32_t index, uint16_t num_descs); void ionic_q_reset(struct ionic_queue *q); -void ionic_q_map(struct ionic_queue *q, void *base, rte_iova_t base_pa); +void ionic_q_map(struct ionic_queue *q, void *base, rte_iova_t base_pa, + void *cmb_base, rte_iova_t cmb_base_pa); void ionic_q_sg_map(struct ionic_queue *q, void *base, rte_iova_t base_pa); static inline uint16_t diff --git a/drivers/net/ionic/ionic_lif.c b/drivers/net/ionic/ionic_lif.c index fe2112c057..2713f8aa24 100644 --- a/drivers/net/ionic/ionic_lif.c +++ b/drivers/net/ionic/ionic_lif.c @@ -572,10 +572,11 @@ ionic_qcq_alloc(struct ionic_lif *lif, { struct ionic_qcq *new; uint32_t q_size, cq_size, sg_size, total_size; - void *q_base, *cq_base, *sg_base; + void *q_base, *cmb_q_base, *cq_base, *sg_base; rte_iova_t q_base_pa = 0; rte_iova_t cq_base_pa = 0; rte_iova_t sg_base_pa = 0; + rte_iova_t cmb_q_base_pa = 0; size_t page_size = rte_mem_page_size(); int err; @@ -666,19 +667,22 @@ ionic_qcq_alloc(struct ionic_lif *lif, IONIC_PRINT(ERR, "Cannot reserve queue from NIC mem"); return -ENOMEM; } - q_base = (void *) + cmb_q_base = (void *) ((uintptr_t)lif->adapter->bars.bar[2].vaddr + (uintptr_t)lif->adapter->cmb_offset); /* CMB PA is a relative address */ - q_base_pa = lif->adapter->cmb_offset; + cmb_q_base_pa = lif->adapter->cmb_offset; lif->adapter->cmb_offset += q_size; + } else { + cmb_q_base = NULL; + cmb_q_base_pa = 0; } IONIC_PRINT(DEBUG, "Q-Base-PA = %#jx CQ-Base-PA = %#jx " "SG-base-PA = %#jx", q_base_pa, cq_base_pa, sg_base_pa); - ionic_q_map(&new->q, q_base, q_base_pa); + ionic_q_map(&new->q, q_base, q_base_pa, cmb_q_base, cmb_q_base_pa); ionic_cq_map(&new->cq, cq_base, cq_base_pa); *qcq = new; @@ -1583,7 +1587,6 @@ ionic_lif_txq_init(struct ionic_tx_qcq *txq) .flags = rte_cpu_to_le_16(IONIC_QINIT_F_ENA), .intr_index = rte_cpu_to_le_16(IONIC_INTR_NONE), .ring_size = rte_log2_u32(q->num_descs), - .ring_base = rte_cpu_to_le_64(q->base_pa), .cq_ring_base = rte_cpu_to_le_64(cq->base_pa), .sg_ring_base = rte_cpu_to_le_64(q->sg_base_pa), }, @@ -1592,8 +1595,12 @@ ionic_lif_txq_init(struct ionic_tx_qcq *txq) if (txq->flags & IONIC_QCQ_F_SG) ctx.cmd.q_init.flags |= rte_cpu_to_le_16(IONIC_QINIT_F_SG); - if (txq->flags & IONIC_QCQ_F_CMB) + if (txq->flags & IONIC_QCQ_F_CMB) { ctx.cmd.q_init.flags |= rte_cpu_to_le_16(IONIC_QINIT_F_CMB); + ctx.cmd.q_init.ring_base = rte_cpu_to_le_64(q->cmb_base_pa); + } else { + ctx.cmd.q_init.ring_base = rte_cpu_to_le_64(q->base_pa); + } IONIC_PRINT(DEBUG, "txq_init.index %d", q->index); IONIC_PRINT(DEBUG, "txq_init.ring_base 0x%" PRIx64 "", q->base_pa); @@ -1638,7 +1645,6 @@ ionic_lif_rxq_init(struct ionic_rx_qcq *rxq) .flags = rte_cpu_to_le_16(IONIC_QINIT_F_ENA), .intr_index = rte_cpu_to_le_16(IONIC_INTR_NONE), .ring_size = rte_log2_u32(q->num_descs), - .ring_base = rte_cpu_to_le_64(q->base_pa), .cq_ring_base = rte_cpu_to_le_64(cq->base_pa), .sg_ring_base = rte_cpu_to_le_64(q->sg_base_pa), }, @@ -1647,8 +1653,12 @@ ionic_lif_rxq_init(struct ionic_rx_qcq *rxq) if (rxq->flags & IONIC_QCQ_F_SG) ctx.cmd.q_init.flags |= rte_cpu_to_le_16(IONIC_QINIT_F_SG); - if (rxq->flags & IONIC_QCQ_F_CMB) + if (rxq->flags & IONIC_QCQ_F_CMB) { ctx.cmd.q_init.flags |= rte_cpu_to_le_16(IONIC_QINIT_F_CMB); + ctx.cmd.q_init.ring_base = rte_cpu_to_le_64(q->cmb_base_pa); + } else { + ctx.cmd.q_init.ring_base = rte_cpu_to_le_64(q->base_pa); + } IONIC_PRINT(DEBUG, "rxq_init.index %d", q->index); IONIC_PRINT(DEBUG, "rxq_init.ring_base 0x%" PRIx64 "", q->base_pa); diff --git a/drivers/net/ionic/ionic_rxtx.h b/drivers/net/ionic/ionic_rxtx.h index 8537141597..5348395956 100644 --- a/drivers/net/ionic/ionic_rxtx.h +++ b/drivers/net/ionic/ionic_rxtx.h @@ -77,4 +77,60 @@ uint16_t ionic_xmit_pkts_sg(void *tx_queue, struct rte_mbuf **tx_pkts, int ionic_rx_fill_sg(struct ionic_rx_qcq *rxq); +static inline void +ionic_rxq_flush(struct ionic_queue *q) +{ + struct ionic_rxq_desc *desc_base = q->base; + struct ionic_rxq_desc *cmb_desc_base = q->cmb_base; + + if (q->cmb_base) { + if (q->head_idx < q->cmb_head_idx) { + /* copy [cmb_head, num_descs) */ + rte_memcpy((void *)&cmb_desc_base[q->cmb_head_idx], + (void *)&desc_base[q->cmb_head_idx], + (q->num_descs - q->cmb_head_idx) * sizeof(*desc_base)); + /* copy [0, head) */ + rte_memcpy((void *)&cmb_desc_base[0], + (void *)&desc_base[0], + q->head_idx * sizeof(*desc_base)); + } else { + /* copy [cmb_head, head) */ + rte_memcpy((void *)&cmb_desc_base[q->cmb_head_idx], + (void *)&desc_base[q->cmb_head_idx], + (q->head_idx - q->cmb_head_idx) * sizeof(*desc_base)); + } + q->cmb_head_idx = q->head_idx; + } + + ionic_q_flush(q); +} + +static inline void +ionic_txq_flush(struct ionic_queue *q) +{ + struct ionic_txq_desc *desc_base = q->base; + struct ionic_txq_desc *cmb_desc_base = q->cmb_base; + + if (q->cmb_base) { + if (q->head_idx < q->cmb_head_idx) { + /* copy [cmb_head, num_descs) */ + rte_memcpy((void *)&cmb_desc_base[q->cmb_head_idx], + (void *)&desc_base[q->cmb_head_idx], + (q->num_descs - q->cmb_head_idx) * sizeof(*desc_base)); + /* copy [0, head) */ + rte_memcpy((void *)&cmb_desc_base[0], + (void *)&desc_base[0], + q->head_idx * sizeof(*desc_base)); + } else { + /* copy [cmb_head, head) */ + rte_memcpy((void *)&cmb_desc_base[q->cmb_head_idx], + (void *)&desc_base[q->cmb_head_idx], + (q->head_idx - q->cmb_head_idx) * sizeof(*desc_base)); + } + q->cmb_head_idx = q->head_idx; + } + + ionic_q_flush(q); +} + #endif /* _IONIC_RXTX_H_ */ diff --git a/drivers/net/ionic/ionic_rxtx_sg.c b/drivers/net/ionic/ionic_rxtx_sg.c index 1392342463..92e1d6e259 100644 --- a/drivers/net/ionic/ionic_rxtx_sg.c +++ b/drivers/net/ionic/ionic_rxtx_sg.c @@ -166,6 +166,7 @@ ionic_xmit_pkts_sg(void *tx_queue, struct rte_mbuf **tx_pkts, { struct ionic_tx_qcq *txq = tx_queue; struct ionic_queue *q = &txq->qcq.q; + struct ionic_txq_desc *desc_base = q->base; struct ionic_tx_stats *stats = &txq->stats; struct rte_mbuf *mbuf; uint32_t bytes_tx = 0; @@ -173,9 +174,7 @@ ionic_xmit_pkts_sg(void *tx_queue, struct rte_mbuf **tx_pkts, uint64_t then, now, hz, delta; int err; - struct ionic_txq_desc *desc_base = q->base; - if (!(txq->flags & IONIC_QCQ_F_CMB)) - rte_prefetch0(&desc_base[q->head_idx]); + rte_prefetch0(&desc_base[q->head_idx]); rte_prefetch0(IONIC_INFO_PTR(q, q->head_idx)); if (nb_pkts) { @@ -196,8 +195,7 @@ ionic_xmit_pkts_sg(void *tx_queue, struct rte_mbuf **tx_pkts, while (nb_tx < nb_pkts) { uint16_t next_idx = Q_NEXT_TO_POST(q, 1); - if (!(txq->flags & IONIC_QCQ_F_CMB)) - rte_prefetch0(&desc_base[next_idx]); + rte_prefetch0(&desc_base[next_idx]); rte_prefetch0(IONIC_INFO_PTR(q, next_idx)); if (nb_tx + 1 < nb_pkts) { @@ -222,7 +220,7 @@ ionic_xmit_pkts_sg(void *tx_queue, struct rte_mbuf **tx_pkts, if (nb_tx > 0) { rte_wmb(); - ionic_q_flush(q); + ionic_txq_flush(q); txq->last_wdog_cycles = rte_get_timer_cycles(); @@ -458,8 +456,7 @@ ionic_rxq_service_sg(struct ionic_rx_qcq *rxq, uint32_t work_to_do, /* Prefetch 4 x 16B comp */ rte_prefetch0(&cq_desc_base[Q_NEXT_TO_SRVC(cq, 4)]); /* Prefetch 4 x 16B descriptors */ - if (!(rxq->flags & IONIC_QCQ_F_CMB)) - rte_prefetch0(&q_desc_base[Q_NEXT_TO_POST(q, 4)]); + rte_prefetch0(&q_desc_base[Q_NEXT_TO_POST(q, 4)]); /* Clean one descriptor */ ionic_rx_clean_one_sg(rxq, cq_desc, rx_svc); @@ -478,7 +475,8 @@ ionic_rxq_service_sg(struct ionic_rx_qcq *rxq, uint32_t work_to_do, /* Update the queue indices and ring the doorbell */ if (work_done) { - ionic_q_flush(q); + ionic_rxq_flush(q); + rxq->last_wdog_cycles = rte_get_timer_cycles(); rxq->wdog_ms = IONIC_Q_WDOG_MS; } else { @@ -542,7 +540,7 @@ ionic_rx_fill_sg(struct ionic_rx_qcq *rxq) q->head_idx = Q_NEXT_TO_POST(q, 1); } - ionic_q_flush(q); + ionic_rxq_flush(q); return err; } diff --git a/drivers/net/ionic/ionic_rxtx_simple.c b/drivers/net/ionic/ionic_rxtx_simple.c index 00152c885a..f12f66f40c 100644 --- a/drivers/net/ionic/ionic_rxtx_simple.c +++ b/drivers/net/ionic/ionic_rxtx_simple.c @@ -139,6 +139,7 @@ ionic_xmit_pkts(void *tx_queue, struct rte_mbuf **tx_pkts, { struct ionic_tx_qcq *txq = tx_queue; struct ionic_queue *q = &txq->qcq.q; + struct ionic_txq_desc *desc_base = q->base; struct ionic_tx_stats *stats = &txq->stats; struct rte_mbuf *mbuf; uint32_t bytes_tx = 0; @@ -146,9 +147,7 @@ ionic_xmit_pkts(void *tx_queue, struct rte_mbuf **tx_pkts, uint64_t then, now, hz, delta; int err; - struct ionic_txq_desc *desc_base = q->base; - if (!(txq->flags & IONIC_QCQ_F_CMB)) - rte_prefetch0(&desc_base[q->head_idx]); + rte_prefetch0(&desc_base[q->head_idx]); rte_prefetch0(&q->info[q->head_idx]); if (nb_pkts) { @@ -169,8 +168,7 @@ ionic_xmit_pkts(void *tx_queue, struct rte_mbuf **tx_pkts, while (nb_tx < nb_pkts) { uint16_t next_idx = Q_NEXT_TO_POST(q, 1); - if (!(txq->flags & IONIC_QCQ_F_CMB)) - rte_prefetch0(&desc_base[next_idx]); + rte_prefetch0(&desc_base[next_idx]); rte_prefetch0(&q->info[next_idx]); if (nb_tx + 1 < nb_pkts) { @@ -195,7 +193,7 @@ ionic_xmit_pkts(void *tx_queue, struct rte_mbuf **tx_pkts, if (nb_tx > 0) { rte_wmb(); - ionic_q_flush(q); + ionic_txq_flush(q); txq->last_wdog_cycles = rte_get_timer_cycles(); @@ -379,8 +377,7 @@ ionic_rxq_service(struct ionic_rx_qcq *rxq, uint32_t work_to_do, /* Prefetch 4 x 16B comp */ rte_prefetch0(&cq_desc_base[Q_NEXT_TO_SRVC(cq, 4)]); /* Prefetch 4 x 16B descriptors */ - if (!(rxq->flags & IONIC_QCQ_F_CMB)) - rte_prefetch0(&q_desc_base[Q_NEXT_TO_POST(q, 4)]); + rte_prefetch0(&q_desc_base[Q_NEXT_TO_POST(q, 4)]); /* Clean one descriptor */ ionic_rx_clean_one(rxq, cq_desc, rx_svc); @@ -399,7 +396,8 @@ ionic_rxq_service(struct ionic_rx_qcq *rxq, uint32_t work_to_do, /* Update the queue indices and ring the doorbell */ if (work_done) { - ionic_q_flush(q); + ionic_rxq_flush(q); + rxq->last_wdog_cycles = rte_get_timer_cycles(); rxq->wdog_ms = IONIC_Q_WDOG_MS; } else { @@ -463,7 +461,7 @@ ionic_rx_fill(struct ionic_rx_qcq *rxq) q->head_idx = Q_NEXT_TO_POST(q, 1); } - ionic_q_flush(q); + ionic_rxq_flush(q); return err; } -- 2.17.1