From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from NAM02-BL2-obe.outbound.protection.outlook.com (mail-eopbgr750053.outbound.protection.outlook.com [40.107.75.53]) by dpdk.org (Postfix) with ESMTP id D7FFF160; Fri, 18 Jan 2019 11:29:35 +0100 (CET) Received: from DM5PR07CA0070.namprd07.prod.outlook.com (2603:10b6:4:ad::35) by BYAPR07MB4759.namprd07.prod.outlook.com (2603:10b6:a02:f0::23) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.1537.25; Fri, 18 Jan 2019 10:29:34 +0000 Received: from BY2NAM05FT026.eop-nam05.prod.protection.outlook.com (2a01:111:f400:7e52::202) by DM5PR07CA0070.outlook.office365.com (2603:10b6:4:ad::35) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384) id 15.20.1537.26 via Frontend Transport; Fri, 18 Jan 2019 10:29:34 +0000 Authentication-Results: spf=fail (sender IP is 199.233.58.38) smtp.mailfrom=marvell.com; dpdk.org; dkim=none (message not signed) header.d=none;dpdk.org; dmarc=fail action=none header.from=marvell.com; Received-SPF: Fail (protection.outlook.com: domain of marvell.com does not designate 199.233.58.38 as permitted sender) receiver=protection.outlook.com; client-ip=199.233.58.38; helo=CAEXCH02.caveonetworks.com; Received: from CAEXCH02.caveonetworks.com (199.233.58.38) by BY2NAM05FT026.mail.protection.outlook.com (10.152.100.163) with Microsoft SMTP Server (version=TLS1_0, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA) id 15.20.1558.9 via Frontend Transport; Fri, 18 Jan 2019 10:29:33 +0000 Received: from dut1171.mv.qlogic.com (10.112.88.18) by CAEXCH02.caveonetworks.com (10.67.98.110) with Microsoft SMTP Server (TLS) id 14.2.347.0; Fri, 18 Jan 2019 02:29:33 -0800 Received: from dut1171.mv.qlogic.com (localhost [127.0.0.1]) by dut1171.mv.qlogic.com (8.14.7/8.14.7) with ESMTP id x0IATWgK027524; Fri, 18 Jan 2019 02:29:32 -0800 Received: (from root@localhost) by dut1171.mv.qlogic.com (8.14.7/8.14.7/Submit) id x0IATWP9027523; Fri, 18 Jan 2019 02:29:32 -0800 From: Shahed Shaikh To: CC: , Date: Fri, 18 Jan 2019 02:29:29 -0800 Message-ID: <20190118102930.27487-1-shshaikh@marvell.com> X-Mailer: git-send-email 2.12.0 MIME-Version: 1.0 Content-Type: text/plain X-EOPAttributedMessage: 0 X-Matching-Connectors: 131922809741216247; (abac79dc-c90b-41ba-8033-08d666125e47); (abac79dc-c90b-41ba-8033-08d666125e47) X-Forefront-Antispam-Report: CIP:199.233.58.38; IPV:CAL; CTRY:US; EFV:NLI; SFV:NSPM; SFS:(10009020)(396003)(136003)(376002)(346002)(39860400002)(2980300002)(1110001)(1109001)(339900001)(199004)(189003)(8936002)(47776003)(50466002)(5660300001)(8676002)(305945005)(14444005)(51416003)(2906002)(498600001)(26826003)(87636003)(81166006)(6916009)(336012)(48376002)(126002)(476003)(50226002)(42186006)(68736007)(486006)(316002)(26005)(81156014)(16586007)(54906003)(2616005)(1076003)(105606002)(36906005)(106466001)(85426001)(2351001)(36756003)(80596001)(97736004)(69596002)(6666004)(356004)(53936002)(450100002)(86362001)(4326008); DIR:OUT; SFP:1101; SCL:1; SRVR:BYAPR07MB4759; H:CAEXCH02.caveonetworks.com; FPR:; SPF:Fail; LANG:en; PTR:InfoDomainNonexistent; A:1; MX:1; X-Microsoft-Exchange-Diagnostics: 1; BY2NAM05FT026; 1:VrOEJDzaaedxww+MvplbY+Nn3AibOWyd6b5yRHft41sdBGUYij6IyT/vXmxcZCqc1MWbNkfLTU7O7QMAn/KeTlllErXnoLA0wO96jjH+5rYpDbX6iKyiLvIOXqKsaBiHAuTFu32fD5eyp4bYfHzrZw== X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: dd2cbb83-7f03-41c6-9e42-08d67d2fd6d0 X-Microsoft-Antispam: BCL:0; PCL:0; RULEID:(2390118)(7020095)(5600109)(711020)(2017052603328); SRVR:BYAPR07MB4759; X-Microsoft-Exchange-Diagnostics: 1; BYAPR07MB4759; 3:V8qaBndIH9y+4ehau2f5RplGCSaDCfEThl/ue9DJzyPCBW9i8JsONm+q9U+YP5kZERv4mwq1gIxU4yRzktK8PQrTWJl6UOLPRr+sCoHlO+QCsu8VE1EoX1Eij3uJ71NTK76jUwWG5AwoIYOoWbXQpDrSmNFf0uE7ncNY4hONfDTffjY/2ScoC+UOAD7CqNuIntRD6fK4iGneVLVS3B3y7RFwMwMUEDGGZF4CoCSuMZDup0N7Bqa9SdhvGiMUTn9+XF9Vsdgs1u/sm64wLcWLrXFzTTSPZmOngC+HDsOlvzTn69w5TFWZbu1G3Cb+IrPPJfhcCM3g4sqLCD254IlBRQ46EaBG64/i7r8FAEym+LjdJwh9vs+Lqt3Yhc1dhFYf; 25:A1ediP594jyLusQ7zAsHgyCft6N+wD9f58v20w1M3FHSlWkMcHgLhf7vz0yNrQvR7vnO1cptdSgMVBf6AgtZtWT/qPdiwG7nYcPSz+WDZEyr1DqB7cthyYFp1IEjIyGq41glOEEbmzSO2r51G7j7J8H2QbJ2jvgp+LXUmt73hV9VnSt694ELy6JtiPCTnhWBL2H2zyEsZtxKXyXIjJI0lakNlABaGS3JQPJu0OhONqsM1OL38TP9JcQ4KqrQQXcVa90RnpLadDgn/L+rxYM8jvr5PVm7qVN5kyJJNc9ctmx0xSHQrdUW0NRWLfrrZQ6n26Co3Macc5P+nf5nK2nVCA== X-MS-TrafficTypeDiagnostic: BYAPR07MB4759: X-Microsoft-Exchange-Diagnostics: 1; BYAPR07MB4759; 31:6orfS3wyI4JYZvUFJKwuDwZqskzDGO7pmBkS8K0EBCXXu03RoqN8gfn/VFBWtl1YBZ+pFeDnx/f3IXzzofpyct+nuDIwuniRpDVuf8OXuetL+oqFWAsKZSyzHWKRNhL6F2ICRHcw6GPkPWs3ey+RABRo1jnuA1aAPXE5MckfLRTmny7+d6BHrlFU6Jlp0eYKv64P+7qOkXGs2MV5Gv9r2s/OhqHoX2Wg3cRrnTCe8NQ=; 4:7u5NmtC3auwZ492qoo8lr8R48gdTpoo4mGr2PPae9/96gYcsatJMntXvfcKkCxCPgW+3DmJ+IQ4bkrSydxNaYkXy9mErTak0pdURj2OxLXoWAZNMV9IWxdZSw6txmxNzoFpzwm11NWCH/Ng7S46gphw7FMp/20Wom+DCVPk/1ZDeork2PfzpwL8AkbhwgVtBjnHtvHXcn0w4wf0kyUM+Xm4jK283E7kaIAijkl8INkVtzQa4K8eG5fn/xPBuGha2DuC+fE7JOmpx6B9qvX3x8NqFtsHDWEhPtBHTOVykl58= X-Microsoft-Antispam-PRVS: X-Forefront-PRVS: 0921D55E4F X-Microsoft-Exchange-Diagnostics: =?us-ascii?Q?1; BYAPR07MB4759; 23:gW4KEu2XORL9Z5RfYxfW9dmYsi668jKg7vDAZZang?= =?us-ascii?Q?UKGf4Ex9y2PocnQq0Fq6MVyzRTw0DsImNGh8xXYHZQib53G/2FaYPZYVHK1t?= =?us-ascii?Q?8fQrARq82uzeanxbtws2g7yoLaWRJcfD/jlSOo9GB/rjJdMBQm+52Z9QOpgZ?= =?us-ascii?Q?QBdywSCneaDcojJ7in1hhcYdBnX6qGkoZCP/2JJP8q8A7RD+vZwyDEyyP2z3?= =?us-ascii?Q?yohY740xiBihqkjyF3iqVa1JYcZt+g+4zuDxrXaxCqQwD641ZMcNQ1iRxiZ5?= =?us-ascii?Q?+URJzO/JQtZC0F1mfUwiVPQ3LhCn8S2zPHawpVB8iVRhVGAn64x5rYwecxFO?= =?us-ascii?Q?+zhk5oIGIW/c3fcfGtbqs1TIZe4QNuQIbKH197avSxjoaLYaqwyy2uW+fGON?= =?us-ascii?Q?YE0ll/7jDRVLNN0qhdEZrTes3EjYsBYQt1YsELNNFXVzx5Ct62M9my+eP6iu?= =?us-ascii?Q?aigzLd00NehAgGmyzpWWXjWCeUxSJznkO0wnBWmmLulvCqz3hhwa83CGL0+P?= =?us-ascii?Q?Qn8FCwpUYGWyW31eImRfjyK8LYGLBwf+I1hjx3h0jEmpT/7jfQzgTGY6hAqs?= =?us-ascii?Q?LTebjYxKS89z3nntu4RE4SSGOsLuGVZlfcN8iubS1A28yGGrXDCS75yOZU2D?= =?us-ascii?Q?N7Joll6XrrzvmKM6MlRnsYCt6d0K7fBmntaNdfYPK7ka4fZ7uSnhWScsPSJY?= =?us-ascii?Q?+ZTGF+Y8RDscqpaJxIxRw0DgdKSSW0YrL4PJ6FXgmwS4yl9Epr24irzGpJCf?= =?us-ascii?Q?qfoP0cgWTVFvfkahwem2X5J17Zzqn6+vhpBtt7jj17KHiQMVDuG2nDXnbtSW?= =?us-ascii?Q?i+LhYGlYsML2/z/bo1itlOaxzjwceJcUzCq4MfuQzZuFY+ouRju2WC/pY2ir?= =?us-ascii?Q?pWTuA+GH4mjwOud7s6sLnRPkrDPeKtJ5AY+/skpYjs8Sm94xytp0q3yDdfuT?= =?us-ascii?Q?0tozPHlR3Ay/DxylcPEBCW8fk2Ry2FnA/IWDV+KcGIkdcNhsgt8HyrbEUPCV?= =?us-ascii?Q?gtAXeeT9Io/pcCL493QKSi7vDfwoYCjINkfh6HkuApCzAt/TZVVQVAywVpNH?= =?us-ascii?Q?Zi8aooLtEvQrh/2YsTH1ASmKA5czKsZfcxBYimeaL446aZtXa1Q6aH6RIdrq?= =?us-ascii?Q?W7ntyywWTwvDu7QBI7B1s/SVAP6KNmGBQF0siN+vCuE5AvWsui3fyk+IaZFp?= =?us-ascii?Q?289anegNGy+NU1vPgATXYGaOaD05iUgjxnJztkRBtDahLBMr8Zg3nBQxA=3D?= =?us-ascii?Q?=3D?= X-Microsoft-Antispam-Message-Info: GWyHq9uxtdWeEu9bQ5u599cLTz7p4nq4l74wiOHEnnUZa+3ilxNEAnPCHLjGgsnVWPGkOAZd7RYp9/XvaBa9O4O8ynwANsCpWuoRiRijF87B7YBg9qixz3o6g7WJUoGKRB+oXfThIuc5whLhH12QhLHlAgNbSaPbdu7tgHJH22oknZmGpminvWuzg8R9DdQsFmDynTuSN/i7BWG3UO+wemK+6MzeyYPjzCKZlNRumeO3WBxm77YfhCObZtOG2G4r0HpBjyH6aix0POR5BBgHxkieoKyWIELGbcqXukDwMLJmFfz8IcJuiRMIpujCVI6f9JmTjta5T9Nuz7Fd2vjCo8JuW/1lShKXXtfu7WSl/FYPI5bmql7+ZCtmjnPOdphXJj0annDUZ7cmVKMVzMilPHglsqdJqNE51uSXQ/SsuV8= X-Microsoft-Exchange-Diagnostics: 1; BYAPR07MB4759; 6:h0DpRHsb4+bbQTLZEq5hgYNuhPeMuPPn1QAEtZIzTs1SqJ79wdtsrIwXL7/NjlT0LsyH5YO2SyK+4oPxojJkojVKff7OhB6vs44BshjqZSZ4Uo55Z/jA3uUStJnZ1b96/L5UsStzPRrgwOoReDze96jmzbo2mm7LPnZ62eQdaEdzeSy+qGMmpEg+2AEj7q2W3TVRIkfAmcixU1K/mALhU9V1aUOU1E1dL3tVY1NcYoa6nK8dJMj96CwA0CxZgjNK0wX0iqZB6W+aYKzf1uYrtLjCX7IvW+1IDMokIb7Iug+ddj50qa+ghS3uFOBK31FQQX8oMuCGMfBGSsgn0Rvx8uxleJSfqBQyg4KW0S1GMnB5PKSG0PfrkVHzXK4kaVn2Sdiqk2qVv+Gs20oQ/mkCwv6P5BHaFoL1yhR6WY57HTNPVjlSL+UBf4R5900gwJQQCDuZtAVv9vdH0ogsIPXuog==; 5:4hAnfWbv1RQySTu4SC8snCswieh+jYVG3jhf64Lwfj/D2fJ0ynsLhanccmpdAVF2gQO4I3Jpf7RC+tTrusFUjkZjuvHNfFWgd6nwHK5hKoZ5oJB6XlQLtwJ6+OlOk36XeYyj48qSk0OrVOnmhc82znqKtrgvL+2yxYb8xnt5GSL/huB2UKgnOdje04+ZFEOzgrOlOcMisGX4HBmBFPn2Yw==; 7:Di4OZz+8LBM7EAlu9Fya4SNXvOyO6W+iajoFaD1fwRzxcDOhQ+cqhN2dljGK1GhjZGRgoZIJHMnCPMqhRXluKz0wLbIoI00PM5Fv/vdYlv8GUsu/qI833TnnYWsJxy52T8rL2363Pa9KVMC7ZUtLAg== SpamDiagnosticOutput: 1:99 SpamDiagnosticMetadata: NSPM X-MS-Exchange-CrossTenant-OriginalArrivalTime: 18 Jan 2019 10:29:33.7622 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: dd2cbb83-7f03-41c6-9e42-08d67d2fd6d0 X-MS-Exchange-CrossTenant-Id: 5afe0b00-7697-4969-b663-5eab37d5f47e X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=5afe0b00-7697-4969-b663-5eab37d5f47e; Ip=[199.233.58.38]; Helo=[CAEXCH02.caveonetworks.com] X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: BYAPR07MB4759 Subject: [dpdk-dev] [PATCH 1/2] net/qede: fix performance bottleneck in Rx path X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 18 Jan 2019 10:29:36 -0000 Allocating replacement buffer per received packet is expensive. Instead, process received packets first and allocate replacement buffers in bulk later. This improves performance by ~25% in terms of PPS on AMD platforms. Fixes: 2ea6f76aff40 ("qede: add core driver") Cc: stable@dpdk.org Signed-off-by: Shahed Shaikh --- drivers/net/qede/qede_rxtx.c | 97 +++++++++++++++++++++++++++++++++----------- drivers/net/qede/qede_rxtx.h | 2 + 2 files changed, 75 insertions(+), 24 deletions(-) diff --git a/drivers/net/qede/qede_rxtx.c b/drivers/net/qede/qede_rxtx.c index 0e33be1..684c4ae 100644 --- a/drivers/net/qede/qede_rxtx.c +++ b/drivers/net/qede/qede_rxtx.c @@ -35,6 +35,52 @@ static inline int qede_alloc_rx_buffer(struct qede_rx_queue *rxq) return 0; } +#define QEDE_MAX_BULK_ALLOC_COUNT 512 + +static inline int qede_alloc_rx_bulk_mbufs(struct qede_rx_queue *rxq, int count) +{ + void *obj_p[QEDE_MAX_BULK_ALLOC_COUNT] __rte_cache_aligned; + struct rte_mbuf *mbuf = NULL; + struct eth_rx_bd *rx_bd; + dma_addr_t mapping; + int i, ret = 0; + uint16_t idx; + + idx = rxq->sw_rx_prod & NUM_RX_BDS(rxq); + + if (count > QEDE_MAX_BULK_ALLOC_COUNT) + count = QEDE_MAX_BULK_ALLOC_COUNT; + + ret = rte_mempool_get_bulk(rxq->mb_pool, obj_p, count); + if (unlikely(ret)) { + PMD_RX_LOG(ERR, rxq, + "Failed to allocate %d rx buffers " + "sw_rx_prod %u sw_rx_cons %u mp entries %u free %u", + count, idx, rxq->sw_rx_cons & NUM_RX_BDS(rxq), + rte_mempool_avail_count(rxq->mb_pool), + rte_mempool_in_use_count(rxq->mb_pool)); + return -ENOMEM; + } + + for (i = 0; i < count; i++) { + mbuf = obj_p[i]; + if (likely(i < count - 1)) + rte_prefetch0(obj_p[i + 1]); + + idx = rxq->sw_rx_prod & NUM_RX_BDS(rxq); + rxq->sw_rx_ring[idx].mbuf = mbuf; + rxq->sw_rx_ring[idx].page_offset = 0; + mapping = rte_mbuf_data_iova_default(mbuf); + rx_bd = (struct eth_rx_bd *) + ecore_chain_produce(&rxq->rx_bd_ring); + rx_bd->addr.hi = rte_cpu_to_le_32(U64_HI(mapping)); + rx_bd->addr.lo = rte_cpu_to_le_32(U64_LO(mapping)); + rxq->sw_rx_prod++; + } + + return 0; +} + /* Criterias for calculating Rx buffer size - * 1) rx_buf_size should not exceed the size of mbuf * 2) In scattered_rx mode - minimum rx_buf_size should be @@ -1131,7 +1177,7 @@ qede_reuse_page(__rte_unused struct qede_dev *qdev, struct qede_rx_queue *rxq, struct qede_rx_entry *curr_cons) { struct eth_rx_bd *rx_bd_prod = ecore_chain_produce(&rxq->rx_bd_ring); - uint16_t idx = rxq->sw_rx_cons & NUM_RX_BDS(rxq); + uint16_t idx = rxq->sw_rx_prod & NUM_RX_BDS(rxq); struct qede_rx_entry *curr_prod; dma_addr_t new_mapping; @@ -1364,7 +1410,6 @@ qede_recv_pkts(void *p_rxq, struct rte_mbuf **rx_pkts, uint16_t nb_pkts) uint8_t bitfield_val; #endif uint8_t tunn_parse_flag; - uint8_t j; struct eth_fast_path_rx_tpa_start_cqe *cqe_start_tpa; uint64_t ol_flags; uint32_t packet_type; @@ -1373,6 +1418,7 @@ qede_recv_pkts(void *p_rxq, struct rte_mbuf **rx_pkts, uint16_t nb_pkts) uint8_t offset, tpa_agg_idx, flags; struct qede_agg_info *tpa_info = NULL; uint32_t rss_hash; + int rx_alloc_count = 0; hw_comp_cons = rte_le_to_cpu_16(*rxq->hw_cons_ptr); sw_comp_cons = ecore_chain_get_cons_idx(&rxq->rx_comp_ring); @@ -1382,6 +1428,25 @@ qede_recv_pkts(void *p_rxq, struct rte_mbuf **rx_pkts, uint16_t nb_pkts) if (hw_comp_cons == sw_comp_cons) return 0; + /* Allocate buffers that we used in previous loop */ + if (rxq->rx_alloc_count) { + if (unlikely(qede_alloc_rx_bulk_mbufs(rxq, + rxq->rx_alloc_count))) { + struct rte_eth_dev *dev; + + PMD_RX_LOG(ERR, rxq, + "New buffer allocation failed," + "dropping incoming packetn"); + dev = &rte_eth_devices[rxq->port_id]; + dev->data->rx_mbuf_alloc_failed += + rxq->rx_alloc_count; + rxq->rx_alloc_errors += rxq->rx_alloc_count; + return 0; + } + qede_update_rx_prod(qdev, rxq); + rxq->rx_alloc_count = 0; + } + while (sw_comp_cons != hw_comp_cons) { ol_flags = 0; packet_type = RTE_PTYPE_UNKNOWN; @@ -1553,16 +1618,7 @@ qede_recv_pkts(void *p_rxq, struct rte_mbuf **rx_pkts, uint16_t nb_pkts) rx_mb->hash.rss = rss_hash; } - if (unlikely(qede_alloc_rx_buffer(rxq) != 0)) { - PMD_RX_LOG(ERR, rxq, - "New buffer allocation failed," - "dropping incoming packet\n"); - qede_recycle_rx_bd_ring(rxq, qdev, fp_cqe->bd_num); - rte_eth_devices[rxq->port_id]. - data->rx_mbuf_alloc_failed++; - rxq->rx_alloc_errors++; - break; - } + rx_alloc_count++; qede_rx_bd_ring_consume(rxq); if (!tpa_start_flg && fp_cqe->bd_num > 1) { @@ -1574,17 +1630,9 @@ qede_recv_pkts(void *p_rxq, struct rte_mbuf **rx_pkts, uint16_t nb_pkts) if (qede_process_sg_pkts(p_rxq, seg1, num_segs, pkt_len - len)) goto next_cqe; - for (j = 0; j < num_segs; j++) { - if (qede_alloc_rx_buffer(rxq)) { - PMD_RX_LOG(ERR, rxq, - "Buffer allocation failed"); - rte_eth_devices[rxq->port_id]. - data->rx_mbuf_alloc_failed++; - rxq->rx_alloc_errors++; - break; - } - rxq->rx_segs++; - } + + rx_alloc_count += num_segs; + rxq->rx_segs += num_segs; } rxq->rx_segs++; /* for the first segment */ @@ -1626,7 +1674,8 @@ qede_recv_pkts(void *p_rxq, struct rte_mbuf **rx_pkts, uint16_t nb_pkts) } } - qede_update_rx_prod(qdev, rxq); + /* Request number of bufferes to be allocated in next loop */ + rxq->rx_alloc_count = rx_alloc_count; rxq->rcv_pkts += rx_pkt; diff --git a/drivers/net/qede/qede_rxtx.h b/drivers/net/qede/qede_rxtx.h index 454daa0..5b249cb 100644 --- a/drivers/net/qede/qede_rxtx.h +++ b/drivers/net/qede/qede_rxtx.h @@ -192,6 +192,8 @@ struct qede_rx_queue { uint16_t queue_id; uint16_t port_id; uint16_t rx_buf_size; + uint16_t rx_alloc_count; + uint16_t unused; uint64_t rcv_pkts; uint64_t rx_segs; uint64_t rx_hw_errors; -- 2.7.4