From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from NAM03-DM3-obe.outbound.protection.outlook.com (mail-dm3nam03on0084.outbound.protection.outlook.com [104.47.41.84]) by dpdk.org (Postfix) with ESMTP id 9DAF37D30 for ; Sun, 28 May 2017 22:02:11 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=CAVIUMNETWORKS.onmicrosoft.com; s=selector1-cavium-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version; bh=605ww+vWmGtumYftT/Id8Am02B6aEUMAhCPK0Qv9ehs=; b=mKiJpk4wM32hg76LlXt1Ug+iZBzw/UvJN59qmNLXESgu+fC4tnGnZwzXsXp2uJA9HOLocQx1pAYdykwRONXKqOzp1DZp4zekB3mOjyjz7ZY1P+vXaGO0m0g1aU3GzpjafGZmMSHVqx05EJJNOR3IE01oR9poHzU+5fI5r1dOlwY= Authentication-Results: dpdk.org; dkim=none (message not signed) header.d=none;dpdk.org; dmarc=none action=none header.from=caviumnetworks.com; Received: from jerin.domain.name (111.93.218.67) by CY1PR0701MB1725.namprd07.prod.outlook.com (10.163.21.14) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256_P256) id 15.1.1124.9; Sun, 28 May 2017 20:02:06 +0000 From: Jerin Jacob To: dev@dpdk.org Cc: bruce.richardson@intel.com, harry.van.haaren@intel.com, hemant.agrawal@nxp.com, gage.eads@intel.com, nipun.gupta@nxp.com, narender.vangati@intel.com, nikhil.rao@intel.com, gprathyusha@caviumnetworks.com, Jerin Jacob Date: Mon, 29 May 2017 01:28:46 +0530 Message-Id: <20170528195854.6064-26-jerin.jacob@caviumnetworks.com> X-Mailer: git-send-email 2.13.0 In-Reply-To: <20170528195854.6064-1-jerin.jacob@caviumnetworks.com> References: <20170528195854.6064-1-jerin.jacob@caviumnetworks.com> MIME-Version: 1.0 Content-Type: text/plain X-Originating-IP: [111.93.218.67] X-ClientProxiedBy: BMXPR01CA0003.INDPRD01.PROD.OUTLOOK.COM (10.174.214.141) To CY1PR0701MB1725.namprd07.prod.outlook.com (10.163.21.14) X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: CY1PR0701MB1725: X-MS-Office365-Filtering-Correlation-Id: eec119b9-71a5-4380-fa36-08d4a6046d19 X-Microsoft-Antispam: UriScan:; BCL:0; PCL:0; RULEID:(22001)(201703131423075)(201703031133081); SRVR:CY1PR0701MB1725; X-Microsoft-Exchange-Diagnostics: 1; CY1PR0701MB1725; 3:vP9dw9bhyVAOvCQWsFnLD7b2OZCig/RHOqJeXZQ4DC1lLyg5yn/P7Eu7Kw2RAzQYoCvqkkbdoHJfOb43bCb9NGkdCf/VM2uNvLvv7SkAJuYRB1kKbPImC/UTQJY5DqpqRecMTegRyjqiWg5LZj0y3DDdRhx2Av89xYeqOy6H4JKev8Ibvi/jN7KeRvwCunWhSJtuScS7/H05SyWu69bOzd+J/f88B6f2yRV7u8H4YhfNGVlp8PwYGb/Rw1i9NzslIRtoRv2SSMZr3XyKCLklrEcIa8DQ+PUOKdpKwKoX5o5J6Pqg1Os6gTziTBPqHFQ+AwvHceHVQIetPD81wy/mIA==; 25:8RfdGIgr7xSIjcKsqYm56AXspAFkqHy2Pv/qvlam1UVZzc1M2nlC2x8HnGWcB47j9reK0+inYV2VfYvwzjZ1XQdyt6j3Fh39at8qw1VdOqRpQwgEnuCGnaAxcp5ukv075KljI+gKKS/UKuSTopTpw7jGVQ63fbu3xSlRP05wB2eWYUhES615dumkQZpzIuKWJ50hSE5NAS3STxnP+P8MDUGlI+Gmi0Z2dopy5zlhbctj9XAg16zxBUaWZssO6aas1eaRhSnp1VQy8skYRdYK9DrS+seIOgboNWFd9lcJv9hI4XD2PKBgjVrplOxZ8xVgihr4AejhBgnmdWdjtLzxLimjbG3c0URfEzirH0jN39NKMKG3qu/ps8cLQLaD41xqC/62iSKDQrpKGnAv+NX/l66eqCmW/AjgtMb/oX8QN+RmO8E5GdDGLYCXbBqX7LV6xuMSnSYrm9R8VAO9GUtN5RicRbk9V3vO3doQ2jV3+kY= X-Microsoft-Exchange-Diagnostics: 1; CY1PR0701MB1725; 31:BYFZ/WB0xP0NSjxdl5pETrGqU1cwuy9uH8YwgDGu5sKMaATnEW3u4JvSaiqU5OOVjCdy1eEfQTkaOsXhLS7d+yacIiymfAFayXNYNp8RcScXuTqbMg4tiT9z6TNWAkBFgzJQyw6Z+w4yWsw5SEO2waIAwBo7aRGIFmsp0g+0eMKFkoBpF1u9hyiqDi2HJ4XEJtl58g15fVhw8NWTa9zV43xp88Vu+YRqYet9q6XZTwc=; 20:RtkmQCL585aBc548BcgcnzvKS170nTdt8Bmwskxg773GCQBlF5cRGy0dSPrmowQMfu/PsS+tkP46yHQVZfVcAc6V5WeIwzsAjlacx1uRycmvxs7a4D5lsPkBV2Dj4jtBmIQ8uesGBTbN3WBgfbewOrbeTlvuQwpmGCsrmGEFN/EtzqUG3RYmtbD/eGMtfz0cPXUJwqCqPKz37eBRPds16tUbVZOZQsKiecVAbBIDrg56xqVcVqiasQqDt2RTWz9gd4LZS5cxzv7NbgXGyMszd4tmMFn+py3Gn5BrLVF5GQotbBvayHeMgh+4cc1a7+Dhh+9QceYj02MT8LnZqcvjFg0aXJSYGWQFL9wJB4fRnMOUdMKZ3HwBm1jzcYRpkaLH73au8WOhux8I6ZDWcdzBgf6eOb1JMFS10PPZd9DAlJcuQfmWP52CO5qbv3kIJZAdKDcJ5wCDiMGLcFkHB5jF0kZH8OjNcRqmrFGLr88znGYV6hGNumgujIZrYNNL9TbGe7k5r266XhTMoHKiM8fei+ShaIWEz7/150LU0zF/YjGo06vhwI/Aj0oAIsl8HPTnC0EiRQGzleP7O7sNXO5ofDmdtILgejNmsGKRVoJnRYQ= X-Microsoft-Antispam-PRVS: X-Exchange-Antispam-Report-Test: UriScan:; X-Exchange-Antispam-Report-CFA-Test: BCL:0; PCL:0; RULEID:(100000700073)(100105000095)(100000701073)(100105300095)(100000702073)(100105100095)(6040450)(601004)(2401047)(5005006)(8121501046)(3002001)(10201501046)(93006095)(100000703073)(100105400095)(6041248)(20161123562025)(20161123555025)(20161123560025)(20161123564025)(201703131423075)(201702281528075)(201703061421075)(201703061406153)(20161123558100)(6072148)(100000704073)(100105200095)(100000705073)(100105500095); SRVR:CY1PR0701MB1725; BCL:0; PCL:0; RULEID:(100000800073)(100110000095)(100000801073)(100110300095)(100000802073)(100110100095)(100000803073)(100110400095)(100000804073)(100110200095)(100000805073)(100110500095); SRVR:CY1PR0701MB1725; X-Microsoft-Exchange-Diagnostics: =?us-ascii?Q?1; CY1PR0701MB1725; 4:JJhSH7jjUNz06rcev+H+UhL11I4qhEWiKAm1RsHz?= =?us-ascii?Q?JRcRxrzXSS4hJTHhAZSRbOZlR7GxioZ6vp64fv6HVAt2HjfokfRYsMeqISqi?= =?us-ascii?Q?iajtES23Dxz0iQ+7BaCkKANEX0vEBDcK2L0d4msAwrWyYPIc2yg9qt8yHAbx?= =?us-ascii?Q?baFS7eLP6UGIVMRHVGE2UYVP3kaX2JdSCEDWcrMOx5+G3PVsn1kdNoiM4uRV?= =?us-ascii?Q?w4QP8FGrN8yf2h4D3LATyTywW5Kv6D3PyMiNoBkjiRpFvBnO3I9W0Z6juAAP?= =?us-ascii?Q?StJKs2SEsJ4E86PYoAW1qqVecXggiH4BRhcvUPVJTdzarzCM1y3Allb/MP5w?= =?us-ascii?Q?3XiXC8NUwfuiZX9M/VoTRgiNaas25Vidi2G59jaY6PcJQYM6bxl+Qsu1Scvd?= =?us-ascii?Q?Vkvcw7lJDQ+o+Cg7GwDIMRJUh/qYUnNENPF1Dr9b+ZyCNFTxxpIlBS7rrSsK?= =?us-ascii?Q?vz1XUc6Ntv7ybfrXfkLjtBecIFYJHRHRei9Y3kOFEUagcn28HxSsOT9sZoxC?= =?us-ascii?Q?Wq2fGXcvc+859L5gn4+aa6BBYqx30gjYFXsSsoUY0Rhinbuwq3UUo0jP/crs?= =?us-ascii?Q?/iHr5FCuO6PkwCfrxEKBp0Ce60sfqOKBLgb69pn8dy/+bw/pDVdGh5cx2p5E?= =?us-ascii?Q?h8+77VS8S9Ap+0J8oR+IIaPcAmwbJTVuLdDiVG7+aw6GFFHanYYP3EK/q5Y0?= =?us-ascii?Q?XzcP99fFcJvq294vPc1p83izh5vwQzYQBIraLzERTrDsUHa/1FDgqldraCtU?= =?us-ascii?Q?GK+ZQmGhXowld1PNun9XtANzbhvmqpTSMABm4tww6FX6UMnlVoLBWHNZ1eKa?= =?us-ascii?Q?yNLchir2CbxsIRwzFum1jH3SfxyXzhx2lmfhGVWLYA4gBt7je4uOrKK7fqgd?= =?us-ascii?Q?NNSZklPlXuGV5GlYjCxG38d8gtoM7/1TAFJjEOzMnQebxo4tWivGpBzEhSkf?= =?us-ascii?Q?v3fFnHS85MBVz5FrxRSeZrUlZIL1Wm7zeAIW6v38iJfy2jGRGZbA6MkJc0U7?= =?us-ascii?Q?sfrsy6Y3nG0v/vS3ytmUCRvPJyHCRviJrxQqsO9bsJTffqlDwpeYRVUZwtgf?= =?us-ascii?Q?PwJ91YWceQsdz1xEEw0+tMffQV5Y1Fr7+rpoX+4fuoO7T6o9rAveR0nJzD8F?= =?us-ascii?Q?N7pMIaezXdg=3D?= X-Forefront-PRVS: 03218BFD9F X-Forefront-Antispam-Report: SFV:NSPM; SFS:(10009020)(4630300001)(6009001)(39450400003)(39850400002)(39410400002)(39840400002)(39400400002)(81166006)(6486002)(66066001)(4326008)(50986999)(76176999)(50226002)(189998001)(72206003)(36756003)(6506006)(38730400002)(5003940100001)(5009440100003)(110136004)(107886003)(42186005)(53416004)(7736002)(8676002)(48376002)(50466002)(1076002)(3846002)(6116002)(6512007)(2361001)(305945005)(2906002)(6666003)(478600001)(25786009)(5660300001)(2351001)(6916009)(42882006)(2950100002)(33646002)(8656002)(47776003)(53936002); DIR:OUT; SFP:1101; SCL:1; SRVR:CY1PR0701MB1725; H:jerin.domain.name; FPR:; SPF:None; MLV:sfv; LANG:en; X-Microsoft-Exchange-Diagnostics: =?us-ascii?Q?1; CY1PR0701MB1725; 23:RsxIb0mUwb+nsm2L36yOpHSkYQjekub+jejIJGk?= =?us-ascii?Q?skSKsG6tf9Yrge5w8pZDv79IS54JfiJCITOYB3EVv5O/qSWqab+Ml61BbVh8?= =?us-ascii?Q?XSU9N7OgF0xeHNECK4+Hd0ePiiS1uIirq7ddoNJYmPymkeasXm8VsYPVv3Ep?= =?us-ascii?Q?0VZjNlnimd6La/42P+wqQj4YyBHWKzAJHiAI3ZUP5TKZyePDV2b9ozavjYUs?= =?us-ascii?Q?3t5uuobmOlCQI4TtYus71MH/PBE38FP/4rXNjQjODjmcAZ57LdXbM4/jTQyI?= =?us-ascii?Q?rC2YRpB8GOCpBWq/+0s4zoh1RDqmH0Dpin23OnEDQBCuwEK/UWGdDO0YVfLz?= =?us-ascii?Q?fiwu8FPB1Ggm6jIi72cjOjDY9FRebaD/Mc2FqDJFBVe+6QM35TBxwZJ6Yfe7?= =?us-ascii?Q?aGMyu+DyKgEsrdPhx/NVXAdehnzo0/Qx82pzMB26Hg/rqezPDyDbxjYVnAA0?= =?us-ascii?Q?kGClInjoxYBF6uszTfsuwhqsLkBU594LkyJUIS6DPa8d1a/PqPv+ZDc5vgYw?= =?us-ascii?Q?1izxbt2Wi+5i+D4Eq0JPo3dwkHWN4IjfFlSrxiBZCzBQrLuCB4un1r+9mR45?= =?us-ascii?Q?VVyDkcGQcEcD4jIR4hHCMOMZDtbywgQSZV5dUrRQ8rBM1ciOCUrmoMzPsYEs?= =?us-ascii?Q?o90wrhQ/2nIyMTNCq5NFmMgozBQihK0AVS40a/aclVQinH0wX44VY/sQiEyX?= =?us-ascii?Q?u4Yvp9Tleu/3n4fBnO7TqhfFUO73mPRS2BfGHjRYL6D+JjOBo67L4/tb7AaF?= =?us-ascii?Q?S2VdFtUAJOPS+ykZDsUnWZCm7X3v3cVtVP4O8ntc4XaM8u9J/bT/yl/ZVYZU?= =?us-ascii?Q?cLWZXsK9Eb9LCEMJNBkDlEbB1ijiHxt6H2lMDn16G+11mdlhIhAfrZE7wcxt?= =?us-ascii?Q?hZGDxLeUW8wL05PBfSnNqotpDK5mm/w82G6q5xn0F2HgWqnvfQ65qowk5VpH?= =?us-ascii?Q?SdntZ0dqsqsCRukoqzJ5rW79GSpXGfGO+jPdGXC70qNb8JsDYgcqSxgIzTQz?= =?us-ascii?Q?NKIgSbonqn22xcJmlURdumCWPlgif3XyIf0fm67NOBcA4N0HxDs5MUikTFjZ?= =?us-ascii?Q?VEfGp3LoY9ibWyFGHWMg2dwwt/nedpwI6ysIKWVlVLIsm9Ngj+QKh4ZsW4oT?= =?us-ascii?Q?LcVWRW+LvCe6KifqYtss1u5ENm9hE/FYvnXgzD+2iS3OSFpi4tWXhvFomoL5?= =?us-ascii?Q?8YkmxHK/Jf8MH8Wd0lD6BcQZ17AFAJc+DyACj?= X-Microsoft-Exchange-Diagnostics: 1; CY1PR0701MB1725; 6:rl94s508jvru1uZUJBNYAaUtSRoH/4CQebPXgHjiehCHcXKOSjowRI0ta6o6zUJlOub3hcyrDrFXYasrQEQ/jwmoGBK1j3KzfvX+e4d/ll9Z4Pzku43mdSP7CMooB+0cxAliCXRlPmqnvu7dVu9XlbhhkKcJQyaladg+L5qNNA7ooloikden5ggqCGWhx+5lY6Vg9fsa8qvKUb89a2KQkp82sl9+pTZr78U26D4BD/fZ2TnCGTnj+6IhfBDlJd8dXLpVdWzvVh+tqaEOVYgIyZlqwZJcWn6qgeOiff0RiPpasPl8+cQBppj/lMh7vjvwbkuI8AdpGSRYanJHiBk54llSl1uGqJIddEyE80k+pzL+CNlBB0HCiEHJyZ3RJAGrEr3zwBXpcuKf3+7Kx01n295mUMlCaOmRqLA5NW9LbMavDzV1MSXPQU5qHl/29ns8vpwflZXVGwFd2UTBSaGHdks57Zy+L7iS+/q4aKeR8Zfw7LryL5xf/9iSszPpmoy79wpXPJSHDqqqDAaG554JAQ== X-Microsoft-Exchange-Diagnostics: 1; CY1PR0701MB1725; 5:+BvOUdGRFJXPACj/ybLBz4THgfID/DU4e2SAggiagFtTGGc9bV2Kcd94GMrSq2XGQXreyLgefX0J3kjdxgJFV13krVDrbqS1qtE6Dk87kUXHEr4Ws8TF+QNhUrrP3mtvBS9nVfbSKe0OfxWZr13GXVCCUMnsQM3wgW93gKJiIjk+l2I4PVclkNo2mb1borJt/twZjPCu24IqFjzIuOiDUtF1wWS2Mdtv/h1uzwaOD4Tj70nlc0MxM0QCbWNAzoMTYUYrQhmjms03a1FiHZ7B1fMC3W0LEKRwx9DrHvPc6iR3tU8r6imAvSxmy2MU8l7DhE27GytdXt3Nur/2Cj1Vp3UPXNVj+WUasdeWR4P2hum0eFrWc9zRXaE7fsu93W3/9vfBnbEkwwEQAs8F+u7dlx7x8CI6m1H7Er9evQdTdfni/JDfP6dFzedmNeumjOltX2XrGz1jN9bR3NjTbHtVmHVgrO1sE0pU0Cvc5yKK4Zxir97OVIZIRh2EFVNjhlBj; 24:IliLopsG3HmKey3AuDyBqBlxIl18i0e+Oh3gOrthgLJO3yb7DANiFsLG6WzxQwxdrAvUOm0jGlDOObRPZpCOg0tsFdiZgcDXtMVB+KMdA+A= SpamDiagnosticOutput: 1:99 SpamDiagnosticMetadata: NSPM X-Microsoft-Exchange-Diagnostics: 1; CY1PR0701MB1725; 7:FH/7Oouzuyy4RK3b9Dqz3OLBs9eRnl3MwEigrXaRJcnXbgoW5T7tCKz8CaaY3uVuNxANjpROY6/NYWZed1vxa/nqt+eLDlkzlIDR5/o8MHvLmbkzudTj2T8rqBMVLT1oF36Hy4sfrKxkZamWbJQ7QLI9FgnowsYCzSv2aF6DJNC/NxD90ZVjRNmGPIsW02FoYgw9KiHnE+e72TfpYmVVYWNBDPivkjjBuBq8z5iNgTfrDlJMEjXcEvljX0lmK/XEzO/KzbieVTFzKhUUx9sybKPqd3CkxBGO9iV5CWu4RvKCvwwh/X6+nRA8fTtkBw1mSTEcNZ9RY+HkxDnaWXee1A== X-OriginatorOrg: caviumnetworks.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 28 May 2017 20:02:06.9902 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-Transport-CrossTenantHeadersStamped: CY1PR0701MB1725 Subject: [dpdk-dev] [PATCH 25/33] app/testeventdev: perf queue: add worker functions X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 28 May 2017 20:02:12 -0000 Signed-off-by: Jerin Jacob --- app/test-eventdev/test_perf_common.h | 60 +++++++++++++++ app/test-eventdev/test_perf_queue.c | 137 +++++++++++++++++++++++++++++++++++ 2 files changed, 197 insertions(+) diff --git a/app/test-eventdev/test_perf_common.h b/app/test-eventdev/test_perf_common.h index f8246953a..9888e5078 100644 --- a/app/test-eventdev/test_perf_common.h +++ b/app/test-eventdev/test_perf_common.h @@ -86,6 +86,66 @@ struct perf_elt { uint64_t timestamp; } __rte_cache_aligned; +#define BURST_SIZE 16 + +#define PERF_WORKER_INIT\ + struct worker_data *w = arg;\ + struct test_perf *t = w->t;\ + struct evt_options *opt = t->opt;\ + const uint8_t dev = w->dev_id;\ + const uint8_t port = w->port_id;\ + uint8_t *const sched_type_list = &t->sched_type_list[0];\ + struct rte_mempool *const pool = t->pool;\ + const uint8_t nb_stages = t->opt->nb_stages;\ + const uint8_t laststage = nb_stages - 1;\ + uint8_t cnt = 0;\ + void *bufs[16] __rte_cache_aligned;\ + int const sz = RTE_DIM(bufs);\ + if (opt->verbose_level > 1)\ + printf("%s(): lcore %d dev_id %d port=%d\n", __func__,\ + rte_lcore_id(), dev, port) + +static inline __attribute__((always_inline)) int +perf_process_last_stage(struct rte_mempool *const pool, + struct rte_event *const ev, struct worker_data *const w, + void *bufs[], int const buf_sz, uint8_t count) +{ + bufs[count++] = ev->event_ptr; + w->processed_pkts++; + rte_smp_wmb(); + + if (unlikely(count == buf_sz)) { + count = 0; + rte_mempool_put_bulk(pool, bufs, buf_sz); + } + return count; +} + +static inline __attribute__((always_inline)) uint8_t +perf_process_last_stage_latency(struct rte_mempool *const pool, + struct rte_event *const ev, struct worker_data *const w, + void *bufs[], int const buf_sz, uint8_t count) +{ + uint64_t latency; + struct perf_elt *const m = ev->event_ptr; + + bufs[count++] = ev->event_ptr; + w->processed_pkts++; + + if (unlikely(count == buf_sz)) { + count = 0; + latency = rte_get_timer_cycles() - m->timestamp; + rte_mempool_put_bulk(pool, bufs, buf_sz); + } else { + latency = rte_get_timer_cycles() - m->timestamp; + } + + w->latency += latency; + rte_smp_wmb(); + return count; +} + + static inline int perf_nb_event_ports(struct evt_options *opt) { diff --git a/app/test-eventdev/test_perf_queue.c b/app/test-eventdev/test_perf_queue.c index 352240c7b..811f7f78d 100644 --- a/app/test-eventdev/test_perf_queue.c +++ b/app/test-eventdev/test_perf_queue.c @@ -41,6 +41,142 @@ perf_queue_nb_event_queues(struct evt_options *opt) return evt_nr_active_lcores(opt->plcores) * opt->nb_stages; } +static inline __attribute__((always_inline)) void +mark_fwd_latency(struct rte_event *const ev, + const uint8_t nb_stages) +{ + if (unlikely((ev->queue_id % nb_stages) == 0)) { + struct perf_elt *const m = ev->event_ptr; + + m->timestamp = rte_get_timer_cycles(); + } +} + +static inline __attribute__((always_inline)) void +fwd_event(struct rte_event *const ev, uint8_t *const sched_type_list, + const uint8_t nb_stages) +{ + ev->queue_id++; + ev->sched_type = sched_type_list[ev->queue_id % nb_stages]; + ev->op = RTE_EVENT_OP_FORWARD; + ev->event_type = RTE_EVENT_TYPE_CPU; +} + +static int +perf_queue_worker(void *arg, const int enable_fwd_latency) +{ + PERF_WORKER_INIT; + struct rte_event ev; + + while (t->done == false) { + uint16_t event = rte_event_dequeue_burst(dev, port, &ev, 1, 0); + + if (!event) { + rte_pause(); + continue; + } + if (enable_fwd_latency) + /* first q in pipeline, mark timestamp to compute fwd latency */ + mark_fwd_latency(&ev, nb_stages); + + /* last stage in pipeline */ + if (unlikely((ev.queue_id % nb_stages) == laststage)) { + if (enable_fwd_latency) + cnt = perf_process_last_stage_latency(pool, + &ev, w, bufs, sz, cnt); + else + cnt = perf_process_last_stage(pool, + &ev, w, bufs, sz, cnt); + } else { + fwd_event(&ev, sched_type_list, nb_stages); + while (rte_event_enqueue_burst(dev, port, &ev, 1) != 1) + rte_pause(); + } + } + return 0; +} + +static int +perf_queue_worker_burst(void *arg, const int enable_fwd_latency) +{ + PERF_WORKER_INIT; + uint16_t i; + /* +1 to avoid prefetch out of array check */ + struct rte_event ev[BURST_SIZE + 1]; + + while (t->done == false) { + uint16_t const nb_rx = rte_event_dequeue_burst(dev, port, ev, + BURST_SIZE, 0); + + if (!nb_rx) { + rte_pause(); + continue; + } + + for (i = 0; i < nb_rx; i++) { + if (enable_fwd_latency) { + rte_prefetch0(ev[i+1].event_ptr); + /* first queue in pipeline. + * mark time stamp to compute fwd latency + */ + mark_fwd_latency(&ev[i], nb_stages); + } + /* last stage in pipeline */ + if (unlikely((ev[i].queue_id % nb_stages) == + laststage)) { + if (enable_fwd_latency) + cnt = perf_process_last_stage_latency( + pool, &ev[i], w, bufs, sz, cnt); + else + cnt = perf_process_last_stage(pool, + &ev[i], w, bufs, sz, cnt); + + ev[i].op = RTE_EVENT_OP_RELEASE; + } else { + fwd_event(&ev[i], sched_type_list, nb_stages); + } + } + + uint16_t enq; + + enq = rte_event_enqueue_burst(dev, port, ev, nb_rx); + while (enq < nb_rx) { + enq += rte_event_enqueue_burst(dev, port, + ev + enq, nb_rx - enq); + } + } + return 0; +} + +static int +worker_wrapper(void *arg) +{ + struct worker_data *w = arg; + struct evt_options *opt = w->t->opt; + + /* FIXME: probe through device capability */ + const int burst = 1; + const int fwd_latency = opt->fwd_latency; + + /* allow compiler to optimize */ + if (!burst && !fwd_latency) + return perf_queue_worker(arg, 0); + else if (!burst && fwd_latency) + return perf_queue_worker(arg, 1); + else if (burst && !fwd_latency) + return perf_queue_worker_burst(arg, 0); + else if (burst && fwd_latency) + return perf_queue_worker_burst(arg, 1); + + rte_panic("invalid worker\n"); +} + +static int +perf_queue_launch_lcores(struct evt_test *test, struct evt_options *opt) +{ + return perf_launch_lcores(test, opt, worker_wrapper); +} + static int perf_queue_eventdev_setup(struct evt_test *test, struct evt_options *opt) { @@ -143,6 +279,7 @@ static const struct evt_test_ops perf_queue = { .test_setup = perf_test_setup, .mempool_setup = perf_mempool_setup, .eventdev_setup = perf_queue_eventdev_setup, + .launch_lcores = perf_queue_launch_lcores, .eventdev_destroy = perf_eventdev_destroy, .mempool_destroy = perf_mempool_destroy, .test_result = perf_test_result, -- 2.13.0