From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from NAM02-BL2-obe.outbound.protection.outlook.com (mail-bl2nam02on0054.outbound.protection.outlook.com [104.47.38.54]) by dpdk.org (Postfix) with ESMTP id AA0AE2BB0 for ; Tue, 22 Nov 2016 07:02:27 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=CAVIUMNETWORKS.onmicrosoft.com; s=selector1-cavium-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version; bh=RzHEP6RHV+CVGaK1elJkgtTIJ4V0vL3jWm5EoMcWkw0=; b=TbKbnWO0hWWcRIcj8mCL4JMzxND9I4wtCB6x1CbqTF4XQMxV+a17t17fA6LEySph5GEhRrt6/9jFe3hYFebEx/KH0dI+R7uDdixxvL22vXGE6c02Ggb+5KuBAY2k7ifEoN9quXwF/pp9Z6mbWP8fNxSS0+YzbF37eN+yxDRimII= Authentication-Results: spf=none (sender IP is ) smtp.mailfrom=Jerin.Jacob@cavium.com; Received: from svelivela-lt.caveonetworks.com (50.233.148.156) by BN3PR0701MB1719.namprd07.prod.outlook.com (10.163.39.18) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384_P384) id 15.1.693.12; Tue, 22 Nov 2016 06:02:24 +0000 Date: Tue, 22 Nov 2016 11:32:13 +0530 From: Jerin Jacob To: Harry van Haaren CC: , Gage Eads , Bruce Richardson Message-ID: <20161122060208.GA27648@svelivela-lt.caveonetworks.com> References: <1479319207-130646-1-git-send-email-harry.van.haaren@intel.com> <1479319207-130646-8-git-send-email-harry.van.haaren@intel.com> MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Disposition: inline In-Reply-To: <1479319207-130646-8-git-send-email-harry.van.haaren@intel.com> User-Agent: Mutt/1.7.1 (2016-10-04) X-Originating-IP: [50.233.148.156] X-ClientProxiedBy: CY4PR14CA0026.namprd14.prod.outlook.com (10.172.162.140) To BN3PR0701MB1719.namprd07.prod.outlook.com (10.163.39.18) X-Microsoft-Exchange-Diagnostics: 1; BN3PR0701MB1719; 2:w3kIrP7Tj2ZqAnHfSRyGK6KPvaRJLw0IfwSyL00H35lNPnq0sJWt9r69ty1Ggv4sztFxsd+TsVkDxLVpHPzg0AXfcD2sRk1kzktlFCo5joL1l5HyqRQeGyUJDMIEng0y6gQiw34CyoaMX+Oeha7GtYU4940LJnUOFAVAC8GnKgU=; 3:VdWE/lpNMIjxdtN+wu7NhMqD3D96+R2uOoCd+P3qoUZz7FN0DIJ7dRtrCE+7mnov5KLrLM78owClSM+EhdHGIWrJcZmg+sDC3vmjs//m4M0Y36QmV5bKCQA/TpzbTOB8ecgjyMyqsTC31DFcI7vnhQaEnclwh8C/yGy/OD5Juho= X-MS-Office365-Filtering-Correlation-Id: 96ec90ab-0a6a-4198-cf9b-08d4129d216f X-Microsoft-Antispam: UriScan:; BCL:0; PCL:0; RULEID:(22001); SRVR:BN3PR0701MB1719; X-Microsoft-Exchange-Diagnostics: 1; BN3PR0701MB1719; 25:kzMQNfPbmGVnrQLP72v3wjWO+jbxtxMXRZydDUGwBCaN+v4emSSsyGT0undTi7Bwu7YWn1z/VhcD35hKNx8TtMy+zBVbfvWZVWl9uoRdTo4YqKsgdR2OW9jAxH/Y5galVM9cW2E7Gz4Xon1JH6fvJ3f+53wHezgxJ8kCXRoz5V+PphGOEK8CEXyONJvb5okJ+2umvl28XMLCJ/NrVljfzxWxZPBxAQ86N6nfRle1tDyjps6ZC9QeGojc95VsifRfONqubJRXsPTLhVBXHVSS7tXWEaGK4FBNXUxhWqrfu862+ylzxgEHS0qGC2OtIJyY+t2ge908kqrfPH/E/nI/xZrKbq+EXbqAghw72kxwdbuBHlQpsNOvD0oubVwcj0u0rr3ZHH6kjb53wI8VeGjlq8LQ/A2/wpVZzEEqqGV2Pl7+ByW6OxzUP5ReONFE9INIes38kK7dd1FLXqXlV4dI0WcKPWUlCEzT4tGi7yLSnS0ujM4qkd2/yLlqwvfZAlYSoB8Ib7ds0elLpVVCQeB39UMoU8Z5TnhQfeyoeUnBnIUS9AGWHGcftxSrqtVOwPbczJJV/YnJXKib0O1F+XLTSxJTcgI1nJeJ8AHfpPTzo17VbcXhRsZ7McxeqAAoEs+ed/nnXSL0yqDdwnokmSVfnY6lgqCmIbzLacXDTuUHtZBGBe76JHlFQs+BOGtNHzeqnWFIyiFRN+7qXxTFJECOXOKSsqLD/qGCymoQ7VFYnVfT8MwjYDF3vPWRGRaLzLTMOQePuoenBWfqqYICClxwMP/L1IIG0UwdG1HuzD+N30c= X-Microsoft-Exchange-Diagnostics: 1; BN3PR0701MB1719; 31:CYItJNObYQ53rsb+1flsCLlILlrsgs5qyRGI/GKzHLSZh0rjy0vXR9FkLw7+UItqP2z4HvVy+ueZwX2r0fcMceZ9AjeUCGCs27w2Wad82e+M9GQjQPtzRb5Z54LLmF3z461ELhXSDMzbyJ3lkiC7vvwx3pXvWGLXXJH33KJgpjtJ6/juAWT/nKAH0YQdSIM/Pk3nlT4sDUeRBgsTxO8bC13MaIQ9V/9Gj6rbD9R99FYbJ2fnQZD2fOoibJ52RF9zUJGxy9A0fQM1W5VcfS9NQZa3B1mr0vzos27zI0eJbb8=; 20:uWS4N3cF5EFOO/WauDjJobquf4bZ3iWdTjlWfbYoI+c9Gpsi9cNHzoc6CSH9Mhoyu/3Ebuvh73BQyBW1Z7lKq3OnPNF2nRYQGcCTgV3RstKfmHHX++aNpX+XtUPdesKRFZHe0Rh496OYH4dVm/APmuct8jukW2tQy2nDIwpo6Qi2qnoJzwdM7QDgput0Bb6AgGRKYDR0OCMNAl8ihFA/iLtaAAM7hpIXlIlgyxLA6GREUTxg+xpY533lUA/f0KAf48Do4d1FxYyM4Q8xlFUefCBcgcFrpqnzIHUzMoEnIZQYyuvMBKFLciInbNwj+v28BsOth6XPUi2/Z/LZn6zbk60lmvSTu5mm/1bMuF7HKf1ZhuQlwazgg7zLeLyWwfax61dZgTiNDoebajAWJJFEIK1GMHcQ894BhfPZ9oqersmkcFECpBoTtk+g/1iexVKAsRZxryl9z6hBNpoIkh8jchXtkLttduUe6i87XQLgs1rhiBcPzp+XbahAzOqOmS1b5tCV+ijGb8nbs4UJqUTOrG1U0FJ/0/GQII17SSAL4Ui0jdngeiCIrzhPq0IZCMyBW3+o0NquCVny5pkLq/VIP7cO/r88Mz9+/cYFlokVSTA= X-Microsoft-Antispam-PRVS: X-Exchange-Antispam-Report-Test: UriScan:(278428928389397)(228905959029699)(17755550239193); X-Exchange-Antispam-Report-CFA-Test: BCL:0; PCL:0; RULEID:(6040307)(6060326)(6045199)(601004)(2401047)(5005006)(8121501046)(10201501046)(3002001)(6041248)(6061324); SRVR:BN3PR0701MB1719; BCL:0; PCL:0; RULEID:; SRVR:BN3PR0701MB1719; X-Microsoft-Exchange-Diagnostics: 1; BN3PR0701MB1719; 4:uRbOpX1mdlGmyNp0RJTtopbTOF2WBbbxCnpvhGmk+AQFBBPFhqtyvJTdmzVXIaYEcyPuvBHKrUuoq3EWYIQaC/8uKZyiXnBUc1nB7iau9MresQI+mSEY1mwfE+o4+AGZ2mCjx0Quezg/Xliv5zDhutQ3trtaedBXrq5QzWS/rcG/syWzpGbvaWnGXEvPx5XBpiMr+OU40gjRxco4A0PPUpQvMQ4L8TZ40B6mdMsRtJz17Qg4iOT0OeYsqZIIT+RYS6Xc2HgUAaDWWTrIsSeVBYN/i5PvJFzOTFaSpyGTrsy/eOV3Vy5iPfPI2hLRgLxayetnznfrCU8rl+ocWjO+5YO1R3DuEsxjFsUDa6MwOD87ciptz7FlzX2tj9RM3cfoh1IuSCQ4IiObhqPETvdxVIf+Co2ulio/0KydleZ1JJDIHQOH1fnhXQHYsGRWaQOZ4Flh/2yBUoVKc7VivrO/H5KBR2cfQDB30qzSYZgZ1p4MEGKw+D6ovS80ppN115s5F72NpKrZ7MYYvyufG5iQMUKHbrUfZLzWbhAiyd6iIjT4rJDJffywWLjCU0FaVVqfX4GH9gKvQ6wo/H28PSmhvA== X-Forefront-PRVS: 0134AD334F X-Forefront-Antispam-Report: SFV:NSPM; SFS:(10009020)(4630300001)(6009001)(7916002)(24454002)(51914003)(199003)(189002)(50986999)(5660300001)(69596002)(6916009)(92566002)(42882006)(1076002)(6116002)(66066001)(33656002)(4001350100001)(83506001)(8676002)(76176999)(38730400001)(110136003)(68736007)(47776003)(2950100002)(2906002)(9686002)(7736002)(229853002)(3846002)(105586002)(23726003)(53416004)(7846002)(77096005)(54356999)(97756001)(81166006)(42186005)(4326007)(101416001)(46406003)(81156014)(189998001)(6666003)(97736004)(50466002)(106356001)(305945005)(18370500001); DIR:OUT; SFP:1101; SCL:1; SRVR:BN3PR0701MB1719; H:svelivela-lt.caveonetworks.com; FPR:; SPF:None; PTR:InfoNoRecords; A:1; MX:1; LANG:en; Received-SPF: None (protection.outlook.com: cavium.com does not designate permitted sender hosts) X-Microsoft-Exchange-Diagnostics: =?us-ascii?Q?1; BN3PR0701MB1719; 23:qL8Zc30M8JFgWdK8Lx+zxRtIuru/HXS0U7tdUfK?= =?us-ascii?Q?PxySZyjASnPTaW8m2o5eU5MS1nROvo2nY28TasoCdKSGdeSXE1WuS/i0CF4z?= =?us-ascii?Q?U/fsZbh1ev6iJa1S6s/i9ki11zVJ/R7mCknqoZWOv8vSFbl3XciT31k9cmZ5?= =?us-ascii?Q?pddW12uaxYO7n6x1ASLoqzcaqBfJVZ5ZRJFKlO35PYL80pkTGr03a4BvuWrU?= =?us-ascii?Q?RTjUTvcX5oXZL9RWKdGPA5Jfp1t4fFETkuHaEpOudXI4RU82QGPEpLIo7sXM?= =?us-ascii?Q?sp/xnRgyJihuu2VXAXZEif1/s3kSGUkuvxoNrJuazQFMNqEF7pHIB6OKHlOk?= =?us-ascii?Q?y/rXVFDn3vxTcGp4dBKMyRMB3H/IUb6hpaWs/wN2q72JrYXTwqdLO0xnmRsD?= =?us-ascii?Q?TivGNih+rx9vrrR0C2gi5EmhWLsgd6vgnNaMGMjgmcxukfAsce2MJc8kt7Mk?= =?us-ascii?Q?BRMPN59stnM5VUcig9EQs3kAXMB0DGoaWftGVI+x3W4D46uvra4OkwaheM9F?= =?us-ascii?Q?o3MKYQMsuekNjVDARYqZdUK18gGe5YkqvGVK7etOuCkLTaWEuhYdJscDzzEy?= =?us-ascii?Q?ytqmcvcVJKtA/lBevm9Tf1+3CnJ9rjoPLxb1BYA3UVnDnP72vBlD0a80NK5A?= =?us-ascii?Q?an3QIXdXcpJJrZ1E0oMKL1i6fes5ow76U3ZXu9m2y086ZfH2/8mP1JCcYBXw?= =?us-ascii?Q?wlZfn4/oICfXb9oLUnWfF01nl6aDAxll1qg8iy9ryq9MlQ8t+a0KtXAuNcLe?= =?us-ascii?Q?8NuG/aE9rvJny4uQLRpOEpAgSxqLW8yKygG37It3UH6iyGPF+DXIyGT8gFsk?= =?us-ascii?Q?4TLaIH/4Uxpkk9i/VOlAp9tIicjwn4rAkTAHZgIVWI3xGi/nCahFLJnAZITw?= =?us-ascii?Q?JGo9CGXfER88s1zQefKjXcHQoZQlf3t0292ojWpFCA04uSilZztKgWJcb/OQ?= =?us-ascii?Q?hGuS2vP4EjIqFRJbLoRSmShuSvlFFNgN1/RvEIS25maB2to/qkKpRBrJBLUY?= =?us-ascii?Q?s2Pl6aR0ZAar4rWql9lLIkUEUd8ShEgLLfkk2FEdB+nJZl5OLolHvQEt1bgZ?= =?us-ascii?Q?+0B7STOol303OyTmTMPoRVJFOzITFnAu7UvjdKVWJyZXcMsklB9Wua0DJtn5?= =?us-ascii?Q?QQP2f4787Bup97DnX5b1d6p4HVtqazd/cOlvELEDCTbK3bnnZkbv+MSdqeWt?= =?us-ascii?Q?XffKTqr4VSUiHzrudLS87wUK8NBxE0Jwda2HI2BpUsBRWgan/KzXGL/qs4kq?= =?us-ascii?Q?1maD4k/IyV7BQRykcmlO/gpOVGToe8puyDrrOwz/A?= X-Microsoft-Exchange-Diagnostics: 1; BN3PR0701MB1719; 6:ZurwpU/LrzbWHmkZ7H/6vXIvFrsowlO303n+NzSeu/T5XQnFTaTpQ+8LZPRO1TDE6VTIfnDvwg7+eWHmS+KhW+XJJ+9xt0gBqpbRGipyX5b2xJqjic3I5Hv4WqTHAcxrexnrA8C/aVraXh0GbpRiGwbf/YfbcTsmcXqBVyQeuHP/Iwa90+0yVqIYPmVu+9L6aZ8JOqHg6jMv1CbPPnbEYKR8sp+Rsjuz9q8a2HDu8aLBpIG+jGYiQ4Fp1sEEQWHsiN3E25H53MS8mXIQq3xwv5W6789aBoITUtx3in4Ga4DJejcKh427/HWPVtFfP4JDRSzkM6wxjTi5L027UCdItfSLzvJOLbhdnal0p+kSpg4=; 5:rXTMxql6wqoGOkSYOGMSBaIlhxugEG6nINcVCAV/2ArCYgE5pV9h/RmMxQpoUDkjxwDw7T81lt7yCupFHu6i5fosbHBr8zoRP3GLfULQwTbtYmx1QTp1eUNAf8LvV7MUrWVog72dIuriiALXNa48ZA==; 24:8HTtNHjPyIpRb0yGDLXCfu7SgRceao61Qi3pQ1fno45kwfjVeQ/RwIjeUQ6iIDJC//0Tf1QH85/QRc0Po88y1q2wkGUOn9Yw8hnltx5D890= SpamDiagnosticOutput: 1:99 SpamDiagnosticMetadata: NSPM X-Microsoft-Exchange-Diagnostics: 1; BN3PR0701MB1719; 7:Omh3cYZeTCQ7h34PeCnoRqVKstbMabAabff4ckUH+B3/tEgjK7spdk2vv7TTURhi3eh6D9armOQcDVX+OKTFKJGvj3d+VqDWJyQAEsDqqF7ExmHOkw1Nm8/NRfC0J6LxLzZkP/NVeCODpiENQ6CVdmxjTlmrrB7Z7AbIB+KT2Mba3b46FsEW4skpZRtGYd6WBHTlmA4vREeR1KY5A+qorfBoAMW8ChGMND8ySnQ2GbF83dUS2yuzUptNlf7rT0dReIPxoa4DR1hP436xVgDkBuCD408gSrkkm1EonJ5wgoqa+WqjU1XiocecWoQ72rQAFVtstLHGzJskBDbNlcdASfM6DAWNdxkouu4GcSzeo8c= X-OriginatorOrg: caviumnetworks.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 22 Nov 2016 06:02:24.0959 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-Transport-CrossTenantHeadersStamped: BN3PR0701MB1719 Subject: Re: [dpdk-dev] [PATCH 7/7] examples/eventdev_pipeline: adding example X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: patches and discussions about DPDK List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 22 Nov 2016 06:02:28 -0000 On Wed, Nov 16, 2016 at 06:00:07PM +0000, Harry van Haaren wrote: > This patch adds a sample app to the examples/ directory, which can be used > as a reference application and for general testing. The application requires > two ethdev ports and expects traffic to be flowing. The application must be > run with the --vdev flags as follows to indicate to EAL that a virtual > eventdev device called "evdev_sw0" is available to be used: > > ./build/eventdev_pipeline --vdev evdev_sw0 > > The general flow of the traffic is as follows: > > Rx core -> Atomic Queue => 4 worker cores => TX core > > A scheduler core is required to do the packet scheduling, making this > configuration require 7 cores (Rx, Tx, Scheduler, and 4 workers). Finally > a master core brings the core count to 8 for this configuration. The Thanks for the example application.I will try to share my views on ethdev integration and usability perspective. Hope we can converge. Some of the high level details first before getting into exact details. 1) From the HW and ethdev integration perspective, The integrated NIC controllers does not need producer core(s) to push the event/packets to event queue. So, I was thinking to use 6WIND rte_flow spec to create the "ethdev port to event queue wiring" connection by extending the output ACTION definition, which specifies event queue its need to enqueued to for the given ethdev port (something your are doing in application). I guess, the producer part of this example can be created as common code, somewhere in rte_flow/ethdev to reuse. We would need this scheme also where when we deal with external nics + HW event manager use case The complete event driven model can be verified and exercised without integrating with eventdev subsystem. So I think, may be we need to focus on functional applications without ethdev to verify the eventdev features like(automatic multicore scaling, dynamic load balancing, pipelining, packet ingress order maintenance and synchronization services) and then integrate with ethdev > + const unsigned cores_needed = num_workers + > + /*main*/1 + > + /*sched*/1 + > + /*TX*/1 + > + /*RX*/1; > + 2) One of the prime aims of the event driven model is to remove the fixed function core mappings and enable automatic multicore scaling, dynamic load balancing etc.I will try to use an example in review section to show the method for removing "consumer core" in this case. > application can be configured for various numbers of flows and worker > cores. Run the application with -h for details. > > Signed-off-by: Gage Eads > Signed-off-by: Bruce Richardson > Signed-off-by: Harry van Haaren > --- > examples/eventdev_pipeline/Makefile | 49 +++ > examples/eventdev_pipeline/main.c | 718 ++++++++++++++++++++++++++++++++++++ > 2 files changed, 767 insertions(+) > create mode 100644 examples/eventdev_pipeline/Makefile > create mode 100644 examples/eventdev_pipeline/main.c > > +static int sched_type = RTE_SCHED_TYPE_ATOMIC; RTE_SCHED_TYPE_ORDERED makes sense as a default. Most common case will have ORDERD at first stage so that it can scale. > + > + > +static int > +worker(void *arg) > +{ > + struct rte_event rcv_events[BATCH_SIZE]; > + > + struct worker_data *data = (struct worker_data *)arg; > + uint8_t event_dev_id = data->event_dev_id; > + uint8_t event_port_id = data->event_port_id; > + int32_t qid = data->qid; > + size_t sent = 0, received = 0; > + > + while (!done) { > + uint16_t i; > + > + uint16_t n = rte_event_dequeue_burst(event_dev_id, > + event_port_id, > + rcv_events, > + RTE_DIM(rcv_events), > + false); > + if (n == 0){ > + rte_pause(); > + /* Flush any buffered events */ > + rte_event_dequeue(event_dev_id, > + event_port_id, > + NULL, > + false); The above can be done in implementation. May not be the candidate for common code. > + continue; > + } > + received += n; > + > + for (i = 0; i < n; i++) { > + struct ether_hdr *eth; > + struct ether_addr addr; > + struct rte_event *ev = &rcv_events[i]; > + > + ev->queue_id = qid; > + ev->flow_id = 0; Another way to deal wit out additional consumer core(it creates issue in scaling and load balancing) is in worker: while(1) { ev = dequeue(port); // stage 1 app processing if (ev.event_type == RTE_EVENT_TYPE_ETHDEV) { // identify the Ethernet port and tx queue the packet needs to go // create the flow based on that ev.flow_id = flow(port_id, tx_queue_id); ev.sched_type = RTE_SCHED_TYPE_ATOMIC; ev.operation = RTE_EVENT_OP_FORWARD; ev.event_type = RTE_EVENT_TYPE_CORE; } // stage 2 app processing else if (ev.event_type == RTE_EVENT_TYPE_CORE) { port_id = function_of(ev.flow_id) ;// look stage 1 processing tx_queue_id = function_of(ev.flow_id) //look stage 1 processing remaining ethdev based tx is same as yours } enqueue(ev); } > + ev->priority = 0; > + ev->sched_type = RTE_SCHED_TYPE_ATOMIC; > + ev->operation = RTE_EVENT_OP_FORWARD; > + > + uint64_t now = rte_rdtsc(); > + while(now + 750 > rte_rdtsc()) {} Why delay ? > + > + /* change mac addresses on packet */ > + eth = rte_pktmbuf_mtod(ev->mbuf, struct ether_hdr *); > + ether_addr_copy(ð->d_addr, &addr); > + ether_addr_copy(ð->s_addr, ð->d_addr); > + ether_addr_copy(&addr, ð->s_addr); > + } > + int ret = rte_event_enqueue_burst(event_dev_id, event_port_id, > + rcv_events, n, false); > + if (ret != n) > + rte_panic("worker %u thread failed to enqueue event\n", > + rte_lcore_id()); > + } > + > + /* Flush the buffered events */ > + rte_event_dequeue(event_dev_id, event_port_id, NULL, false); > + > + if (!quiet) > + printf(" worker %u thread done. RX=%zu TX=%zu\n", > + rte_lcore_id(), received, sent); > + > + return 0; > +} > + > +static int > +scheduler(void *arg) > +{ Maybe better to abstract as "service core" or something like I mentioned earlier, as HW implementation does not need this > + RTE_SET_USED(arg); > + size_t loops = 0; > + > + while (!done) { > + /* Assumes an event dev ID of 0 */ > + rte_event_schedule(0); > + loops++; > + } > + > + printf(" scheduler thread done. loops=%zu\n", loops); > + > + return 0; > +} > + > + > +static int > +producer(void *arg) > +{ > + > + struct prod_data *data = (struct prod_data *)arg; > + size_t npackets = num_packets; > + unsigned i; > + uint64_t mbuf_seqno = 0; > + size_t sent = 0; > + uint8_t eth_port = 0; > + uint8_t event_dev_id = data->event_dev_id; > + uint8_t event_port_id = data->event_port_id; > + int fid_counter = 0; > + > + while (!done) { > + int ret; > + unsigned num_ports = data->num_ports; > + int32_t qid = data->qid; > + struct rte_event events[BATCH_SIZE]; > + struct rte_mbuf *mbufs[BATCH_SIZE]; > + > + uint16_t nb_rx = rte_eth_rx_burst(eth_port, 0, mbufs, BATCH_SIZE); > + if (++eth_port == num_ports) > + eth_port = 0; > + if (nb_rx == 0) { > + rte_pause(); > + /* Flush any buffered events */ > + rte_event_dequeue(event_dev_id, > + event_port_id, > + NULL, > + false); > + continue; > + } > + > + for (i = 0; i < nb_rx; i++) { > + struct rte_mbuf *m = mbufs[i]; > + struct rte_event *ev = &events[i]; > + > + ev->queue_id = qid; > + ev->flow_id = fid_counter++ % 6; To me, flow_id should be a function of port_id and rx queue number here. right? > + ev->priority = 0; > + m->udata64 = mbuf_seqno++; Why update mbuf_seqno++ here. Shouldn't be something inside the implementation? > + ev->mbuf = m; > + ev->sched_type = sched_type; > + ev->operation = RTE_EVENT_OP_NEW; > + } > + > + do { > + ret = rte_event_enqueue_burst(event_dev_id, > + event_port_id, > + events, > + nb_rx, > + false); > + } while (ret == -ENOSPC); I guess, -ENOSPC can be checked inside the implementation. I guess, we can pass the info required in the configuration stage to decide the timeout. May not be the candidate for common code. > + if (ret != nb_rx) > + rte_panic("producer thread failed to enqueue *all* events\n"); > + > + sent += nb_rx; > + > + if (num_packets > 0 && npackets > 0) { > + npackets -= nb_rx; > + if (npackets == 0) > + break; > + } > + } > + > + /* Flush any buffered events */ > + while (!done) > + rte_event_dequeue(event_dev_id, event_port_id, NULL, false); > + > + printf(" prod thread done! TX=%zu across %u flows\n", sent, num_fids); > + > + return 0; > +} > + > +static uint8_t > +setup_event_dev(struct prod_data *prod_data, > + struct cons_data *cons_data, > + struct worker_data *worker_data) > +{ > + config.nb_events_limit = 256; In real application, we may need to pass as command line > + config.dequeue_wait_ns = 0; > + > + ret = rte_event_dev_configure(id, &config); > + if (ret) > + rte_panic("Failed to configure the event dev\n"); > + > + /* Create queues */ > + queue_config.event_queue_cfg = RTE_EVENT_QUEUE_CFG_ATOMIC_ONLY; > + queue_config.priority = 0; > + > + qid0 = 0; > + ret = rte_event_queue_setup(id, qid0, &queue_config); > + if (ret < 0) > + rte_panic("Failed to create the scheduled QID\n"); > + > + queue_config.event_queue_cfg = RTE_EVENT_QUEUE_CFG_SINGLE_CONSUMER; > + queue_config.priority = 0; > + > + cons_qid = 1; > + ret = rte_event_queue_setup(id, cons_qid, &queue_config); > + if (ret < 0) > + rte_panic("Failed to create the cons directed QID\n"); > + > + queue_config.event_queue_cfg = RTE_EVENT_QUEUE_CFG_SINGLE_CONSUMER; I guess its more of RTE_EVENT_QUEUE_CFG_SINGLE_PRODUCER case, Does it make sense to add RTE_EVENT_QUEUE_CFG_SINGLE_PRODUCER in spec, if you are enqueueing only through that port. see next comment. > + queue_config.priority = 0; > + > + prod_qid = 2; > + ret = rte_event_queue_setup(id, prod_qid, &queue_config); > + if (ret < 0) > + rte_panic("Failed to create the prod directed QID\n"); > + Looks like prod_qid is just created as a dummy, The actual producer is en-queuing on qid0.Something not adding up. > + /* Create ports */ > +#define LB_PORT_DEPTH 16 > +#define DIR_PORT_DEPTH 32 > + port_config.enqueue_queue_depth = LB_PORT_DEPTH; > + port_config.dequeue_queue_depth = LB_PORT_DEPTH; We need to check the info->max_enqueue_queue_depth. Jerin