From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from NAM02-CY1-obe.outbound.protection.outlook.com (mail-cys01nam02on0087.outbound.protection.outlook.com [104.47.37.87]) by dpdk.org (Postfix) with ESMTP id 6214D1DBF for ; Mon, 18 Dec 2017 22:45:34 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=CAVIUMNETWORKS.onmicrosoft.com; s=selector1-cavium-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version; bh=wLORRso+1XTy5+gr+VgBuaQ7o1Sd+VUJMrsP/VulVDc=; b=d/s8O1aynRu0cq85OMZuR9+vbNEOLt1BnsJLSvWhuE6WSdtODsFv8jHBW0gj03tY3tWxnEGrwYtt3b7n+qZDAfNXhQj5vqs6h6znFbE8qnCa9Wa97igs1ph4mn9UwdO/jvRa34xgbUiP8oEnvJCke1JI0J1fdhgjFNmJYnL3yAo= Authentication-Results: spf=none (sender IP is ) smtp.mailfrom=Pavan.Bhagavatula@cavium.com; Received: from localhost.localdomain (111.93.218.67) by DM5PR07MB3467.namprd07.prod.outlook.com (10.164.153.22) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384_P256) id 15.20.323.15; Mon, 18 Dec 2017 21:45:28 +0000 From: Pavan Nikhilesh To: jerin.jacob@caviumnetworks.com, santosh.shukla@caviumnetworks.com, bruce.richardson@intel.com, harry.van.haaren@intel.com, gage.eads@intel.com, hemant.agrawal@nxp.com, nipun.gupta@nxp.com, liang.j.ma@intel.com Cc: dev@dpdk.org, Pavan Nikhilesh Date: Tue, 19 Dec 2017 03:14:04 +0530 Message-Id: <20171218214405.26763-11-pbhagavatula@caviumnetworks.com> X-Mailer: git-send-email 2.14.1 In-Reply-To: <20171218214405.26763-1-pbhagavatula@caviumnetworks.com> References: <20171130072406.15605-1-pbhagavatula@caviumnetworks.com> <20171218214405.26763-1-pbhagavatula@caviumnetworks.com> MIME-Version: 1.0 Content-Type: text/plain X-Originating-IP: [111.93.218.67] X-ClientProxiedBy: HK2PR02CA0163.apcprd02.prod.outlook.com (10.171.30.23) To DM5PR07MB3467.namprd07.prod.outlook.com (10.164.153.22) X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: 2aa3e1ca-dcbc-4dcd-8452-08d54660aa46 X-Microsoft-Antispam: UriScan:; BCL:0; PCL:0; RULEID:(5600026)(4604075)(4534020)(4602075)(4627115)(201703031133081)(201702281549075)(2017052603307); SRVR:DM5PR07MB3467; X-Microsoft-Exchange-Diagnostics: 1; DM5PR07MB3467; 3:lzpjZ6lXtlozar2FE2IX6tYa9UsPWPGd38LVVtrTtG32KzqgGhpO4vwmd1hcnAG2BrQrwyF4zcugFx+2ZsA110hr7gfd2AnuvOdteTjeabei0QDLm3SQjj254pJAPhOl+J8LM9aFB2UoCVANnUkkUtC32raWCVtxHCnjrzTLMSRh2JnqkartztF83KvPSEiAbj5nxi35K9ioV28B7LrtTzlt1vdv9JZzgWS9m9qz3k8gBcbfloc9Fk1c7OqcpRba; 25:TbIpmTMA81tUT4mw53wiCk4Uo58ESEWDCcSLrFN0l/cfr4nl5OgpwcRsxBCpHO6NDjhbW/5NmKCssGYOUq2l0ey/SnQ1+S6JLglQvcKU9iJWdqZlSjWu/ORRffTQwLzrJZ4m0V4incSJ3NgeqRvPM6dlfqH1TN1vPu1gYMepprPSVbCSRR/8Lm6FWpSA0SFh3/vDYl412q5Uwp4v3DJD4phOeNxKCmIPrT4RyoNvTZNiZGN0WjsJYncFsHbOBbyYRzor0QxSP6vHerDpFnuJhazzZstBA0m7ewhxJHJgMYUraCL6fPBbeK8MczU7u/owOTIvZRQaxQJWVmK0vmdI0A==; 31:9myrbeJecj5Hqz10FNlqcxIJI57psJh4+37GKaIT8iiPnSuuzuIIDoOB+Xn0sauZOsOp30UOt94ibd9dITmWFuCqmDPMbIwkaUCMlZyikurHKOlaC/6/Tu9WXoV0ka6lzsVcvNWo5ltuSMl0YEPb7VtlVg0gAHokKY9I9mMuVfhKB8X5QuP4NhtbU/xKViHqFz5BTJuIdLF2VdjZuRXzq647shxrX1PE1ZXN1DkR1/Y= X-MS-TrafficTypeDiagnostic: DM5PR07MB3467: X-Microsoft-Exchange-Diagnostics: 1; DM5PR07MB3467; 20:ES8i/Sj23T3GlblPm6r4Dkc0JW0xlkgXSK5D/bHE4FH+PgUbClvQsW3WEqrJhZnPZtJ5g126Ac9ZCJlhv57PLzZYIpLO1C1pox3a0dyPkVitLZ11froETRzaFMP65VFkXeLZ2qqiUqxIhKcBoEHwRng0fXEKetqxHgC7fuaaMkLuH8w5v8I6PX6Kr3+Vis1x0Tb14lv0qqRxz34jQa6zExSpx/MClB+2L5HPpCLsWPivPPQRVsKrZDSzDjDaYq2+Pg9WRsK7Si1Pzm7CmKQgVG30iW54UTLWi0DM35fFXjdBX3gWa7pN0yLWHFswW4WcCA+LlTOOaEiGbrqLgt6F7WpiLIqvMlDuuBLLpmiq/d0LrfCtutWEQ5Ur2ZC8fASuhogavOFZCGWb8C/5h0W5WORFTdKLs72yWjI9ENSWITogE31t38dwAGgew76CkfhqUBqHp3kCta1wQ/UP2mrjt10J4vz7VKpA+YRRldjeL0AEHVmBCMB7JAcpCNqeJ8wVOcmdpm4MugogOXJJb0kB+f5iiCf0h6FeRz+pvt7vhX+F9+s1LU264tGlV8QbD+nHTB6ZbXVMilsmyKZqIKpg+rFqvnh+TzU+cM91Y7L6zzA=; 4:QJ1WyyYpPqAOAK+Vs5L0M0bjpg5vIxL6z3OZXZVYW2nUz/AVZLmM+EsgDzxuG2qWucVRx52B67FeGTl2pbESbrKFMxNRNOP1XdCKHG6xp1MyqppICBGY5lsy+AvCWs8WGbvv9FK2HAik4eOG5F0vWm2MhgJ8GcGvJKRdcVFdHXalKeQSpsUJzYF/NftEE1jd6V5eAAt2lA/fdBgk4IFvicSxgGk76LGQObIjB4Ve0/6yYuHw6+FKgbXPIfYYTAnabkn+mOxUH5UekEWYiL+knw== X-Microsoft-Antispam-PRVS: X-Exchange-Antispam-Report-Test: UriScan:; X-Exchange-Antispam-Report-CFA-Test: BCL:0; PCL:0; RULEID:(6040450)(2401047)(8121501046)(5005006)(10201501046)(93006095)(3002001)(3231023)(6041248)(201703131423075)(201702281528075)(201703061421075)(201703061406153)(20161123558100)(20161123562025)(20161123564025)(20161123555025)(20161123560025)(6072148)(201708071742011); SRVR:DM5PR07MB3467; BCL:0; PCL:0; RULEID:(100000803101)(100110400095); SRVR:DM5PR07MB3467; X-Forefront-PRVS: 0525BB0ADF X-Forefront-Antispam-Report: SFV:NSPM; SFS:(10009020)(6069001)(396003)(366004)(376002)(39860400002)(346002)(199004)(189003)(4326008)(2906002)(53936002)(305945005)(25786009)(1076002)(8936002)(50226002)(16526018)(6506007)(386003)(68736007)(52116002)(478600001)(51416003)(48376002)(5009440100003)(3846002)(6116002)(72206003)(36756003)(6486002)(107886003)(8676002)(5660300001)(106356001)(105586002)(8656006)(2950100002)(42882006)(6512007)(6666003)(59450400001)(16586007)(7736002)(66066001)(47776003)(76176011)(50466002)(81166006)(81156014)(316002)(97736004)(42262002); DIR:OUT; SFP:1101; SCL:1; SRVR:DM5PR07MB3467; H:localhost.localdomain; FPR:; SPF:None; PTR:InfoNoRecords; A:1; MX:1; LANG:en; Received-SPF: None (protection.outlook.com: cavium.com does not designate permitted sender hosts) X-Microsoft-Exchange-Diagnostics: =?us-ascii?Q?1; DM5PR07MB3467; 23:Ue91ussiRLKZmIoi8DjdtpzMMJO+KooE+QS41Gqq9?= =?us-ascii?Q?/VelWRFO3X02TwOk44zZ117zAa1aKKo3jw9lCj7TW4iXsCVndSzKCD31mk3W?= =?us-ascii?Q?Y6R5vtpEDdBzAUUmUacdQcA8JXnbpnXr6/jTxoMmxbnmA7lIRPaOQx+3DLN4?= =?us-ascii?Q?z0KaLFGMSrsYaaCik9hfqszotc2MooFsPOumzxvIqiPHpUKFE0WIIiubfw5F?= =?us-ascii?Q?FpQ3P6vPPIpLichok8ahkG2ZBwvrFhbdsK5mPwT44Gcz6NEdPp9bBn5gy496?= =?us-ascii?Q?3OwHehCNIsEEC/WrF6o5jzk4DD7UELtJhCvwCiYg50l8EPLBZ6GBGxKSby2m?= =?us-ascii?Q?z4WDolJwCyDGjiU3X0sHifRR3XdbYyrifGY4XNunC/5at514vRIP+CpM1YTC?= =?us-ascii?Q?0+MZHI3p6H0bWkyl6lv2hu2HHhN2o+PLQGl/j4adddVrfqlkbtgO/dA/WUwg?= =?us-ascii?Q?zXAhW1oiYdhuQGMTPDAOCyB8S36qvzLbF0mcaTkB3Kg0naFr0CnRp7RiNeBq?= =?us-ascii?Q?TINodZci7039gBcIoBdPVfpcR8OrM3PE1ef+Q14KH5PGsnyn/5Mb0ut/jywc?= =?us-ascii?Q?7l8vjAgZ16Z4McGTGxfwPbBlQIaFa5JaIqNTTkKRBG8bUbw0aEYKjH6Jl2Iq?= =?us-ascii?Q?HsAv7XzcCHxaKVt9+WWHr3O4jDLiJH2gU/tuLzz0/QPa6xFdLd/HeEUA6U4G?= =?us-ascii?Q?a7ElkbRDA645aB4O9bcOB2vWVESUaAObXB5yGY79+UVRHILfuq5rIKx1c7sR?= =?us-ascii?Q?WOttbLfnY7sHlqIMS6dLGYuQBxaBItKlqgfFm+hQcx2pvScfDx617U6t0y8j?= =?us-ascii?Q?u6rj/5HVFNcG42BuJ6BYb4zyxt3OeOFnU4GI1l1TDKSEHTyP+QJbcQLhKX0y?= =?us-ascii?Q?yFZm8YSZudfi+Qm2MpHl00cXcuUltKnFKyHtY/jkthHNUFPYz6G5dmGvR1kb?= =?us-ascii?Q?HMuqnVM/DYVCi7r+vj5qEpOs4y7XrPufCvXw7DmPqA/Y6cMmy6/Rb2WjBxpe?= =?us-ascii?Q?Gl6vb7ubL2e2rvOKtxfTvqtxiciKYX1CCtm2VHQDKudkLVIStyl42gqk6dEC?= =?us-ascii?Q?LxkDmtoj+9gSnEf1PIJJWsZJ5vSFwCSFjrXguJgveqeeQRN85E0zp1stbOg8?= =?us-ascii?Q?ygfK8TsogHrdmwRdXKebMJr8dYYwkjIlP9ZLQE0zIxAdO51YzkTVoqH8mS6X?= =?us-ascii?Q?nlVmA3A0bfwS+r741o+utKKp3Z/NUZ4NaPt?= X-Microsoft-Exchange-Diagnostics: 1; DM5PR07MB3467; 6:FKtYc2f3ocOJNIBXRdmTcmIy2L3AtL1rEklZQhnFEi+0NLrNFpgPrsHF3n7/Z+dK9EZbqHZn4bMatwxolguFNNpyRG/nQMYphgshAumQf8+I4Oxm8qqXOjiEgmxiksd0fxg+AXkwE0Gi1kl1jO8LovjJ3gNlNVybwIlUt2sFB1rYrsC6gwlEdncctPbLv4oAxNpeiJrOfER0+fp/kt+9UrsKq8+LHhSO7YrIqpAPyICkH32XHBC+vCwNMbwhMldWDdwE6x2WDG2wKYyMP3tHnkT6+cHhOi5FStCfQ5QscHqJBe7iThLRgzL54272LxaAMxppXkb8hM7Qre0HGgyWY+B5zZLI7bkNWRmswW4kb3o=; 5:IoHHLvayzVVZF6eGbDrC+JMqO8lkg+PoiL5fY35cwsLK1M1UNHDUILM31NWto/rBpJ0WR2QgRG5UDnND47MYuC1B4EYvIEo3BAJi7OZJWOQ5cq3nEM7FBQb1g0eq6gsWkmnweVnjErUseP57K6TQPKxugzD9fmlwB7pcWUZJSn4=; 24:2q0HufQXA1xh6woJfLZ8fSzomfm12tzHBYA3tFkqZ6XOlU5QmbhN0rJ4UJ+r4hL1PGI51rUXl+1xwY9N0N2xi5fgL9GotDaQe8ym7nSJz8Y=; 7:+mEfzjWpExXZUZCsO6u0THfYtHZ7c5Gj548rrO6R/AKQ0Dj3w6+T4euuJ5W4Y4+/EterQv9VZ8geqDHjzNMhirIajkNAxkPodb1j79ZZMSsP8qZbAyDvIVUu2e/ujcZ4RVYFoDh5/fYhRuQQ4EhoJ/KQXKiz4q940hc8ifNhVOrBUhS9MR8A0OqkgRAWLpuTMDrlGlRLfmHr6m6OvysbP+arxu+hI20d2NkOnvIEWalVMniJuS/woS2mFZAwdg67 SpamDiagnosticOutput: 1:99 SpamDiagnosticMetadata: NSPM X-OriginatorOrg: caviumnetworks.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 18 Dec 2017 21:45:28.6597 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 2aa3e1ca-dcbc-4dcd-8452-08d54660aa46 X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 711e4ccf-2e9b-4bcf-a551-4094005b6194 X-MS-Exchange-Transport-CrossTenantHeadersStamped: DM5PR07MB3467 Subject: [dpdk-dev] [PATCH v2 11/12] app/eventdev: add pipeline atq worker functions X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 18 Dec 2017 21:45:35 -0000 Signed-off-by: Pavan Nikhilesh --- app/test-eventdev/test_pipeline_atq.c | 354 +++++++++++++++++++++++++++++++++- 1 file changed, 353 insertions(+), 1 deletion(-) diff --git a/app/test-eventdev/test_pipeline_atq.c b/app/test-eventdev/test_pipeline_atq.c index f625b04b1..6507ac911 100644 --- a/app/test-eventdev/test_pipeline_atq.c +++ b/app/test-eventdev/test_pipeline_atq.c @@ -42,10 +42,362 @@ pipeline_atq_nb_event_queues(struct evt_options *opt) return (eth_count * opt->nb_stages); } +static int +pipeline_atq_worker_single_stage_safe(void *arg) +{ + struct worker_data *w = arg; + struct test_pipeline *t = w->t; + const uint8_t dev = w->dev_id; + const uint8_t port = w->port_id; + struct rte_event ev; + + while (t->done == false) { + uint16_t event = rte_event_dequeue_burst(dev, port, &ev, 1, 0); + + if (!event) { + rte_pause(); + continue; + } + + if (ev.sched_type == RTE_SCHED_TYPE_ATOMIC) { + pipeline_tx_pkt_safe(ev.mbuf); + w->processed_pkts++; + continue; + } + pipeline_fwd_event(&ev, RTE_SCHED_TYPE_ATOMIC); + pipeline_event_enqueue(dev, port, &ev); + } + + return 0; +} + +static int +pipeline_atq_worker_single_stage_unsafe(void *arg) +{ + struct worker_data *w = arg; + struct test_pipeline *t = w->t; + const uint8_t dev = w->dev_id; + const uint8_t port = w->port_id; + struct rte_event ev; + + while (t->done == false) { + uint16_t event = rte_event_dequeue_burst(dev, port, &ev, 1, 0); + + if (!event) { + rte_pause(); + continue; + } + + if (ev.sched_type == RTE_SCHED_TYPE_ATOMIC) { + pipeline_tx_pkt_unsafe(ev.mbuf, t); + w->processed_pkts++; + } else { + pipeline_fwd_event(&ev, RTE_SCHED_TYPE_ATOMIC); + pipeline_event_enqueue(dev, port, &ev); + } + } + + return 0; +} + +static int +pipeline_atq_worker_single_stage_burst_safe(void *arg) +{ + int i; + struct worker_data *w = arg; + struct test_pipeline *t = w->t; + const uint8_t dev = w->dev_id; + const uint8_t port = w->port_id; + struct rte_event ev[BURST_SIZE]; + + while (t->done == false) { + uint16_t nb_rx = rte_event_dequeue_burst(dev, port, ev, + BURST_SIZE, 0); + + if (!nb_rx) { + rte_pause(); + continue; + } + + for (i = 0; i < nb_rx; i++) { + rte_prefetch0(ev[i + 1].mbuf); + if (ev[i].sched_type == RTE_SCHED_TYPE_ATOMIC) { + + pipeline_tx_pkt_safe(ev[i].mbuf); + ev[i].op = RTE_EVENT_OP_RELEASE; + w->processed_pkts++; + } else + pipeline_fwd_event(&ev[i], + RTE_SCHED_TYPE_ATOMIC); + } + + pipeline_event_enqueue_burst(dev, port, ev, nb_rx); + } + + return 0; +} + +static int +pipeline_atq_worker_single_stage_burst_unsafe(void *arg) +{ + int i; + struct worker_data *w = arg; + struct test_pipeline *t = w->t; + const uint8_t dev = w->dev_id; + const uint8_t port = w->port_id; + struct rte_event ev[BURST_SIZE]; + const uint16_t nb_ports = rte_eth_dev_count(); + + while (t->done == false) { + uint16_t nb_rx = rte_event_dequeue_burst(dev, port, ev, + BURST_SIZE, 0); + + if (!nb_rx) { + pipeline_tx_flush(t, nb_ports); + rte_pause(); + continue; + } + + for (i = 0; i < nb_rx; i++) { + rte_prefetch0(ev[i + 1].mbuf); + if (ev[i].sched_type == RTE_SCHED_TYPE_ATOMIC) { + + pipeline_tx_unsafe_burst(ev[i].mbuf, t); + ev[i].op = RTE_EVENT_OP_RELEASE; + w->processed_pkts++; + } else + pipeline_fwd_event(&ev[i], + RTE_SCHED_TYPE_ATOMIC); + } + + pipeline_event_enqueue_burst(dev, port, ev, nb_rx); + } + + return 0; +} + +static int +pipeline_atq_worker_multi_stage_safe(void *arg) +{ + struct worker_data *w = arg; + struct test_pipeline *t = w->t; + const uint8_t dev = w->dev_id; + const uint8_t port = w->port_id; + const uint8_t last_queue = t->opt->nb_stages - 1; + const uint8_t nb_stages = t->opt->nb_stages; + uint8_t *const sched_type_list = &t->sched_type_list[0]; + uint8_t cq_id; + struct rte_event ev; + + + while (t->done == false) { + uint16_t event = rte_event_dequeue_burst(dev, port, &ev, 1, 0); + + if (!event) { + rte_pause(); + continue; + } + + cq_id = ev.queue_id % nb_stages; + + if (cq_id == last_queue) { + if (ev.sched_type == RTE_SCHED_TYPE_ATOMIC) { + + pipeline_tx_pkt_safe(ev.mbuf); + w->processed_pkts++; + continue; + } + pipeline_fwd_event(&ev, RTE_SCHED_TYPE_ATOMIC); + } else { + ev.queue_id++; + pipeline_fwd_event(&ev, sched_type_list[cq_id]); + } + + pipeline_event_enqueue(dev, port, &ev); + } + return 0; +} + +static int +pipeline_atq_worker_multi_stage_unsafe(void *arg) +{ + struct worker_data *w = arg; + struct test_pipeline *t = w->t; + const uint8_t dev = w->dev_id; + const uint8_t port = w->port_id; + const uint8_t last_queue = t->opt->nb_stages - 1; + const uint8_t nb_stages = t->opt->nb_stages; + uint8_t *const sched_type_list = &t->sched_type_list[0]; + uint8_t cq_id; + struct rte_event ev; + + + while (t->done == false) { + uint16_t event = rte_event_dequeue_burst(dev, port, &ev, 1, 0); + + if (!event) { + rte_pause(); + continue; + } + + cq_id = ev.queue_id % nb_stages; + + if (cq_id == last_queue) { + if (ev.sched_type == RTE_SCHED_TYPE_ATOMIC) { + + pipeline_tx_pkt_unsafe(ev.mbuf, t); + w->processed_pkts++; + continue; + } + pipeline_fwd_event(&ev, RTE_SCHED_TYPE_ATOMIC); + } else { + ev.queue_id++; + pipeline_fwd_event(&ev, sched_type_list[cq_id]); + } + + pipeline_event_enqueue(dev, port, &ev); + } + return 0; +} + +static int +pipeline_atq_worker_multi_stage_burst_safe(void *arg) +{ + int i; + struct worker_data *w = arg; + struct test_pipeline *t = w->t; + const uint8_t dev = w->dev_id; + const uint8_t port = w->port_id; + uint8_t *const sched_type_list = &t->sched_type_list[0]; + const uint8_t last_queue = t->opt->nb_stages - 1; + const uint8_t nb_stages = t->opt->nb_stages; + uint8_t cq_id; + struct rte_event ev[BURST_SIZE + 1]; + + while (t->done == false) { + uint16_t nb_rx = rte_event_dequeue_burst(dev, port, ev, + BURST_SIZE, 0); + + if (!nb_rx) { + rte_pause(); + continue; + } + + for (i = 0; i < nb_rx; i++) { + rte_prefetch0(ev[i + 1].mbuf); + cq_id = ev[i].queue_id % nb_stages; + + if (cq_id == last_queue) { + if (ev[i].sched_type == RTE_SCHED_TYPE_ATOMIC) { + + pipeline_tx_unsafe_burst(ev[i].mbuf, t); + ev[i].op = RTE_EVENT_OP_RELEASE; + w->processed_pkts++; + continue; + } + + pipeline_fwd_event(&ev[i], + RTE_SCHED_TYPE_ATOMIC); + } else { + ev[i].queue_id++; + pipeline_fwd_event(&ev[i], + sched_type_list[cq_id]); + } + } + + pipeline_event_enqueue_burst(dev, port, ev, nb_rx); + } + return 0; +} + +static int +pipeline_atq_worker_multi_stage_burst_unsafe(void *arg) +{ + int i; + struct worker_data *w = arg; + struct test_pipeline *t = w->t; + const uint8_t dev = w->dev_id; + const uint8_t port = w->port_id; + uint8_t *const sched_type_list = &t->sched_type_list[0]; + const uint8_t last_queue = t->opt->nb_stages - 1; + const uint8_t nb_stages = t->opt->nb_stages; + uint8_t cq_id; + struct rte_event ev[BURST_SIZE + 1]; + const uint16_t nb_ports = rte_eth_dev_count(); + + while (t->done == false) { + uint16_t nb_rx = rte_event_dequeue_burst(dev, port, ev, + BURST_SIZE, 0); + + if (!nb_rx) { + pipeline_tx_flush(t, nb_ports); + rte_pause(); + continue; + } + + for (i = 0; i < nb_rx; i++) { + rte_prefetch0(ev[i + 1].mbuf); + cq_id = ev[i].queue_id % nb_stages; + + if (cq_id == last_queue) { + if (ev[i].sched_type == RTE_SCHED_TYPE_ATOMIC) { + + pipeline_tx_unsafe_burst(ev[i].mbuf, t); + ev[i].op = RTE_EVENT_OP_RELEASE; + w->processed_pkts++; + continue; + } + + pipeline_fwd_event(&ev[i], + RTE_SCHED_TYPE_ATOMIC); + } else { + ev[i].queue_id++; + pipeline_fwd_event(&ev[i], + sched_type_list[cq_id]); + } + } + + pipeline_event_enqueue_burst(dev, port, ev, nb_rx); + } + return 0; +} + static int worker_wrapper(void *arg) { - RTE_SET_USED(arg); + struct worker_data *w = arg; + struct evt_options *opt = w->t->opt; + const bool burst = evt_has_burst_mode(w->dev_id); + const bool mt_safe = !w->t->mt_unsafe; + const uint8_t nb_stages = opt->nb_stages; + RTE_SET_USED(opt); + + /* allow compiler to optimize */ + if (nb_stages == 1) { + if (!burst && mt_safe) + return pipeline_atq_worker_single_stage_safe(arg); + else if (!burst && !mt_safe) + return pipeline_atq_worker_single_stage_unsafe( + arg); + else if (burst && mt_safe) + return pipeline_atq_worker_single_stage_burst_safe( + arg); + else if (burst && !mt_safe) + return pipeline_atq_worker_single_stage_burst_unsafe( + arg); + } else { + if (!burst && mt_safe) + return pipeline_atq_worker_multi_stage_safe(arg); + else if (!burst && !mt_safe) + return pipeline_atq_worker_multi_stage_unsafe(arg); + if (burst && mt_safe) + return pipeline_atq_worker_multi_stage_burst_safe( + arg); + else if (burst && !mt_safe) + return pipeline_atq_worker_multi_stage_burst_unsafe( + arg); + + } rte_panic("invalid worker\n"); } -- 2.14.1