From: Jerin Jacob <jerin.jacob@caviumnetworks.com>
To: Harry van Haaren <harry.van.haaren@intel.com>
Cc: <dev@dpdk.org>, Bruce Richardson <bruce.richardson@intel.com>,
David Hunt <david.hunt@intel.com>, <nipun.gupta@nxp.com>,
<hemant.agrawal@nxp.com>, <gage.eads@intel.com>
Subject: Re: [dpdk-dev] [PATCH v2 15/15] app/test: add unit tests for SW eventdev driver
Date: Wed, 8 Feb 2017 15:53:08 +0530 [thread overview]
Message-ID: <20170208102306.GA19597@localhost.localdomain> (raw)
In-Reply-To: <1485879273-86228-16-git-send-email-harry.van.haaren@intel.com>
On Tue, Jan 31, 2017 at 04:14:33PM +0000, Harry van Haaren wrote:
> From: Bruce Richardson <bruce.richardson@intel.com>
>
> Since the sw driver is a standalone lookaside device that has no HW
> requirements, we can provide a set of unit tests that test its
> functionality across the different queue types and with different input
> scenarios.
>
Thanks for SW driver specific test cases. It provided me a good insight
of expected application behavior from SW driver perspective and in turn it created
some challenge in portable applications.
I would like highlight a main difference between the implementation and get a
consensus on how to abstract it?
Based on existing header file, We can do event pipelining in two different ways
a) Flow-based event pipelining
b) queue_id based event pipelining
I will provide an example to showcase application flow in both modes.
Based on my understanding from SW driver source code, it supports only
queue_id based event pipelining. I guess, Flow based event pipelining will
work semantically with SW driver but it will be very slow.
I think, the reason for the difference is the capability of the context definition.
SW model the context is - queue_id
Cavium HW model the context is queue_id + flow_id + sub_event_type +
event_type
AFAIK, queue_id based event pipelining will work with NXP HW but I am not
sure about flow based event pipelining model with NXP HW. Appreciate any
input this?
In Cavium HW, We support both modes.
As an open question, Should we add a capability flag to advertise the supported
models and let application choose the model based on implementation capability. The
downside is, a small portion of stage advance code will be different but we
can reuse the STAGE specific application code(I think it a fair
trade off)
Bruce, Harry, Gage, Hemant, Nipun
Thoughts? Or any other proposal?
I will take an non trivial realworld NW use case show the difference.
A standard IPSec outbound processing will have minimum 4 to 5 stages
stage_0:
--------
a) Takes the pkts from ethdev and push to eventdev as
RTE_EVENT_OP_NEW
b) Some HW implementation, This will be done by HW. In SW implementation
it done by service cores
stage_1:(ORDERED)
------------------
a) Receive pkts from stage_0 in ORDERED flow and it process in parallel on N
of cores
b) Find a SA belongs that packet move to next stage for SA specific
outbound operations.Outbound processing starts with updating the
sequence number in the critical section and followed by packet encryption in
parallel.
stage_2(ATOMIC) based on SA
----------------------------
a) Update the sequence number and move to ORDERED sched_type for packet
encryption in parallel
stage_3(ORDERED) based on SA
----------------------------
a) Encrypt the packets in parallel
b) Do output route look-up and figure out tx port and queue to transmit
the packet
c) Move to ATOMIC stage based on tx port and tx queue_id to transmit
the packet _without_ losing the ingress ordering
stage_4(ATOMIC) based on tx port/tx queue
-----------------------------------------
a) enqueue the encrypted packet to ethdev tx port/tx_queue
1) queue_id based event pipelining
=================================
stage_1_work(assigned to event queue 1)# N ports/N cores establish
link to queue 1 through rte_event_port_link()
on_each_cores_linked_to_queue1(stage1)
while(1)
{
/* STAGE 1 processing */
nr_events = rte_event_dequeue_burst(ev,..);
if (!nr_events);
continue;
sa = find_sa_from_packet(ev.mbuf);
/* move to next stage(ATOMIC) */
ev.event_type = RTE_EVENT_TYPE_CPU;
ev.sub_event_type = 2;
ev.sched_type = RTE_SCHED_TYPE_ATOMIC;
ev.flow_id = sa;
ev.op = RTE_EVENT_OP_FORWARD;
ev.queue_id = 2;
/* move to stage 2(event queue 2) */
rte_event_enqueue_burst(ev,..);
}
on_each_cores_linked_to_queue2(stage2)
while(1)
{
/* STAGE 2 processing */
nr_events = rte_event_dequeue_burst(ev,..);
if (!nr_events);
continue;
sa_specific_atomic_processing(sa /* ev.flow_id */);/* seq number update in critical section */
/* move to next stage(ORDERED) */
ev.event_type = RTE_EVENT_TYPE_CPU;
ev.sub_event_type = 3;
ev.sched_type = RTE_SCHED_TYPE_ORDERED;
ev.flow_id = sa;
ev.op = RTE_EVENT_OP_FORWARD;
ev.queue_id = 3;
/* move to stage 3(event queue 3) */
rte_event_enqueue_burst(ev,..);
}
on_each_cores_linked_to_queue3(stage3)
while(1)
{
/* STAGE 3 processing */
nr_events = rte_event_dequeue_burst(ev,..);
if (!nr_events);
continue;
sa_specific_ordered_processing(sa /*ev.flow_id */);/* packets encryption in parallel */
/* move to next stage(ATOMIC) */
ev.event_type = RTE_EVENT_TYPE_CPU;
ev.sub_event_type = 4;
ev.sched_type = RTE_SCHED_TYPE_ATOMIC;
output_tx_port_queue = find_output_tx_queue_and_tx_port(ev.mbuff);
ev.flow_id = output_tx_port_queue;
ev.op = RTE_EVENT_OP_FORWARD;
ev.queue_id = 4;
/* move to stage 4(event queue 4) */
rte_event_enqueue_burst(ev,...);
}
on_each_cores_linked_to_queue4(stage4)
while(1)
{
/* STAGE 4 processing */
nr_events = rte_event_dequeue_burst(ev,..);
if (!nr_events);
continue;
rte_eth_tx_buffer();
}
2) flow-based event pipelining
=============================
- No need to partition queues for different stages
- All the cores can operate on all the stages, Thus enables
automatic multicore scaling, true dynamic load balancing,
- Fairly large number of SA(kind of 2^16 to 2^20) can be processed in parallel
Something existing IPSec application has constraints on
http://dpdk.org/doc/guides-16.04/sample_app_ug/ipsec_secgw.html
on_each_worker_cores()
while(1)
{
rte_event_dequeue_burst(ev,..)
if (!nr_events);
continue;
/* STAGE 1 processing */
if(ev.event_type == RTE_EVENT_TYPE_ETHDEV) {
sa = find_it_from_packet(ev.mbuf);
/* move to next stage2(ATOMIC) */
ev.event_type = RTE_EVENT_TYPE_CPU;
ev.sub_event_type = 2;
ev.sched_type = RTE_SCHED_TYPE_ATOMIC;
ev.flow_id = sa;
ev.op = RTE_EVENT_OP_FORWARD;
rte_event_enqueue_burst(ev..);
} else if(ev.event_type == RTE_EVENT_TYPE_CPU && ev.sub_event_type == 2) { /* stage 2 */
sa_specific_atomic_processing(sa /* ev.flow_id */);/* seq number update in critical section */
/* move to next stage(ORDERED) */
ev.event_type = RTE_EVENT_TYPE_CPU;
ev.sub_event_type = 3;
ev.sched_type = RTE_SCHED_TYPE_ORDERED;
ev.flow_id = sa;
ev.op = RTE_EVENT_OP_FORWARD;
rte_event_enqueue_burst(ev,..);
} else if(ev.event_type == RTE_EVENT_TYPE_CPU && ev.sub_event_type == 3) { /* stage 3 */
sa_specific_ordered_processing(sa /* ev.flow_id */);/* like encrypting packets in parallel */
/* move to next stage(ATOMIC) */
ev.event_type = RTE_EVENT_TYPE_CPU;
ev.sub_event_type = 4;
ev.sched_type = RTE_SCHED_TYPE_ATOMIC;
output_tx_port_queue = find_output_tx_queue_and_tx_port(ev.mbuff);
ev.flow_id = output_tx_port_queue;
ev.op = RTE_EVENT_OP_FORWARD;
rte_event_enqueue_burst(ev,..);
} else if(ev.event_type == RTE_EVENT_TYPE_CPU && ev.sub_event_type == 4) { /* stage 4 */
rte_eth_tx_buffer();
}
}
/Jerin
Cavium
next prev parent reply other threads:[~2017-02-08 10:23 UTC|newest]
Thread overview: 91+ messages / expand[flat|nested] mbox.gz Atom feed top
[not found] <1484580885-148524-1-git-send-email-harry.van.haaren@intel.com>
2017-01-31 16:14 ` [dpdk-dev] [PATCH v2 00/15] next-eventdev: event/sw software eventdev Harry van Haaren
2017-01-31 16:14 ` [dpdk-dev] [PATCH v2 01/15] eventdev: remove unneeded dependencies Harry van Haaren
2017-02-06 8:12 ` Jerin Jacob
2017-02-08 14:35 ` Jerin Jacob
2017-01-31 16:14 ` [dpdk-dev] [PATCH v2 02/15] eventdev: add APIs for extended stats Harry van Haaren
2017-02-06 8:22 ` Jerin Jacob
2017-02-06 10:37 ` Van Haaren, Harry
2017-02-07 6:24 ` Jerin Jacob
2017-02-09 14:04 ` Van Haaren, Harry
2017-01-31 16:14 ` [dpdk-dev] [PATCH v2 03/15] event/sw: add new software-only eventdev driver Harry van Haaren
2017-02-06 8:32 ` Jerin Jacob
2017-01-31 16:14 ` [dpdk-dev] [PATCH v2 04/15] event/sw: add device capabilities function Harry van Haaren
2017-02-06 8:34 ` Jerin Jacob
2017-01-31 16:14 ` [dpdk-dev] [PATCH v2 05/15] event/sw: add configure function Harry van Haaren
2017-01-31 16:14 ` [dpdk-dev] [PATCH v2 06/15] event/sw: add fns to return default port/queue config Harry van Haaren
2017-01-31 16:14 ` [dpdk-dev] [PATCH v2 07/15] event/sw: add support for event queues Harry van Haaren
2017-02-06 9:25 ` Jerin Jacob
2017-02-06 10:25 ` Van Haaren, Harry
2017-02-07 6:58 ` Jerin Jacob
2017-02-07 9:58 ` Van Haaren, Harry
2017-01-31 16:14 ` [dpdk-dev] [PATCH v2 08/15] event/sw: add support for event ports Harry van Haaren
2017-01-31 16:14 ` [dpdk-dev] [PATCH v2 09/15] event/sw: add support for linking queues to ports Harry van Haaren
2017-02-06 9:37 ` Jerin Jacob
2017-01-31 16:14 ` [dpdk-dev] [PATCH v2 10/15] event/sw: add worker core functions Harry van Haaren
2017-01-31 16:14 ` [dpdk-dev] [PATCH v2 11/15] event/sw: add scheduling logic Harry van Haaren
2017-01-31 16:14 ` [dpdk-dev] [PATCH v2 12/15] event/sw: add start stop and close functions Harry van Haaren
2017-01-31 16:14 ` [dpdk-dev] [PATCH v2 13/15] event/sw: add dump function for easier debugging Harry van Haaren
2017-01-31 16:14 ` [dpdk-dev] [PATCH v2 14/15] event/sw: add xstats support Harry van Haaren
2017-01-31 16:14 ` [dpdk-dev] [PATCH v2 15/15] app/test: add unit tests for SW eventdev driver Harry van Haaren
2017-02-08 10:23 ` Jerin Jacob [this message]
2017-02-08 10:44 ` Van Haaren, Harry
2017-02-13 10:28 ` Jerin Jacob
2017-02-13 10:45 ` Bruce Richardson
2017-02-13 11:21 ` Jerin Jacob
2017-02-08 18:02 ` Nipun Gupta
2017-02-13 11:37 ` Jerin Jacob
2017-02-11 9:13 ` Jerin Jacob
2017-02-06 8:07 ` [dpdk-dev] [PATCH v2 00/15] next-eventdev: event/sw software eventdev Jerin Jacob
2017-02-06 10:14 ` Van Haaren, Harry
2017-02-17 14:53 ` [dpdk-dev] [PATCH v3 00/17] " Harry van Haaren
2017-02-17 14:53 ` [dpdk-dev] [PATCH v3 01/17] eventdev: fix API docs and test for timeout ticks Harry van Haaren
2017-02-20 15:22 ` Mcnamara, John
2017-03-06 10:33 ` Jerin Jacob
2017-03-10 15:24 ` Van Haaren, Harry
2017-03-08 10:35 ` [dpdk-dev] [PATCH] eventdev: improve API docs " Harry van Haaren
2017-03-13 8:48 ` Jerin Jacob
2017-02-17 14:53 ` [dpdk-dev] [PATCH v3 02/17] eventdev: increase size of enq deq conf variables Harry van Haaren
2017-02-19 12:05 ` Jerin Jacob
2017-02-17 14:53 ` [dpdk-dev] [PATCH v3 03/17] app/test: eventdev link all queues before start Harry van Haaren
2017-02-19 12:09 ` Jerin Jacob
2017-02-17 14:53 ` [dpdk-dev] [PATCH v3 04/17] eventdev: add APIs for extended stats Harry van Haaren
2017-02-19 12:32 ` Jerin Jacob
2017-02-20 12:12 ` Van Haaren, Harry
2017-02-20 12:34 ` Jerin Jacob
2017-02-17 14:54 ` [dpdk-dev] [PATCH v3 05/17] event/sw: add new software-only eventdev driver Harry van Haaren
2017-02-19 12:37 ` Jerin Jacob
2017-02-17 14:54 ` [dpdk-dev] [PATCH v3 06/17] event/sw: add device capabilities function Harry van Haaren
2017-02-17 14:54 ` [dpdk-dev] [PATCH v3 07/17] event/sw: add configure function Harry van Haaren
2017-02-17 14:54 ` [dpdk-dev] [PATCH v3 08/17] event/sw: add fns to return default port/queue config Harry van Haaren
2017-02-17 14:54 ` [dpdk-dev] [PATCH v3 09/17] event/sw: add support for event queues Harry van Haaren
2017-02-17 14:54 ` [dpdk-dev] [PATCH v3 10/17] event/sw: add support for event ports Harry van Haaren
2017-02-17 14:54 ` [dpdk-dev] [PATCH v3 11/17] event/sw: add support for linking queues to ports Harry van Haaren
2017-02-17 14:54 ` [dpdk-dev] [PATCH v3 12/17] event/sw: add worker core functions Harry van Haaren
2017-02-17 14:54 ` [dpdk-dev] [PATCH v3 13/17] event/sw: add scheduling logic Harry van Haaren
2017-02-17 14:54 ` [dpdk-dev] [PATCH v3 14/17] event/sw: add start stop and close functions Harry van Haaren
2017-02-17 14:54 ` [dpdk-dev] [PATCH v3 15/17] event/sw: add dump function for easier debugging Harry van Haaren
2017-02-17 14:54 ` [dpdk-dev] [PATCH v3 16/17] event/sw: add xstats support Harry van Haaren
2017-02-17 14:54 ` [dpdk-dev] [PATCH v3 17/17] app/test: add unit tests for SW eventdev driver Harry van Haaren
2017-03-10 19:43 ` [dpdk-dev] [PATCH v4 00/17] next-eventdev: event/sw software eventdev Harry van Haaren
2017-03-10 19:43 ` [dpdk-dev] [PATCH v4 01/17] eventdev: increase size of enq deq conf variables Harry van Haaren
2017-03-13 8:47 ` Jerin Jacob
2017-03-10 19:43 ` [dpdk-dev] [PATCH v4 02/17] app/test: eventdev link all queues before start Harry van Haaren
2017-03-20 4:46 ` Jerin Jacob
2017-03-23 10:18 ` Jerin Jacob
2017-03-10 19:43 ` [dpdk-dev] [PATCH v4 03/17] test/eventdev: rework timeout ticks test Harry van Haaren
2017-03-10 19:43 ` [dpdk-dev] [PATCH v4 04/17] eventdev: add APIs for extended stats Harry van Haaren
2017-03-17 12:22 ` Jerin Jacob
2017-03-23 10:20 ` Jerin Jacob
2017-03-10 19:43 ` [dpdk-dev] [PATCH v4 05/17] event/sw: add new software-only eventdev driver Harry van Haaren
2017-03-10 19:43 ` [dpdk-dev] [PATCH v4 06/17] event/sw: add device capabilities function Harry van Haaren
2017-03-10 19:43 ` [dpdk-dev] [PATCH v4 07/17] event/sw: add configure function Harry van Haaren
2017-03-10 19:43 ` [dpdk-dev] [PATCH v4 08/17] event/sw: add fns to return default port/queue config Harry van Haaren
2017-03-10 19:43 ` [dpdk-dev] [PATCH v4 09/17] event/sw: add support for event queues Harry van Haaren
2017-03-10 19:43 ` [dpdk-dev] [PATCH v4 10/17] event/sw: add support for event ports Harry van Haaren
2017-03-10 19:43 ` [dpdk-dev] [PATCH v4 11/17] event/sw: add support for linking queues to ports Harry van Haaren
2017-03-10 19:43 ` [dpdk-dev] [PATCH v4 12/17] event/sw: add worker core functions Harry van Haaren
2017-03-10 19:43 ` [dpdk-dev] [PATCH v4 13/17] event/sw: add scheduling logic Harry van Haaren
2017-03-10 19:43 ` [dpdk-dev] [PATCH v4 14/17] event/sw: add start stop and close functions Harry van Haaren
2017-03-10 19:43 ` [dpdk-dev] [PATCH v4 15/17] event/sw: add dump function for easier debugging Harry van Haaren
2017-03-10 19:43 ` [dpdk-dev] [PATCH v4 16/17] event/sw: add xstats support Harry van Haaren
2017-03-10 19:43 ` [dpdk-dev] [PATCH v4 17/17] app/test: add unit tests for SW eventdev driver Harry van Haaren
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20170208102306.GA19597@localhost.localdomain \
--to=jerin.jacob@caviumnetworks.com \
--cc=bruce.richardson@intel.com \
--cc=david.hunt@intel.com \
--cc=dev@dpdk.org \
--cc=gage.eads@intel.com \
--cc=harry.van.haaren@intel.com \
--cc=hemant.agrawal@nxp.com \
--cc=nipun.gupta@nxp.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).