From: Nipun Gupta <nipun.gupta@nxp.com>
To: Jerin Jacob <jerin.jacob@caviumnetworks.com>,
Harry van Haaren <harry.van.haaren@intel.com>
Cc: "dev@dpdk.org" <dev@dpdk.org>,
Bruce Richardson <bruce.richardson@intel.com>,
David Hunt <david.hunt@intel.com>,
"Hemant Agrawal" <hemant.agrawal@nxp.com>,
"gage.eads@intel.com" <gage.eads@intel.com>
Subject: Re: [dpdk-dev] [PATCH v2 15/15] app/test: add unit tests for SW eventdev driver
Date: Wed, 8 Feb 2017 18:02:26 +0000 [thread overview]
Message-ID: <AM5PR0401MB2514B4AB9E7A7E810D5959F5E6420@AM5PR0401MB2514.eurprd04.prod.outlook.com> (raw)
In-Reply-To: <20170208102306.GA19597@localhost.localdomain>
> -----Original Message-----
> From: Jerin Jacob [mailto:jerin.jacob@caviumnetworks.com]
> Sent: Wednesday, February 08, 2017 15:53
> To: Harry van Haaren <harry.van.haaren@intel.com>
> Cc: dev@dpdk.org; Bruce Richardson <bruce.richardson@intel.com>; David
> Hunt <david.hunt@intel.com>; Nipun Gupta <nipun.gupta@nxp.com>; Hemant
> Agrawal <hemant.agrawal@nxp.com>; gage.eads@intel.com
> Subject: Re: [PATCH v2 15/15] app/test: add unit tests for SW eventdev driver
>
> On Tue, Jan 31, 2017 at 04:14:33PM +0000, Harry van Haaren wrote:
> > From: Bruce Richardson <bruce.richardson@intel.com>
> >
> > Since the sw driver is a standalone lookaside device that has no HW
> > requirements, we can provide a set of unit tests that test its
> > functionality across the different queue types and with different input
> > scenarios.
> >
>
> Thanks for SW driver specific test cases. It provided me a good insight
> of expected application behavior from SW driver perspective and in turn it
> created
> some challenge in portable applications.
>
> I would like highlight a main difference between the implementation and get a
> consensus on how to abstract it?
>
> Based on existing header file, We can do event pipelining in two different ways
> a) Flow-based event pipelining
> b) queue_id based event pipelining
>
> I will provide an example to showcase application flow in both modes.
> Based on my understanding from SW driver source code, it supports only
> queue_id based event pipelining. I guess, Flow based event pipelining will
> work semantically with SW driver but it will be very slow.
>
> I think, the reason for the difference is the capability of the context definition.
> SW model the context is - queue_id
> Cavium HW model the context is queue_id + flow_id + sub_event_type +
> event_type
>
> AFAIK, queue_id based event pipelining will work with NXP HW but I am not
> sure about flow based event pipelining model with NXP HW. Appreciate any
> input this?
[Nipun] Yes Jerin, that's right. NXP HW will not be suitable for flow based event pipelining.
>
> In Cavium HW, We support both modes.
>
> As an open question, Should we add a capability flag to advertise the supported
> models and let application choose the model based on implementation
> capability. The
> downside is, a small portion of stage advance code will be different but we
> can reuse the STAGE specific application code(I think it a fair
> trade off)
>
> Bruce, Harry, Gage, Hemant, Nipun
> Thoughts? Or any other proposal?
>
> I will take an non trivial realworld NW use case show the difference.
> A standard IPSec outbound processing will have minimum 4 to 5 stages
>
> stage_0:
> --------
> a) Takes the pkts from ethdev and push to eventdev as
> RTE_EVENT_OP_NEW
> b) Some HW implementation, This will be done by HW. In SW implementation
> it done by service cores
>
> stage_1:(ORDERED)
> ------------------
> a) Receive pkts from stage_0 in ORDERED flow and it process in parallel on N
> of cores
> b) Find a SA belongs that packet move to next stage for SA specific
> outbound operations.Outbound processing starts with updating the
> sequence number in the critical section and followed by packet encryption in
> parallel.
>
> stage_2(ATOMIC) based on SA
> ----------------------------
> a) Update the sequence number and move to ORDERED sched_type for packet
> encryption in parallel
>
> stage_3(ORDERED) based on SA
> ----------------------------
> a) Encrypt the packets in parallel
> b) Do output route look-up and figure out tx port and queue to transmit
> the packet
> c) Move to ATOMIC stage based on tx port and tx queue_id to transmit
> the packet _without_ losing the ingress ordering
>
> stage_4(ATOMIC) based on tx port/tx queue
> -----------------------------------------
> a) enqueue the encrypted packet to ethdev tx port/tx_queue
>
>
> 1) queue_id based event pipelining
> =================================
>
> stage_1_work(assigned to event queue 1)# N ports/N cores establish
> link to queue 1 through rte_event_port_link()
>
> on_each_cores_linked_to_queue1(stage1)
> while(1)
> {
> /* STAGE 1 processing */
> nr_events = rte_event_dequeue_burst(ev,..);
> if (!nr_events);
> continue;
>
> sa = find_sa_from_packet(ev.mbuf);
>
> /* move to next stage(ATOMIC) */
> ev.event_type = RTE_EVENT_TYPE_CPU;
> ev.sub_event_type = 2;
> ev.sched_type = RTE_SCHED_TYPE_ATOMIC;
> ev.flow_id = sa;
> ev.op = RTE_EVENT_OP_FORWARD;
> ev.queue_id = 2;
> /* move to stage 2(event queue 2) */
> rte_event_enqueue_burst(ev,..);
> }
>
> on_each_cores_linked_to_queue2(stage2)
> while(1)
> {
> /* STAGE 2 processing */
> nr_events = rte_event_dequeue_burst(ev,..);
> if (!nr_events);
> continue;
>
> sa_specific_atomic_processing(sa /* ev.flow_id */);/* seq number
> update in critical section */
>
> /* move to next stage(ORDERED) */
> ev.event_type = RTE_EVENT_TYPE_CPU;
> ev.sub_event_type = 3;
> ev.sched_type = RTE_SCHED_TYPE_ORDERED;
> ev.flow_id = sa;
[Nipun] Queue1 has flow_id as an 'sa' with sched_type as RTE_SCHED_TYPE_ATOMIC and
Queue2 has same flow_id but with sched_type as RTE_SCHED_TYPE_ORDERED.
Does this mean that same flow_id be associated with separate RTE_SCHED_TYPE_* as sched_type?
My understanding is that one flow can either be parallel or atomic or ordered.
The rte_eventdev.h states that sched_type is associated with flow_id, which also seems legitimate:
uint8_t sched_type:2;
/**< Scheduler synchronization type (RTE_SCHED_TYPE_*)
* associated with flow id on a given event queue
* for the enqueue and dequeue operation.
*/
> ev.op = RTE_EVENT_OP_FORWARD;
> ev.queue_id = 3;
> /* move to stage 3(event queue 3) */
> rte_event_enqueue_burst(ev,..);
> }
>
> on_each_cores_linked_to_queue3(stage3)
> while(1)
> {
> /* STAGE 3 processing */
> nr_events = rte_event_dequeue_burst(ev,..);
> if (!nr_events);
> continue;
>
> sa_specific_ordered_processing(sa /*ev.flow_id */);/* packets
> encryption in parallel */
>
> /* move to next stage(ATOMIC) */
> ev.event_type = RTE_EVENT_TYPE_CPU;
> ev.sub_event_type = 4;
> ev.sched_type = RTE_SCHED_TYPE_ATOMIC;
> output_tx_port_queue =
> find_output_tx_queue_and_tx_port(ev.mbuff);
> ev.flow_id = output_tx_port_queue;
> ev.op = RTE_EVENT_OP_FORWARD;
> ev.queue_id = 4;
> /* move to stage 4(event queue 4) */
> rte_event_enqueue_burst(ev,...);
> }
>
> on_each_cores_linked_to_queue4(stage4)
> while(1)
> {
> /* STAGE 4 processing */
> nr_events = rte_event_dequeue_burst(ev,..);
> if (!nr_events);
> continue;
>
> rte_eth_tx_buffer();
> }
>
> 2) flow-based event pipelining
> =============================
>
> - No need to partition queues for different stages
> - All the cores can operate on all the stages, Thus enables
> automatic multicore scaling, true dynamic load balancing,
> - Fairly large number of SA(kind of 2^16 to 2^20) can be processed in parallel
> Something existing IPSec application has constraints on
> http://dpdk.org/doc/guides-16.04/sample_app_ug/ipsec_secgw.html
>
> on_each_worker_cores()
> while(1)
> {
> rte_event_dequeue_burst(ev,..)
> if (!nr_events);
> continue;
>
> /* STAGE 1 processing */
> if(ev.event_type == RTE_EVENT_TYPE_ETHDEV) {
> sa = find_it_from_packet(ev.mbuf);
> /* move to next stage2(ATOMIC) */
> ev.event_type = RTE_EVENT_TYPE_CPU;
> ev.sub_event_type = 2;
> ev.sched_type = RTE_SCHED_TYPE_ATOMIC;
> ev.flow_id = sa;
> ev.op = RTE_EVENT_OP_FORWARD;
> rte_event_enqueue_burst(ev..);
>
> } else if(ev.event_type == RTE_EVENT_TYPE_CPU &&
> ev.sub_event_type == 2) { /* stage 2 */
[Nipun] I didn't got that in this case on which event queue (and eventually
its associated event ports) will the RTE_EVENT_TYPE_CPU type events be received on?
Adding on to what Harry also mentions in other mail, If same code is run in the case you
mentioned in '#1 - queue_id based event pipelining', after specifying the ev.queue_id
with appropriate value then also #1 would be good. Isn't it?
>
> sa_specific_atomic_processing(sa /* ev.flow_id */);/* seq
> number update in critical section */
> /* move to next stage(ORDERED) */
> ev.event_type = RTE_EVENT_TYPE_CPU;
> ev.sub_event_type = 3;
> ev.sched_type = RTE_SCHED_TYPE_ORDERED;
> ev.flow_id = sa;
> ev.op = RTE_EVENT_OP_FORWARD;
> rte_event_enqueue_burst(ev,..);
>
> } else if(ev.event_type == RTE_EVENT_TYPE_CPU &&
> ev.sub_event_type == 3) { /* stage 3 */
>
> sa_specific_ordered_processing(sa /* ev.flow_id */);/* like
> encrypting packets in parallel */
> /* move to next stage(ATOMIC) */
> ev.event_type = RTE_EVENT_TYPE_CPU;
> ev.sub_event_type = 4;
> ev.sched_type = RTE_SCHED_TYPE_ATOMIC;
> output_tx_port_queue =
> find_output_tx_queue_and_tx_port(ev.mbuff);
> ev.flow_id = output_tx_port_queue;
> ev.op = RTE_EVENT_OP_FORWARD;
> rte_event_enqueue_burst(ev,..);
>
> } else if(ev.event_type == RTE_EVENT_TYPE_CPU &&
> ev.sub_event_type == 4) { /* stage 4 */
> rte_eth_tx_buffer();
> }
> }
>
> /Jerin
> Cavium
next prev parent reply other threads:[~2017-02-08 18:02 UTC|newest]
Thread overview: 91+ messages / expand[flat|nested] mbox.gz Atom feed top
[not found] <1484580885-148524-1-git-send-email-harry.van.haaren@intel.com>
2017-01-31 16:14 ` [dpdk-dev] [PATCH v2 00/15] next-eventdev: event/sw software eventdev Harry van Haaren
2017-01-31 16:14 ` [dpdk-dev] [PATCH v2 01/15] eventdev: remove unneeded dependencies Harry van Haaren
2017-02-06 8:12 ` Jerin Jacob
2017-02-08 14:35 ` Jerin Jacob
2017-01-31 16:14 ` [dpdk-dev] [PATCH v2 02/15] eventdev: add APIs for extended stats Harry van Haaren
2017-02-06 8:22 ` Jerin Jacob
2017-02-06 10:37 ` Van Haaren, Harry
2017-02-07 6:24 ` Jerin Jacob
2017-02-09 14:04 ` Van Haaren, Harry
2017-01-31 16:14 ` [dpdk-dev] [PATCH v2 03/15] event/sw: add new software-only eventdev driver Harry van Haaren
2017-02-06 8:32 ` Jerin Jacob
2017-01-31 16:14 ` [dpdk-dev] [PATCH v2 04/15] event/sw: add device capabilities function Harry van Haaren
2017-02-06 8:34 ` Jerin Jacob
2017-01-31 16:14 ` [dpdk-dev] [PATCH v2 05/15] event/sw: add configure function Harry van Haaren
2017-01-31 16:14 ` [dpdk-dev] [PATCH v2 06/15] event/sw: add fns to return default port/queue config Harry van Haaren
2017-01-31 16:14 ` [dpdk-dev] [PATCH v2 07/15] event/sw: add support for event queues Harry van Haaren
2017-02-06 9:25 ` Jerin Jacob
2017-02-06 10:25 ` Van Haaren, Harry
2017-02-07 6:58 ` Jerin Jacob
2017-02-07 9:58 ` Van Haaren, Harry
2017-01-31 16:14 ` [dpdk-dev] [PATCH v2 08/15] event/sw: add support for event ports Harry van Haaren
2017-01-31 16:14 ` [dpdk-dev] [PATCH v2 09/15] event/sw: add support for linking queues to ports Harry van Haaren
2017-02-06 9:37 ` Jerin Jacob
2017-01-31 16:14 ` [dpdk-dev] [PATCH v2 10/15] event/sw: add worker core functions Harry van Haaren
2017-01-31 16:14 ` [dpdk-dev] [PATCH v2 11/15] event/sw: add scheduling logic Harry van Haaren
2017-01-31 16:14 ` [dpdk-dev] [PATCH v2 12/15] event/sw: add start stop and close functions Harry van Haaren
2017-01-31 16:14 ` [dpdk-dev] [PATCH v2 13/15] event/sw: add dump function for easier debugging Harry van Haaren
2017-01-31 16:14 ` [dpdk-dev] [PATCH v2 14/15] event/sw: add xstats support Harry van Haaren
2017-01-31 16:14 ` [dpdk-dev] [PATCH v2 15/15] app/test: add unit tests for SW eventdev driver Harry van Haaren
2017-02-08 10:23 ` Jerin Jacob
2017-02-08 10:44 ` Van Haaren, Harry
2017-02-13 10:28 ` Jerin Jacob
2017-02-13 10:45 ` Bruce Richardson
2017-02-13 11:21 ` Jerin Jacob
2017-02-08 18:02 ` Nipun Gupta [this message]
2017-02-13 11:37 ` Jerin Jacob
2017-02-11 9:13 ` Jerin Jacob
2017-02-06 8:07 ` [dpdk-dev] [PATCH v2 00/15] next-eventdev: event/sw software eventdev Jerin Jacob
2017-02-06 10:14 ` Van Haaren, Harry
2017-02-17 14:53 ` [dpdk-dev] [PATCH v3 00/17] " Harry van Haaren
2017-02-17 14:53 ` [dpdk-dev] [PATCH v3 01/17] eventdev: fix API docs and test for timeout ticks Harry van Haaren
2017-02-20 15:22 ` Mcnamara, John
2017-03-06 10:33 ` Jerin Jacob
2017-03-10 15:24 ` Van Haaren, Harry
2017-03-08 10:35 ` [dpdk-dev] [PATCH] eventdev: improve API docs " Harry van Haaren
2017-03-13 8:48 ` Jerin Jacob
2017-02-17 14:53 ` [dpdk-dev] [PATCH v3 02/17] eventdev: increase size of enq deq conf variables Harry van Haaren
2017-02-19 12:05 ` Jerin Jacob
2017-02-17 14:53 ` [dpdk-dev] [PATCH v3 03/17] app/test: eventdev link all queues before start Harry van Haaren
2017-02-19 12:09 ` Jerin Jacob
2017-02-17 14:53 ` [dpdk-dev] [PATCH v3 04/17] eventdev: add APIs for extended stats Harry van Haaren
2017-02-19 12:32 ` Jerin Jacob
2017-02-20 12:12 ` Van Haaren, Harry
2017-02-20 12:34 ` Jerin Jacob
2017-02-17 14:54 ` [dpdk-dev] [PATCH v3 05/17] event/sw: add new software-only eventdev driver Harry van Haaren
2017-02-19 12:37 ` Jerin Jacob
2017-02-17 14:54 ` [dpdk-dev] [PATCH v3 06/17] event/sw: add device capabilities function Harry van Haaren
2017-02-17 14:54 ` [dpdk-dev] [PATCH v3 07/17] event/sw: add configure function Harry van Haaren
2017-02-17 14:54 ` [dpdk-dev] [PATCH v3 08/17] event/sw: add fns to return default port/queue config Harry van Haaren
2017-02-17 14:54 ` [dpdk-dev] [PATCH v3 09/17] event/sw: add support for event queues Harry van Haaren
2017-02-17 14:54 ` [dpdk-dev] [PATCH v3 10/17] event/sw: add support for event ports Harry van Haaren
2017-02-17 14:54 ` [dpdk-dev] [PATCH v3 11/17] event/sw: add support for linking queues to ports Harry van Haaren
2017-02-17 14:54 ` [dpdk-dev] [PATCH v3 12/17] event/sw: add worker core functions Harry van Haaren
2017-02-17 14:54 ` [dpdk-dev] [PATCH v3 13/17] event/sw: add scheduling logic Harry van Haaren
2017-02-17 14:54 ` [dpdk-dev] [PATCH v3 14/17] event/sw: add start stop and close functions Harry van Haaren
2017-02-17 14:54 ` [dpdk-dev] [PATCH v3 15/17] event/sw: add dump function for easier debugging Harry van Haaren
2017-02-17 14:54 ` [dpdk-dev] [PATCH v3 16/17] event/sw: add xstats support Harry van Haaren
2017-02-17 14:54 ` [dpdk-dev] [PATCH v3 17/17] app/test: add unit tests for SW eventdev driver Harry van Haaren
2017-03-10 19:43 ` [dpdk-dev] [PATCH v4 00/17] next-eventdev: event/sw software eventdev Harry van Haaren
2017-03-10 19:43 ` [dpdk-dev] [PATCH v4 01/17] eventdev: increase size of enq deq conf variables Harry van Haaren
2017-03-13 8:47 ` Jerin Jacob
2017-03-10 19:43 ` [dpdk-dev] [PATCH v4 02/17] app/test: eventdev link all queues before start Harry van Haaren
2017-03-20 4:46 ` Jerin Jacob
2017-03-23 10:18 ` Jerin Jacob
2017-03-10 19:43 ` [dpdk-dev] [PATCH v4 03/17] test/eventdev: rework timeout ticks test Harry van Haaren
2017-03-10 19:43 ` [dpdk-dev] [PATCH v4 04/17] eventdev: add APIs for extended stats Harry van Haaren
2017-03-17 12:22 ` Jerin Jacob
2017-03-23 10:20 ` Jerin Jacob
2017-03-10 19:43 ` [dpdk-dev] [PATCH v4 05/17] event/sw: add new software-only eventdev driver Harry van Haaren
2017-03-10 19:43 ` [dpdk-dev] [PATCH v4 06/17] event/sw: add device capabilities function Harry van Haaren
2017-03-10 19:43 ` [dpdk-dev] [PATCH v4 07/17] event/sw: add configure function Harry van Haaren
2017-03-10 19:43 ` [dpdk-dev] [PATCH v4 08/17] event/sw: add fns to return default port/queue config Harry van Haaren
2017-03-10 19:43 ` [dpdk-dev] [PATCH v4 09/17] event/sw: add support for event queues Harry van Haaren
2017-03-10 19:43 ` [dpdk-dev] [PATCH v4 10/17] event/sw: add support for event ports Harry van Haaren
2017-03-10 19:43 ` [dpdk-dev] [PATCH v4 11/17] event/sw: add support for linking queues to ports Harry van Haaren
2017-03-10 19:43 ` [dpdk-dev] [PATCH v4 12/17] event/sw: add worker core functions Harry van Haaren
2017-03-10 19:43 ` [dpdk-dev] [PATCH v4 13/17] event/sw: add scheduling logic Harry van Haaren
2017-03-10 19:43 ` [dpdk-dev] [PATCH v4 14/17] event/sw: add start stop and close functions Harry van Haaren
2017-03-10 19:43 ` [dpdk-dev] [PATCH v4 15/17] event/sw: add dump function for easier debugging Harry van Haaren
2017-03-10 19:43 ` [dpdk-dev] [PATCH v4 16/17] event/sw: add xstats support Harry van Haaren
2017-03-10 19:43 ` [dpdk-dev] [PATCH v4 17/17] app/test: add unit tests for SW eventdev driver Harry van Haaren
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=AM5PR0401MB2514B4AB9E7A7E810D5959F5E6420@AM5PR0401MB2514.eurprd04.prod.outlook.com \
--to=nipun.gupta@nxp.com \
--cc=bruce.richardson@intel.com \
--cc=david.hunt@intel.com \
--cc=dev@dpdk.org \
--cc=gage.eads@intel.com \
--cc=harry.van.haaren@intel.com \
--cc=hemant.agrawal@nxp.com \
--cc=jerin.jacob@caviumnetworks.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).