From: Bruce Richardson <bruce.richardson@intel.com>
To: "Mattias Rönnblom" <hofors@lysator.liu.se>
Cc: Jerin Jacob <jerinjacobk@gmail.com>, <dev@dpdk.org>,
<jerinj@marvell.com>, <mattias.ronnblom@ericsson.com>,
<abdullah.sevincer@intel.com>, <sachin.saxena@oss.nxp.com>,
<hemant.agrawal@nxp.com>, <pbhagavatula@marvell.com>,
<pravin.pathak@intel.com>
Subject: Re: [PATCH v3 09/11] eventdev: improve comments on scheduling types
Date: Tue, 20 Feb 2024 17:23:23 +0000 [thread overview]
Message-ID: <ZdTgC_ozWA5ZrN14@bricha3-mobl1.ger.corp.intel.com> (raw)
In-Reply-To: <0a94b2e5-1c66-4f89-8d28-123ce26217f1@lysator.liu.se>
On Thu, Feb 08, 2024 at 11:04:03AM +0100, Mattias Rönnblom wrote:
> On 2024-02-08 10:18, Jerin Jacob wrote:
> > On Fri, Feb 2, 2024 at 6:11 PM Bruce Richardson
> > <bruce.richardson@intel.com> wrote:
> > >
> > > The description of ordered and atomic scheduling given in the eventdev
> > > doxygen documentation was not always clear. Try and simplify this so
> > > that it is clearer for the end-user of the application
> > >
> > > Signed-off-by: Bruce Richardson <bruce.richardson@intel.com>
> > >
> > > ---
> > > V3: extensive rework following feedback. Please re-review!
> > > ---
> > > lib/eventdev/rte_eventdev.h | 73 +++++++++++++++++++++++--------------
> > > 1 file changed, 45 insertions(+), 28 deletions(-)
> > >
> > > diff --git a/lib/eventdev/rte_eventdev.h b/lib/eventdev/rte_eventdev.h
> > > index a7d8c28015..8d72765ae7 100644
> > > --- a/lib/eventdev/rte_eventdev.h
> > > +++ b/lib/eventdev/rte_eventdev.h
> > > @@ -1347,25 +1347,35 @@ struct rte_event_vector {
> > > /**< Ordered scheduling
> > > *
> > > * Events from an ordered flow of an event queue can be scheduled to multiple
> > > - * ports for concurrent processing while maintaining the original event order.
> > > + * ports for concurrent processing while maintaining the original event order,
> > > + * i.e. the order in which they were first enqueued to that queue.
> > > * This scheme enables the user to achieve high single flow throughput by
> > > - * avoiding SW synchronization for ordering between ports which bound to cores.
> > > - *
> > > - * The source flow ordering from an event queue is maintained when events are
> > > - * enqueued to their destination queue within the same ordered flow context.
> > > - * An event port holds the context until application call
> > > - * rte_event_dequeue_burst() from the same port, which implicitly releases
> > > - * the context.
> > > - * User may allow the scheduler to release the context earlier than that
> > > - * by invoking rte_event_enqueue_burst() with RTE_EVENT_OP_RELEASE operation.
> > > - *
> > > - * Events from the source queue appear in their original order when dequeued
> > > - * from a destination queue.
> > > - * Event ordering is based on the received event(s), but also other
> > > - * (newly allocated or stored) events are ordered when enqueued within the same
> > > - * ordered context. Events not enqueued (e.g. released or stored) within the
> > > - * context are considered missing from reordering and are skipped at this time
> > > - * (but can be ordered again within another context).
> > > + * avoiding SW synchronization for ordering between ports which are polled
> > > + * by different cores.
> >
> > I prefer the following version to remove "polled" and to be more explicit.
> >
> > avoiding SW synchronization for ordering between ports which are
> > dequeuing events
> > using @ref rte_event_deque_burst() across different cores.
> >
>
> "This scheme allows events pertaining to the same, potentially large flow to
> be processed in parallel on multiple cores without incurring any
> application-level order restoration logic overhead."
>
Ack.
> > > + *
> > > + * After events are dequeued from a set of ports, as those events are re-enqueued
> > > + * to another queue (with the op field set to @ref RTE_EVENT_OP_FORWARD), the event
> > > + * device restores the original event order - including events returned from all
> > > + * ports in the set - before the events arrive on the destination queue.
> >
> > _arrrive_ is bit vague since we have enqueue operation. How about,
> > "before the events actually deposited on the destination queue."
> >
I'll use the term "placed" rather than "deposited".
> >
> > > + *
> > > + * Any events not forwarded i.e. dropped explicitly via RELEASE or implicitly
> > > + * released by the next dequeue operation on a port, are skipped by the reordering
> > > + * stage and do not affect the reordering of other returned events.
> > > + *
> > > + * Any NEW events sent on a port are not ordered with respect to FORWARD events sent
> > > + * on the same port, since they have no original event order. They also are not
> > > + * ordered with respect to NEW events enqueued on other ports.
> > > + * However, NEW events to the same destination queue from the same port are guaranteed
> > > + * to be enqueued in the order they were submitted via rte_event_enqueue_burst().
> > > + *
> > > + * NOTE:
> > > + * In restoring event order of forwarded events, the eventdev API guarantees that
> > > + * all events from the same flow (i.e. same @ref rte_event.flow_id,
> > > + * @ref rte_event.priority and @ref rte_event.queue_id) will be put in the original
> > > + * order before being forwarded to the destination queue.
> > > + * Some eventdevs may implement stricter ordering to achieve this aim,
> > > + * for example, restoring the order across *all* flows dequeued from the same ORDERED
> > > + * queue.
> > > *
> > > * @see rte_event_queue_setup(), rte_event_dequeue_burst(), RTE_EVENT_OP_RELEASE
> > > */
> > > @@ -1373,18 +1383,25 @@ struct rte_event_vector {
> > > #define RTE_SCHED_TYPE_ATOMIC 1
> > > /**< Atomic scheduling
> > > *
> > > - * Events from an atomic flow of an event queue can be scheduled only to a
> > > + * Events from an atomic flow, identified by a combination of @ref rte_event.flow_id,
> > > + * @ref rte_event.queue_id and @ref rte_event.priority, can be scheduled only to a
> > > * single port at a time. The port is guaranteed to have exclusive (atomic)
> > > * access to the associated flow context, which enables the user to avoid SW
> > > - * synchronization. Atomic flows also help to maintain event ordering
> > > - * since only one port at a time can process events from a flow of an
> > > - * event queue.
> > > - *
> > > - * The atomic queue synchronization context is dedicated to the port until
> > > - * application call rte_event_dequeue_burst() from the same port,
> > > - * which implicitly releases the context. User may allow the scheduler to
> > > - * release the context earlier than that by invoking rte_event_enqueue_burst()
> > > - * with RTE_EVENT_OP_RELEASE operation.
> > > + * synchronization. Atomic flows also maintain event ordering
> > > + * since only one port at a time can process events from each flow of an
> > > + * event queue, and events within a flow are not reordered within the scheduler.
> > > + *
> > > + * An atomic flow is locked to a port when events from that flow are first
> > > + * scheduled to that port. That lock remains in place until the
> > > + * application calls rte_event_dequeue_burst() from the same port,
> > > + * which implicitly releases the lock (if @ref RTE_EVENT_PORT_CFG_DISABLE_IMPL_REL flag is not set).
> > > + * User may allow the scheduler to release the lock earlier than that by invoking
> > > + * rte_event_enqueue_burst() with RTE_EVENT_OP_RELEASE operation for each event from that flow.
> > > + *
> > > + * NOTE: The lock is only released once the last event from the flow, outstanding on the port,
> >
> > I think, Note can start with something like below,
> >
> > When there are multiple atomic events dequeue from @ref
> > rte_event_dequeue_burst()
> > for the same event queue, and it has same flow id then the lock is ....
> >
>
> Yes, or maybe describing the whole lock/unlock state.
>
> "The conceptual per-queue-per-flow lock is in a locked state as long (and
> only as long) as one or more events pertaining to that flow were scheduled
> to the port in question, but are not yet released."
>
> Maybe it needs to be more meaty, describing what released means. I don't
> have the full context of the documentation in my head when I'm writing this.
>
I'd rather not go into what "released" means, but I'll reword this a bit in
v4. As part of that, I'll also put in a reference to forwarding events also
releasing the lock.
/Bruce
next prev parent reply other threads:[~2024-02-20 17:24 UTC|newest]
Thread overview: 123+ messages / expand[flat|nested] mbox.gz Atom feed top
2024-01-18 13:45 [PATCH v1 0/7] improve eventdev API specification/documentation Bruce Richardson
2024-01-18 13:45 ` [PATCH v1 1/7] eventdev: improve doxygen introduction text Bruce Richardson
2024-01-18 13:45 ` [PATCH v1 2/7] eventdev: move text on driver internals to proper section Bruce Richardson
2024-01-18 13:45 ` [PATCH v1 3/7] eventdev: update documentation on device capability flags Bruce Richardson
2024-01-18 13:45 ` [PATCH v1 4/7] eventdev: cleanup doxygen comments on info structure Bruce Richardson
2024-01-18 13:49 ` Bruce Richardson
2024-01-18 13:45 ` [PATCH v1 5/7] eventdev: improve function documentation for query fns Bruce Richardson
2024-01-18 13:45 ` [PATCH v1 6/7] eventdev: improve doxygen comments on configure struct Bruce Richardson
2024-01-18 13:45 ` [PATCH v1 7/7] eventdev: fix documentation for counting single-link ports Bruce Richardson
2024-01-19 17:43 ` [PATCH v2 00/11] improve eventdev API specification/documentation Bruce Richardson
2024-01-19 17:43 ` [PATCH v2 01/11] eventdev: improve doxygen introduction text Bruce Richardson
2024-01-23 8:57 ` Mattias Rönnblom
2024-01-23 9:06 ` Bruce Richardson
2024-01-24 11:37 ` Mattias Rönnblom
2024-01-31 13:45 ` Bruce Richardson
2024-01-19 17:43 ` [PATCH v2 02/11] eventdev: move text on driver internals to proper section Bruce Richardson
2024-01-19 17:43 ` [PATCH v2 03/11] eventdev: update documentation on device capability flags Bruce Richardson
2024-01-23 9:18 ` Mattias Rönnblom
2024-01-23 9:34 ` Bruce Richardson
2024-01-31 14:09 ` Bruce Richardson
2024-02-02 8:58 ` Mattias Rönnblom
2024-02-02 11:20 ` Bruce Richardson
2024-01-19 17:43 ` [PATCH v2 04/11] eventdev: cleanup doxygen comments on info structure Bruce Richardson
2024-01-23 9:35 ` Mattias Rönnblom
2024-01-23 9:43 ` Bruce Richardson
2024-01-24 11:51 ` Mattias Rönnblom
2024-01-31 14:37 ` Bruce Richardson
2024-02-02 9:24 ` Mattias Rönnblom
2024-02-02 10:30 ` Bruce Richardson
2024-01-19 17:43 ` [PATCH v2 05/11] eventdev: improve function documentation for query fns Bruce Richardson
2024-01-23 9:40 ` Mattias Rönnblom
2024-01-19 17:43 ` [PATCH v2 06/11] eventdev: improve doxygen comments on configure struct Bruce Richardson
2024-01-23 9:46 ` Mattias Rönnblom
2024-01-31 16:15 ` Bruce Richardson
2024-01-19 17:43 ` [PATCH v2 07/11] eventdev: fix documentation for counting single-link ports Bruce Richardson
2024-01-23 9:48 ` Mattias Rönnblom
2024-01-23 9:56 ` Bruce Richardson
2024-01-31 16:18 ` Bruce Richardson
2024-01-19 17:43 ` [PATCH v2 08/11] eventdev: improve doxygen comments on config fns Bruce Richardson
2024-01-23 10:00 ` Mattias Rönnblom
2024-01-23 10:07 ` Bruce Richardson
2024-01-19 17:43 ` [PATCH v2 09/11] eventdev: improve doxygen comments for control APIs Bruce Richardson
2024-01-23 10:10 ` Mattias Rönnblom
2024-01-19 17:43 ` [PATCH v2 10/11] eventdev: RFC clarify comments on scheduling types Bruce Richardson
2024-01-23 16:19 ` Mattias Rönnblom
2024-01-24 11:21 ` Bruce Richardson
2024-01-31 17:54 ` Bruce Richardson
2024-01-19 17:43 ` [PATCH v2 11/11] eventdev: RFC clarify docs on event object fields Bruce Richardson
2024-01-24 11:34 ` Mattias Rönnblom
2024-02-01 16:59 ` Bruce Richardson
2024-02-02 9:38 ` Mattias Rönnblom
2024-02-02 11:33 ` Bruce Richardson
2024-02-02 12:02 ` Bruce Richardson
2024-02-01 17:02 ` Bruce Richardson
2024-02-02 9:14 ` Bruce Richardson
2024-02-02 9:22 ` Jerin Jacob
2024-02-02 9:36 ` Bruce Richardson
2024-02-02 9:45 ` Mattias Rönnblom
2024-02-02 10:32 ` Bruce Richardson
2024-02-01 9:35 ` Bruce Richardson
2024-02-01 15:00 ` Jerin Jacob
2024-02-01 15:24 ` Bruce Richardson
2024-02-01 16:20 ` Jerin Jacob
2024-02-02 12:39 ` [PATCH v3 00/11] improve eventdev API specification/documentation Bruce Richardson
2024-02-02 12:39 ` [PATCH v3 01/11] eventdev: improve doxygen introduction text Bruce Richardson
2024-02-07 10:14 ` Jerin Jacob
2024-02-08 9:50 ` Mattias Rönnblom
2024-02-09 8:43 ` Jerin Jacob
2024-02-10 7:24 ` Mattias Rönnblom
2024-02-20 16:28 ` Bruce Richardson
2024-02-20 16:33 ` Bruce Richardson
2024-02-02 12:39 ` [PATCH v3 02/11] eventdev: move text on driver internals to proper section Bruce Richardson
2024-02-02 12:39 ` [PATCH v3 03/11] eventdev: update documentation on device capability flags Bruce Richardson
2024-02-07 10:30 ` Jerin Jacob
2024-02-20 16:42 ` Bruce Richardson
2024-02-02 12:39 ` [PATCH v3 04/11] eventdev: cleanup doxygen comments on info structure Bruce Richardson
2024-02-02 12:39 ` [PATCH v3 05/11] eventdev: improve function documentation for query fns Bruce Richardson
2024-02-02 12:39 ` [PATCH v3 06/11] eventdev: improve doxygen comments on configure struct Bruce Richardson
2024-02-02 12:39 ` [PATCH v3 07/11] eventdev: improve doxygen comments on config fns Bruce Richardson
2024-02-02 12:39 ` [PATCH v3 08/11] eventdev: improve doxygen comments for control APIs Bruce Richardson
2024-02-02 12:39 ` [PATCH v3 09/11] eventdev: improve comments on scheduling types Bruce Richardson
2024-02-08 9:18 ` Jerin Jacob
2024-02-08 10:04 ` Mattias Rönnblom
2024-02-20 17:23 ` Bruce Richardson [this message]
2024-02-02 12:39 ` [PATCH v3 10/11] eventdev: clarify docs on event object fields and op types Bruce Richardson
2024-02-09 9:14 ` Jerin Jacob
2024-02-20 17:39 ` Bruce Richardson
2024-02-21 9:31 ` Jerin Jacob
2024-02-21 10:28 ` Bruce Richardson
2024-02-20 17:50 ` Bruce Richardson
2024-02-20 18:03 ` Bruce Richardson
2024-02-02 12:39 ` [PATCH v3 11/11] eventdev: drop comment for anon union from doxygen Bruce Richardson
2024-02-21 10:32 ` [PATCH v4 00/12] improve eventdev API specification/documentation Bruce Richardson
2024-02-21 10:32 ` [PATCH v4 01/12] eventdev: improve doxygen introduction text Bruce Richardson
2024-02-26 4:51 ` [EXT] " Pavan Nikhilesh Bhagavatula
2024-02-26 9:59 ` Bruce Richardson
2024-02-29 16:13 ` Jerin Jacob
2024-02-21 10:32 ` [PATCH v4 02/12] eventdev: move text on driver internals to proper section Bruce Richardson
2024-02-26 5:01 ` [EXT] " Pavan Nikhilesh Bhagavatula
2024-02-21 10:32 ` [PATCH v4 03/12] eventdev: update documentation on device capability flags Bruce Richardson
2024-02-26 5:07 ` [EXT] " Pavan Nikhilesh Bhagavatula
2024-02-21 10:32 ` [PATCH v4 04/12] eventdev: cleanup doxygen comments on info structure Bruce Richardson
2024-02-26 5:18 ` [EXT] " Pavan Nikhilesh Bhagavatula
2024-02-21 10:32 ` [PATCH v4 05/12] eventdev: improve function documentation for query fns Bruce Richardson
2024-02-26 5:18 ` [EXT] " Pavan Nikhilesh Bhagavatula
2024-02-21 10:32 ` [PATCH v4 06/12] eventdev: improve doxygen comments on configure struct Bruce Richardson
2024-02-26 6:36 ` [EXT] " Pavan Nikhilesh Bhagavatula
2024-02-21 10:32 ` [PATCH v4 07/12] eventdev: improve doxygen comments on config fns Bruce Richardson
2024-02-26 6:43 ` [EXT] " Pavan Nikhilesh Bhagavatula
2024-02-26 6:44 ` Pavan Nikhilesh Bhagavatula
2024-02-21 10:32 ` [PATCH v4 08/12] eventdev: improve doxygen comments for control APIs Bruce Richardson
2024-02-26 6:44 ` [EXT] " Pavan Nikhilesh Bhagavatula
2024-02-21 10:32 ` [PATCH v4 09/12] eventdev: improve comments on scheduling types Bruce Richardson
2024-02-26 6:49 ` [EXT] " Pavan Nikhilesh Bhagavatula
2024-02-21 10:32 ` [PATCH v4 10/12] eventdev: clarify docs on event object fields and op types Bruce Richardson
2024-02-26 6:52 ` [EXT] " Pavan Nikhilesh Bhagavatula
2024-02-21 10:32 ` [PATCH v4 11/12] eventdev: drop comment for anon union from doxygen Bruce Richardson
2024-02-26 6:52 ` [EXT] " Pavan Nikhilesh Bhagavatula
2024-02-21 10:32 ` [PATCH v4 12/12] eventdev: fix doxygen processing of event vector struct Bruce Richardson
2024-02-26 6:53 ` [EXT] " Pavan Nikhilesh Bhagavatula
2024-03-04 15:35 ` Thomas Monjalon
2024-03-04 15:49 ` Bruce Richardson
2024-02-23 12:36 ` [PATCH v4 00/12] improve eventdev API specification/documentation Jerin Jacob
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=ZdTgC_ozWA5ZrN14@bricha3-mobl1.ger.corp.intel.com \
--to=bruce.richardson@intel.com \
--cc=abdullah.sevincer@intel.com \
--cc=dev@dpdk.org \
--cc=hemant.agrawal@nxp.com \
--cc=hofors@lysator.liu.se \
--cc=jerinj@marvell.com \
--cc=jerinjacobk@gmail.com \
--cc=mattias.ronnblom@ericsson.com \
--cc=pbhagavatula@marvell.com \
--cc=pravin.pathak@intel.com \
--cc=sachin.saxena@oss.nxp.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).