DPDK patches and discussions
 help / color / mirror / Atom feed
From: "Eads, Gage" <gage.eads@intel.com>
To: Jerin Jacob <jerin.jacob@caviumnetworks.com>
Cc: "dev@dpdk.org" <dev@dpdk.org>,
	"Richardson, Bruce" <bruce.richardson@intel.com>,
	"Van Haaren, Harry" <harry.van.haaren@intel.com>,
	"hemant.agrawal@nxp.com" <hemant.agrawal@nxp.com>
Subject: Re: [dpdk-dev] [PATCH 2/4] eventdev: implement the northbound APIs
Date: Mon, 28 Nov 2016 15:53:08 +0000	[thread overview]
Message-ID: <9184057F7FC11744A2107296B6B8EB1E01E33E96@FMSMSX108.amr.corp.intel.com> (raw)
In-Reply-To: <20161122234331.GA20501@svelivela-lt.caveonetworks.com>

(Bruce's adviced heeded :))

>  -----Original Message-----
>  From: Jerin Jacob [mailto:jerin.jacob@caviumnetworks.com]
>  Sent: Tuesday, November 22, 2016 5:44 PM
>  To: Eads, Gage <gage.eads@intel.com>
>  Cc: dev@dpdk.org; Richardson, Bruce <bruce.richardson@intel.com>; Van
>  Haaren, Harry <harry.van.haaren@intel.com>; hemant.agrawal@nxp.com
>  Subject: Re: [dpdk-dev] [PATCH 2/4] eventdev: implement the northbound APIs
>  
>  On Tue, Nov 22, 2016 at 10:48:32PM +0000, Eads, Gage wrote:
>  >
>  >
>  > >  -----Original Message-----
>  > >  From: Jerin Jacob [mailto:jerin.jacob@caviumnetworks.com]
>  > >  Sent: Tuesday, November 22, 2016 2:00 PM
>  > >  To: Eads, Gage <gage.eads@intel.com>
>  > >  Cc: dev@dpdk.org; Richardson, Bruce <bruce.richardson@intel.com>;
>  > > Van  Haaren, Harry <harry.van.haaren@intel.com>;
>  > > hemant.agrawal@nxp.com
>  > >  Subject: Re: [dpdk-dev] [PATCH 2/4] eventdev: implement the
>  > > northbound APIs
>  > >
>  > >  On Tue, Nov 22, 2016 at 07:43:03PM +0000, Eads, Gage wrote:
>  > >  > >  > >  > > One open issue I noticed is the "typical workflow"
>  > >  > > description starting in  > >  rte_eventdev.h:204 conflicts with
>  > > the  > > centralized software PMD that Harry  > >  posted last week.
>  > >  > > Specifically, that PMD expects a single core to call the  > >
>  > > > > schedule function. We could extend the documentation to account
>  > > for  > > this  > >  alternative style of scheduler invocation, or
>  > > discuss  > > ways to make the  software  > >  PMD work with the
>  > > documented  > > workflow. I prefer the former, but either  way I  >
>  > > >  think we  > > ought to expose the scheduler's expected usage to
>  > > the user --  > > perhaps  > >  through an RTE_EVENT_DEV_CAP flag?
>  > >  > >  > >  >
>  > >  > >  > >  > I prefer former too, you can propose the documentation
>  > > > > change required  for  > >  software PMD.
>  > >  > >  >
>  > >  > >  > Sure, proposal follows. The "typical workflow" isn't the
>  > > most  > > optimal by  having a conditional in the fast-path, of
>  > > course, but it  > > demonstrates the idea  simply.
>  > >  > >  >
>  > >  > >  > (line 204)
>  > >  > >  >  * An event driven based application has following typical
>  > > > > workflow on  > >  fastpath:
>  > >  > >  >  * \code{.c}
>  > >  > >  >  *      while (1) {
>  > >  > >  >  *
>  > >  > >  >  *              if (dev_info.event_dev_cap &
>  > >  > >  >  *                      RTE_EVENT_DEV_CAP_DISTRIBUTED_SCHED)
>  > >  > >  >  *                      rte_event_schedule(dev_id);
>  > >  > >
>  > >  > >  Yes, I like the idea of RTE_EVENT_DEV_CAP_DISTRIBUTED_SCHED.
>  > >  > >  It  can be input to application/subsystem to  launch separate
>  > > > > core(s) for schedule functions.
>  > >  > >  But, I think, the "dev_info.event_dev_cap &  > >
>  > > RTE_EVENT_DEV_CAP_DISTRIBUTED_SCHED"
>  > >  > >  check can be moved inside the implementation(to make the
>  > > better  > > decisions  and  avoiding consuming cycles on HW based
>  schedulers.
>  > >  >
>  > >  > How would this check work? Wouldn't it prevent any core from
>  > > running the  software scheduler in the centralized case?
>  > >
>  > >  I guess you may not need RTE_EVENT_DEV_CAP here, instead need flag
>  > > for  device configure here
>  > >
>  > >  #define RTE_EVENT_DEV_CFG_DISTRIBUTED_SCHED (1ULL << 1)
>  > >
>  > >  struct rte_event_dev_config config;  config.event_dev_cfg =
>  > > RTE_EVENT_DEV_CFG_DISTRIBUTED_SCHED;
>  > >  rte_event_dev_configure(.., &config);
>  > >
>  > >  on the driver side on configure,
>  > >  if (config.event_dev_cfg & RTE_EVENT_DEV_CFG_DISTRIBUTED_SCHED)
>  > >  	eventdev->schedule = NULL;
>  > >  else // centralized case
>  > >  	eventdev->schedule = your_centrized_schedule_function;
>  > >
>  > >  Does that work?
>  >
>  > Hm, I fear the API would give users the impression that they can select the
>  scheduling behavior of a given eventdev, when a software scheduler is more
>  likely to be either distributed or centralized -- not both.
>  
>  Even if it is capability flag then also it is per "device". Right ?
>  capability flag is more of read only too. Am i missing something here?
>  

Correct, the capability flag I'm envisioning is per-device and read-only. 

>  >
>  > What if we use the capability flag, and define rte_event_schedule() as the
>  scheduling function for centralized schedulers and rte_event_dequeue() as the
>  scheduling function for distributed schedulers? That way, the datapath could be
>  the simple dequeue -> process -> enqueue. Applications would check the
>  capability flag at configuration time to decide whether or not to launch an
>  lcore that calls rte_event_schedule().
>  
>  I am all for simple "dequeue -> process -> enqueue".
>  rte_event_schedule() added for SW scheduler only,  now it may not make sense
>  to add one more check on top of "rte_event_schedule()" to see it is really need
>  or not in fastpath?
>  

Yes, the additional check shouldn't be needed. In terms of the 'typical workflow' description, this is what I have in mind:

*
 * An event driven based application has following typical workflow on fastpath:
 * \code{.c}
 *  while (1) {
 *
 *      rte_event_dequeue(...);
 *
 *      (event processing)
 *
 *      rte_event_enqueue(...);
 *  }
 * \endcode
 *
 * The events are injected to event device through the *enqueue* operation by
 * event producers in the system. The typical event producers are ethdev
 * subsystem for generating packet events, core(SW) for generating events based
 * on different stages of application processing, cryptodev for generating
 * crypto work completion notification etc
 *
 * The *dequeue* operation gets one or more events from the event ports.
 * The application process the events and send to downstream event queue through
 * rte_event_enqueue() if it is an intermediate stage of event processing, on
 * the final stage, the application may send to different subsystem like ethdev
 * to send the packet/event on the wire using ethdev rte_eth_tx_burst() API.
 *
 * The point at which events are scheduled to ports depends on the device. For
 * hardware devices, scheduling occurs asynchronously. Software schedulers can
 * either be distributed (each worker thread schedules events to its own port)
 * or centralized (a dedicated thread schedules to all ports). Distributed
 * software schedulers perform the scheduling in rte_event_dequeue(), whereas
 * centralized scheduler logic is located in rte_event_schedule(). The
 * RTE_EVENT_DEV_CAP_DISTRIBUTED_SCHED capability flag indicates whether a
 * device is centralized and thus needs a dedicated scheduling thread that
 * repeatedly calls rte_event_schedule().
 *
 */

  reply	other threads:[~2016-11-28 15:53 UTC|newest]

Thread overview: 109+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2016-11-18  5:44 [dpdk-dev] [PATCH 0/4] libeventdev API and northbound implementation Jerin Jacob
2016-11-18  5:44 ` [dpdk-dev] [PATCH 1/4] eventdev: introduce event driven programming model Jerin Jacob
2016-11-23 18:39   ` Thomas Monjalon
2016-11-24  1:59     ` Jerin Jacob
2016-11-24 12:26       ` Bruce Richardson
2016-11-24 15:35       ` Thomas Monjalon
2016-11-25  0:23         ` Jerin Jacob
2016-11-25 11:00           ` Bruce Richardson
2016-11-25 13:09             ` Thomas Monjalon
2016-11-26  0:57               ` Jerin Jacob
2016-11-28  9:10                 ` Bruce Richardson
2016-11-26  2:54             ` Jerin Jacob
2016-11-28  9:16               ` Bruce Richardson
2016-11-28 11:30                 ` Thomas Monjalon
2016-11-29  4:01                 ` Jerin Jacob
2016-11-29 10:00                   ` Bruce Richardson
2016-11-25 11:59           ` Van Haaren, Harry
2016-11-25 12:09             ` Richardson, Bruce
2016-11-24 16:24   ` Bruce Richardson
2016-11-24 19:30     ` Jerin Jacob
2016-12-06  3:52   ` [dpdk-dev] [PATCH v2 0/6] libeventdev API and northbound implementation Jerin Jacob
2016-12-06  3:52     ` [dpdk-dev] [PATCH v2 1/6] eventdev: introduce event driven programming model Jerin Jacob
2016-12-06 16:51       ` Bruce Richardson
2016-12-07 18:53         ` Jerin Jacob
2016-12-08  9:30           ` Bruce Richardson
2016-12-08 20:41             ` Jerin Jacob
2016-12-09 15:11               ` Bruce Richardson
2016-12-14  6:55                 ` Jerin Jacob
2016-12-07 10:57       ` Van Haaren, Harry
2016-12-08  1:24         ` Jerin Jacob
2016-12-08 11:02           ` Van Haaren, Harry
2016-12-14 13:13             ` Jerin Jacob
2016-12-14 15:15               ` Bruce Richardson
2016-12-15 16:54               ` Van Haaren, Harry
2016-12-07 11:12       ` Bruce Richardson
2016-12-08  1:48         ` Jerin Jacob
2016-12-08  9:57           ` Bruce Richardson
2016-12-14  6:40             ` Jerin Jacob
2016-12-14 15:19       ` Bruce Richardson
2016-12-15 13:39         ` Jerin Jacob
2016-12-06  3:52     ` [dpdk-dev] [PATCH v2 2/6] eventdev: define southbound driver interface Jerin Jacob
2016-12-06  3:52     ` [dpdk-dev] [PATCH v2 3/6] eventdev: implement the northbound APIs Jerin Jacob
2016-12-06 17:17       ` Bruce Richardson
2016-12-07 17:02         ` Jerin Jacob
2016-12-08  9:59           ` Bruce Richardson
2016-12-14  6:28             ` Jerin Jacob
2016-12-06  3:52     ` [dpdk-dev] [PATCH v2 4/6] eventdev: implement PMD registration functions Jerin Jacob
2016-12-06  3:52     ` [dpdk-dev] [PATCH v2 5/6] event/skeleton: add skeleton eventdev driver Jerin Jacob
2016-12-06  3:52     ` [dpdk-dev] [PATCH v2 6/6] app/test: unit test case for eventdev APIs Jerin Jacob
2016-12-06 16:46     ` [dpdk-dev] [PATCH v2 0/6] libeventdev API and northbound implementation Bruce Richardson
2016-12-21  9:25     ` [dpdk-dev] [PATCH v4 " Jerin Jacob
2016-12-21  9:25       ` [dpdk-dev] [PATCH v4 1/6] eventdev: introduce event driven programming model Jerin Jacob
2017-01-25 16:32         ` Eads, Gage
2017-01-25 16:36           ` Richardson, Bruce
2017-01-25 16:53             ` Eads, Gage
2017-01-25 22:36               ` Eads, Gage
2017-01-26  9:39                 ` Jerin Jacob
2017-01-26 20:39                   ` Eads, Gage
2017-01-27 10:03                     ` Bruce Richardson
2017-01-30 10:42                     ` Jerin Jacob
2017-02-02 11:18         ` Nipun Gupta
2017-02-02 14:09           ` Jerin Jacob
2017-02-03  6:38             ` Nipun Gupta
2017-02-03 10:58               ` Hemant Agrawal
2017-02-07  4:59                 ` Jerin Jacob
2016-12-21  9:25       ` [dpdk-dev] [PATCH v4 2/6] eventdev: define southbound driver interface Jerin Jacob
2017-02-02 11:19         ` Nipun Gupta
2017-02-02 11:34           ` Bruce Richardson
2017-02-02 12:53             ` Nipun Gupta
2017-02-02 13:58               ` Bruce Richardson
2017-02-03  5:59                 ` Nipun Gupta
2016-12-21  9:25       ` [dpdk-dev] [PATCH v4 3/6] eventdev: implement the northbound APIs Jerin Jacob
2017-02-02 11:19         ` Nipun Gupta
2017-02-02 14:32           ` Jerin Jacob
2017-02-03  6:59             ` Nipun Gupta
2016-12-21  9:25       ` [dpdk-dev] [PATCH v4 4/6] eventdev: implement PMD registration functions Jerin Jacob
2017-02-02 11:20         ` Nipun Gupta
2017-02-05 13:04           ` Jerin Jacob
2016-12-21  9:25       ` [dpdk-dev] [PATCH v4 5/6] event/skeleton: add skeleton eventdev driver Jerin Jacob
2016-12-21  9:25       ` [dpdk-dev] [PATCH v4 6/6] app/test: unit test case for eventdev APIs Jerin Jacob
2016-11-18  5:45 ` [dpdk-dev] [PATCH 2/4] eventdev: implement the northbound APIs Jerin Jacob
2016-11-21 17:45   ` Eads, Gage
2016-11-21 19:13     ` Jerin Jacob
2016-11-21 19:31       ` Jerin Jacob
2016-11-22 15:15         ` Eads, Gage
2016-11-22 18:19           ` Jerin Jacob
2016-11-22 19:43             ` Eads, Gage
2016-11-22 20:00               ` Jerin Jacob
2016-11-22 22:48                 ` Eads, Gage
2016-11-22 23:43                   ` Jerin Jacob
2016-11-28 15:53                     ` Eads, Gage [this message]
2016-11-29  2:01                       ` Jerin Jacob
2016-11-29  3:43                       ` Jerin Jacob
2016-11-29  5:46                         ` Eads, Gage
2016-11-23  9:57           ` Bruce Richardson
2016-11-23 19:18   ` Thomas Monjalon
2016-11-25  4:17     ` Jerin Jacob
2016-11-25  9:55       ` Richardson, Bruce
2016-11-25 23:08         ` Jerin Jacob
2016-11-18  5:45 ` [dpdk-dev] [PATCH 3/4] event/skeleton: add skeleton eventdev driver Jerin Jacob
2016-11-18  5:45 ` [dpdk-dev] [PATCH 4/4] app/test: unit test case for eventdev APIs Jerin Jacob
2016-11-18 15:25 ` [dpdk-dev] [PATCH 0/4] libeventdev API and northbound implementation Bruce Richardson
2016-11-18 16:04   ` Bruce Richardson
2016-11-18 19:27     ` Jerin Jacob
2016-11-21  9:40       ` Thomas Monjalon
2016-11-21  9:57         ` Bruce Richardson
2016-11-22  0:11           ` Thomas Monjalon
2016-11-22  2:00       ` Yuanhan Liu
2016-11-22  9:05         ` Shreyansh Jain

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=9184057F7FC11744A2107296B6B8EB1E01E33E96@FMSMSX108.amr.corp.intel.com \
    --to=gage.eads@intel.com \
    --cc=bruce.richardson@intel.com \
    --cc=dev@dpdk.org \
    --cc=harry.van.haaren@intel.com \
    --cc=hemant.agrawal@nxp.com \
    --cc=jerin.jacob@caviumnetworks.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).