From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mga09.intel.com (mga09.intel.com [134.134.136.24]) by dpdk.org (Postfix) with ESMTP id C0AD82BB0 for ; Tue, 22 Nov 2016 23:48:35 +0100 (CET) Received: from fmsmga005.fm.intel.com ([10.253.24.32]) by orsmga102.jf.intel.com with ESMTP; 22 Nov 2016 14:48:34 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.31,682,1473145200"; d="scan'208";a="34597338" Received: from fmsmsx108.amr.corp.intel.com ([10.18.124.206]) by fmsmga005.fm.intel.com with ESMTP; 22 Nov 2016 14:48:33 -0800 Received: from FMSMSX110.amr.corp.intel.com (10.18.116.10) by FMSMSX108.amr.corp.intel.com (10.18.124.206) with Microsoft SMTP Server (TLS) id 14.3.248.2; Tue, 22 Nov 2016 14:48:33 -0800 Received: from fmsmsx108.amr.corp.intel.com ([169.254.9.109]) by FMSMSX110.amr.corp.intel.com ([169.254.14.199]) with mapi id 14.03.0248.002; Tue, 22 Nov 2016 14:48:33 -0800 From: "Eads, Gage" To: Jerin Jacob CC: "dev@dpdk.org" , "Richardson, Bruce" , "Van Haaren, Harry" , "hemant.agrawal@nxp.com" Thread-Topic: [dpdk-dev] [PATCH 2/4] eventdev: implement the northbound APIs Thread-Index: AQHSQV8LELAhdxcND0O3wvTpAFVOiaDlc3UhgAApkrA= Date: Tue, 22 Nov 2016 22:48:32 +0000 Message-ID: <9184057F7FC11744A2107296B6B8EB1E01E331A3@FMSMSX108.amr.corp.intel.com> References: <1479447902-3700-1-git-send-email-jerin.jacob@caviumnetworks.com> <1479447902-3700-3-git-send-email-jerin.jacob@caviumnetworks.com> <9184057F7FC11744A2107296B6B8EB1E01E31739@FMSMSX108.amr.corp.intel.com> <20161121191358.GA9044@svelivela-lt.caveonetworks.com> <20161121193133.GA9895@svelivela-lt.caveonetworks.com> <9184057F7FC11744A2107296B6B8EB1E01E31C40@FMSMSX108.amr.corp.intel.com> <20161122181913.GA9456@svelivela-lt.caveonetworks.com> <9184057F7FC11744A2107296B6B8EB1E01E32F3E@FMSMSX108.amr.corp.intel.com> <20161122200022.GA12168@svelivela-lt.caveonetworks.com> In-Reply-To: <20161122200022.GA12168@svelivela-lt.caveonetworks.com> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-titus-metadata-40: eyJDYXRlZ29yeUxhYmVscyI6IiIsIk1ldGFkYXRhIjp7Im5zIjoiaHR0cDpcL1wvd3d3LnRpdHVzLmNvbVwvbnNcL0ludGVsMyIsImlkIjoiMGNkZWI4ODYtNmM5OC00NTFiLThlNjYtYjlhZmM3OWQ4ODJjIiwicHJvcHMiOlt7Im4iOiJDVFBDbGFzc2lmaWNhdGlvbiIsInZhbHMiOlt7InZhbHVlIjoiQ1RQX0lDIn1dfV19LCJTdWJqZWN0TGFiZWxzIjpbXSwiVE1DVmVyc2lvbiI6IjE1LjkuNi42IiwiVHJ1c3RlZExhYmVsSGFzaCI6IkRTZHcwclVZVFdmXC82RDNFeFdpRzNJdXQ2TlBmQnVMbnlEK1IxUlwvRWhqVT0ifQ== x-ctpclassification: CTP_IC x-originating-ip: [10.1.200.108] Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Subject: Re: [dpdk-dev] [PATCH 2/4] eventdev: implement the northbound APIs X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: patches and discussions about DPDK List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 22 Nov 2016 22:48:36 -0000 > -----Original Message----- > From: Jerin Jacob [mailto:jerin.jacob@caviumnetworks.com] > Sent: Tuesday, November 22, 2016 2:00 PM > To: Eads, Gage > Cc: dev@dpdk.org; Richardson, Bruce ; Van > Haaren, Harry ; hemant.agrawal@nxp.com > Subject: Re: [dpdk-dev] [PATCH 2/4] eventdev: implement the northbound A= PIs > =20 > On Tue, Nov 22, 2016 at 07:43:03PM +0000, Eads, Gage wrote: > > > > > > > One open issue I noticed is the "typical workflow" > > > description starting in > > rte_eventdev.h:204 conflicts with the > > > centralized software PMD that Harry > > posted last week. > > > Specifically, that PMD expects a single core to call the > > > > > schedule function. We could extend the documentation to account for > > > this > > alternative style of scheduler invocation, or discuss > > > ways to make the software > > PMD work with the documented > > > workflow. I prefer the former, but either way I > > think we > > > ought to expose the scheduler's expected usage to the user -- > > > perhaps > > through an RTE_EVENT_DEV_CAP flag? > > > > > > > > > > > > I prefer former too, you can propose the documentation > > > change required for > > software PMD. > > > > > > > > Sure, proposal follows. The "typical workflow" isn't the most > > > optimal by having a conditional in the fast-path, of course, but it > > > demonstrates the idea simply. > > > > > > > > (line 204) > > > > * An event driven based application has following typical > > > workflow on > > > fastpath: > > > > * \code{.c} > > > > * while (1) { > > > > * > > > > * if (dev_info.event_dev_cap & > > > > * RTE_EVENT_DEV_CAP_DISTRIBUTED_SCHED) > > > > * rte_event_schedule(dev_id); > > > > > > Yes, I like the idea of RTE_EVENT_DEV_CAP_DISTRIBUTED_SCHED. > > > It can be input to application/subsystem to launch separate > > > core(s) for schedule functions. > > > But, I think, the "dev_info.event_dev_cap & > > > RTE_EVENT_DEV_CAP_DISTRIBUTED_SCHED" > > > check can be moved inside the implementation(to make the better > > > decisions and avoiding consuming cycles on HW based schedulers. > > > > How would this check work? Wouldn't it prevent any core from running t= he > software scheduler in the centralized case? > =20 > I guess you may not need RTE_EVENT_DEV_CAP here, instead need flag for > device configure here > =20 > #define RTE_EVENT_DEV_CFG_DISTRIBUTED_SCHED (1ULL << 1) > =20 > struct rte_event_dev_config config; > config.event_dev_cfg =3D RTE_EVENT_DEV_CFG_DISTRIBUTED_SCHED; > rte_event_dev_configure(.., &config); > =20 > on the driver side on configure, > if (config.event_dev_cfg & RTE_EVENT_DEV_CFG_DISTRIBUTED_SCHED) > eventdev->schedule =3D NULL; > else // centralized case > eventdev->schedule =3D your_centrized_schedule_function; > =20 > Does that work? Hm, I fear the API would give users the impression that they can select the= scheduling behavior of a given eventdev, when a software scheduler is more= likely to be either distributed or centralized -- not both. What if we use the capability flag, and define rte_event_schedule() as the = scheduling function for centralized schedulers and rte_event_dequeue() as t= he scheduling function for distributed schedulers? That way, the datapath c= ould be the simple dequeue -> process -> enqueue. Applications would check = the capability flag at configuration time to decide whether or not to launc= h an lcore that calls rte_event_schedule(). > =20 > > > > > > > > > * > > > > * rte_event_dequeue(...); > > > > * > > > > * (event processing) > > > > * > > > > * rte_event_enqueue(...); > > > > * } > > > > * \endcode > > > > * > > > > * The *schedule* operation is intended to do event scheduling, > > > and the > * *dequeue* operation returns the scheduled events. An > > > implementation > * is free to define the semantics between > > > *schedule* and *dequeue*. For > * example, a system based on a > > > hardware scheduler can define its > * rte_event_schedule() to be > > > an NOOP, whereas a software scheduler can use > * the *schedule* > > > operation to schedule events. The > * > > > RTE_EVENT_DEV_CAP_DISTRIBUTED_SCHED capability flag indicates > > > whether > * rte_event_schedule() should be called by all cores or > > > by a single (typically > * dedicated) core. > > > > > > > > (line 308) > > > > #define RTE_EVENT_DEV_CAP_DISTRIBUTED_SCHED (1ULL < 2) > /**< > > > Event scheduling implementation is distributed and all cores must > > > execute > * rte_event_schedule(). If unset, the implementation is > > > centralized and > * a single core must execute the schedule > > > operation. > > > > * > > > > * \see rte_event_schedule() > > > > */ > > > > > > > > > > > > > > > > On same note, If software PMD based workflow need a > > > separate core(s) for > > > schedule function then, Can we hide > > > that from API specification and pass an > > > argument to SW pmd > > > to define the scheduling core(s)? > > > > > > > > > > > > Something like --vdev=3Deventsw0,schedule_cmask=3D0x2 > > > > > > > > An API for controlling the scheduler coremask instead of (or > > > perhaps in addition to) the vdev argument would be good, to allow > > > runtime control. I can imagine apps that scale the number of cores > > > based on load, and in doing so may want to migrate the scheduler to= a > different core. > > > > > > Yes, an API for number of scheduler core looks OK. But if we are > > > going to have service core approach then we just need to specify at > > > one place as application will not creating the service functions. > > > > > > > > > > > > > > > > > Just a thought, > > > > > > > > > > Perhaps, We could introduce generic "service" cores concept to > > > DPDK to hide > > the > > requirement where the implementation > > > needs dedicated core to do certain > > work. I guess it would > > > useful for other NPU integration in DPDK. > > > > > > > > > > > > > That's an interesting idea. As you suggested in the other thread, > > > this concept could be extended to the "producer" code in the > > > example for configurations where the NIC requires software to feed > > > into the eventdev. And to the other subsystems mentioned in your or= iginal > PDF, crypto and timer. > > > > > > Yes. Producers should come in service core category. I think, that > > > enables us to have better NPU integration.(same application code for > > > NPU vs non NPU) > > >