From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mga11.intel.com (mga11.intel.com [192.55.52.93]) by dpdk.org (Postfix) with ESMTP id 898FF2C2F for ; Wed, 17 May 2017 14:36:33 +0200 (CEST) Received: from orsmga002.jf.intel.com ([10.7.209.21]) by fmsmga102.fm.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 17 May 2017 05:36:32 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.38,354,1491289200"; d="scan'208";a="87905688" Received: from bricha3-mobl3.ger.corp.intel.com ([10.237.221.42]) by orsmga002.jf.intel.com with SMTP; 17 May 2017 05:36:28 -0700 Received: by (sSMTP sendmail emulation); Wed, 17 May 2017 13:36:28 +0100 Date: Wed, 17 May 2017 13:36:27 +0100 From: Bruce Richardson To: Thomas Monjalon Cc: Harry van Haaren , dev@dpdk.org, hemant.agrawal@nxp.com, nipun.gupta@nxp.com, narender.vangati@intel.com, jerin.jacob@caviumnetworks.com, gage.eads@intel.com, Keith Wiles Message-ID: <20170517123627.GA10692@bricha3-MOBL3.ger.corp.intel.com> References: <1493810961-139469-1-git-send-email-harry.van.haaren@intel.com> <5129240.aMJQgb24zL@xps> <20170517103228.GA14292@bricha3-MOBL3.ger.corp.intel.com> <3696663.2JsLXSjzmN@xps> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <3696663.2JsLXSjzmN@xps> Organization: Intel Research and =?iso-8859-1?Q?De=ACvel?= =?iso-8859-1?Q?opment?= Ireland Ltd. User-Agent: Mutt/1.8.0 (2017-02-23) Subject: Re: [dpdk-dev] [RFC] Service Cores concept X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 17 May 2017 12:36:34 -0000 On Wed, May 17, 2017 at 01:28:14PM +0200, Thomas Monjalon wrote: > 17/05/2017 12:32, Bruce Richardson: > > On Wed, May 17, 2017 at 12:11:10AM +0200, Thomas Monjalon wrote: > > > 03/05/2017 13:29, Harry van Haaren: > > > > The concept is to allow a software function register itself with EAL as > > > > a "service", which requires CPU time to perform its duties. Multiple > > > > services can be registered in an application, if more than one service > > > > exists. The application can retrieve a list of services, and decide how > > > > many "service cores" to use. The number of service cores is removed > > > > from the application usage, and they are mapped to services based on > > > > an application supplied coremask. > > > > > > > > The application now continues as normal, without having to manually > > > > schedule and implement arbitration of CPU time for the SW services. > > > > > > I think it should not be the DPDK responsibility to schedule threads. > > > The mainloops and scheduling are application design choices. > > > > > > If I understand well the idea of your proposal, it is a helper for > > > the application to configure the thread scheduling of known services. > > > So I think we could add interrupt processing and other thread creations > > > in this concept. > > > Could we also embed the rte_eal_mp_remote_launch() calls in this concept? > > > > > > There are a couple of parts of this: > > 1. Allowing libraries and drivers to register the fact that they require > > background processing, e.g. as a SW fallback for functionality that > > would otherwise be implemented in hardware > > 2. Providing support for easily multi-plexing these independent > > functions from different libs onto a different core, compared to the > > normal operation of DPDK of firing a single run-forever function on each > > core. > > 3. Providing support for the application to configure the running of > > these background services on specific cores. > > 4. Once configured, hiding these services and the cores they run on from > > the rest of the application, so that the rest of the app logic does not > > need to change depending on whether service cores are in use or not. For > > instance, removing the service cores from the core list in foreach-lcore > > loops, and preventing the EAL from trying to run app functions on the > > cores when the app calls mp_remote_launch. > > > > Overall, the objective is to provide us a way to have software > > equivalents of hardware functions in as transparent a manner as > > possible. There is a certain amount of scheduling being done by the > > DPDK, but it is still very much under the control of the app. > > > > As for other things being able to use this concept, definite +1 for > > interrupt threads and similar. I would not see mp_remote_launch as being > > affected here in any significant way (except from the hiding service > > cores from it, obviously) > > OK to register CPU needs for services (including interrupts processing). > > Then we could take this opportunity to review how threads are managed. > We will have three types of cores: > - not used > - reserved for services > - used for polling / application processing > It is fine to reserve/define CPU from DPDK point of view. > > Then DPDK launch threads on cores. Maybe we should allow the application > to choose how threads are launched and managed. > Keith was talking about a plugin approach for thread management I think. For thread management, I'd view us as extending what we have with some EAL APIs rather than replacing what is there already. What I think we could with would be APIs to: * spawn an additional thread on a core i.e. add a bit to the coremask * shutdown a thread on a core i.e. remove a bit from the coremask * register an existing thread with DPDK, i.e. give it an lcore_id internally so that it can use DPDK data structures as a first-class citizen. However, while needed, this is separate work from the service cores concept. Regards, /Bruce