From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mga02.intel.com (mga02.intel.com [134.134.136.20]) by dpdk.org (Postfix) with ESMTP id F0F4619F5 for ; Wed, 28 Mar 2018 11:21:26 +0200 (CEST) X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from orsmga003.jf.intel.com ([10.7.209.27]) by orsmga101.jf.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 28 Mar 2018 02:21:25 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.48,370,1517904000"; d="scan'208";a="38785845" Received: from aburakov-mobl.ger.corp.intel.com (HELO [10.237.220.112]) ([10.237.220.112]) by orsmga003.jf.intel.com with ESMTP; 28 Mar 2018 02:21:24 -0700 To: "Tan, Jianfeng" , "Van Haaren, Harry" , Thomas Monjalon Cc: "dev@dpdk.org" , "Ananyev, Konstantin" References: <3396888.LEadjR7LpM@xps> <2563064.olffpIeeN1@xps> <944ebaf4-6dc3-5069-1d7e-2ee7bbcd8adc@intel.com> From: "Burakov, Anatoly" Message-ID: <9c6ae3bf-6f45-8148-40c0-757c02fe2102@intel.com> Date: Wed, 28 Mar 2018 10:21:23 +0100 User-Agent: Mozilla/5.0 (Windows NT 10.0; WOW64; rv:52.0) Gecko/20100101 Thunderbird/52.6.0 MIME-Version: 1.0 In-Reply-To: <944ebaf4-6dc3-5069-1d7e-2ee7bbcd8adc@intel.com> Content-Type: text/plain; charset=windows-1252; format=flowed Content-Language: en-US Content-Transfer-Encoding: 7bit Subject: Re: [dpdk-dev] [PATCH v6 2/2] eal: add asynchronous request API to DPDK IPC X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 28 Mar 2018 09:21:27 -0000 On 28-Mar-18 9:55 AM, Tan, Jianfeng wrote: > Hi Thomas and Harry, > > > On 3/28/2018 4:22 PM, Van Haaren, Harry wrote: >>> From: Thomas Monjalon [mailto:thomas@monjalon.net] >>> Sent: Wednesday, March 28, 2018 8:30 AM >>> To: Tan, Jianfeng >>> Cc: Burakov, Anatoly ; dev@dpdk.org; Ananyev, >>> Konstantin ; Van Haaren, Harry >>> >>> Subject: Re: [dpdk-dev] [PATCH v6 2/2] eal: add asynchronous request >>> API to >>> DPDK IPC >>> >>> 28/03/2018 04:08, Tan, Jianfeng: >>>> Hi Thomas , >>>> >>>> From: Thomas Monjalon [mailto:thomas@monjalon.net] >>>>> 27/03/2018 15:59, Anatoly Burakov: >>>>>> Under the hood, we create a separate thread to deal with replies to >>>>>> asynchronous requests, that will just wait to be notified by the >>>>>> main thread, or woken up on a timer. >>>>> I really don't like that a library is creating a thread. >>>>> We don't even know where the thread is created (which core). >>>>> Can it be a rte_service? or in the interrupt thread? >>>> Agree that we'd better not adding so many threads in a library. >>>> >>>> I was considering to merge all the threads into the interrupt thread, >>> however, we don't have an interrupt thread in freebsd. Further, we don't >>> implement alarm API in freebsd. That's why I tend to current >>> implementation, >>> and optimize it later. >>> >>> I would prefer we improve the current code now instead of polluting more >>> with more uncontrolled threads. >>> >>>> For rte_service, it may be not a good idea to reply on it as it needs >>> explicit API calls to setup. >>> >>> I don't see the issue of the explicit API. >>> The IPC is a new service. > > My concern is that not every DPDK application sets up rte_service, but > IPC will be used for very fundamental functions, like memory allocation. > We could not possibly ask all DPDK applications to add rte_service now. > > And also take Harry's comments below into consideration, most likely, we > will move these threads into interrupt thread now by adding > >> Although I do like to see new services, if we want to enable "core" >> dpdk functionality with Services, we need a proper designed solution >> for that. Service cores is not intended for "occasional" work - there >> is no method to block and sleep on a specific service until work >> becomes available, so this would imply a busy-polling. Using a service >> (hence busy polling) for rte_malloc()-based memory mapping requests is >> inefficient, and total overkill :) >> >> For this patch I suggest to use some blocking-read capable mechanism. > > The problem here is that we add too many threads; blocking-read does not > decrease # of threads. > >> >> The above said, in the longer term it would be good to have a design >> that allows new file-descriptors to be added to a "dpdk core" thread, >> which performs occasional lengthy work if the FD has data available. > > Interrupt thread vs rte_service, which is the direction to go? We > actually have some others threads, in vhost and even virtio-user; we can > also avoid those threads if we have a clear direction. > > Thanks, > Jianfeng > Hi all, First of all, @Thomas, this is not a "new library" - it's part of EAL. We're going to be removing a few threads from EAL as it is because of IPC (Jianfeng has already submitted patches for those), so i don't think it's such a big deal to have two IPC threads instead of one. I'm open to suggestions on how to make this work without a second thread, but i don't see it. We've discussed possibility of using rte_service internally, but decided against it for reasons already outlined by Harry - it's not a suitable mechanism for this kind of thing, not as it is. Using interrupt thread for this _will_ work, however this will require a a lot more changes, as currently alarm API allocates everything through rte_malloc, while we want to use IPC for rte_malloc (which would make it a circular dependency). So it'll probably be more API and more complexity for dealing with malloc vs rte_malloc allocations. Hence the least-bad approach taken here: a new thread. -- Thanks, Anatoly