From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mga04.intel.com (mga04.intel.com [192.55.52.120]) by dpdk.org (Postfix) with ESMTP id E50A0322C for ; Wed, 28 Mar 2018 11:10:15 +0200 (CEST) X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from orsmga003.jf.intel.com ([10.7.209.27]) by fmsmga104.fm.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 28 Mar 2018 02:10:14 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.48,370,1517904000"; d="scan'208";a="38783770" Received: from irsmsx105.ger.corp.intel.com ([163.33.3.28]) by orsmga003.jf.intel.com with ESMTP; 28 Mar 2018 02:10:13 -0700 Received: from irsmsx112.ger.corp.intel.com (10.108.20.5) by irsmsx105.ger.corp.intel.com (163.33.3.28) with Microsoft SMTP Server (TLS) id 14.3.319.2; Wed, 28 Mar 2018 10:10:12 +0100 Received: from irsmsx101.ger.corp.intel.com ([169.254.1.176]) by irsmsx112.ger.corp.intel.com ([169.254.1.226]) with mapi id 14.03.0319.002; Wed, 28 Mar 2018 10:10:12 +0100 From: "Van Haaren, Harry" To: "Tan, Jianfeng" , Thomas Monjalon CC: "Burakov, Anatoly" , "dev@dpdk.org" , "Ananyev, Konstantin" Thread-Topic: [dpdk-dev] [PATCH v6 2/2] eal: add asynchronous request API to DPDK IPC Thread-Index: AQHTxel7DHKbmMO08ke0/uE2vr7zt6Pk1t0AgABZl4CAABvcYP///AqAgAARGxA= Date: Wed, 28 Mar 2018 09:10:12 +0000 Message-ID: References: <3396888.LEadjR7LpM@xps> <2563064.olffpIeeN1@xps> <944ebaf4-6dc3-5069-1d7e-2ee7bbcd8adc@intel.com> In-Reply-To: <944ebaf4-6dc3-5069-1d7e-2ee7bbcd8adc@intel.com> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-titus-metadata-40: eyJDYXRlZ29yeUxhYmVscyI6IiIsIk1ldGFkYXRhIjp7Im5zIjoiaHR0cDpcL1wvd3d3LnRpdHVzLmNvbVwvbnNcL0ludGVsMyIsImlkIjoiNjczYmU3NDMtZWQwMy00YmZhLWIwNjYtN2Y4MDM1MGViNzRkIiwicHJvcHMiOlt7Im4iOiJDVFBDbGFzc2lmaWNhdGlvbiIsInZhbHMiOlt7InZhbHVlIjoiQ1RQX05UIn1dfV19LCJTdWJqZWN0TGFiZWxzIjpbXSwiVE1DVmVyc2lvbiI6IjE3LjIuNS4xOCIsIlRydXN0ZWRMYWJlbEhhc2giOiI2dE1iY3ZJRFFUU0VnNW9DU0ZveEs4WXhhZDAxeURZRkFxcmM5b1ZsazNscXhUZ3RUazFaU1g0XC9yVEw0bjRYTSJ9 x-ctpclassification: CTP_NT dlp-product: dlpe-windows dlp-version: 11.0.200.100 dlp-reaction: no-action x-originating-ip: [163.33.239.182] Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Subject: Re: [dpdk-dev] [PATCH v6 2/2] eal: add asynchronous request API to DPDK IPC X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 28 Mar 2018 09:10:17 -0000 > -----Original Message----- > From: Tan, Jianfeng > Sent: Wednesday, March 28, 2018 9:55 AM > To: Van Haaren, Harry ; Thomas Monjalon > > Cc: Burakov, Anatoly ; dev@dpdk.org; Ananyev, > Konstantin > Subject: Re: [dpdk-dev] [PATCH v6 2/2] eal: add asynchronous request API = to > DPDK IPC >=20 > Hi Thomas and Harry, Hey, > On 3/28/2018 4:22 PM, Van Haaren, Harry wrote: > >> From: Thomas Monjalon [mailto:thomas@monjalon.net] > >> Sent: Wednesday, March 28, 2018 8:30 AM > >> To: Tan, Jianfeng > >> Cc: Burakov, Anatoly ; dev@dpdk.org; Ananye= v, > >> Konstantin ; Van Haaren, Harry > >> > >> Subject: Re: [dpdk-dev] [PATCH v6 2/2] eal: add asynchronous request A= PI > to > >> DPDK IPC > >> > >> 28/03/2018 04:08, Tan, Jianfeng: > >>> Hi Thomas , > >>> > >>> From: Thomas Monjalon [mailto:thomas@monjalon.net] > >>>> 27/03/2018 15:59, Anatoly Burakov: > >>>>> Under the hood, we create a separate thread to deal with replies to > >>>>> asynchronous requests, that will just wait to be notified by the > >>>>> main thread, or woken up on a timer. > >>>> I really don't like that a library is creating a thread. > >>>> We don't even know where the thread is created (which core). > >>>> Can it be a rte_service? or in the interrupt thread? > >>> Agree that we'd better not adding so many threads in a library. > >>> > >>> I was considering to merge all the threads into the interrupt thread, > >> however, we don't have an interrupt thread in freebsd. Further, we don= 't > >> implement alarm API in freebsd. That's why I tend to current > implementation, > >> and optimize it later. > >> > >> I would prefer we improve the current code now instead of polluting mo= re > >> with more uncontrolled threads. > >> > >>> For rte_service, it may be not a good idea to reply on it as it needs > >> explicit API calls to setup. > >> > >> I don't see the issue of the explicit API. > >> The IPC is a new service. >=20 > My concern is that not every DPDK application sets up rte_service, but > IPC will be used for very fundamental functions, like memory allocation. > We could not possibly ask all DPDK applications to add rte_service now. >=20 > And also take Harry's comments below into consideration, most likely, we > will move these threads into interrupt thread now by adding I don't suggest moving everything into interrupt thread. We need to ensure the interrupt thread (aka, link status interrupt thread) = is not busy for too long. Certain work may cause un-acceptable delay in the interr= upt thread, hence perhaps we should have 2 core DPDK threads: 1) EAL interrupt thread: handles only interrupts, so is ~always blocked on = some FDs 2) EAL "core" thread: handles the IPC work here, and any other irregular wo= rk that can take some time to complete. Adding all core DPDK work to this thread= enables us to refactor and create a scalable / coherent design. > > Although I do like to see new services, if we want to enable "core" dpd= k > functionality with Services, we need a proper designed solution for that. > Service cores is not intended for "occasional" work - there is no method = to > block and sleep on a specific service until work becomes available, so th= is > would imply a busy-polling. Using a service (hence busy polling) for > rte_malloc()-based memory mapping requests is inefficient, and total > overkill :) > > > > For this patch I suggest to use some blocking-read capable mechanism. >=20 > The problem here is that we add too many threads; blocking-read does not > decrease # of threads. To correct my statement just above - some method capable of blocking reads = (as opposed to busy polling). In my mind, this method *must* allow waiting on multiple FDs= , as to *require* only 1 thread. We could use more if it makes sense to do so= . > > The above said, in the longer term it would be good to have a design th= at > allows new file-descriptors to be added to a "dpdk core" thread, which > performs occasional lengthy work if the FD has data available. >=20 > Interrupt thread vs rte_service, which is the direction to go? We > actually have some others threads, in vhost and even virtio-user; we can > also avoid those threads if we have a clear direction. I don't think that service is the correct hammer for this problem. As to interrupt thread or creating a new thread, what makes more sense give= n the current codebase? Is the current implementation an acceptable short-ter= m solution, that gets reworked to be more generic in future?