From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mga01.intel.com (mga01.intel.com [192.55.52.88]) by dpdk.org (Postfix) with ESMTP id 2DB7D7E7C for ; Thu, 18 Dec 2014 17:10:48 +0100 (CET) Received: from fmsmga002.fm.intel.com ([10.253.24.26]) by fmsmga101.fm.intel.com with ESMTP; 18 Dec 2014 08:04:19 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.07,601,1413270000"; d="scan'208";a="649946300" Received: from bricha3-mobl3.ger.corp.intel.com ([10.237.220.74]) by fmsmga002.fm.intel.com with SMTP; 18 Dec 2014 08:04:04 -0800 Received: by (sSMTP sendmail emulation); Thu, 18 Dec 2014 16:04:03 +0025 Date: Thu, 18 Dec 2014 16:04:02 +0000 From: Bruce Richardson To: Olivier MATZ Message-ID: <20141218160400.GB2460@bricha3-MOBL3> References: <1418263490-21088-1-git-send-email-cunming.liang@intel.com> <7C4248CAE043B144B1CD242D275626532FE15298@IRSMSX104.ger.corp.intel.com> <7C4248CAE043B144B1CD242D275626532FE232BA@IRSMSX104.ger.corp.intel.com> <7C4248CAE043B144B1CD242D275626532FE27C3B@IRSMSX104.ger.corp.intel.com> <20141218143225.GA2460@bricha3-MOBL3> <5492EE90.4040502@6wind.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <5492EE90.4040502@6wind.com> Organization: Intel Shannon Ltd. User-Agent: Mutt/1.5.23 (2014-03-12) Cc: "dev@dpdk.org" Subject: Re: [dpdk-dev] [RFC PATCH 0/7] support multi-phtread per lcore X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: patches and discussions about DPDK List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 18 Dec 2014 16:10:52 -0000 On Thu, Dec 18, 2014 at 04:11:12PM +0100, Olivier MATZ wrote: > Hi, > > On 12/18/2014 03:32 PM, Bruce Richardson wrote: > > On Thu, Dec 18, 2014 at 12:20:07PM +0000, Walukiewicz, Miroslaw wrote: > >> I have another question regarding your patch. > >> > >> Could we extend values returned by rte_lcore_id() to set them per thread (really the DPDK lcore is a pthread but started on specific core) instead of creating linear thread id. > >> > >> The patch would be much simpler and will work same way. The only change would be extending rte_lcore_id when rte_pthread_create() is called. > >> > >> The value __lcore_id has really an attribute __thread that means it is valid not only per CPU core but also per thread. > >> > >> The mempools, timers, statistics would work without any modifications in that environment. > >> > >> I do not see any reason why old legacy DPDK applications would not work in that model. > >> > >> Mirek > > > > Definite +1 here. > > One remark though: it looks that the rte_rings (and therefore the > rte_mempools) are designed with the assumption that the execution > units are alone on their cores. > > As explained in [1], there is a risk that a pthread is interrupted > by the kernel at a bad moment. Therefore another thread can be > blocked, spinning on a variable to change its value. > > The same could also occurs with spinlocks which are not designed > to wakeup another pthread when the lock is held (like pthread_locks). > > And finally, having several pthreads per core implies that the > application should be designed with large queues: if a pthread is > not scheduled during 10ms, it represents 100K packets at 10M PPS. > > I don't say it's impossible to do it, but I think it's not so > simple :) > > Regards, > Olivier > > [1] http://dpdk.org/ml/archives/dev/2013-November/000714.html Yes, completely agree, but because there are so many potential pitfalls, I think we should take small steps without making major changes. Hence my agreement with Mirek's suggestion as the simplest way forward that gives us good possibilities and flexibility without major work. BTW: For the multi-thread per core case, I think we'll need to look at threads making extensive use of yields to help force the context switches at proper times. /Bruce