From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mga07.intel.com (mga07.intel.com [134.134.136.100]) by dpdk.org (Postfix) with ESMTP id DD3DD2BBE for ; Tue, 24 Jan 2017 11:02:50 +0100 (CET) Received: from fmsmga004.fm.intel.com ([10.253.24.48]) by orsmga105.jf.intel.com with ESMTP; 24 Jan 2017 02:02:49 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.33,278,1477983600"; d="scan'208";a="216939764" Received: from bricha3-mobl3.ger.corp.intel.com ([10.237.221.61]) by fmsmga004.fm.intel.com with SMTP; 24 Jan 2017 02:02:46 -0800 Received: by (sSMTP sendmail emulation); Tue, 24 Jan 2017 10:02:46 +0000 Date: Tue, 24 Jan 2017 10:02:46 +0000 From: Bruce Richardson To: Olivier MATZ Cc: "Wiles, Keith" , "Yigit, Ferruh" , "dev@dpdk.org" , Thomas Monjalon Message-ID: <20170124100245.GA151532@bricha3-MOBL3.ger.corp.intel.com> References: <20170122015034.19824-1-ferruh.yigit@intel.com> <20170123181813.00567e47@platinum> <5C15197B-7CD8-490D-B756-B0580CBFFA50@intel.com> <20170124094235.7fda08cc@glumotte.dev.6wind.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20170124094235.7fda08cc@glumotte.dev.6wind.com> Organization: Intel Research and =?iso-8859-1?Q?De=ACvel?= =?iso-8859-1?Q?opment?= Ireland Ltd. User-Agent: Mutt/1.7.1 (2016-10-04) Subject: Re: [dpdk-dev] [PATCH] mk: parallelize make config X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 24 Jan 2017 10:02:51 -0000 On Tue, Jan 24, 2017 at 09:42:35AM +0100, Olivier MATZ wrote: > Hi Keith, > > On Mon, 23 Jan 2017 17:50:27 +0000, "Wiles, Keith" > wrote: > > > On Jan 23, 2017, at 10:18 AM, Olivier Matz > > > wrote: > > > > > > Hi Ferruh, > > > > > > On Sun, 22 Jan 2017 01:50:34 +0000, Ferruh Yigit > > > wrote: > > >> make config dependency resolving was always running serial, > > >> parallelize it for better performance. > > >> > > >> $ time make T=x86_64-native-linuxapp-gcc config > > >> real 0m12.633s > > >> > > >> $ time make -j8 T=x86_64-native-linuxapp-gcc config > > >> real 0m1.826s > > >> > > >> Signed-off-by: Ferruh Yigit > > > > > > I have a patch that fix the same issue (configuration takes to > > > long), but done differently. It is more intrusive, since it rework > > > the way DEPDIRS are used, but it does not require to use -j. > > > > I tested the patch and the performance is very quick and seems to > > work very nicely. I have not looked at the other patch yet are they > > both compatible and could be combined or not? > > No, I don't think they could be combined. > > Ferruh's approach looks less risky, since it's a parallelization > of the config. My approach changes the way dependencies are processed, > and also how they are declared. I think it's faster, but more intrusive. > I think we may hit a case of diminishing returns. On my system, the config time goes from 12 seconds down to less than 1 second (large amounts of parallelism). Even with limiting parallelism to just 4 make processes, the time for config drops to 3 seconds. I don't see huge value in trying to shave off any more time than that, given the amount of extra complexity involved. /Bruce