From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mga04.intel.com (mga04.intel.com [192.55.52.120]) by dpdk.org (Postfix) with ESMTP id 1CA11108D for ; Tue, 24 Jan 2017 13:15:10 +0100 (CET) Received: from fmsmga006.fm.intel.com ([10.253.24.20]) by fmsmga104.fm.intel.com with ESMTP; 24 Jan 2017 04:15:09 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.33,278,1477983600"; d="scan'208";a="57560133" Received: from bricha3-mobl3.ger.corp.intel.com ([10.237.221.61]) by fmsmga006.fm.intel.com with SMTP; 24 Jan 2017 04:15:07 -0800 Received: by (sSMTP sendmail emulation); Tue, 24 Jan 2017 12:15:07 +0000 Date: Tue, 24 Jan 2017 12:15:06 +0000 From: Bruce Richardson To: Jerin Jacob Cc: Olivier Matz , dev@dpdk.org, ferruh.yigit@intel.com, thomas.monjalon@6wind.com Message-ID: <20170124121506.GA160692@bricha3-MOBL3.ger.corp.intel.com> References: <20170123181813.00567e47@platinum> <1485191953-2613-1-git-send-email-olivier.matz@6wind.com> <20170124114013.GA4609@localhost.localdomain> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Disposition: inline Content-Transfer-Encoding: 8bit In-Reply-To: <20170124114013.GA4609@localhost.localdomain> Organization: Intel Research and =?iso-8859-1?Q?De=ACvel?= =?iso-8859-1?Q?opment?= Ireland Ltd. User-Agent: Mutt/1.7.1 (2016-10-04) Subject: Re: [dpdk-dev] [PATCH] mk: optimize directory dependencies X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 24 Jan 2017 12:15:11 -0000 On Tue, Jan 24, 2017 at 05:10:15PM +0530, Jerin Jacob wrote: > On Mon, Jan 23, 2017 at 06:19:13PM +0100, Olivier Matz wrote: > > Before this patch, the management of dependencies between directories > > had several issues: > > > > - the generation of .depdirs, done at configuration is slow: it can take > > more than one minute on some slow targets (usually ~10s on a standard > > PC). > > > > - for instance, it is possible to expressed a dependency like: > > - app/foo depends on lib/librte_foo > > - and lib/librte_foo depends on app/bar > > But this won't work because the directories are traversed with a > > depth-first algorithm, so we have to choose between doing 'app' before > > or after 'lib'. > > > > - the script depdirs-rule.sh is too complex. > > > > - we cannot use "make -d" for debug, because the output of make is used for > > the generation of .depdirs. > > > > This patch moves the DEPDIRS-* variables in the upper Makefile, making > > the dependencies much easier to calculate. A DEPDIRS variable is still > > used to process library dependencies in LDLIBS. > > > > After this commit, "make config" is almost immediate. > > > > Signed-off-by: Olivier Matz > > Tested both approach on ThunderX. This patch looks better > > Tested-by: Jerin Jacob > > ➜ [master]GB-2S [dpdk-master] $ time make config T=arm64-thunderx-linuxapp-gcc > Configuration done > > real 0m18.112s > user 0m2.810s > sys 0m0.660s > > ➜ [master]GB-2S [dpdk-master] $ pwclient git-am 19859 > Applying patch #19859 using 'git am' > Description: [dpdk-dev] mk: parallelize make config > Applying: mk: parallelize make config > > ➜ [master]GB-2S [dpdk-master] $ rm -rf build > ➜ [master]GB-2S [dpdk-master] $ time make config T=arm64-thunderx-linuxapp-gcc -j 8 > Configuration done > > real 0m2.812s > user 0m3.020s > sys 0m0.870s > > ➜ [master]GB-2S [dpdk-master] $ rm -rf build > ➜ [master]GB-2S [dpdk-master] $ time make config T=arm64-thunderx-linuxapp-gcc -j 16 > Configuration done > > real 0m1.748s > user 0m3.040s > sys 0m1.020s > > ➜ [master]GB-2S [dpdk-master] $ rm -rf build > ➜ [master]GB-2S [dpdk-master] $ time make config T=arm64-thunderx-linuxapp-gcc -j 32 > Configuration done > > real 0m1.422s > user 0m3.380s > sys 0m1.080s > > ➜ [master]GB-2S [dpdk-master] $ pwclient git-am 19918 > Applying patch #19918 using 'git am' > Description: [dpdk-dev] mk: optimize directory dependencies > Applying: mk: optimize directory dependencies > > ➜ [master]GB-2S [dpdk-master] $ rm -rf build > ➜ [master]GB-2S [dpdk-master] $ time make config T=arm64-thunderx-linuxapp-gcc > Configuration done > > real 0m0.064s > user 0m0.000s > sys 0m0.000s > ➜ [master]GB-2S [dpdk-master] $ rm -rf build > ➜ [master]GB-2S [dpdk-master] $ time make config T=arm64-thunderx-linuxapp-gcc -j 8 > Configuration done > > real 0m0.055s > user 0m0.000s > sys 0m0.000s > I agree that Olivier's patch is faster. However, I think I prefer having the library dependencies in the makefiles for the libs themselves rather than up a level. Given that we are only looking at ~2second of a difference here in your tests - assuming -j flag - what is the actual build time differences? My suspicion is that after Ferruh's simpler patch is applied, the config time becomes such a small part of the build, that the extra benefits from Oliviers work is not worth the extra complexity. Regards, /Bruce