From mboxrd@z Thu Jan  1 00:00:00 1970
Return-Path: <bruce.richardson@intel.com>
Received: from mga06.intel.com (mga06.intel.com [134.134.136.31])
 by dpdk.org (Postfix) with ESMTP id 1631A108D
 for <dev@dpdk.org>; Tue, 24 Jan 2017 12:26:37 +0100 (CET)
Received: from fmsmga001.fm.intel.com ([10.253.24.23])
 by orsmga104.jf.intel.com with ESMTP; 24 Jan 2017 03:26:36 -0800
X-ExtLoop1: 1
X-IronPort-AV: E=Sophos;i="5.33,278,1477983600"; d="scan'208";a="1098179192"
Received: from bricha3-mobl3.ger.corp.intel.com ([10.237.221.61])
 by fmsmga001.fm.intel.com with SMTP; 24 Jan 2017 03:26:34 -0800
Received: by  (sSMTP sendmail emulation); Tue, 24 Jan 2017 11:26:33 +0000
Date: Tue, 24 Jan 2017 11:26:33 +0000
From: Bruce Richardson <bruce.richardson@intel.com>
To: Robin Jarry <robin.jarry@6wind.com>
Cc: Olivier Matz <olivier.matz@6wind.com>, dev@dpdk.org,
 ferruh.yigit@intel.com, thomas.monjalon@6wind.com
Message-ID: <20170124112633.GA156840@bricha3-MOBL3.ger.corp.intel.com>
References: <20170123181813.00567e47@platinum>
 <1485191953-2613-1-git-send-email-olivier.matz@6wind.com>
 <20170124111949.6jcif4uliwg6bgxv@6wind.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <20170124111949.6jcif4uliwg6bgxv@6wind.com>
Organization: Intel Research and =?iso-8859-1?Q?De=ACvel?=
 =?iso-8859-1?Q?opment?= Ireland Ltd.
User-Agent: Mutt/1.7.1 (2016-10-04)
Subject: Re: [dpdk-dev] [PATCH] mk: optimize directory dependencies
X-BeenThere: dev@dpdk.org
X-Mailman-Version: 2.1.15
Precedence: list
List-Id: DPDK patches and discussions <dev.dpdk.org>
List-Unsubscribe: <http://dpdk.org/ml/options/dev>,
 <mailto:dev-request@dpdk.org?subject=unsubscribe>
List-Archive: <http://dpdk.org/ml/archives/dev/>
List-Post: <mailto:dev@dpdk.org>
List-Help: <mailto:dev-request@dpdk.org?subject=help>
List-Subscribe: <http://dpdk.org/ml/listinfo/dev>,
 <mailto:dev-request@dpdk.org?subject=subscribe>
X-List-Received-Date: Tue, 24 Jan 2017 11:26:38 -0000

On Tue, Jan 24, 2017 at 12:19:49PM +0100, Robin Jarry wrote:
> Hi Olivier,
> 
> 2017-01-23, Olivier Matz:
> > Before this patch, the management of dependencies between directories
> > had several issues:
> > 
> > - the generation of .depdirs, done at configuration is slow: it can take
> >  more than one minute on some slow targets (usually ~10s on a standard
> >  PC).
> 
> Indeed, on a Qualcomm development board where disk I/O is quite slow:
> 
>  $ git describe
>  v17.02-rc1-3-g61207d014fc9
> 
>  $ time make config T=arm64-armv8a-linuxapp-gcc
>  real    1m4.308s
> 

Wow, what is the build time in that case?

> > - for instance, it is possible to expressed a dependency like:
> 
> s/expressed/express/
> 
> > - we cannot use "make -d" for debug, because the output of make is used
> > for the generation of .depdirs.
> 
> That is really annoying when debugging makefiles.
> 
> After applying this patch:
> 
>  $ git am mk-optimize-directory-dependencies.patch
>  Applying: mk: optimize directory dependencies
>  $ rm -rf build/
> 
>  $ time make config T=arm64-armv8a-linuxapp-gcc
>  real    0m0.111s
> 
> Almost 600 times faster than before!
> 
> I prefer this solution to the one proposed by Ferruh (which is interesting
> but requires to run parallel make). Here is a test with the other patch:
> 
>  $ git am mk-parallelize-make-config.patch
>  Applying: mk: parallelize make config
>  $ rm -rf build/
>  $ grep -c processor /proc/cpuinfo
>  24
> 
>  $ time make config T=arm64-armv8a-linuxapp-gcc -j24
>  real    0m11.253s
> 
> Here only 6 times faster than before, even when using 24 parallel processes.
> 
> Tested-by: Robin Jarry <robin.jarry@6wind.com>

Hi Robin,

what are the differences in the patches like when doing a build rather
than just a config? If the build is minutes long because of slow IO,
is the extra 10 seconds really going to make that much of a difference?

Regards,
/Bruce