From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-wm0-f43.google.com (mail-wm0-f43.google.com [74.125.82.43]) by dpdk.org (Postfix) with ESMTP id 61F92BD2C for ; Sun, 29 Jan 2017 16:29:39 +0100 (CET) Received: by mail-wm0-f43.google.com with SMTP id r126so181120322wmr.0 for ; Sun, 29 Jan 2017 07:29:39 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=6wind-com.20150623.gappssmtp.com; s=20150623; h=from:to:cc:subject:date:message-id:user-agent:in-reply-to :references:mime-version:content-transfer-encoding; bh=MMyJ3G1D0RI5IpYbKGchUcF32C3BM89V9bGL0U4O1pY=; b=ONGFQtFc/c3Tzq0MdlJrF4vasaIb/7+Jcj3ooC597aXYJdY3QVpfl8pyW9TSi57zpW z5ecNPQo3ZWCt9EK7Xu0wLdoAEiwPcWTsdsKuzK5zqQwxC5zJOOoYUL2wUKWxAPynUWA Gs3F4ZRxzXhimzpgaClfA6tAS7rnPLvA0IIKds0mbtyLHfpZmh9mJrTSioBj4eH0MCOP OWI7/r0JwrvJu0SCqaRBfJmiAVrz2u1ioL98UitrUo4eOY1nF90lEQzsUU4yBrtngPoa hTXqob1opFayIwh297e4dQratpibrxBfHFhepP7R109oe91mahV+wFXYvxA0A9M5nzxo ko/g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:user-agent :in-reply-to:references:mime-version:content-transfer-encoding; bh=MMyJ3G1D0RI5IpYbKGchUcF32C3BM89V9bGL0U4O1pY=; b=ZFF+utIdseRRecvtDEaIXA27Q8sI483Hdf+8DdkpDmhKA7Q9KQdbzrhoH2KBXbbygU BC2AFUtTT17ghBSEjpijZy6zeuTM7LN3pynDOZ+JxmaURBpVwlI136WmCEYuPg5StbFm 02PwVq6PwKcwxnLNWuRKshmaDzmqUtHvWslDGO7YP0nuijFHOr41QVWTwLZQLXzY72w+ 058SqDaxtdhxBCGya1LzH37jn1pw5a8JPjNOH6/ABOnQlA0ckWiJ0/f3IwERmuEsd33n 5qyEBQ/OvD6Vc1sZWaP+pkvLOxlpEz/3+rDBgH6mJMC+mLaO9tgQoBwjUwkbBjUPUrw8 kjbQ== X-Gm-Message-State: AIkVDXI1M5bJo4/V+kef1IF72CyCm+m++uWm/avEwpvr8qqGguCHqKmXhHsR347YA8gX1A4D X-Received: by 10.223.162.205 with SMTP id t13mr17063472wra.155.1485703778949; Sun, 29 Jan 2017 07:29:38 -0800 (PST) Received: from xps13.localnet (48.114.118.80.rev.sfr.net. [80.118.114.48]) by smtp.gmail.com with ESMTPSA id x69sm14128702wma.15.2017.01.29.07.29.37 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Sun, 29 Jan 2017 07:29:38 -0800 (PST) From: Thomas Monjalon To: Ferruh Yigit Cc: dev@dpdk.org Date: Sun, 29 Jan 2017 16:29:35 +0100 Message-ID: <1800797.YCEvINBUOG@xps13> User-Agent: KMail/4.14.10 (Linux/4.5.4-1-ARCH; KDE/4.14.11; x86_64; ; ) In-Reply-To: <20170122015034.19824-1-ferruh.yigit@intel.com> References: <20170122015034.19824-1-ferruh.yigit@intel.com> MIME-Version: 1.0 Content-Transfer-Encoding: 7Bit Content-Type: text/plain; charset="us-ascii" Subject: Re: [dpdk-dev] [PATCH] mk: parallelize make config X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 29 Jan 2017 15:29:39 -0000 2017-01-22 01:50, Ferruh Yigit: > make config dependency resolving was always running serial, > parallelize it for better performance. It could be interesting to explain why it was not parallelized, and how you made it possible. The test script should be updated as below: --- a/devtools/test-build.sh +++ b/devtools/test-build.sh - make T=$2 O=$1 config + make -j$J T=$2 O=$1 config > --- a/mk/internal/rte.depdirs-post.mk > +++ b/mk/internal/rte.depdirs-post.mk > @@ -29,11 +29,12 @@ > # (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE > # OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. > > -.PHONY: depdirs > -depdirs: > - @for d in $(DEPDIRS-y); do \ > - $(RTE_SDK)/buildtools/depdirs-rule.sh $(S) $$d ; \ > - done > +.PHONY: depdirs $(DEPDIRS-y) > +depdirs: $(DEPDIRS-y) > + @echo "" Why this echo "" ? > + > +$(DEPDIRS-y): > + @$(RTE_SDK)/buildtools/depdirs-rule.sh $(S) $@ > --- a/mk/rte.sdkdepdirs.mk > +++ b/mk/rte.sdkdepdirs.mk > @@ -36,19 +36,22 @@ ifeq (,$(wildcard $(RTE_OUTPUT)/Makefile)) > $(error "need a make config first") > endif > > +DEPDIRS = $(addsuffix /.depdirs, $(addprefix $(BUILDDIR)/,$(ROOTDIRS-y))) These DEPDIRS are files, although DEPDIRS in other contexts are directories. I think it should be renamed. DEPDIR_FILES? > # use a "for" in a shell to process dependencies: we don't want this > # task to be run in parallel. You forgot to remove this obsolete comment. > .PHONY: depdirs > depdirs: $(RTE_OUTPUT)/.depdirs > -$(RTE_OUTPUT)/.depdirs: $(RTE_OUTPUT)/.config > - @rm -f $(RTE_OUTPUT)/.depdirs ; \ > - for d in $(ROOTDIRS-y); do \ > - if [ -f $(RTE_SRCDIR)/$$d/Makefile ]; then \ > - [ -d $(BUILDDIR)/$$d ] || mkdir -p $(BUILDDIR)/$$d ; \ > - $(MAKE) S=$$d -f $(RTE_SRCDIR)/$$d/Makefile depdirs \ > - >> $(RTE_OUTPUT)/.depdirs ; \ > - fi ; \ > - done > +$(RTE_OUTPUT)/.depdirs: $(DEPDIRS) > + @rm -f $@ > + @for f in $(DEPDIRS); do cat $$f >> $@; done > + @sort -u -o $@ $@ > + > +$(DEPDIRS): $(RTE_OUTPUT)/.config > + @f=$(lastword $(subst /, ,$(dir $@))); \ Could you use $(notdir $(@D)) ? > + [ -d $(BUILDDIR)/$$f ] || mkdir -p $(BUILDDIR)/$$f; \ > + rm -f $@; \ Why this removal? > + $(MAKE) S=$$f -f $(RTE_SRCDIR)/$$f/Makefile depdirs >> $@ This part is a bit complicated. Could it be simplified by better naming $f? > --- a/mk/rte.subdir.mk > +++ b/mk/rte.subdir.mk > @@ -76,7 +76,7 @@ clean: _postclean > # include .depdirs and define rules to order priorities between build > # of directories. > # > -include $(RTE_OUTPUT)/.depdirs > +-include $(RTE_OUTPUT)/.depdirs > > define depdirs_rule > $(1): $(sort $(patsubst $(S)/%,%,$(LOCAL_DEPDIRS-$(S)/$(1)))) > @@ -84,16 +84,15 @@ endef > > $(foreach d,$(DIRS-y),$(eval $(call depdirs_rule,$(d)))) > > +DEPDIRS = $(wildcard $(addprefix $(S)/,$(DIRS-y))) > > # use a "for" in a shell to process dependencies: we don't want this > # task to be run in parallel. You forgot to remove this obsolete comment. > -.PHONY: depdirs > -depdirs: > - @for d in $(DIRS-y); do \ > - if [ -f $(SRCDIR)/$$d/Makefile ]; then \ > - $(MAKE) S=$S/$$d -f $(SRCDIR)/$$d/Makefile depdirs ; \ > - fi ; \ > - done > +.PHONY: depdirs $(DEPDIRS) > +depdirs: $(DEPDIRS) > + > +$(DEPDIRS): > + @$(MAKE) S=$@ -f $(RTE_SRCDIR)/$@/Makefile depdirs