From: Ferruh Yigit <ferruh.yigit@intel.com>
To: dev@dpdk.org, Thomas Monjalon <thomas.monjalon@6wind.com>
Cc: Ferruh Yigit <ferruh.yigit@intel.com>
Subject: [dpdk-dev] [PATCH] mk: parallelize make config
Date: Sun, 22 Jan 2017 01:50:34 +0000 [thread overview]
Message-ID: <20170122015034.19824-1-ferruh.yigit@intel.com> (raw)
make config dependency resolving was always running serial,
parallelize it for better performance.
$ time make T=x86_64-native-linuxapp-gcc config
real 0m12.633s
$ time make -j8 T=x86_64-native-linuxapp-gcc config
real 0m1.826s
Signed-off-by: Ferruh Yigit <ferruh.yigit@intel.com>
---
mk/internal/rte.depdirs-post.mk | 11 ++++++-----
mk/rte.sdkdepdirs.mk | 21 ++++++++++++---------
mk/rte.subdir.mk | 15 +++++++--------
3 files changed, 25 insertions(+), 22 deletions(-)
diff --git a/mk/internal/rte.depdirs-post.mk b/mk/internal/rte.depdirs-post.mk
index 102a369..eb73ad3 100644
--- a/mk/internal/rte.depdirs-post.mk
+++ b/mk/internal/rte.depdirs-post.mk
@@ -29,11 +29,12 @@
# (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
# OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
-.PHONY: depdirs
-depdirs:
- @for d in $(DEPDIRS-y); do \
- $(RTE_SDK)/buildtools/depdirs-rule.sh $(S) $$d ; \
- done
+.PHONY: depdirs $(DEPDIRS-y)
+depdirs: $(DEPDIRS-y)
+ @echo ""
+
+$(DEPDIRS-y):
+ @$(RTE_SDK)/buildtools/depdirs-rule.sh $(S) $@
.PHONY: depgraph
depgraph:
diff --git a/mk/rte.sdkdepdirs.mk b/mk/rte.sdkdepdirs.mk
index bebaf2a..daf42eb 100644
--- a/mk/rte.sdkdepdirs.mk
+++ b/mk/rte.sdkdepdirs.mk
@@ -36,19 +36,22 @@ ifeq (,$(wildcard $(RTE_OUTPUT)/Makefile))
$(error "need a make config first")
endif
+DEPDIRS = $(addsuffix /.depdirs, $(addprefix $(BUILDDIR)/,$(ROOTDIRS-y)))
+
# use a "for" in a shell to process dependencies: we don't want this
# task to be run in parallel.
.PHONY: depdirs
depdirs: $(RTE_OUTPUT)/.depdirs
-$(RTE_OUTPUT)/.depdirs: $(RTE_OUTPUT)/.config
- @rm -f $(RTE_OUTPUT)/.depdirs ; \
- for d in $(ROOTDIRS-y); do \
- if [ -f $(RTE_SRCDIR)/$$d/Makefile ]; then \
- [ -d $(BUILDDIR)/$$d ] || mkdir -p $(BUILDDIR)/$$d ; \
- $(MAKE) S=$$d -f $(RTE_SRCDIR)/$$d/Makefile depdirs \
- >> $(RTE_OUTPUT)/.depdirs ; \
- fi ; \
- done
+$(RTE_OUTPUT)/.depdirs: $(DEPDIRS)
+ @rm -f $@
+ @for f in $(DEPDIRS); do cat $$f >> $@; done
+ @sort -u -o $@ $@
+
+$(DEPDIRS): $(RTE_OUTPUT)/.config
+ @f=$(lastword $(subst /, ,$(dir $@))); \
+ [ -d $(BUILDDIR)/$$f ] || mkdir -p $(BUILDDIR)/$$f; \
+ rm -f $@; \
+ $(MAKE) S=$$f -f $(RTE_SRCDIR)/$$f/Makefile depdirs >> $@
.PHONY: depgraph
depgraph:
diff --git a/mk/rte.subdir.mk b/mk/rte.subdir.mk
index 256e64e..3bb3019 100644
--- a/mk/rte.subdir.mk
+++ b/mk/rte.subdir.mk
@@ -76,7 +76,7 @@ clean: _postclean
# include .depdirs and define rules to order priorities between build
# of directories.
#
-include $(RTE_OUTPUT)/.depdirs
+-include $(RTE_OUTPUT)/.depdirs
define depdirs_rule
$(1): $(sort $(patsubst $(S)/%,%,$(LOCAL_DEPDIRS-$(S)/$(1))))
@@ -84,16 +84,15 @@ endef
$(foreach d,$(DIRS-y),$(eval $(call depdirs_rule,$(d))))
+DEPDIRS = $(wildcard $(addprefix $(S)/,$(DIRS-y)))
# use a "for" in a shell to process dependencies: we don't want this
# task to be run in parallel.
-.PHONY: depdirs
-depdirs:
- @for d in $(DIRS-y); do \
- if [ -f $(SRCDIR)/$$d/Makefile ]; then \
- $(MAKE) S=$S/$$d -f $(SRCDIR)/$$d/Makefile depdirs ; \
- fi ; \
- done
+.PHONY: depdirs $(DEPDIRS)
+depdirs: $(DEPDIRS)
+
+$(DEPDIRS):
+ @$(MAKE) S=$@ -f $(RTE_SRCDIR)/$@/Makefile depdirs
.PHONY: depgraph
depgraph:
--
2.9.3
next reply other threads:[~2017-01-22 1:50 UTC|newest]
Thread overview: 38+ messages / expand[flat|nested] mbox.gz Atom feed top
2017-01-22 1:50 Ferruh Yigit [this message]
2017-01-23 17:18 ` Olivier Matz
2017-01-23 17:19 ` [dpdk-dev] [PATCH] mk: optimize directory dependencies Olivier Matz
2017-01-24 11:19 ` Robin Jarry
2017-01-24 11:26 ` Bruce Richardson
2017-01-24 12:31 ` Robin Jarry
2017-01-24 11:40 ` Jerin Jacob
2017-01-24 12:15 ` Bruce Richardson
2017-01-24 12:56 ` Jerin Jacob
2017-01-24 13:26 ` Richardson, Bruce
2017-01-24 14:50 ` Olivier MATZ
2017-01-24 14:55 ` Wiles, Keith
2017-03-01 11:25 ` Thomas Monjalon
2017-03-01 12:10 ` Bruce Richardson
2017-03-01 12:30 ` Olivier Matz
2017-01-24 13:05 ` Ferruh Yigit
2017-03-17 17:13 ` Olivier Matz
2017-03-17 17:47 ` Robin Jarry
2017-03-20 8:31 ` Olivier Matz
2017-03-24 13:21 ` [dpdk-dev] [PATCH v2] " Olivier Matz
2017-03-27 21:33 ` Thomas Monjalon
2017-03-28 10:34 ` Ferruh Yigit
2017-03-30 8:51 ` Olivier Matz
2017-03-30 9:27 ` Ferruh Yigit
2017-03-30 12:11 ` Olivier Matz
2017-03-30 12:32 ` [dpdk-dev] [PATCH] mk: fix dependencies to optional configs Olivier Matz
2017-03-30 12:37 ` Ferruh Yigit
2017-03-30 13:37 ` Thomas Monjalon
2017-01-23 17:50 ` [dpdk-dev] [PATCH] mk: parallelize make config Wiles, Keith
2017-01-24 8:42 ` Olivier MATZ
2017-01-24 10:02 ` Bruce Richardson
2017-01-23 19:03 ` Michał Mirosław
2017-01-30 9:41 ` Ferruh Yigit
2017-01-24 10:52 ` Bruce Richardson
2017-01-29 15:29 ` Thomas Monjalon
2017-01-30 9:46 ` Ferruh Yigit
2017-01-30 10:21 ` [dpdk-dev] [PATCH v2] " Ferruh Yigit
2017-01-30 18:13 ` Thomas Monjalon
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20170122015034.19824-1-ferruh.yigit@intel.com \
--to=ferruh.yigit@intel.com \
--cc=dev@dpdk.org \
--cc=thomas.monjalon@6wind.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).