[dpdk-dev] [PATCH 0/4] reproducible builds - docs and linker script

[dpdk-dev] [PATCH 0/4] reproducible builds - docs and linker script

[lboccass]

From: Luca Boccassi <luca.boccassi@gmail.com>

In the past couple of years a concerted effort among almost all Linux
distros has been striving toward achieving reproducible builds. [1]
This involves changes to the toolchain, new tools and CI systems. [2]

This first series fixes unreproducibility problems in DPDK with
regards to documentation and the linker script generation.

Outstanding issues are the use of __FILE__ in conjuction with
absolute file paths when pointing the compiler to the sources, and
public headers being listed in the DWARF files under either the
source location or the make install location, seemingly randomly.
The first issue is being worked on with a new standard for
reproducible source paths. [3] Perhaps the build system could be
adjusted to use relative paths, I will investigate further.
The second one I've got no clue - will keep investigating, help is
most welcome.

[1] https://reproducible-builds.org/
[2] https://reproducible-builds.org/tools/
[3] https://reproducible-builds.org/specs/build-path-prefix-map/

Luca Boccassi (4):
  mk: use make silent flag to print HTML doc version
  mk: fix excluding .doctrees when installing docs
  mk: sort list of shared objects in linker script
  mk: sort list of files in examples.dox

 mk/rte.combinedlib.mk | 2 +-
 mk/rte.sdkdoc.mk      | 4 ++--
 mk/rte.sdkinstall.mk  | 2 +-
 3 files changed, 4 insertions(+), 4 deletions(-)

-- 
2.11.0

[dpdk-dev] [PATCH 1/4] mk: use make silent flag to print HTML doc version

[lboccass]

From: Luca Boccassi <luca.boccassi@gmail.com>

Depending on the environment, make might echo the command being ran.
In mk/rte.sdkdoc.mk make is used to print the DPDK version to be
piped to doxygen. This causes the following to be written:

<div id="projectname">DPDK
&#160;<span id="projectnumber">/usr/bin/make-f/build/dpdk-jYjqnr/dpdk-16.11.2/mk/rte.sdkconfig.mkshowversion</span>
</div>

Use -s (--silent) to prevent echoing.

Signed-off-by: Luca Boccassi <luca.boccassi@gmail.com>
---
 mk/rte.sdkdoc.mk | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/mk/rte.sdkdoc.mk b/mk/rte.sdkdoc.mk
index fb8f91555..c0eaa3502 100644
--- a/mk/rte.sdkdoc.mk
+++ b/mk/rte.sdkdoc.mk
@@ -73,7 +73,7 @@ api-html: $(API_EXAMPLES)
 	$(Q)mkdir -p $(RTE_OUTPUT)/doc/html
 	$(Q)(cat $(RTE_SDK)/doc/api/doxy-api.conf     && \
 	    printf 'PROJECT_NUMBER = '                && \
-	                      $(MAKE) -rR showversion && \
+	                     $(MAKE) -rRs showversion && \
 	    echo INPUT           += $(API_EXAMPLES)   && \
 	    echo OUTPUT_DIRECTORY = $(RTE_OUTPUT)/doc && \
 	    echo HTML_OUTPUT      = html/api          && \
-- 
2.11.0

[dpdk-dev] [PATCH 2/4] mk: fix excluding .doctrees when installing docs

[lboccass]

From: Luca Boccassi <luca.boccassi@gmail.com>

The --exclude parameter must be passed before the input directory to
tar, otherwise it's silently ignored and the .doctrees directory is
installed by make install-doc.

Signed-off-by: Luca Boccassi <luca.boccassi@gmail.com>
---
 mk/rte.sdkinstall.mk | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/mk/rte.sdkinstall.mk b/mk/rte.sdkinstall.mk
index dbac2a277..4e97feff9 100644
--- a/mk/rte.sdkinstall.mk
+++ b/mk/rte.sdkinstall.mk
@@ -162,7 +162,7 @@ install-sdk:
 install-doc:
 ifneq ($(wildcard $O/doc/html),)
 	$(Q)$(call rte_mkdir, $(DESTDIR)$(docdir))
-	$(Q)tar -cf -      -C $O/doc html --exclude 'html/guides/.*' | \
+	$(Q)tar -cf -      -C $O/doc --exclude 'html/guides/.*' html | \
 	    tar -xf -      -C $(DESTDIR)$(docdir) --strip-components=1 \
 		--keep-newer-files
 endif
-- 
2.11.0

[dpdk-dev] [PATCH 3/4] mk: sort list of shared objects in linker script

[lboccass]

From: Luca Boccassi <luca.boccassi@gmail.com>

The output of wildcard might not be stable and depend on the
filesystem and other factors.
This means the content libdpdk.so linker script might change between
builds from the same sources.
Run the list through sort to ensure reproducibility.

Signed-off-by: Luca Boccassi <luca.boccassi@gmail.com>
---
 mk/rte.combinedlib.mk | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/mk/rte.combinedlib.mk b/mk/rte.combinedlib.mk
index 449358b33..2ab7ee8a1 100644
--- a/mk/rte.combinedlib.mk
+++ b/mk/rte.combinedlib.mk
@@ -42,7 +42,7 @@ endif
 RTE_LIBNAME := dpdk
 COMBINEDLIB := lib$(RTE_LIBNAME)$(EXT)
 
-LIBS := $(filter-out $(COMBINEDLIB), $(notdir $(wildcard $(RTE_OUTPUT)/lib/*$(EXT))))
+LIBS := $(filter-out $(COMBINEDLIB), $(sort $(notdir $(wildcard $(RTE_OUTPUT)/lib/*$(EXT)))))
 
 all: FORCE
 	$(Q)echo "GROUP ( $(LIBS) )" > $(RTE_OUTPUT)/lib/$(COMBINEDLIB)
-- 
2.11.0

[dpdk-dev] [PATCH 4/4] mk: sort list of files in examples.dox

[lboccass]

From: Luca Boccassi <luca.boccassi@gmail.com>

The result of find might not be stable depending on external
conditions.
Pipe it through LC_ALL=C sort to ensure reproducible results when
generating examples.dox.

Signed-off-by: Luca Boccassi <luca.boccassi@gmail.com>
---
 mk/rte.sdkdoc.mk | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/mk/rte.sdkdoc.mk b/mk/rte.sdkdoc.mk
index c0eaa3502..de31b78cf 100644
--- a/mk/rte.sdkdoc.mk
+++ b/mk/rte.sdkdoc.mk
@@ -93,7 +93,7 @@ $(API_EXAMPLES): api-html-clean
 	$(Q)mkdir -p $(@D)
 	@printf '/**\n' > $(API_EXAMPLES)
 	@printf '@page examples DPDK Example Programs\n\n' >> $(API_EXAMPLES)
-	@find examples -type f -name '*.c' -printf '@example %p\n' >> $(API_EXAMPLES)
+	@find examples -type f -name '*.c' -printf '@example %p\n' | LC_ALL=C sort >> $(API_EXAMPLES)
 	@printf '*/\n' >> $(API_EXAMPLES)
 
 guides-pdf-clean: guides-pdf-img-clean
-- 
2.11.0

[dpdk-dev] [PATCH v2 0/7] Reproducible build

[lboccass]

From: Luca Boccassi <luca.boccassi@gmail.com>

In the past couple of years a concerted effort among almost all Linux
distros has been striving toward achieving reproducible builds. [1]
This involves changes to the toolchain, new tools and CI systems. [2]

v1 fixed the documentation, examples and linker script generation.
v2 fixes all problems, which were caused by unstable order of headers
inclusion, source files listing and object file listing when passing
them to the compiler.
DPDK's build, at least with the default configuration, is fully
reproducible with this patch series as tested by the Reproducible
Builds developers experimental toolchain.

[1] https://reproducible-builds.org/
[2] https://reproducible-builds.org/tools/
[3] https://wiki.debian.org/ReproducibleBuilds/ExperimentalToolchain#Us

Luca Boccassi (7):
  mk: fix excluding .doctrees when installing docs
  mk: sort list of shared objects in linker script
  mk: sort list of files in examples.dox
  mk: sort headers before wildcard inclusion
  mk: sort source files before passing them to the compiler
  mk: sort object files when building deps lists
  mk: always rebuild in the same order

 drivers/net/cxgbe/Makefile                                 | 2 +-
 drivers/net/e1000/Makefile                                 | 2 +-
 drivers/net/fm10k/Makefile                                 | 2 +-
 drivers/net/i40e/Makefile                                  | 2 +-
 drivers/net/ixgbe/Makefile                                 | 2 +-
 drivers/net/qede/Makefile                                  | 2 +-
 drivers/net/sfc/Makefile                                   | 2 +-
 drivers/net/thunderx/Makefile                              | 2 +-
 examples/ip_pipeline/Makefile                              | 2 +-
 examples/multi_process/client_server_mp/mp_server/Makefile | 2 +-
 examples/server_node_efd/server/Makefile                   | 2 +-
 lib/librte_eal/common/Makefile                             | 2 +-
 mk/internal/rte.compile-pre.mk                             | 8 ++++----
 mk/rte.app.mk                                              | 4 ++--
 mk/rte.combinedlib.mk                                      | 2 +-
 mk/rte.hostapp.mk                                          | 4 ++--
 mk/rte.sdkdoc.mk                                           | 2 +-
 mk/rte.sdkinstall.mk                                       | 2 +-
 mk/rte.shared.mk                                           | 4 ++--
 19 files changed, 25 insertions(+), 25 deletions(-)

-- 
2.11.0

[dpdk-dev] [PATCH v2 1/7] mk: fix excluding .doctrees when installing docs

[lboccass]

From: Luca Boccassi <luca.boccassi@gmail.com>

The --exclude parameter must be passed before the input directory to
tar, otherwise it's silently ignored and the .doctrees directory is
installed by make install-doc.

Signed-off-by: Luca Boccassi <luca.boccassi@gmail.com>
---
 mk/rte.sdkinstall.mk | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/mk/rte.sdkinstall.mk b/mk/rte.sdkinstall.mk
index dbac2a277..4e97feff9 100644
--- a/mk/rte.sdkinstall.mk
+++ b/mk/rte.sdkinstall.mk
@@ -162,7 +162,7 @@ install-sdk:
 install-doc:
 ifneq ($(wildcard $O/doc/html),)
 	$(Q)$(call rte_mkdir, $(DESTDIR)$(docdir))
-	$(Q)tar -cf -      -C $O/doc html --exclude 'html/guides/.*' | \
+	$(Q)tar -cf -      -C $O/doc --exclude 'html/guides/.*' html | \
 	    tar -xf -      -C $(DESTDIR)$(docdir) --strip-components=1 \
 		--keep-newer-files
 endif
-- 
2.11.0

[dpdk-dev] [PATCH v2 2/7] mk: sort list of shared objects in linker script

[lboccass]

From: Luca Boccassi <luca.boccassi@gmail.com>

The output of wildcard might not be stable and depend on the
filesystem and other factors.
This means the content libdpdk.so linker script might change between
builds from the same sources.
Run the list through sort to ensure reproducibility.

Signed-off-by: Luca Boccassi <luca.boccassi@gmail.com>
---
 mk/rte.combinedlib.mk | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/mk/rte.combinedlib.mk b/mk/rte.combinedlib.mk
index 449358b33..2ab7ee8a1 100644
--- a/mk/rte.combinedlib.mk
+++ b/mk/rte.combinedlib.mk
@@ -42,7 +42,7 @@ endif
 RTE_LIBNAME := dpdk
 COMBINEDLIB := lib$(RTE_LIBNAME)$(EXT)
 
-LIBS := $(filter-out $(COMBINEDLIB), $(notdir $(wildcard $(RTE_OUTPUT)/lib/*$(EXT))))
+LIBS := $(filter-out $(COMBINEDLIB), $(sort $(notdir $(wildcard $(RTE_OUTPUT)/lib/*$(EXT)))))
 
 all: FORCE
 	$(Q)echo "GROUP ( $(LIBS) )" > $(RTE_OUTPUT)/lib/$(COMBINEDLIB)
-- 
2.11.0

[dpdk-dev] [PATCH v2 3/7] mk: sort list of files in examples.dox

[lboccass]

From: Luca Boccassi <luca.boccassi@gmail.com>

The result of find might not be stable depending on external
conditions.
Pipe it through LC_ALL=C sort to ensure reproducible results when
generating examples.dox.

Signed-off-by: Luca Boccassi <luca.boccassi@gmail.com>
---
 mk/rte.sdkdoc.mk | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/mk/rte.sdkdoc.mk b/mk/rte.sdkdoc.mk
index c0eaa3502..de31b78cf 100644
--- a/mk/rte.sdkdoc.mk
+++ b/mk/rte.sdkdoc.mk
@@ -93,7 +93,7 @@ $(API_EXAMPLES): api-html-clean
 	$(Q)mkdir -p $(@D)
 	@printf '/**\n' > $(API_EXAMPLES)
 	@printf '@page examples DPDK Example Programs\n\n' >> $(API_EXAMPLES)
-	@find examples -type f -name '*.c' -printf '@example %p\n' >> $(API_EXAMPLES)
+	@find examples -type f -name '*.c' -printf '@example %p\n' | LC_ALL=C sort >> $(API_EXAMPLES)
 	@printf '*/\n' >> $(API_EXAMPLES)
 
 guides-pdf-clean: guides-pdf-img-clean
-- 
2.11.0

[dpdk-dev] [PATCH v2 4/7] mk: sort headers before wildcard inclusion

[lboccass]

From: Luca Boccassi <luca.boccassi@gmail.com>

In order to achieve fully reproducible builds, always use the same
inclusion order for headers in the Makefiles.

Signed-off-by: Luca Boccassi <luca.boccassi@gmail.com>
---
 examples/ip_pipeline/Makefile                              | 2 +-
 examples/multi_process/client_server_mp/mp_server/Makefile | 2 +-
 examples/server_node_efd/server/Makefile                   | 2 +-
 lib/librte_eal/common/Makefile                             | 2 +-
 4 files changed, 4 insertions(+), 4 deletions(-)

diff --git a/examples/ip_pipeline/Makefile b/examples/ip_pipeline/Makefile
index dc7e0ddd7..12ce0a1d5 100644
--- a/examples/ip_pipeline/Makefile
+++ b/examples/ip_pipeline/Makefile
@@ -43,7 +43,7 @@ APP = ip_pipeline
 
 VPATH += $(SRCDIR)/pipeline
 
-INC += $(wildcard *.h) $(wildcard pipeline/*.h)
+INC += $(sort $(wildcard *.h)) $(sort $(wildcard pipeline/*.h))
 
 # all source are stored in SRCS-y
 SRCS-$(CONFIG_RTE_LIBRTE_PIPELINE) := main.c
diff --git a/examples/multi_process/client_server_mp/mp_server/Makefile b/examples/multi_process/client_server_mp/mp_server/Makefile
index 5552999b5..160c17b68 100644
--- a/examples/multi_process/client_server_mp/mp_server/Makefile
+++ b/examples/multi_process/client_server_mp/mp_server/Makefile
@@ -49,7 +49,7 @@ APP = mp_server
 # all source are stored in SRCS-y
 SRCS-y := main.c init.c args.c
 
-INC := $(wildcard *.h)
+INC := $(sort $(wildcard *.h))
 
 CFLAGS += $(WERROR_FLAGS) -O3
 CFLAGS += -I$(SRCDIR)/../shared
diff --git a/examples/server_node_efd/server/Makefile b/examples/server_node_efd/server/Makefile
index a2f2f361b..9f1fe2894 100644
--- a/examples/server_node_efd/server/Makefile
+++ b/examples/server_node_efd/server/Makefile
@@ -49,7 +49,7 @@ APP = server
 # all source are stored in SRCS-y
 SRCS-y := main.c init.c args.c
 
-INC := $(wildcard *.h)
+INC := $(sort $(wildcard *.h))
 
 CFLAGS += $(WERROR_FLAGS) -O3
 CFLAGS += -I$(SRCDIR)/../shared
diff --git a/lib/librte_eal/common/Makefile b/lib/librte_eal/common/Makefile
index a5bd1089a..4b712600a 100644
--- a/lib/librte_eal/common/Makefile
+++ b/lib/librte_eal/common/Makefile
@@ -48,7 +48,7 @@ GENERIC_INC += rte_vect.h rte_io.h
 
 # defined in mk/arch/$(RTE_ARCH)/rte.vars.mk
 ARCH_DIR ?= $(RTE_ARCH)
-ARCH_INC := $(notdir $(wildcard $(RTE_SDK)/lib/librte_eal/common/include/arch/$(ARCH_DIR)/*.h))
+ARCH_INC := $(sort $(notdir $(wildcard $(RTE_SDK)/lib/librte_eal/common/include/arch/$(ARCH_DIR)/*.h)))
 
 SYMLINK-$(CONFIG_RTE_LIBRTE_EAL)-include := $(addprefix include/,$(INC))
 SYMLINK-$(CONFIG_RTE_LIBRTE_EAL)-include += \
-- 
2.11.0

[dpdk-dev] [PATCH v2 5/7] mk: sort source files before passing them to the compiler

[lboccass]

From: Luca Boccassi <luca.boccassi@gmail.com>

In order to achieve reproducible builds, always use the same
order when listing files for compilation.

Signed-off-by: Luca Boccassi <luca.boccassi@gmail.com>
---
 drivers/net/cxgbe/Makefile    | 2 +-
 drivers/net/e1000/Makefile    | 2 +-
 drivers/net/fm10k/Makefile    | 2 +-
 drivers/net/i40e/Makefile     | 2 +-
 drivers/net/ixgbe/Makefile    | 2 +-
 drivers/net/qede/Makefile     | 2 +-
 drivers/net/sfc/Makefile      | 2 +-
 drivers/net/thunderx/Makefile | 2 +-
 8 files changed, 8 insertions(+), 8 deletions(-)

diff --git a/drivers/net/cxgbe/Makefile b/drivers/net/cxgbe/Makefile
index 7cef6279c..b4666b5af 100644
--- a/drivers/net/cxgbe/Makefile
+++ b/drivers/net/cxgbe/Makefile
@@ -67,7 +67,7 @@ endif
 # Add extra flags for base driver files (also known as shared code)
 # to disable warnings in them
 #
-BASE_DRIVER_OBJS=$(patsubst %.c,%.o,$(notdir $(wildcard $(SRCDIR)/base/*.c)))
+BASE_DRIVER_OBJS=$(sort $(patsubst %.c,%.o,$(notdir $(wildcard $(SRCDIR)/base/*.c))))
 $(foreach obj, $(BASE_DRIVER_OBJS), $(eval CFLAGS_$(obj)+=$(CFLAGS_BASE_DRIVER)))
 
 VPATH += $(SRCDIR)/base
diff --git a/drivers/net/e1000/Makefile b/drivers/net/e1000/Makefile
index ffdf36d37..59d96bca1 100644
--- a/drivers/net/e1000/Makefile
+++ b/drivers/net/e1000/Makefile
@@ -68,7 +68,7 @@ endif
 # Add extra flags for base driver files (also known as shared code)
 # to disable warnings in them
 #
-BASE_DRIVER_OBJS=$(patsubst %.c,%.o,$(notdir $(wildcard $(SRCDIR)/base/*.c)))
+BASE_DRIVER_OBJS=$(sort $(patsubst %.c,%.o,$(notdir $(wildcard $(SRCDIR)/base/*.c))))
 $(foreach obj, $(BASE_DRIVER_OBJS), $(eval CFLAGS_$(obj)+=$(CFLAGS_BASE_DRIVER)))
 
 VPATH += $(SRCDIR)/base
diff --git a/drivers/net/fm10k/Makefile b/drivers/net/fm10k/Makefile
index e0024f052..0bc124eb1 100644
--- a/drivers/net/fm10k/Makefile
+++ b/drivers/net/fm10k/Makefile
@@ -80,7 +80,7 @@ endif
 #
 # Add extra flags for base driver source files to disable warnings in them
 #
-BASE_DRIVER_OBJS=$(patsubst %.c,%.o,$(notdir $(wildcard $(SRCDIR)/base/*.c)))
+BASE_DRIVER_OBJS=$(sort $(patsubst %.c,%.o,$(notdir $(wildcard $(SRCDIR)/base/*.c))))
 $(foreach obj, $(BASE_DRIVER_OBJS), $(eval CFLAGS_$(obj)+=$(CFLAGS_BASE_DRIVER)))
 
 VPATH += $(SRCDIR)/base
diff --git a/drivers/net/i40e/Makefile b/drivers/net/i40e/Makefile
index 56f210d6d..06eedc592 100644
--- a/drivers/net/i40e/Makefile
+++ b/drivers/net/i40e/Makefile
@@ -78,7 +78,7 @@ endif
 
 CFLAGS_i40e_lan_hmc.o += -Wno-error
 endif
-OBJS_BASE_DRIVER=$(patsubst %.c,%.o,$(notdir $(wildcard $(SRCDIR)/base/*.c)))
+OBJS_BASE_DRIVER=$(sort $(patsubst %.c,%.o,$(notdir $(wildcard $(SRCDIR)/base/*.c))))
 $(foreach obj, $(OBJS_BASE_DRIVER), $(eval CFLAGS_$(obj)+=$(CFLAGS_BASE_DRIVER)))
 
 VPATH += $(SRCDIR)/base
diff --git a/drivers/net/ixgbe/Makefile b/drivers/net/ixgbe/Makefile
index 6a651b923..f5c370ce5 100644
--- a/drivers/net/ixgbe/Makefile
+++ b/drivers/net/ixgbe/Makefile
@@ -87,7 +87,7 @@ endif
 # Add extra flags for base driver files (also known as shared code)
 # to disable warnings in them
 #
-BASE_DRIVER_OBJS=$(patsubst %.c,%.o,$(notdir $(wildcard $(SRCDIR)/base/*.c)))
+BASE_DRIVER_OBJS=$(sort $(patsubst %.c,%.o,$(notdir $(wildcard $(SRCDIR)/base/*.c))))
 $(foreach obj, $(BASE_DRIVER_OBJS), $(eval CFLAGS_$(obj)+=$(CFLAGS_BASE_DRIVER)))
 
 VPATH += $(SRCDIR)/base
diff --git a/drivers/net/qede/Makefile b/drivers/net/qede/Makefile
index f03441d9a..83ff95474 100644
--- a/drivers/net/qede/Makefile
+++ b/drivers/net/qede/Makefile
@@ -78,7 +78,7 @@ endif
 # to disable warnings in them
 #
 #
-BASE_DRIVER_OBJS=$(patsubst %.c,%.o,$(notdir $(wildcard $(SRCDIR)/base/*.c)))
+BASE_DRIVER_OBJS=$(sort $(patsubst %.c,%.o,$(notdir $(wildcard $(SRCDIR)/base/*.c))))
 $(foreach obj, $(BASE_DRIVER_OBJS), $(eval CFLAGS_$(obj)+=$(CFLAGS_BASE_DRIVER)))
 
 VPATH += $(SRCDIR)/base
diff --git a/drivers/net/sfc/Makefile b/drivers/net/sfc/Makefile
index 57aa963ba..8cfd14d45 100644
--- a/drivers/net/sfc/Makefile
+++ b/drivers/net/sfc/Makefile
@@ -71,7 +71,7 @@ endif
 # List of base driver object files for which
 # special CFLAGS above should be applied
 #
-BASE_DRIVER_OBJS=$(patsubst %.c,%.o,$(notdir $(wildcard $(SRCDIR)/base/*.c)))
+BASE_DRIVER_OBJS=$(sort $(patsubst %.c,%.o,$(notdir $(wildcard $(SRCDIR)/base/*.c))))
 $(foreach obj, $(BASE_DRIVER_OBJS), \
   $(eval CFLAGS_$(obj)+=$(CFLAGS_BASE_DRIVER)))
 
diff --git a/drivers/net/thunderx/Makefile b/drivers/net/thunderx/Makefile
index 706250b8b..ac384a624 100644
--- a/drivers/net/thunderx/Makefile
+++ b/drivers/net/thunderx/Makefile
@@ -45,7 +45,7 @@ EXPORT_MAP := rte_pmd_thunderx_nicvf_version.map
 
 LIBABIVER := 1
 
-OBJS_BASE_DRIVER=$(patsubst %.c,%.o,$(notdir $(wildcard $(SRCDIR)/base/*.c)))
+OBJS_BASE_DRIVER=$(sort $(patsubst %.c,%.o,$(notdir $(wildcard $(SRCDIR)/base/*.c))))
 $(foreach obj, $(OBJS_BASE_DRIVER), $(eval CFLAGS_$(obj)+=$(CFLAGS_BASE_DRIVER)))
 
 VPATH += $(SRCDIR)/base
-- 
2.11.0

[dpdk-dev] [PATCH v2 6/7] mk: sort object files when building deps lists

[lboccass]

From: Luca Boccassi <luca.boccassi@gmail.com>

In order to achieve reproducible builds, always use the same
order when listing object files to build dependencies lists.

Signed-off-by: Luca Boccassi <luca.boccassi@gmail.com>
---
 mk/rte.app.mk     | 4 ++--
 mk/rte.hostapp.mk | 4 ++--
 mk/rte.shared.mk  | 4 ++--
 3 files changed, 6 insertions(+), 6 deletions(-)

diff --git a/mk/rte.app.mk b/mk/rte.app.mk
index bcaf1b382..54134dea4 100644
--- a/mk/rte.app.mk
+++ b/mk/rte.app.mk
@@ -263,8 +263,8 @@ LDLIBS_NAMES += $(patsubst -Wl$(comma)-l%,lib%.a,$(filter -Wl$(comma)-l%,$(LDLIB
 
 # list of found libraries files (useful for deps). If not found, the
 # library is silently ignored and dep won't be checked
-LDLIBS_FILES := $(wildcard $(foreach dir,$(LDLIBS_PATH),\
-	$(addprefix $(dir)/,$(LDLIBS_NAMES))))
+LDLIBS_FILES := $(sort $(wildcard $(foreach dir,$(LDLIBS_PATH),\
+	$(addprefix $(dir)/,$(LDLIBS_NAMES)))))
 
 #
 # Compile executable file if needed
diff --git a/mk/rte.hostapp.mk b/mk/rte.hostapp.mk
index 5cb4909cb..f58173c31 100644
--- a/mk/rte.hostapp.mk
+++ b/mk/rte.hostapp.mk
@@ -69,9 +69,9 @@ O_TO_EXE_DO = @set -e; \
 -include .$(HOSTAPP).cmd
 
 # list of .a files that are linked to this application
-LDLIBS_FILES := $(wildcard \
+LDLIBS_FILES := $(sort $(wildcard \
 	$(addprefix $(RTE_OUTPUT)/lib/, \
-	$(patsubst -l%,lib%.a,$(filter -l%,$(LDLIBS)))))
+	$(patsubst -l%,lib%.a,$(filter -l%,$(LDLIBS))))))
 
 #
 # Compile executable file if needed
diff --git a/mk/rte.shared.mk b/mk/rte.shared.mk
index 87ccf0ba4..4e680bc03 100644
--- a/mk/rte.shared.mk
+++ b/mk/rte.shared.mk
@@ -85,8 +85,8 @@ LDLIBS_NAMES += $(patsubst -Wl$(comma)-l%,lib%.a,$(filter -Wl$(comma)-l%,$(LDLIB
 
 # list of found libraries files (useful for deps). If not found, the
 # library is silently ignored and dep won't be checked
-LDLIBS_FILES := $(wildcard $(foreach dir,$(LDLIBS_PATH),\
-	$(addprefix $(dir)/,$(LDLIBS_NAMES))))
+LDLIBS_FILES := $(sort $(wildcard $(foreach dir,$(LDLIBS_PATH),\
+	$(addprefix $(dir)/,$(LDLIBS_NAMES)))))
 
 #
 # Archive objects in .so file if needed
-- 
2.11.0

[dpdk-dev] [PATCH v2 7/7] mk: always rebuild in the same order

[lboccass]

From: Luca Boccassi <luca.boccassi@gmail.com>

In order to achieve reproducible builds, always check dependencies in
the same order.

Signed-off-by: Luca Boccassi <luca.boccassi@gmail.com>
---
 mk/internal/rte.compile-pre.mk | 8 ++++----
 1 file changed, 4 insertions(+), 4 deletions(-)

diff --git a/mk/internal/rte.compile-pre.mk b/mk/internal/rte.compile-pre.mk
index da8dda498..5d519100c 100644
--- a/mk/internal/rte.compile-pre.mk
+++ b/mk/internal/rte.compile-pre.mk
@@ -108,13 +108,13 @@ C_TO_O_DO = @set -e; \
 compare = $(strip $(subst $(1),,$(2)) $(subst $(2),,$(1)))
 
 # return a non-empty string if the dst file does not exist
-file_missing = $(call compare,$(wildcard $@),$@)
+file_missing = $(call compare,$(sort $(wildcard $@)),$@)
 
 # return a non-empty string if cmdline changed
 cmdline_changed = $(call compare,$(strip $(cmd_$@)),$(strip $(1)))
 
 # return a non-empty string if a dependency file does not exist
-depfile_missing = $(call compare,$(wildcard $(dep_$@)),$(dep_$@))
+depfile_missing = $(call compare,$(sort $(wildcard $(dep_$@))),$(dep_$@))
 
 # return an empty string if no prereq is newer than target
 #     - $^ -> names of all the prerequisites
@@ -123,7 +123,7 @@ depfile_missing = $(call compare,$(wildcard $(dep_$@)),$(dep_$@))
 #       exist (filter-out removes existing ones from the list)
 #     - $? -> names of all the prerequisites newer than target
 depfile_newer = $(strip $(filter-out FORCE,$? \
-	$(filter-out $(wildcard $^),$^)))
+	$(filter-out $(sort $(wildcard $^)),$^)))
 
 # return 1 if parameter is a non-empty string, else 0
 boolean = $(if $1,1,0)
@@ -134,7 +134,7 @@ boolean = $(if $1,1,0)
 # user (by default it is empty)
 #
 .SECONDEXPANSION:
-%.o: %.c $$(wildcard $$(dep_$$@)) $$(DEP_$$(@)) FORCE
+%.o: %.c $$(sort $$(wildcard $$(dep_$$@))) $$(DEP_$$(@)) FORCE
 	@[ -d $(dir $@) ] || mkdir -p $(dir $@)
 	$(if $(D),\
 		@echo -n "$< -> $@ " ; \
-- 
2.11.0

Re: [dpdk-dev] [PATCH v2 0/7] Reproducible build

[Luca Boccassi]

On Fri, 2017-06-23 at 19:16 +0100, lboccass@brocade.com wrote:
> From: Luca Boccassi <luca.boccassi@gmail.com>
> 
> In the past couple of years a concerted effort among almost all Linux
> distros has been striving toward achieving reproducible builds. [1]
> This involves changes to the toolchain, new tools and CI systems. [2]
> 
> v1 fixed the documentation, examples and linker script generation.
> v2 fixes all problems, which were caused by unstable order of headers
> inclusion, source files listing and object file listing when passing
> them to the compiler.
> DPDK's build, at least with the default configuration, is fully
> reproducible with this patch series as tested by the Reproducible
> Builds developers experimental toolchain.
> 
> [1] https://reproducible-builds.org/
> [2] https://reproducible-builds.org/tools/
> [3] https://wiki.debian.org/ReproducibleBuilds/ExperimentalToolchain#
> Us
> 
> Luca Boccassi (7):
>   mk: fix excluding .doctrees when installing docs
>   mk: sort list of shared objects in linker script
>   mk: sort list of files in examples.dox
>   mk: sort headers before wildcard inclusion
>   mk: sort source files before passing them to the compiler
>   mk: sort object files when building deps lists
>   mk: always rebuild in the same order

Agh git send-email ate one patch! Third time's the charm...

Kind regards,
Luca Boccassi

[dpdk-dev] [PATCH v3 0/8] Reproducible build

[lboccass]

From: Luca Boccassi <luca.boccassi@gmail.com>

In the past couple of years a concerted effort among almost all Linux
distros has been striving toward achieving reproducible builds. [1]
This involves changes to the toolchain, new tools and CI systems. [2]

v1 fixed the documentation, examples and linker script generation.
v2 fixes all problems, which were caused by unstable order of headers
inclusion, source files listing and object file listing when passing
them to the compiler.
DPDK's build, at least with the default configuration, is fully
reproducible with this patch series as tested by the Reproducible
Builds developers experimental toolchain.

v3 restores the first patch, which was eaten by git send-email.

[1] https://reproducible-builds.org/
[2] https://reproducible-builds.org/tools/
[3] https://wiki.debian.org/ReproducibleBuilds/ExperimentalToolchain#Us

Luca Boccassi (8):
  mk: use make silent flag to print HTML doc version
  mk: fix excluding .doctrees when installing docs
  mk: sort list of shared objects in linker script
  mk: sort list of files in examples.dox
  mk: sort headers before wildcard inclusion
  mk: sort source files before passing them to the compiler
  mk: sort object files when building deps lists
  mk: always rebuild in the same order

 drivers/net/cxgbe/Makefile                                 | 2 +-
 drivers/net/e1000/Makefile                                 | 2 +-
 drivers/net/fm10k/Makefile                                 | 2 +-
 drivers/net/i40e/Makefile                                  | 2 +-
 drivers/net/ixgbe/Makefile                                 | 2 +-
 drivers/net/qede/Makefile                                  | 2 +-
 drivers/net/sfc/Makefile                                   | 2 +-
 drivers/net/thunderx/Makefile                              | 2 +-
 examples/ip_pipeline/Makefile                              | 2 +-
 examples/multi_process/client_server_mp/mp_server/Makefile | 2 +-
 examples/server_node_efd/server/Makefile                   | 2 +-
 lib/librte_eal/common/Makefile                             | 2 +-
 mk/internal/rte.compile-pre.mk                             | 8 ++++----
 mk/rte.app.mk                                              | 4 ++--
 mk/rte.combinedlib.mk                                      | 2 +-
 mk/rte.hostapp.mk                                          | 4 ++--
 mk/rte.sdkdoc.mk                                           | 4 ++--
 mk/rte.sdkinstall.mk                                       | 2 +-
 mk/rte.shared.mk                                           | 4 ++--
 19 files changed, 26 insertions(+), 26 deletions(-)

-- 
2.11.0

[dpdk-dev] [PATCH v3 1/8] mk: use make silent flag to print HTML doc version

[lboccass]

From: Luca Boccassi <luca.boccassi@gmail.com>

Depending on the environment, make might echo the command being ran.
In mk/rte.sdkdoc.mk make is used to print the DPDK version to be
piped to doxygen. This causes the following to be written:

<div id="projectname">DPDK
&#160;<span id="projectnumber">/usr/bin/make-f/build/dpdk-jYjqnr/
 dpdk-16.11.2/mk/rte.sdkconfig.mkshowversion</span>
</div>

Use -s (--silent) to prevent echoing.

Signed-off-by: Luca Boccassi <luca.boccassi@gmail.com>
---
 mk/rte.sdkdoc.mk | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/mk/rte.sdkdoc.mk b/mk/rte.sdkdoc.mk
index fb8f91555..c0eaa3502 100644
--- a/mk/rte.sdkdoc.mk
+++ b/mk/rte.sdkdoc.mk
@@ -73,7 +73,7 @@ api-html: $(API_EXAMPLES)
 	$(Q)mkdir -p $(RTE_OUTPUT)/doc/html
 	$(Q)(cat $(RTE_SDK)/doc/api/doxy-api.conf     && \
 	    printf 'PROJECT_NUMBER = '                && \
-	                      $(MAKE) -rR showversion && \
+	                     $(MAKE) -rRs showversion && \
 	    echo INPUT           += $(API_EXAMPLES)   && \
 	    echo OUTPUT_DIRECTORY = $(RTE_OUTPUT)/doc && \
 	    echo HTML_OUTPUT      = html/api          && \
-- 
2.11.0

[dpdk-dev] [PATCH v3 2/8] mk: fix excluding .doctrees when installing docs

[lboccass]

From: Luca Boccassi <luca.boccassi@gmail.com>

The --exclude parameter must be passed before the input directory to
tar, otherwise it's silently ignored and the .doctrees directory is
installed by make install-doc.

Signed-off-by: Luca Boccassi <luca.boccassi@gmail.com>
---
 mk/rte.sdkinstall.mk | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/mk/rte.sdkinstall.mk b/mk/rte.sdkinstall.mk
index dbac2a277..4e97feff9 100644
--- a/mk/rte.sdkinstall.mk
+++ b/mk/rte.sdkinstall.mk
@@ -162,7 +162,7 @@ install-sdk:
 install-doc:
 ifneq ($(wildcard $O/doc/html),)
 	$(Q)$(call rte_mkdir, $(DESTDIR)$(docdir))
-	$(Q)tar -cf -      -C $O/doc html --exclude 'html/guides/.*' | \
+	$(Q)tar -cf -      -C $O/doc --exclude 'html/guides/.*' html | \
 	    tar -xf -      -C $(DESTDIR)$(docdir) --strip-components=1 \
 		--keep-newer-files
 endif
-- 
2.11.0

[dpdk-dev] [PATCH v3 3/8] mk: sort list of shared objects in linker script

[lboccass]

From: Luca Boccassi <luca.boccassi@gmail.com>

The output of wildcard might not be stable and depend on the
filesystem and other factors.
This means the content libdpdk.so linker script might change between
builds from the same sources.
Run the list through sort to ensure reproducibility.

Signed-off-by: Luca Boccassi <luca.boccassi@gmail.com>
---
 mk/rte.combinedlib.mk | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/mk/rte.combinedlib.mk b/mk/rte.combinedlib.mk
index 449358b33..2ab7ee8a1 100644
--- a/mk/rte.combinedlib.mk
+++ b/mk/rte.combinedlib.mk
@@ -42,7 +42,7 @@ endif
 RTE_LIBNAME := dpdk
 COMBINEDLIB := lib$(RTE_LIBNAME)$(EXT)
 
-LIBS := $(filter-out $(COMBINEDLIB), $(notdir $(wildcard $(RTE_OUTPUT)/lib/*$(EXT))))
+LIBS := $(filter-out $(COMBINEDLIB), $(sort $(notdir $(wildcard $(RTE_OUTPUT)/lib/*$(EXT)))))
 
 all: FORCE
 	$(Q)echo "GROUP ( $(LIBS) )" > $(RTE_OUTPUT)/lib/$(COMBINEDLIB)
-- 
2.11.0

[dpdk-dev] [PATCH v3 4/8] mk: sort list of files in examples.dox

[lboccass]

From: Luca Boccassi <luca.boccassi@gmail.com>

The result of find might not be stable depending on external
conditions.
Pipe it through LC_ALL=C sort to ensure reproducible results when
generating examples.dox.

Signed-off-by: Luca Boccassi <luca.boccassi@gmail.com>
---
 mk/rte.sdkdoc.mk | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/mk/rte.sdkdoc.mk b/mk/rte.sdkdoc.mk
index c0eaa3502..de31b78cf 100644
--- a/mk/rte.sdkdoc.mk
+++ b/mk/rte.sdkdoc.mk
@@ -93,7 +93,7 @@ $(API_EXAMPLES): api-html-clean
 	$(Q)mkdir -p $(@D)
 	@printf '/**\n' > $(API_EXAMPLES)
 	@printf '@page examples DPDK Example Programs\n\n' >> $(API_EXAMPLES)
-	@find examples -type f -name '*.c' -printf '@example %p\n' >> $(API_EXAMPLES)
+	@find examples -type f -name '*.c' -printf '@example %p\n' | LC_ALL=C sort >> $(API_EXAMPLES)
 	@printf '*/\n' >> $(API_EXAMPLES)
 
 guides-pdf-clean: guides-pdf-img-clean
-- 
2.11.0

[dpdk-dev] [PATCH v3 5/8] mk: sort headers before wildcard inclusion

[lboccass]

From: Luca Boccassi <luca.boccassi@gmail.com>

In order to achieve fully reproducible builds, always use the same
inclusion order for headers in the Makefiles.

Signed-off-by: Luca Boccassi <luca.boccassi@gmail.com>
---
 examples/ip_pipeline/Makefile                              | 2 +-
 examples/multi_process/client_server_mp/mp_server/Makefile | 2 +-
 examples/server_node_efd/server/Makefile                   | 2 +-
 lib/librte_eal/common/Makefile                             | 2 +-
 4 files changed, 4 insertions(+), 4 deletions(-)

diff --git a/examples/ip_pipeline/Makefile b/examples/ip_pipeline/Makefile
index dc7e0ddd7..12ce0a1d5 100644
--- a/examples/ip_pipeline/Makefile
+++ b/examples/ip_pipeline/Makefile
@@ -43,7 +43,7 @@ APP = ip_pipeline
 
 VPATH += $(SRCDIR)/pipeline
 
-INC += $(wildcard *.h) $(wildcard pipeline/*.h)
+INC += $(sort $(wildcard *.h)) $(sort $(wildcard pipeline/*.h))
 
 # all source are stored in SRCS-y
 SRCS-$(CONFIG_RTE_LIBRTE_PIPELINE) := main.c
diff --git a/examples/multi_process/client_server_mp/mp_server/Makefile b/examples/multi_process/client_server_mp/mp_server/Makefile
index 5552999b5..160c17b68 100644
--- a/examples/multi_process/client_server_mp/mp_server/Makefile
+++ b/examples/multi_process/client_server_mp/mp_server/Makefile
@@ -49,7 +49,7 @@ APP = mp_server
 # all source are stored in SRCS-y
 SRCS-y := main.c init.c args.c
 
-INC := $(wildcard *.h)
+INC := $(sort $(wildcard *.h))
 
 CFLAGS += $(WERROR_FLAGS) -O3
 CFLAGS += -I$(SRCDIR)/../shared
diff --git a/examples/server_node_efd/server/Makefile b/examples/server_node_efd/server/Makefile
index a2f2f361b..9f1fe2894 100644
--- a/examples/server_node_efd/server/Makefile
+++ b/examples/server_node_efd/server/Makefile
@@ -49,7 +49,7 @@ APP = server
 # all source are stored in SRCS-y
 SRCS-y := main.c init.c args.c
 
-INC := $(wildcard *.h)
+INC := $(sort $(wildcard *.h))
 
 CFLAGS += $(WERROR_FLAGS) -O3
 CFLAGS += -I$(SRCDIR)/../shared
diff --git a/lib/librte_eal/common/Makefile b/lib/librte_eal/common/Makefile
index a5bd1089a..4b712600a 100644
--- a/lib/librte_eal/common/Makefile
+++ b/lib/librte_eal/common/Makefile
@@ -48,7 +48,7 @@ GENERIC_INC += rte_vect.h rte_io.h
 
 # defined in mk/arch/$(RTE_ARCH)/rte.vars.mk
 ARCH_DIR ?= $(RTE_ARCH)
-ARCH_INC := $(notdir $(wildcard $(RTE_SDK)/lib/librte_eal/common/include/arch/$(ARCH_DIR)/*.h))
+ARCH_INC := $(sort $(notdir $(wildcard $(RTE_SDK)/lib/librte_eal/common/include/arch/$(ARCH_DIR)/*.h)))
 
 SYMLINK-$(CONFIG_RTE_LIBRTE_EAL)-include := $(addprefix include/,$(INC))
 SYMLINK-$(CONFIG_RTE_LIBRTE_EAL)-include += \
-- 
2.11.0

[dpdk-dev] [PATCH v3 6/8] mk: sort source files before passing them to the compiler

[lboccass]

From: Luca Boccassi <luca.boccassi@gmail.com>

In order to achieve reproducible builds, always use the same
order when listing files for compilation.

Signed-off-by: Luca Boccassi <luca.boccassi@gmail.com>
---
 drivers/net/cxgbe/Makefile    | 2 +-
 drivers/net/e1000/Makefile    | 2 +-
 drivers/net/fm10k/Makefile    | 2 +-
 drivers/net/i40e/Makefile     | 2 +-
 drivers/net/ixgbe/Makefile    | 2 +-
 drivers/net/qede/Makefile     | 2 +-
 drivers/net/sfc/Makefile      | 2 +-
 drivers/net/thunderx/Makefile | 2 +-
 8 files changed, 8 insertions(+), 8 deletions(-)

diff --git a/drivers/net/cxgbe/Makefile b/drivers/net/cxgbe/Makefile
index 7cef6279c..b4666b5af 100644
--- a/drivers/net/cxgbe/Makefile
+++ b/drivers/net/cxgbe/Makefile
@@ -67,7 +67,7 @@ endif
 # Add extra flags for base driver files (also known as shared code)
 # to disable warnings in them
 #
-BASE_DRIVER_OBJS=$(patsubst %.c,%.o,$(notdir $(wildcard $(SRCDIR)/base/*.c)))
+BASE_DRIVER_OBJS=$(sort $(patsubst %.c,%.o,$(notdir $(wildcard $(SRCDIR)/base/*.c))))
 $(foreach obj, $(BASE_DRIVER_OBJS), $(eval CFLAGS_$(obj)+=$(CFLAGS_BASE_DRIVER)))
 
 VPATH += $(SRCDIR)/base
diff --git a/drivers/net/e1000/Makefile b/drivers/net/e1000/Makefile
index ffdf36d37..59d96bca1 100644
--- a/drivers/net/e1000/Makefile
+++ b/drivers/net/e1000/Makefile
@@ -68,7 +68,7 @@ endif
 # Add extra flags for base driver files (also known as shared code)
 # to disable warnings in them
 #
-BASE_DRIVER_OBJS=$(patsubst %.c,%.o,$(notdir $(wildcard $(SRCDIR)/base/*.c)))
+BASE_DRIVER_OBJS=$(sort $(patsubst %.c,%.o,$(notdir $(wildcard $(SRCDIR)/base/*.c))))
 $(foreach obj, $(BASE_DRIVER_OBJS), $(eval CFLAGS_$(obj)+=$(CFLAGS_BASE_DRIVER)))
 
 VPATH += $(SRCDIR)/base
diff --git a/drivers/net/fm10k/Makefile b/drivers/net/fm10k/Makefile
index e0024f052..0bc124eb1 100644
--- a/drivers/net/fm10k/Makefile
+++ b/drivers/net/fm10k/Makefile
@@ -80,7 +80,7 @@ endif
 #
 # Add extra flags for base driver source files to disable warnings in them
 #
-BASE_DRIVER_OBJS=$(patsubst %.c,%.o,$(notdir $(wildcard $(SRCDIR)/base/*.c)))
+BASE_DRIVER_OBJS=$(sort $(patsubst %.c,%.o,$(notdir $(wildcard $(SRCDIR)/base/*.c))))
 $(foreach obj, $(BASE_DRIVER_OBJS), $(eval CFLAGS_$(obj)+=$(CFLAGS_BASE_DRIVER)))
 
 VPATH += $(SRCDIR)/base
diff --git a/drivers/net/i40e/Makefile b/drivers/net/i40e/Makefile
index 56f210d6d..06eedc592 100644
--- a/drivers/net/i40e/Makefile
+++ b/drivers/net/i40e/Makefile
@@ -78,7 +78,7 @@ endif
 
 CFLAGS_i40e_lan_hmc.o += -Wno-error
 endif
-OBJS_BASE_DRIVER=$(patsubst %.c,%.o,$(notdir $(wildcard $(SRCDIR)/base/*.c)))
+OBJS_BASE_DRIVER=$(sort $(patsubst %.c,%.o,$(notdir $(wildcard $(SRCDIR)/base/*.c))))
 $(foreach obj, $(OBJS_BASE_DRIVER), $(eval CFLAGS_$(obj)+=$(CFLAGS_BASE_DRIVER)))
 
 VPATH += $(SRCDIR)/base
diff --git a/drivers/net/ixgbe/Makefile b/drivers/net/ixgbe/Makefile
index 6a651b923..f5c370ce5 100644
--- a/drivers/net/ixgbe/Makefile
+++ b/drivers/net/ixgbe/Makefile
@@ -87,7 +87,7 @@ endif
 # Add extra flags for base driver files (also known as shared code)
 # to disable warnings in them
 #
-BASE_DRIVER_OBJS=$(patsubst %.c,%.o,$(notdir $(wildcard $(SRCDIR)/base/*.c)))
+BASE_DRIVER_OBJS=$(sort $(patsubst %.c,%.o,$(notdir $(wildcard $(SRCDIR)/base/*.c))))
 $(foreach obj, $(BASE_DRIVER_OBJS), $(eval CFLAGS_$(obj)+=$(CFLAGS_BASE_DRIVER)))
 
 VPATH += $(SRCDIR)/base
diff --git a/drivers/net/qede/Makefile b/drivers/net/qede/Makefile
index f03441d9a..83ff95474 100644
--- a/drivers/net/qede/Makefile
+++ b/drivers/net/qede/Makefile
@@ -78,7 +78,7 @@ endif
 # to disable warnings in them
 #
 #
-BASE_DRIVER_OBJS=$(patsubst %.c,%.o,$(notdir $(wildcard $(SRCDIR)/base/*.c)))
+BASE_DRIVER_OBJS=$(sort $(patsubst %.c,%.o,$(notdir $(wildcard $(SRCDIR)/base/*.c))))
 $(foreach obj, $(BASE_DRIVER_OBJS), $(eval CFLAGS_$(obj)+=$(CFLAGS_BASE_DRIVER)))
 
 VPATH += $(SRCDIR)/base
diff --git a/drivers/net/sfc/Makefile b/drivers/net/sfc/Makefile
index 57aa963ba..8cfd14d45 100644
--- a/drivers/net/sfc/Makefile
+++ b/drivers/net/sfc/Makefile
@@ -71,7 +71,7 @@ endif
 # List of base driver object files for which
 # special CFLAGS above should be applied
 #
-BASE_DRIVER_OBJS=$(patsubst %.c,%.o,$(notdir $(wildcard $(SRCDIR)/base/*.c)))
+BASE_DRIVER_OBJS=$(sort $(patsubst %.c,%.o,$(notdir $(wildcard $(SRCDIR)/base/*.c))))
 $(foreach obj, $(BASE_DRIVER_OBJS), \
   $(eval CFLAGS_$(obj)+=$(CFLAGS_BASE_DRIVER)))
 
diff --git a/drivers/net/thunderx/Makefile b/drivers/net/thunderx/Makefile
index 706250b8b..ac384a624 100644
--- a/drivers/net/thunderx/Makefile
+++ b/drivers/net/thunderx/Makefile
@@ -45,7 +45,7 @@ EXPORT_MAP := rte_pmd_thunderx_nicvf_version.map
 
 LIBABIVER := 1
 
-OBJS_BASE_DRIVER=$(patsubst %.c,%.o,$(notdir $(wildcard $(SRCDIR)/base/*.c)))
+OBJS_BASE_DRIVER=$(sort $(patsubst %.c,%.o,$(notdir $(wildcard $(SRCDIR)/base/*.c))))
 $(foreach obj, $(OBJS_BASE_DRIVER), $(eval CFLAGS_$(obj)+=$(CFLAGS_BASE_DRIVER)))
 
 VPATH += $(SRCDIR)/base
-- 
2.11.0

[dpdk-dev] [PATCH v3 7/8] mk: sort object files when building deps lists

[lboccass]

From: Luca Boccassi <luca.boccassi@gmail.com>

In order to achieve reproducible builds, always use the same
order when listing object files to build dependencies lists.

Signed-off-by: Luca Boccassi <luca.boccassi@gmail.com>
---
 mk/rte.app.mk     | 4 ++--
 mk/rte.hostapp.mk | 4 ++--
 mk/rte.shared.mk  | 4 ++--
 3 files changed, 6 insertions(+), 6 deletions(-)

diff --git a/mk/rte.app.mk b/mk/rte.app.mk
index bcaf1b382..54134dea4 100644
--- a/mk/rte.app.mk
+++ b/mk/rte.app.mk
@@ -263,8 +263,8 @@ LDLIBS_NAMES += $(patsubst -Wl$(comma)-l%,lib%.a,$(filter -Wl$(comma)-l%,$(LDLIB
 
 # list of found libraries files (useful for deps). If not found, the
 # library is silently ignored and dep won't be checked
-LDLIBS_FILES := $(wildcard $(foreach dir,$(LDLIBS_PATH),\
-	$(addprefix $(dir)/,$(LDLIBS_NAMES))))
+LDLIBS_FILES := $(sort $(wildcard $(foreach dir,$(LDLIBS_PATH),\
+	$(addprefix $(dir)/,$(LDLIBS_NAMES)))))
 
 #
 # Compile executable file if needed
diff --git a/mk/rte.hostapp.mk b/mk/rte.hostapp.mk
index 5cb4909cb..f58173c31 100644
--- a/mk/rte.hostapp.mk
+++ b/mk/rte.hostapp.mk
@@ -69,9 +69,9 @@ O_TO_EXE_DO = @set -e; \
 -include .$(HOSTAPP).cmd
 
 # list of .a files that are linked to this application
-LDLIBS_FILES := $(wildcard \
+LDLIBS_FILES := $(sort $(wildcard \
 	$(addprefix $(RTE_OUTPUT)/lib/, \
-	$(patsubst -l%,lib%.a,$(filter -l%,$(LDLIBS)))))
+	$(patsubst -l%,lib%.a,$(filter -l%,$(LDLIBS))))))
 
 #
 # Compile executable file if needed
diff --git a/mk/rte.shared.mk b/mk/rte.shared.mk
index 87ccf0ba4..4e680bc03 100644
--- a/mk/rte.shared.mk
+++ b/mk/rte.shared.mk
@@ -85,8 +85,8 @@ LDLIBS_NAMES += $(patsubst -Wl$(comma)-l%,lib%.a,$(filter -Wl$(comma)-l%,$(LDLIB
 
 # list of found libraries files (useful for deps). If not found, the
 # library is silently ignored and dep won't be checked
-LDLIBS_FILES := $(wildcard $(foreach dir,$(LDLIBS_PATH),\
-	$(addprefix $(dir)/,$(LDLIBS_NAMES))))
+LDLIBS_FILES := $(sort $(wildcard $(foreach dir,$(LDLIBS_PATH),\
+	$(addprefix $(dir)/,$(LDLIBS_NAMES)))))
 
 #
 # Archive objects in .so file if needed
-- 
2.11.0

[dpdk-dev] [PATCH v3 8/8] mk: always rebuild in the same order

[lboccass]

From: Luca Boccassi <luca.boccassi@gmail.com>

In order to achieve reproducible builds, always check dependencies in
the same order.

Signed-off-by: Luca Boccassi <luca.boccassi@gmail.com>
---
 mk/internal/rte.compile-pre.mk | 8 ++++----
 1 file changed, 4 insertions(+), 4 deletions(-)

diff --git a/mk/internal/rte.compile-pre.mk b/mk/internal/rte.compile-pre.mk
index da8dda498..5d519100c 100644
--- a/mk/internal/rte.compile-pre.mk
+++ b/mk/internal/rte.compile-pre.mk
@@ -108,13 +108,13 @@ C_TO_O_DO = @set -e; \
 compare = $(strip $(subst $(1),,$(2)) $(subst $(2),,$(1)))
 
 # return a non-empty string if the dst file does not exist
-file_missing = $(call compare,$(wildcard $@),$@)
+file_missing = $(call compare,$(sort $(wildcard $@)),$@)
 
 # return a non-empty string if cmdline changed
 cmdline_changed = $(call compare,$(strip $(cmd_$@)),$(strip $(1)))
 
 # return a non-empty string if a dependency file does not exist
-depfile_missing = $(call compare,$(wildcard $(dep_$@)),$(dep_$@))
+depfile_missing = $(call compare,$(sort $(wildcard $(dep_$@))),$(dep_$@))
 
 # return an empty string if no prereq is newer than target
 #     - $^ -> names of all the prerequisites
@@ -123,7 +123,7 @@ depfile_missing = $(call compare,$(wildcard $(dep_$@)),$(dep_$@))
 #       exist (filter-out removes existing ones from the list)
 #     - $? -> names of all the prerequisites newer than target
 depfile_newer = $(strip $(filter-out FORCE,$? \
-	$(filter-out $(wildcard $^),$^)))
+	$(filter-out $(sort $(wildcard $^)),$^)))
 
 # return 1 if parameter is a non-empty string, else 0
 boolean = $(if $1,1,0)
@@ -134,7 +134,7 @@ boolean = $(if $1,1,0)
 # user (by default it is empty)
 #
 .SECONDEXPANSION:
-%.o: %.c $$(wildcard $$(dep_$$@)) $$(DEP_$$(@)) FORCE
+%.o: %.c $$(sort $$(wildcard $$(dep_$$@))) $$(DEP_$$(@)) FORCE
 	@[ -d $(dir $@) ] || mkdir -p $(dir $@)
 	$(if $(D),\
 		@echo -n "$< -> $@ " ; \
-- 
2.11.0

Re: [dpdk-dev] [PATCH v3 1/8] mk: use make silent flag to print HTML doc version

[Mcnamara, John]

> -----Original Message-----
> From: dev [mailto:dev-bounces@dpdk.org] On Behalf Of lboccass@brocade.com
> Sent: Friday, June 23, 2017 7:42 PM
> To: dev@dpdk.org
> Cc: Luca Boccassi <luca.boccassi@gmail.com>
> Subject: [dpdk-dev] [PATCH v3 1/8] mk: use make silent flag to print HTML
> doc version
> 
> From: Luca Boccassi <luca.boccassi@gmail.com>

Acked-by: John McNamara <john.mcnamara@intel.com>

Re: [dpdk-dev] [PATCH v3 2/8] mk: fix excluding .doctrees when installing docs

[Mcnamara, John]

> -----Original Message-----
> From: dev [mailto:dev-bounces@dpdk.org] On Behalf Of lboccass@brocade.com
> Sent: Friday, June 23, 2017 7:42 PM
> To: dev@dpdk.org
> Cc: Luca Boccassi <luca.boccassi@gmail.com>
> Subject: [dpdk-dev] [PATCH v3 2/8] mk: fix excluding .doctrees when
> installing docs
> 
> From: Luca Boccassi <luca.boccassi@gmail.com>
> 
> The --exclude parameter must be passed before the input directory to tar,
> otherwise it's silently ignored and the .doctrees directory is installed
> by make install-doc.
> 
> Signed-off-by: Luca Boccassi <luca.boccassi@gmail.com>


Nice fixes.

Acked-by: John McNamara <john.mcnamara@intel.com>

Re: [dpdk-dev] [PATCH v3 4/8] mk: sort list of files in examples.dox

[Mcnamara, John]

> -----Original Message-----
> From: dev [mailto:dev-bounces@dpdk.org] On Behalf Of lboccass@brocade.com
> Sent: Friday, June 23, 2017 7:42 PM
> To: dev@dpdk.org
> Cc: Luca Boccassi <luca.boccassi@gmail.com>
> Subject: [dpdk-dev] [PATCH v3 4/8] mk: sort list of files in examples.dox
> 
> From: Luca Boccassi <luca.boccassi@gmail.com>
> 
> The result of find might not be stable depending on external conditions.
> Pipe it through LC_ALL=C sort to ensure reproducible results when
> generating examples.dox.
> 
> Signed-off-by: Luca Boccassi <luca.boccassi@gmail.com>

Acked-by: John McNamara <john.mcnamara@intel.com>

Re: [dpdk-dev] [PATCH v3 5/8] mk: sort headers before wildcard inclusion

[Mcnamara, John]

> -----Original Message-----
> From: dev [mailto:dev-bounces@dpdk.org] On Behalf Of lboccass@brocade.com
> Sent: Friday, June 23, 2017 7:42 PM
> To: dev@dpdk.org
> Cc: Luca Boccassi <luca.boccassi@gmail.com>
> Subject: [dpdk-dev] [PATCH v3 5/8] mk: sort headers before wildcard
> inclusion
> 
> From: Luca Boccassi <luca.boccassi@gmail.com>
> 
> In order to achieve fully reproducible builds, always use the same
> inclusion order for headers in the Makefiles.
> 
> Signed-off-by: Luca Boccassi <luca.boccassi@gmail.com>

Acked-by: John McNamara <john.mcnamara@intel.com>

Re: [dpdk-dev] [PATCH v3 4/8] mk: sort list of files in examples.dox

[Mcnamara, John]

> -----Original Message-----
> From: dev [mailto:dev-bounces@dpdk.org] On Behalf Of Mcnamara, John
> Sent: Monday, June 26, 2017 3:49 PM
> To: lboccass@brocade.com; dev@dpdk.org
> Cc: Luca Boccassi <luca.boccassi@gmail.com>
> Subject: Re: [dpdk-dev] [PATCH v3 4/8] mk: sort list of files in
> examples.dox
> 
> 
> 
> > -----Original Message-----
> > From: dev [mailto:dev-bounces@dpdk.org] On Behalf Of
> > lboccass@brocade.com
> > Sent: Friday, June 23, 2017 7:42 PM
> > To: dev@dpdk.org
> > Cc: Luca Boccassi <luca.boccassi@gmail.com>
> > Subject: [dpdk-dev] [PATCH v3 4/8] mk: sort list of files in
> > examples.dox
> >
> > From: Luca Boccassi <luca.boccassi@gmail.com>
> >
> > The result of find might not be stable depending on external conditions.
> > Pipe it through LC_ALL=C sort to ensure reproducible results when
> > generating examples.dox.
> >
> > Signed-off-by: Luca Boccassi <luca.boccassi@gmail.com>
> 
> Acked-by: John McNamara <john.mcnamara@intel.com>
> 

P.S. Could you mark the older versions of this patchset as Superseded in Patchwork. Thanks.

http://dpdk.org/dev/patchwork/project/dpdk/list/?submitter=566

Re: [dpdk-dev] [PATCH v3 0/8] Reproducible build

[Thomas Monjalon]

23/06/2017 20:41, lboccass@brocade.com:
> Luca Boccassi (8):
>   mk: use make silent flag to print HTML doc version
>   mk: fix excluding .doctrees when installing docs
>   mk: sort list of shared objects in linker script
>   mk: sort list of files in examples.dox
>   mk: sort headers before wildcard inclusion
>   mk: sort source files before passing them to the compiler
>   mk: sort object files when building deps lists
>   mk: always rebuild in the same order

These patches look good.
If there is a mistake hidden in these changes, better to test it
as soon as possible in various user environments.

Applied shortly, thanks

Re: [dpdk-dev] [PATCH v3 0/8] Reproducible build

[Thomas Monjalon]

27/06/2017 00:11, Thomas Monjalon:
> 23/06/2017 20:41, lboccass@brocade.com:
> > Luca Boccassi (8):
> >   mk: use make silent flag to print HTML doc version
> >   mk: fix excluding .doctrees when installing docs
> >   mk: sort list of shared objects in linker script
> >   mk: sort list of files in examples.dox
> >   mk: sort headers before wildcard inclusion
> >   mk: sort source files before passing them to the compiler
> >   mk: sort object files when building deps lists
> >   mk: always rebuild in the same order
> 
> These patches look good.
> If there is a mistake hidden in these changes, better to test it
> as soon as possible in various user environments.
> 
> Applied shortly, thanks

Removed shortly from master branch :)
I've just spotted some bugs in last 2 patches.

Only first two patches are applied because they are fixes.

Re: [dpdk-dev] [PATCH v3 7/8] mk: sort object files when building deps lists

[Thomas Monjalon]

23/06/2017 20:41, lboccass@brocade.com:
> From: Luca Boccassi <luca.boccassi@gmail.com>
> 
> In order to achieve reproducible builds, always use the same
> order when listing object files to build dependencies lists.
> 
> Signed-off-by: Luca Boccassi <luca.boccassi@gmail.com>
> ---
>  mk/rte.app.mk     | 4 ++--
>  mk/rte.hostapp.mk | 4 ++--
>  mk/rte.shared.mk  | 4 ++--
>  3 files changed, 6 insertions(+), 6 deletions(-)
> 
> --- a/mk/rte.app.mk
> +++ b/mk/rte.app.mk
> @@ -263,8 +263,8 @@ LDLIBS_NAMES += $(patsubst -Wl$(comma)-l%,lib%.a,$(filter -Wl$(comma)-l%,$(LDLIB
>  
>  # list of found libraries files (useful for deps). If not found, the
>  # library is silently ignored and dep won't be checked
> -LDLIBS_FILES := $(wildcard $(foreach dir,$(LDLIBS_PATH),\
> -	$(addprefix $(dir)/,$(LDLIBS_NAMES))))
> +LDLIBS_FILES := $(sort $(wildcard $(foreach dir,$(LDLIBS_PATH),\
> +	$(addprefix $(dir)/,$(LDLIBS_NAMES)))))

You cannot sort libraries.
Check - for instance - this comment above in this file:
	# Eliminate duplicates without sorting, only keep the last occurrence
	filter-libs = \

Why sorting them?
What is random in libraries list?

Re: [dpdk-dev] [PATCH v3 8/8] mk: always rebuild in the same order

[Thomas Monjalon]

23/06/2017 20:41, lboccass@brocade.com:
> From: Luca Boccassi <luca.boccassi@gmail.com>
> 
> In order to achieve reproducible builds, always check dependencies in
> the same order.
> 
> Signed-off-by: Luca Boccassi <luca.boccassi@gmail.com>
> ---
>  mk/internal/rte.compile-pre.mk | 8 ++++----
>  1 file changed, 4 insertions(+), 4 deletions(-)

It seems something is missing in this patch,
because it is always rebuilding all on the second build.

Re: [dpdk-dev] [PATCH v3 8/8] mk: always rebuild in the same order

[Luca Boccassi]

On Tue, 2017-06-27 at 01:22 +0200, Thomas Monjalon wrote:
> 23/06/2017 20:41, lboccass@brocade.com:
> > From: Luca Boccassi <luca.boccassi@gmail.com>
> > 
> > In order to achieve reproducible builds, always check dependencies
> > in
> > the same order.
> > 
> > Signed-off-by: Luca Boccassi <luca.boccassi@gmail.com>
> > ---
> >  mk/internal/rte.compile-pre.mk | 8 ++++----
> >  1 file changed, 4 insertions(+), 4 deletions(-)
> 
> It seems something is missing in this patch,
> because it is always rebuilding all on the second build.

Indeed, thanks for spotting my mistake!

Sorting before comparing was changing the compared strings, so that was
not working. Duh!
Moving the sort as the "outer" call solves the problem, now rebuilds
correctly skip already built objects.

-- 
Kind regards,
Luca Boccassi

Re: [dpdk-dev] [PATCH v3 7/8] mk: sort object files when building deps lists

[Luca Boccassi]

On Tue, 2017-06-27 at 01:20 +0200, Thomas Monjalon wrote:
> 23/06/2017 20:41, lboccass@brocade.com:
> > From: Luca Boccassi <luca.boccassi@gmail.com>
> > 
> > In order to achieve reproducible builds, always use the same
> > order when listing object files to build dependencies lists.
> > 
> > Signed-off-by: Luca Boccassi <luca.boccassi@gmail.com>
> > ---
> >  mk/rte.app.mk     | 4 ++--
> >  mk/rte.hostapp.mk | 4 ++--
> >  mk/rte.shared.mk  | 4 ++--
> >  3 files changed, 6 insertions(+), 6 deletions(-)
> > 
> > --- a/mk/rte.app.mk
> > +++ b/mk/rte.app.mk
> > @@ -263,8 +263,8 @@ LDLIBS_NAMES += $(patsubst -Wl$(comma)-
> > l%,lib%.a,$(filter -Wl$(comma)-l%,$(LDLIB
> >  
> >  # list of found libraries files (useful for deps). If not found,
> > the
> >  # library is silently ignored and dep won't be checked
> > -LDLIBS_FILES := $(wildcard $(foreach dir,$(LDLIBS_PATH),\
> > -	$(addprefix $(dir)/,$(LDLIBS_NAMES))))
> > +LDLIBS_FILES := $(sort $(wildcard $(foreach dir,$(LDLIBS_PATH),\
> > +	$(addprefix $(dir)/,$(LDLIBS_NAMES)))))
> 
> You cannot sort libraries.
> Check - for instance - this comment above in this file:
> 	# Eliminate duplicates without sorting, only keep the last
> occurrence
> 	filter-libs = \

Not sure I follow - what's the reason for avoiding to sort the list of
libs to link against?

> Why sorting them?
> What is random in libraries list?

The issue is that the output of wildcard is not fully deterministic. It
can depend on the filesystem, and even on the locale settings [1].
Before GNU Make 3.82 (2009) it used to automatically sort the output,
but that was removed (to make it faster... sigh). [2]



-- 
Kind regards,
Luca Boccassi

[1] https://reproducible-builds.org/docs/stable-inputs/
[2] http://comments.gmane.org/gmane.comp.gnu.make.bugs/4260

Re: [dpdk-dev] [PATCH v3 7/8] mk: sort object files when building deps lists

[Thomas Monjalon]

27/06/2017 12:43, Luca Boccassi:
> On Tue, 2017-06-27 at 01:20 +0200, Thomas Monjalon wrote:
> > 23/06/2017 20:41, lboccass@brocade.com:
> > > From: Luca Boccassi <luca.boccassi@gmail.com>
> > > 
> > > In order to achieve reproducible builds, always use the same
> > > order when listing object files to build dependencies lists.
> > > 
> > > Signed-off-by: Luca Boccassi <luca.boccassi@gmail.com>
> > > ---
> > >  mk/rte.app.mk     | 4 ++--
> > >  mk/rte.hostapp.mk | 4 ++--
> > >  mk/rte.shared.mk  | 4 ++--
> > >  3 files changed, 6 insertions(+), 6 deletions(-)
> > > 
> > > --- a/mk/rte.app.mk
> > > +++ b/mk/rte.app.mk
> > > @@ -263,8 +263,8 @@ LDLIBS_NAMES += $(patsubst -Wl$(comma)-
> > > l%,lib%.a,$(filter -Wl$(comma)-l%,$(LDLIB
> > >  
> > >  # list of found libraries files (useful for deps). If not found,
> > > the
> > >  # library is silently ignored and dep won't be checked
> > > -LDLIBS_FILES := $(wildcard $(foreach dir,$(LDLIBS_PATH),\
> > > -	$(addprefix $(dir)/,$(LDLIBS_NAMES))))
> > > +LDLIBS_FILES := $(sort $(wildcard $(foreach dir,$(LDLIBS_PATH),\
> > > +	$(addprefix $(dir)/,$(LDLIBS_NAMES)))))
> > 
> > You cannot sort libraries.
> > Check - for instance - this comment above in this file:
> > 	# Eliminate duplicates without sorting, only keep the last
> > occurrence
> > 	filter-libs = \
> 
> Not sure I follow - what's the reason for avoiding to sort the list of
> libs to link against?

Sorry, the ordering issue is with LDLIBS not LDLIBS_FILES.

> > Why sorting them?
> > What is random in libraries list?
> 
> The issue is that the output of wildcard is not fully deterministic. It
> can depend on the filesystem, and even on the locale settings [1].
> Before GNU Make 3.82 (2009) it used to automatically sort the output,
> but that was removed (to make it faster... sigh). [2]

It is not a true wildcard here. It is just filtering files which
do not exist.
I think you do not need this patch for deterministic build.

Re: [dpdk-dev] [PATCH v3 7/8] mk: sort object files when building deps lists

[Luca Boccassi]

On Tue, 2017-06-27 at 15:52 +0200, Thomas Monjalon wrote:
> 27/06/2017 12:43, Luca Boccassi:
> > On Tue, 2017-06-27 at 01:20 +0200, Thomas Monjalon wrote:
> > > 23/06/2017 20:41, lboccass@brocade.com:
> > > > From: Luca Boccassi <luca.boccassi@gmail.com>
> > > > 
> > > > In order to achieve reproducible builds, always use the same
> > > > order when listing object files to build dependencies lists.
> > > > 
> > > > Signed-off-by: Luca Boccassi <luca.boccassi@gmail.com>
> > > > ---
> > > >  mk/rte.app.mk     | 4 ++--
> > > >  mk/rte.hostapp.mk | 4 ++--
> > > >  mk/rte.shared.mk  | 4 ++--
> > > >  3 files changed, 6 insertions(+), 6 deletions(-)
> > > > 
> > > > --- a/mk/rte.app.mk
> > > > +++ b/mk/rte.app.mk
> > > > @@ -263,8 +263,8 @@ LDLIBS_NAMES += $(patsubst -Wl$(comma)-
> > > > l%,lib%.a,$(filter -Wl$(comma)-l%,$(LDLIB
> > > >  
> > > >  # list of found libraries files (useful for deps). If not
> > > > found,
> > > > the
> > > >  # library is silently ignored and dep won't be checked
> > > > -LDLIBS_FILES := $(wildcard $(foreach dir,$(LDLIBS_PATH),\
> > > > -	$(addprefix $(dir)/,$(LDLIBS_NAMES))))
> > > > +LDLIBS_FILES := $(sort $(wildcard $(foreach
> > > > dir,$(LDLIBS_PATH),\
> > > > +	$(addprefix $(dir)/,$(LDLIBS_NAMES)))))
> > > 
> > > You cannot sort libraries.
> > > Check - for instance - this comment above in this file:
> > > 	# Eliminate duplicates without sorting, only keep the last
> > > occurrence
> > > 	filter-libs = \
> > 
> > Not sure I follow - what's the reason for avoiding to sort the list
> > of
> > libs to link against?
> 
> Sorry, the ordering issue is with LDLIBS not LDLIBS_FILES.
> 
> > > Why sorting them?
> > > What is random in libraries list?
> > 
> > The issue is that the output of wildcard is not fully
> > deterministic. It
> > can depend on the filesystem, and even on the locale settings [1].
> > Before GNU Make 3.82 (2009) it used to automatically sort the
> > output,
> > but that was removed (to make it faster... sigh). [2]
> 
> It is not a true wildcard here. It is just filtering files which
> do not exist.
> I think you do not need this patch for deterministic build.

But then those lists are passed down in the .SECONDEXPANSION rule
right?

I am trying to find out an alternative solution. The problem to solve
is that the build system picks the public headers path (which is
embedded in the object files as notation and in the debug symbol) from
a seemingly random location between the make install path and the
source path (build/include/rte_foo.h vs lib/librte_foo/rte_foo.h) and
this makes the build not reproducible.

Nonetheless, while I work more on the last 4 patches, could you please
have a look and eventually take patch 3 and 4? They are needed to
respectively have a deterministic list of files in the libdpdk.so
linker script and a list of source files in one of the example
documentation files.

Thanks!

-- 
Kind regards,
Luca Boccassi

Re: [dpdk-dev] [PATCH v3 7/8] mk: sort object files when building deps lists

[Thomas Monjalon]

27/06/2017 16:51, Luca Boccassi:
> On Tue, 2017-06-27 at 15:52 +0200, Thomas Monjalon wrote:
> > 27/06/2017 12:43, Luca Boccassi:
> > > On Tue, 2017-06-27 at 01:20 +0200, Thomas Monjalon wrote:
> > > > 23/06/2017 20:41, lboccass@brocade.com:
> > > > > From: Luca Boccassi <luca.boccassi@gmail.com>
> > > > > 
> > > > > In order to achieve reproducible builds, always use the same
> > > > > order when listing object files to build dependencies lists.
> > > > > 
> > > > > Signed-off-by: Luca Boccassi <luca.boccassi@gmail.com>
> > > > > ---
> > > > >  mk/rte.app.mk     | 4 ++--
> > > > >  mk/rte.hostapp.mk | 4 ++--
> > > > >  mk/rte.shared.mk  | 4 ++--
> > > > >  3 files changed, 6 insertions(+), 6 deletions(-)
> > > > > 
> > > > > --- a/mk/rte.app.mk
> > > > > +++ b/mk/rte.app.mk
> > > > > @@ -263,8 +263,8 @@ LDLIBS_NAMES += $(patsubst -Wl$(comma)-
> > > > > l%,lib%.a,$(filter -Wl$(comma)-l%,$(LDLIB
> > > > >  
> > > > >  # list of found libraries files (useful for deps). If not
> > > > > found,
> > > > > the
> > > > >  # library is silently ignored and dep won't be checked
> > > > > -LDLIBS_FILES := $(wildcard $(foreach dir,$(LDLIBS_PATH),\
> > > > > -	$(addprefix $(dir)/,$(LDLIBS_NAMES))))
> > > > > +LDLIBS_FILES := $(sort $(wildcard $(foreach
> > > > > dir,$(LDLIBS_PATH),\
> > > > > +	$(addprefix $(dir)/,$(LDLIBS_NAMES)))))
> > > > 
> > > > You cannot sort libraries.
> > > > Check - for instance - this comment above in this file:
> > > > 	# Eliminate duplicates without sorting, only keep the last
> > > > occurrence
> > > > 	filter-libs = \
> > > 
> > > Not sure I follow - what's the reason for avoiding to sort the list
> > > of
> > > libs to link against?
> > 
> > Sorry, the ordering issue is with LDLIBS not LDLIBS_FILES.
> > 
> > > > Why sorting them?
> > > > What is random in libraries list?
> > > 
> > > The issue is that the output of wildcard is not fully
> > > deterministic. It
> > > can depend on the filesystem, and even on the locale settings [1].
> > > Before GNU Make 3.82 (2009) it used to automatically sort the
> > > output,
> > > but that was removed (to make it faster... sigh). [2]
> > 
> > It is not a true wildcard here. It is just filtering files which
> > do not exist.
> > I think you do not need this patch for deterministic build.
> 
> But then those lists are passed down in the .SECONDEXPANSION rule
> right?

I do not follow you.
Please explain what is the benefit of the patch in the next version.

> I am trying to find out an alternative solution. The problem to solve
> is that the build system picks the public headers path (which is
> embedded in the object files as notation and in the debug symbol) from
> a seemingly random location between the make install path and the
> source path (build/include/rte_foo.h vs lib/librte_foo/rte_foo.h) and
> this makes the build not reproducible.
> 
> Nonetheless, while I work more on the last 4 patches, could you please
> have a look and eventually take patch 3 and 4? They are needed to
> respectively have a deterministic list of files in the libdpdk.so
> linker script and a list of source files in one of the example
> documentation files.

I think it's better to consider and apply the whole series making
a reproducible build.
The rule is to avoid splitting series without good reason,
so tracking patches is easier.

[dpdk-dev] [PATCH v4 0/6] Reproducible build

[lboccass]

From: Luca Boccassi <luca.boccassi@gmail.com>

In the past couple of years a concerted effort among almost all Linux
distros has been striving toward achieving reproducible builds. [1]
This involves changes to the toolchain, new tools and CI systems. [2]

v1 fixed the documentation, examples and linker script generation.
v2 fixes all problems, which were caused by unstable order of headers
inclusion, source files listing and object file listing when passing
them to the compiler.
DPDK's build, at least with the default configuration, is fully
reproducible with this patch series as tested by the Reproducible
Builds developers experimental toolchain. [3]

v3 restores the first patch, which was eaten by git send-email.

v4 drops the patch that reorders rebuilds, and adds a patch to make
the inclusion of headers deterministic with regards to GCC embedding
the full file path when expading __FILE__ and when writing the
directory listing in the DWARF objects.
It also drops the first 2 patches which have already been merged.

[1] https://reproducible-builds.org/
[2] https://reproducible-builds.org/tools/
[3] https://wiki.debian.org/ReproducibleBuilds/ExperimentalToolchain#Us

Luca Boccassi (6):
  mk: sort list of shared objects in linker script
  mk: sort list of files in examples.dox
  mk: sort headers before wildcard inclusion
  mk: sort source files before passing them to the compiler
  mk: sort object files when building deps lists
  mk: set -ISCDIR before -IRTE_OUT/include in CFLAGS

 drivers/net/cxgbe/Makefile                                 |  2 +-
 drivers/net/e1000/Makefile                                 |  2 +-
 drivers/net/fm10k/Makefile                                 |  2 +-
 drivers/net/i40e/Makefile                                  |  2 +-
 drivers/net/ixgbe/Makefile                                 |  2 +-
 drivers/net/qede/Makefile                                  |  2 +-
 drivers/net/sfc/Makefile                                   |  2 +-
 drivers/net/thunderx/Makefile                              |  2 +-
 examples/ip_pipeline/Makefile                              |  2 +-
 examples/multi_process/client_server_mp/mp_server/Makefile |  2 +-
 examples/server_node_efd/server/Makefile                   |  2 +-
 lib/librte_acl/Makefile                                    | 10 ++++++++--
 lib/librte_bitratestats/Makefile                           |  9 ++++++++-
 lib/librte_cmdline/Makefile                                |  9 ++++++++-
 lib/librte_distributor/Makefile                            | 10 ++++++++--
 lib/librte_eal/common/Makefile                             |  2 +-
 lib/librte_efd/Makefile                                    | 10 ++++++++--
 lib/librte_hash/Makefile                                   | 10 ++++++++--
 lib/librte_ip_frag/Makefile                                | 10 ++++++++--
 lib/librte_jobstats/Makefile                               | 10 ++++++++--
 lib/librte_kni/Makefile                                    |  9 ++++++++-
 lib/librte_kvargs/Makefile                                 |  9 ++++++++-
 lib/librte_latencystats/Makefile                           |  9 ++++++++-
 lib/librte_lpm/Makefile                                    | 10 ++++++++--
 lib/librte_mbuf/Makefile                                   |  9 ++++++++-
 lib/librte_mempool/Makefile                                |  9 ++++++++-
 lib/librte_metrics/Makefile                                |  9 ++++++++-
 lib/librte_net/Makefile                                    |  9 ++++++++-
 lib/librte_pdump/Makefile                                  | 10 ++++++++--
 lib/librte_power/Makefile                                  |  9 ++++++++-
 lib/librte_reorder/Makefile                                | 10 ++++++++--
 lib/librte_ring/Makefile                                   |  9 ++++++++-
 lib/librte_timer/Makefile                                  |  9 ++++++++-
 lib/librte_vhost/Makefile                                  |  9 ++++++++-
 mk/rte.app.mk                                              |  4 ++--
 mk/rte.combinedlib.mk                                      |  2 +-
 mk/rte.hostapp.mk                                          |  4 ++--
 mk/rte.sdkdoc.mk                                           |  2 +-
 mk/rte.shared.mk                                           |  4 ++--
 39 files changed, 196 insertions(+), 51 deletions(-)

-- 
2.11.0

[dpdk-dev] [PATCH v4 1/6] mk: sort list of shared objects in linker script

[lboccass]

From: Luca Boccassi <luca.boccassi@gmail.com>

The output of wildcard might not be stable and depend on the
filesystem and other factors.
This means the content libdpdk.so linker script might change between
builds from the same sources.
Run the list through sort to ensure reproducibility.

Signed-off-by: Luca Boccassi <luca.boccassi@gmail.com>
---
 mk/rte.combinedlib.mk | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/mk/rte.combinedlib.mk b/mk/rte.combinedlib.mk
index 449358b33..2ab7ee8a1 100644
--- a/mk/rte.combinedlib.mk
+++ b/mk/rte.combinedlib.mk
@@ -42,7 +42,7 @@ endif
 RTE_LIBNAME := dpdk
 COMBINEDLIB := lib$(RTE_LIBNAME)$(EXT)
 
-LIBS := $(filter-out $(COMBINEDLIB), $(notdir $(wildcard $(RTE_OUTPUT)/lib/*$(EXT))))
+LIBS := $(filter-out $(COMBINEDLIB), $(sort $(notdir $(wildcard $(RTE_OUTPUT)/lib/*$(EXT)))))
 
 all: FORCE
 	$(Q)echo "GROUP ( $(LIBS) )" > $(RTE_OUTPUT)/lib/$(COMBINEDLIB)
-- 
2.11.0

[dpdk-dev] [PATCH v4 2/6] mk: sort list of files in examples.dox

[lboccass]

From: Luca Boccassi <luca.boccassi@gmail.com>

The result of find might not be stable depending on external
conditions.
Pipe it through LC_ALL=C sort to ensure reproducible results when
generating examples.dox.

Signed-off-by: Luca Boccassi <luca.boccassi@gmail.com>
---
 mk/rte.sdkdoc.mk | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/mk/rte.sdkdoc.mk b/mk/rte.sdkdoc.mk
index c0eaa3502..de31b78cf 100644
--- a/mk/rte.sdkdoc.mk
+++ b/mk/rte.sdkdoc.mk
@@ -93,7 +93,7 @@ $(API_EXAMPLES): api-html-clean
 	$(Q)mkdir -p $(@D)
 	@printf '/**\n' > $(API_EXAMPLES)
 	@printf '@page examples DPDK Example Programs\n\n' >> $(API_EXAMPLES)
-	@find examples -type f -name '*.c' -printf '@example %p\n' >> $(API_EXAMPLES)
+	@find examples -type f -name '*.c' -printf '@example %p\n' | LC_ALL=C sort >> $(API_EXAMPLES)
 	@printf '*/\n' >> $(API_EXAMPLES)
 
 guides-pdf-clean: guides-pdf-img-clean
-- 
2.11.0

[dpdk-dev] [PATCH v4 3/6] mk: sort headers before wildcard inclusion

[lboccass]

From: Luca Boccassi <luca.boccassi@gmail.com>

In order to achieve fully reproducible builds, always use the same
inclusion order for headers in the Makefiles.

Signed-off-by: Luca Boccassi <luca.boccassi@gmail.com>
---
 examples/ip_pipeline/Makefile                              | 2 +-
 examples/multi_process/client_server_mp/mp_server/Makefile | 2 +-
 examples/server_node_efd/server/Makefile                   | 2 +-
 lib/librte_eal/common/Makefile                             | 2 +-
 4 files changed, 4 insertions(+), 4 deletions(-)

diff --git a/examples/ip_pipeline/Makefile b/examples/ip_pipeline/Makefile
index dc7e0ddd7..12ce0a1d5 100644
--- a/examples/ip_pipeline/Makefile
+++ b/examples/ip_pipeline/Makefile
@@ -43,7 +43,7 @@ APP = ip_pipeline
 
 VPATH += $(SRCDIR)/pipeline
 
-INC += $(wildcard *.h) $(wildcard pipeline/*.h)
+INC += $(sort $(wildcard *.h)) $(sort $(wildcard pipeline/*.h))
 
 # all source are stored in SRCS-y
 SRCS-$(CONFIG_RTE_LIBRTE_PIPELINE) := main.c
diff --git a/examples/multi_process/client_server_mp/mp_server/Makefile b/examples/multi_process/client_server_mp/mp_server/Makefile
index 5552999b5..160c17b68 100644
--- a/examples/multi_process/client_server_mp/mp_server/Makefile
+++ b/examples/multi_process/client_server_mp/mp_server/Makefile
@@ -49,7 +49,7 @@ APP = mp_server
 # all source are stored in SRCS-y
 SRCS-y := main.c init.c args.c
 
-INC := $(wildcard *.h)
+INC := $(sort $(wildcard *.h))
 
 CFLAGS += $(WERROR_FLAGS) -O3
 CFLAGS += -I$(SRCDIR)/../shared
diff --git a/examples/server_node_efd/server/Makefile b/examples/server_node_efd/server/Makefile
index a2f2f361b..9f1fe2894 100644
--- a/examples/server_node_efd/server/Makefile
+++ b/examples/server_node_efd/server/Makefile
@@ -49,7 +49,7 @@ APP = server
 # all source are stored in SRCS-y
 SRCS-y := main.c init.c args.c
 
-INC := $(wildcard *.h)
+INC := $(sort $(wildcard *.h))
 
 CFLAGS += $(WERROR_FLAGS) -O3
 CFLAGS += -I$(SRCDIR)/../shared
diff --git a/lib/librte_eal/common/Makefile b/lib/librte_eal/common/Makefile
index a5bd1089a..4b712600a 100644
--- a/lib/librte_eal/common/Makefile
+++ b/lib/librte_eal/common/Makefile
@@ -48,7 +48,7 @@ GENERIC_INC += rte_vect.h rte_io.h
 
 # defined in mk/arch/$(RTE_ARCH)/rte.vars.mk
 ARCH_DIR ?= $(RTE_ARCH)
-ARCH_INC := $(notdir $(wildcard $(RTE_SDK)/lib/librte_eal/common/include/arch/$(ARCH_DIR)/*.h))
+ARCH_INC := $(sort $(notdir $(wildcard $(RTE_SDK)/lib/librte_eal/common/include/arch/$(ARCH_DIR)/*.h)))
 
 SYMLINK-$(CONFIG_RTE_LIBRTE_EAL)-include := $(addprefix include/,$(INC))
 SYMLINK-$(CONFIG_RTE_LIBRTE_EAL)-include += \
-- 
2.11.0

[dpdk-dev] [PATCH v4 4/6] mk: sort source files before passing them to the compiler

[lboccass]

From: Luca Boccassi <luca.boccassi@gmail.com>

In order to achieve reproducible builds, always use the same
order when listing files for compilation.

Signed-off-by: Luca Boccassi <luca.boccassi@gmail.com>
---
 drivers/net/cxgbe/Makefile    | 2 +-
 drivers/net/e1000/Makefile    | 2 +-
 drivers/net/fm10k/Makefile    | 2 +-
 drivers/net/i40e/Makefile     | 2 +-
 drivers/net/ixgbe/Makefile    | 2 +-
 drivers/net/qede/Makefile     | 2 +-
 drivers/net/sfc/Makefile      | 2 +-
 drivers/net/thunderx/Makefile | 2 +-
 8 files changed, 8 insertions(+), 8 deletions(-)

diff --git a/drivers/net/cxgbe/Makefile b/drivers/net/cxgbe/Makefile
index 7cef6279c..b4666b5af 100644
--- a/drivers/net/cxgbe/Makefile
+++ b/drivers/net/cxgbe/Makefile
@@ -67,7 +67,7 @@ endif
 # Add extra flags for base driver files (also known as shared code)
 # to disable warnings in them
 #
-BASE_DRIVER_OBJS=$(patsubst %.c,%.o,$(notdir $(wildcard $(SRCDIR)/base/*.c)))
+BASE_DRIVER_OBJS=$(sort $(patsubst %.c,%.o,$(notdir $(wildcard $(SRCDIR)/base/*.c))))
 $(foreach obj, $(BASE_DRIVER_OBJS), $(eval CFLAGS_$(obj)+=$(CFLAGS_BASE_DRIVER)))
 
 VPATH += $(SRCDIR)/base
diff --git a/drivers/net/e1000/Makefile b/drivers/net/e1000/Makefile
index ffdf36d37..59d96bca1 100644
--- a/drivers/net/e1000/Makefile
+++ b/drivers/net/e1000/Makefile
@@ -68,7 +68,7 @@ endif
 # Add extra flags for base driver files (also known as shared code)
 # to disable warnings in them
 #
-BASE_DRIVER_OBJS=$(patsubst %.c,%.o,$(notdir $(wildcard $(SRCDIR)/base/*.c)))
+BASE_DRIVER_OBJS=$(sort $(patsubst %.c,%.o,$(notdir $(wildcard $(SRCDIR)/base/*.c))))
 $(foreach obj, $(BASE_DRIVER_OBJS), $(eval CFLAGS_$(obj)+=$(CFLAGS_BASE_DRIVER)))
 
 VPATH += $(SRCDIR)/base
diff --git a/drivers/net/fm10k/Makefile b/drivers/net/fm10k/Makefile
index e0024f052..0bc124eb1 100644
--- a/drivers/net/fm10k/Makefile
+++ b/drivers/net/fm10k/Makefile
@@ -80,7 +80,7 @@ endif
 #
 # Add extra flags for base driver source files to disable warnings in them
 #
-BASE_DRIVER_OBJS=$(patsubst %.c,%.o,$(notdir $(wildcard $(SRCDIR)/base/*.c)))
+BASE_DRIVER_OBJS=$(sort $(patsubst %.c,%.o,$(notdir $(wildcard $(SRCDIR)/base/*.c))))
 $(foreach obj, $(BASE_DRIVER_OBJS), $(eval CFLAGS_$(obj)+=$(CFLAGS_BASE_DRIVER)))
 
 VPATH += $(SRCDIR)/base
diff --git a/drivers/net/i40e/Makefile b/drivers/net/i40e/Makefile
index 56f210d6d..06eedc592 100644
--- a/drivers/net/i40e/Makefile
+++ b/drivers/net/i40e/Makefile
@@ -78,7 +78,7 @@ endif
 
 CFLAGS_i40e_lan_hmc.o += -Wno-error
 endif
-OBJS_BASE_DRIVER=$(patsubst %.c,%.o,$(notdir $(wildcard $(SRCDIR)/base/*.c)))
+OBJS_BASE_DRIVER=$(sort $(patsubst %.c,%.o,$(notdir $(wildcard $(SRCDIR)/base/*.c))))
 $(foreach obj, $(OBJS_BASE_DRIVER), $(eval CFLAGS_$(obj)+=$(CFLAGS_BASE_DRIVER)))
 
 VPATH += $(SRCDIR)/base
diff --git a/drivers/net/ixgbe/Makefile b/drivers/net/ixgbe/Makefile
index 6a651b923..f5c370ce5 100644
--- a/drivers/net/ixgbe/Makefile
+++ b/drivers/net/ixgbe/Makefile
@@ -87,7 +87,7 @@ endif
 # Add extra flags for base driver files (also known as shared code)
 # to disable warnings in them
 #
-BASE_DRIVER_OBJS=$(patsubst %.c,%.o,$(notdir $(wildcard $(SRCDIR)/base/*.c)))
+BASE_DRIVER_OBJS=$(sort $(patsubst %.c,%.o,$(notdir $(wildcard $(SRCDIR)/base/*.c))))
 $(foreach obj, $(BASE_DRIVER_OBJS), $(eval CFLAGS_$(obj)+=$(CFLAGS_BASE_DRIVER)))
 
 VPATH += $(SRCDIR)/base
diff --git a/drivers/net/qede/Makefile b/drivers/net/qede/Makefile
index f03441d9a..83ff95474 100644
--- a/drivers/net/qede/Makefile
+++ b/drivers/net/qede/Makefile
@@ -78,7 +78,7 @@ endif
 # to disable warnings in them
 #
 #
-BASE_DRIVER_OBJS=$(patsubst %.c,%.o,$(notdir $(wildcard $(SRCDIR)/base/*.c)))
+BASE_DRIVER_OBJS=$(sort $(patsubst %.c,%.o,$(notdir $(wildcard $(SRCDIR)/base/*.c))))
 $(foreach obj, $(BASE_DRIVER_OBJS), $(eval CFLAGS_$(obj)+=$(CFLAGS_BASE_DRIVER)))
 
 VPATH += $(SRCDIR)/base
diff --git a/drivers/net/sfc/Makefile b/drivers/net/sfc/Makefile
index 57aa963ba..8cfd14d45 100644
--- a/drivers/net/sfc/Makefile
+++ b/drivers/net/sfc/Makefile
@@ -71,7 +71,7 @@ endif
 # List of base driver object files for which
 # special CFLAGS above should be applied
 #
-BASE_DRIVER_OBJS=$(patsubst %.c,%.o,$(notdir $(wildcard $(SRCDIR)/base/*.c)))
+BASE_DRIVER_OBJS=$(sort $(patsubst %.c,%.o,$(notdir $(wildcard $(SRCDIR)/base/*.c))))
 $(foreach obj, $(BASE_DRIVER_OBJS), \
   $(eval CFLAGS_$(obj)+=$(CFLAGS_BASE_DRIVER)))
 
diff --git a/drivers/net/thunderx/Makefile b/drivers/net/thunderx/Makefile
index 706250b8b..ac384a624 100644
--- a/drivers/net/thunderx/Makefile
+++ b/drivers/net/thunderx/Makefile
@@ -45,7 +45,7 @@ EXPORT_MAP := rte_pmd_thunderx_nicvf_version.map
 
 LIBABIVER := 1
 
-OBJS_BASE_DRIVER=$(patsubst %.c,%.o,$(notdir $(wildcard $(SRCDIR)/base/*.c)))
+OBJS_BASE_DRIVER=$(sort $(patsubst %.c,%.o,$(notdir $(wildcard $(SRCDIR)/base/*.c))))
 $(foreach obj, $(OBJS_BASE_DRIVER), $(eval CFLAGS_$(obj)+=$(CFLAGS_BASE_DRIVER)))
 
 VPATH += $(SRCDIR)/base
-- 
2.11.0

[dpdk-dev] [PATCH v4 5/6] mk: sort object files when building deps lists

[lboccass]

From: Luca Boccassi <luca.boccassi@gmail.com>

In order to achieve reproducible builds, always use the same
order when listing object files to build dependencies lists.

Signed-off-by: Luca Boccassi <luca.boccassi@gmail.com>
---
 mk/rte.app.mk     | 4 ++--
 mk/rte.hostapp.mk | 4 ++--
 mk/rte.shared.mk  | 4 ++--
 3 files changed, 6 insertions(+), 6 deletions(-)

diff --git a/mk/rte.app.mk b/mk/rte.app.mk
index bcaf1b382..54134dea4 100644
--- a/mk/rte.app.mk
+++ b/mk/rte.app.mk
@@ -263,8 +263,8 @@ LDLIBS_NAMES += $(patsubst -Wl$(comma)-l%,lib%.a,$(filter -Wl$(comma)-l%,$(LDLIB
 
 # list of found libraries files (useful for deps). If not found, the
 # library is silently ignored and dep won't be checked
-LDLIBS_FILES := $(wildcard $(foreach dir,$(LDLIBS_PATH),\
-	$(addprefix $(dir)/,$(LDLIBS_NAMES))))
+LDLIBS_FILES := $(sort $(wildcard $(foreach dir,$(LDLIBS_PATH),\
+	$(addprefix $(dir)/,$(LDLIBS_NAMES)))))
 
 #
 # Compile executable file if needed
diff --git a/mk/rte.hostapp.mk b/mk/rte.hostapp.mk
index 5cb4909cb..f58173c31 100644
--- a/mk/rte.hostapp.mk
+++ b/mk/rte.hostapp.mk
@@ -69,9 +69,9 @@ O_TO_EXE_DO = @set -e; \
 -include .$(HOSTAPP).cmd
 
 # list of .a files that are linked to this application
-LDLIBS_FILES := $(wildcard \
+LDLIBS_FILES := $(sort $(wildcard \
 	$(addprefix $(RTE_OUTPUT)/lib/, \
-	$(patsubst -l%,lib%.a,$(filter -l%,$(LDLIBS)))))
+	$(patsubst -l%,lib%.a,$(filter -l%,$(LDLIBS))))))
 
 #
 # Compile executable file if needed
diff --git a/mk/rte.shared.mk b/mk/rte.shared.mk
index 87ccf0ba4..4e680bc03 100644
--- a/mk/rte.shared.mk
+++ b/mk/rte.shared.mk
@@ -85,8 +85,8 @@ LDLIBS_NAMES += $(patsubst -Wl$(comma)-l%,lib%.a,$(filter -Wl$(comma)-l%,$(LDLIB
 
 # list of found libraries files (useful for deps). If not found, the
 # library is silently ignored and dep won't be checked
-LDLIBS_FILES := $(wildcard $(foreach dir,$(LDLIBS_PATH),\
-	$(addprefix $(dir)/,$(LDLIBS_NAMES))))
+LDLIBS_FILES := $(sort $(wildcard $(foreach dir,$(LDLIBS_PATH),\
+	$(addprefix $(dir)/,$(LDLIBS_NAMES)))))
 
 #
 # Archive objects in .so file if needed
-- 
2.11.0

[dpdk-dev] [PATCH v4 6/6] mk: set -ISCDIR before -IRTE_OUT/include in CFLAGS

[lboccass]

From: Luca Boccassi <luca.boccassi@gmail.com>

A race condition can happen during parallel builds, where a header
might be installed in RTE_OUT/include before CFLAGS is recursively
expanded. This causes GCC to sometimes pick the header path as
SRCDIR/... and sometimes as RTE_OUT/include/... making the build
unreproducible, as the full path is used for the expansion of
__FILE__ and in the DWARF directory listing.
Always pass -ISRCDIR first to CFLAGS so that it's deterministic.

Signed-off-by: Luca Boccassi <luca.boccassi@gmail.com>
---
 lib/librte_acl/Makefile          | 10 ++++++++--
 lib/librte_bitratestats/Makefile |  9 ++++++++-
 lib/librte_cmdline/Makefile      |  9 ++++++++-
 lib/librte_distributor/Makefile  | 10 ++++++++--
 lib/librte_efd/Makefile          | 10 ++++++++--
 lib/librte_hash/Makefile         | 10 ++++++++--
 lib/librte_ip_frag/Makefile      | 10 ++++++++--
 lib/librte_jobstats/Makefile     | 10 ++++++++--
 lib/librte_kni/Makefile          |  9 ++++++++-
 lib/librte_kvargs/Makefile       |  9 ++++++++-
 lib/librte_latencystats/Makefile |  9 ++++++++-
 lib/librte_lpm/Makefile          | 10 ++++++++--
 lib/librte_mbuf/Makefile         |  9 ++++++++-
 lib/librte_mempool/Makefile      |  9 ++++++++-
 lib/librte_metrics/Makefile      |  9 ++++++++-
 lib/librte_net/Makefile          |  9 ++++++++-
 lib/librte_pdump/Makefile        | 10 ++++++++--
 lib/librte_power/Makefile        |  9 ++++++++-
 lib/librte_reorder/Makefile      | 10 ++++++++--
 lib/librte_ring/Makefile         |  9 ++++++++-
 lib/librte_timer/Makefile        |  9 ++++++++-
 lib/librte_vhost/Makefile        |  9 ++++++++-
 22 files changed, 176 insertions(+), 31 deletions(-)

diff --git a/lib/librte_acl/Makefile b/lib/librte_acl/Makefile
index e2dacd606..470ef6d21 100644
--- a/lib/librte_acl/Makefile
+++ b/lib/librte_acl/Makefile
@@ -34,8 +34,14 @@ include $(RTE_SDK)/mk/rte.vars.mk
 # library name
 LIB = librte_acl.a
 
-CFLAGS += -O3
-CFLAGS += $(WERROR_FLAGS) -I$(SRCDIR)
+# Include the source dir first, so that headers paths are always picked
+# from there. By including it last race conditions might happen during paralle
+# builds, and headers might be already installed in RTE_OUT/include when the
+# variable is recursively expanded, thus causing GCC to sometimes use the
+# SRCDIR path and sometimes the RTE_OUT/include, making the builds not
+# reproducible.
+CFLAGS := -I$(SRCDIR) $(CFLAGS)
+CFLAGS += $(WERROR_FLAGS) -O3
 
 EXPORT_MAP := rte_acl_version.map
 
diff --git a/lib/librte_bitratestats/Makefile b/lib/librte_bitratestats/Makefile
index 58a20ea09..26a3f4908 100644
--- a/lib/librte_bitratestats/Makefile
+++ b/lib/librte_bitratestats/Makefile
@@ -34,7 +34,14 @@ include $(RTE_SDK)/mk/rte.vars.mk
 # library name
 LIB = librte_bitratestats.a
 
-CFLAGS += $(WERROR_FLAGS) -I$(SRCDIR) -O3
+# Include the source dir first, so that headers paths are always picked
+# from there. By including it last race conditions might happen during paralle
+# builds, and headers might be already installed in RTE_OUT/include when the
+# variable is recursively expanded, thus causing GCC to sometimes use the
+# SRCDIR path and sometimes the RTE_OUT/include, making the builds not
+# reproducible.
+CFLAGS := -I$(SRCDIR) $(CFLAGS)
+CFLAGS += $(WERROR_FLAGS) -O3
 
 EXPORT_MAP := rte_bitratestats_version.map
 
diff --git a/lib/librte_cmdline/Makefile b/lib/librte_cmdline/Makefile
index 644f68e47..9dd75f2d2 100644
--- a/lib/librte_cmdline/Makefile
+++ b/lib/librte_cmdline/Makefile
@@ -34,7 +34,14 @@ include $(RTE_SDK)/mk/rte.vars.mk
 # library name
 LIB = librte_cmdline.a
 
-CFLAGS += $(WERROR_FLAGS) -I$(SRCDIR) -O3
+# Include the source dir first, so that headers paths are always picked
+# from there. By including it last race conditions might happen during paralle
+# builds, and headers might be already installed in RTE_OUT/include when the
+# variable is recursively expanded, thus causing GCC to sometimes use the
+# SRCDIR path and sometimes the RTE_OUT/include, making the builds not
+# reproducible.
+CFLAGS := -I$(SRCDIR) $(CFLAGS)
+CFLAGS += $(WERROR_FLAGS) -O3
 
 EXPORT_MAP := rte_cmdline_version.map
 
diff --git a/lib/librte_distributor/Makefile b/lib/librte_distributor/Makefile
index 3ffb911ce..87418f254 100644
--- a/lib/librte_distributor/Makefile
+++ b/lib/librte_distributor/Makefile
@@ -34,8 +34,14 @@ include $(RTE_SDK)/mk/rte.vars.mk
 # library name
 LIB = librte_distributor.a
 
-CFLAGS += -O3
-CFLAGS += $(WERROR_FLAGS) -I$(SRCDIR)
+# Include the source dir first, so that headers paths are always picked
+# from there. By including it last race conditions might happen during paralle
+# builds, and headers might be already installed in RTE_OUT/include when the
+# variable is recursively expanded, thus causing GCC to sometimes use the
+# SRCDIR path and sometimes the RTE_OUT/include, making the builds not
+# reproducible.
+CFLAGS := -I$(SRCDIR) $(CFLAGS)
+CFLAGS += $(WERROR_FLAGS) -O3
 
 EXPORT_MAP := rte_distributor_version.map
 
diff --git a/lib/librte_efd/Makefile b/lib/librte_efd/Makefile
index b9277bc5d..b169e3240 100644
--- a/lib/librte_efd/Makefile
+++ b/lib/librte_efd/Makefile
@@ -34,8 +34,14 @@ include $(RTE_SDK)/mk/rte.vars.mk
 # library name
 LIB = librte_efd.a
 
-CFLAGS += -O3
-CFLAGS += $(WERROR_FLAGS) -I$(SRCDIR)
+# Include the source dir first, so that headers paths are always picked
+# from there. By including it last race conditions might happen during paralle
+# builds, and headers might be already installed in RTE_OUT/include when the
+# variable is recursively expanded, thus causing GCC to sometimes use the
+# SRCDIR path and sometimes the RTE_OUT/include, making the builds not
+# reproducible.
+CFLAGS := -I$(SRCDIR) $(CFLAGS)
+CFLAGS += $(WERROR_FLAGS) -O3
 
 EXPORT_MAP := rte_efd_version.map
 
diff --git a/lib/librte_hash/Makefile b/lib/librte_hash/Makefile
index d856aa26d..e408dcc46 100644
--- a/lib/librte_hash/Makefile
+++ b/lib/librte_hash/Makefile
@@ -34,8 +34,14 @@ include $(RTE_SDK)/mk/rte.vars.mk
 # library name
 LIB = librte_hash.a
 
-CFLAGS += -O3
-CFLAGS += $(WERROR_FLAGS) -I$(SRCDIR)
+# Include the source dir first, so that headers paths are always picked
+# from there. By including it last race conditions might happen during paralle
+# builds, and headers might be already installed in RTE_OUT/include when the
+# variable is recursively expanded, thus causing GCC to sometimes use the
+# SRCDIR path and sometimes the RTE_OUT/include, making the builds not
+# reproducible.
+CFLAGS := -I$(SRCDIR) $(CFLAGS)
+CFLAGS += $(WERROR_FLAGS) -O3
 
 EXPORT_MAP := rte_hash_version.map
 
diff --git a/lib/librte_ip_frag/Makefile b/lib/librte_ip_frag/Makefile
index 4e693bf8f..de45ec2d3 100644
--- a/lib/librte_ip_frag/Makefile
+++ b/lib/librte_ip_frag/Makefile
@@ -34,8 +34,14 @@ include $(RTE_SDK)/mk/rte.vars.mk
 # library name
 LIB = librte_ip_frag.a
 
-CFLAGS += -O3
-CFLAGS += $(WERROR_FLAGS) -I$(SRCDIR)
+# Include the source dir first, so that headers paths are always picked
+# from there. By including it last race conditions might happen during paralle
+# builds, and headers might be already installed in RTE_OUT/include when the
+# variable is recursively expanded, thus causing GCC to sometimes use the
+# SRCDIR path and sometimes the RTE_OUT/include, making the builds not
+# reproducible.
+CFLAGS := -I$(SRCDIR) $(CFLAGS)
+CFLAGS += $(WERROR_FLAGS) -O3
 
 EXPORT_MAP := rte_ipfrag_version.map
 
diff --git a/lib/librte_jobstats/Makefile b/lib/librte_jobstats/Makefile
index 561a0678c..57329b18e 100644
--- a/lib/librte_jobstats/Makefile
+++ b/lib/librte_jobstats/Makefile
@@ -34,8 +34,14 @@ include $(RTE_SDK)/mk/rte.vars.mk
 # library name
 LIB = librte_jobstats.a
 
-CFLAGS += -O3
-CFLAGS += $(WERROR_FLAGS) -I$(SRCDIR)
+# Include the source dir first, so that headers paths are always picked
+# from there. By including it last race conditions might happen during paralle
+# builds, and headers might be already installed in RTE_OUT/include when the
+# variable is recursively expanded, thus causing GCC to sometimes use the
+# SRCDIR path and sometimes the RTE_OUT/include, making the builds not
+# reproducible.
+CFLAGS := -I$(SRCDIR) $(CFLAGS)
+CFLAGS += $(WERROR_FLAGS) -O3
 
 EXPORT_MAP := rte_jobstats_version.map
 
diff --git a/lib/librte_kni/Makefile b/lib/librte_kni/Makefile
index 70f1ca8f6..130d6fd74 100644
--- a/lib/librte_kni/Makefile
+++ b/lib/librte_kni/Makefile
@@ -34,7 +34,14 @@ include $(RTE_SDK)/mk/rte.vars.mk
 # library name
 LIB = librte_kni.a
 
-CFLAGS += $(WERROR_FLAGS) -I$(SRCDIR) -O3 -fno-strict-aliasing
+# Include the source dir first, so that headers paths are always picked
+# from there. By including it last race conditions might happen during paralle
+# builds, and headers might be already installed in RTE_OUT/include when the
+# variable is recursively expanded, thus causing GCC to sometimes use the
+# SRCDIR path and sometimes the RTE_OUT/include, making the builds not
+# reproducible.
+CFLAGS := -I$(SRCDIR) $(CFLAGS)
+CFLAGS += $(WERROR_FLAGS) -O3 -fno-strict-aliasing
 
 EXPORT_MAP := rte_kni_version.map
 
diff --git a/lib/librte_kvargs/Makefile b/lib/librte_kvargs/Makefile
index 564dd3102..7c332c110 100644
--- a/lib/librte_kvargs/Makefile
+++ b/lib/librte_kvargs/Makefile
@@ -36,7 +36,14 @@ include $(RTE_SDK)/mk/rte.vars.mk
 # library name
 LIB = librte_kvargs.a
 
-CFLAGS += $(WERROR_FLAGS) -I$(SRCDIR) -O3
+# Include the source dir first, so that headers paths are always picked
+# from there. By including it last race conditions might happen during paralle
+# builds, and headers might be already installed in RTE_OUT/include when the
+# variable is recursively expanded, thus causing GCC to sometimes use the
+# SRCDIR path and sometimes the RTE_OUT/include, making the builds not
+# reproducible.
+CFLAGS := -I$(SRCDIR) $(CFLAGS)
+CFLAGS += $(WERROR_FLAGS) -O3
 
 EXPORT_MAP := rte_kvargs_version.map
 
diff --git a/lib/librte_latencystats/Makefile b/lib/librte_latencystats/Makefile
index eaacbb731..46a8ecd34 100644
--- a/lib/librte_latencystats/Makefile
+++ b/lib/librte_latencystats/Makefile
@@ -33,7 +33,14 @@ include $(RTE_SDK)/mk/rte.vars.mk
 # library name
 LIB = librte_latencystats.a
 
-CFLAGS += $(WERROR_FLAGS) -I$(SRCDIR) -O3
+# Include the source dir first, so that headers paths are always picked
+# from there. By including it last race conditions might happen during paralle
+# builds, and headers might be already installed in RTE_OUT/include when the
+# variable is recursively expanded, thus causing GCC to sometimes use the
+# SRCDIR path and sometimes the RTE_OUT/include, making the builds not
+# reproducible.
+CFLAGS := -I$(SRCDIR) $(CFLAGS)
+CFLAGS += $(WERROR_FLAGS) -O3
 LDLIBS += -lm
 LDLIBS += -lpthread
 
diff --git a/lib/librte_lpm/Makefile b/lib/librte_lpm/Makefile
index 32be46b3b..6a97fdc97 100644
--- a/lib/librte_lpm/Makefile
+++ b/lib/librte_lpm/Makefile
@@ -34,8 +34,14 @@ include $(RTE_SDK)/mk/rte.vars.mk
 # library name
 LIB = librte_lpm.a
 
-CFLAGS += -O3
-CFLAGS += $(WERROR_FLAGS) -I$(SRCDIR)
+# Include the source dir first, so that headers paths are always picked
+# from there. By including it last race conditions might happen during paralle
+# builds, and headers might be already installed in RTE_OUT/include when the
+# variable is recursively expanded, thus causing GCC to sometimes use the
+# SRCDIR path and sometimes the RTE_OUT/include, making the builds not
+# reproducible.
+CFLAGS := -I$(SRCDIR) $(CFLAGS)
+CFLAGS += $(WERROR_FLAGS) -O3
 
 EXPORT_MAP := rte_lpm_version.map
 
diff --git a/lib/librte_mbuf/Makefile b/lib/librte_mbuf/Makefile
index 548273054..4d51191ee 100644
--- a/lib/librte_mbuf/Makefile
+++ b/lib/librte_mbuf/Makefile
@@ -34,7 +34,14 @@ include $(RTE_SDK)/mk/rte.vars.mk
 # library name
 LIB = librte_mbuf.a
 
-CFLAGS += $(WERROR_FLAGS) -I$(SRCDIR) -O3
+# Include the source dir first, so that headers paths are always picked
+# from there. By including it last race conditions might happen during paralle
+# builds, and headers might be already installed in RTE_OUT/include when the
+# variable is recursively expanded, thus causing GCC to sometimes use the
+# SRCDIR path and sometimes the RTE_OUT/include, making the builds not
+# reproducible.
+CFLAGS := -I$(SRCDIR) $(CFLAGS)
+CFLAGS += $(WERROR_FLAGS) -O3
 
 EXPORT_MAP := rte_mbuf_version.map
 
diff --git a/lib/librte_mempool/Makefile b/lib/librte_mempool/Makefile
index 7b5bdfee7..33678192f 100644
--- a/lib/librte_mempool/Makefile
+++ b/lib/librte_mempool/Makefile
@@ -34,7 +34,14 @@ include $(RTE_SDK)/mk/rte.vars.mk
 # library name
 LIB = librte_mempool.a
 
-CFLAGS += $(WERROR_FLAGS) -I$(SRCDIR) -O3
+# Include the source dir first, so that headers paths are always picked
+# from there. By including it last race conditions might happen during paralle
+# builds, and headers might be already installed in RTE_OUT/include when the
+# variable is recursively expanded, thus causing GCC to sometimes use the
+# SRCDIR path and sometimes the RTE_OUT/include, making the builds not
+# reproducible.
+CFLAGS := -I$(SRCDIR) $(CFLAGS)
+CFLAGS += $(WERROR_FLAGS) -O3
 
 EXPORT_MAP := rte_mempool_version.map
 
diff --git a/lib/librte_metrics/Makefile b/lib/librte_metrics/Makefile
index d4990e839..195bd4d56 100644
--- a/lib/librte_metrics/Makefile
+++ b/lib/librte_metrics/Makefile
@@ -34,7 +34,14 @@ include $(RTE_SDK)/mk/rte.vars.mk
 # library name
 LIB = librte_metrics.a
 
-CFLAGS += $(WERROR_FLAGS) -I$(SRCDIR) -O3
+# Include the source dir first, so that headers paths are always picked
+# from there. By including it last race conditions might happen during paralle
+# builds, and headers might be already installed in RTE_OUT/include when the
+# variable is recursively expanded, thus causing GCC to sometimes use the
+# SRCDIR path and sometimes the RTE_OUT/include, making the builds not
+# reproducible.
+CFLAGS := -I$(SRCDIR) $(CFLAGS)
+CFLAGS += $(WERROR_FLAGS) -O3
 
 EXPORT_MAP := rte_metrics_version.map
 
diff --git a/lib/librte_net/Makefile b/lib/librte_net/Makefile
index 56727c4df..a08a7dd8b 100644
--- a/lib/librte_net/Makefile
+++ b/lib/librte_net/Makefile
@@ -33,7 +33,14 @@ include $(RTE_SDK)/mk/rte.vars.mk
 
 LIB = librte_net.a
 
-CFLAGS += $(WERROR_FLAGS) -I$(SRCDIR) -O3
+# Include the source dir first, so that headers paths are always picked
+# from there. By including it last race conditions might happen during paralle
+# builds, and headers might be already installed in RTE_OUT/include when the
+# variable is recursively expanded, thus causing GCC to sometimes use the
+# SRCDIR path and sometimes the RTE_OUT/include, making the builds not
+# reproducible.
+CFLAGS := -I$(SRCDIR) $(CFLAGS)
+CFLAGS += $(WERROR_FLAGS) -O3
 
 EXPORT_MAP := rte_net_version.map
 LIBABIVER := 1
diff --git a/lib/librte_pdump/Makefile b/lib/librte_pdump/Makefile
index 1c03bcbb7..8923d7499 100644
--- a/lib/librte_pdump/Makefile
+++ b/lib/librte_pdump/Makefile
@@ -34,8 +34,14 @@ include $(RTE_SDK)/mk/rte.vars.mk
 # library name
 LIB = librte_pdump.a
 
-CFLAGS += $(WERROR_FLAGS) -I$(SRCDIR) -O3
-CFLAGS += -D_GNU_SOURCE
+# Include the source dir first, so that headers paths are always picked
+# from there. By including it last race conditions might happen during paralle
+# builds, and headers might be already installed in RTE_OUT/include when the
+# variable is recursively expanded, thus causing GCC to sometimes use the
+# SRCDIR path and sometimes the RTE_OUT/include, making the builds not
+# reproducible.
+CFLAGS := -I$(SRCDIR) $(CFLAGS)
+CFLAGS += $(WERROR_FLAGS) -O3 -D_GNU_SOURCE
 LDLIBS += -lpthread
 
 EXPORT_MAP := rte_pdump_version.map
diff --git a/lib/librte_power/Makefile b/lib/librte_power/Makefile
index 06cd10e86..b76ad689e 100644
--- a/lib/librte_power/Makefile
+++ b/lib/librte_power/Makefile
@@ -34,7 +34,14 @@ include $(RTE_SDK)/mk/rte.vars.mk
 # library name
 LIB = librte_power.a
 
-CFLAGS += $(WERROR_FLAGS) -I$(SRCDIR) -O3 -fno-strict-aliasing
+# Include the source dir first, so that headers paths are always picked
+# from there. By including it last race conditions might happen during paralle
+# builds, and headers might be already installed in RTE_OUT/include when the
+# variable is recursively expanded, thus causing GCC to sometimes use the
+# SRCDIR path and sometimes the RTE_OUT/include, making the builds not
+# reproducible.
+CFLAGS := -I$(SRCDIR) $(CFLAGS)
+CFLAGS += $(WERROR_FLAGS) -O3 -fno-strict-aliasing
 
 EXPORT_MAP := rte_power_version.map
 
diff --git a/lib/librte_reorder/Makefile b/lib/librte_reorder/Makefile
index 4e44e72f0..51b5d490d 100644
--- a/lib/librte_reorder/Makefile
+++ b/lib/librte_reorder/Makefile
@@ -34,8 +34,14 @@ include $(RTE_SDK)/mk/rte.vars.mk
 # library name
 LIB = librte_reorder.a
 
-CFLAGS += -O3
-CFLAGS += $(WERROR_FLAGS) -I$(SRCDIR)
+# Include the source dir first, so that headers paths are always picked
+# from there. By including it last race conditions might happen during paralle
+# builds, and headers might be already installed in RTE_OUT/include when the
+# variable is recursively expanded, thus causing GCC to sometimes use the
+# SRCDIR path and sometimes the RTE_OUT/include, making the builds not
+# reproducible.
+CFLAGS := -I$(SRCDIR) $(CFLAGS)
+CFLAGS += $(WERROR_FLAGS) -O3
 
 EXPORT_MAP := rte_reorder_version.map
 
diff --git a/lib/librte_ring/Makefile b/lib/librte_ring/Makefile
index 3e2f4b873..e5248a2c1 100644
--- a/lib/librte_ring/Makefile
+++ b/lib/librte_ring/Makefile
@@ -34,7 +34,14 @@ include $(RTE_SDK)/mk/rte.vars.mk
 # library name
 LIB = librte_ring.a
 
-CFLAGS += $(WERROR_FLAGS) -I$(SRCDIR) -O3
+# Include the source dir first, so that headers paths are always picked
+# from there. By including it last race conditions might happen during paralle
+# builds, and headers might be already installed in RTE_OUT/include when the
+# variable is recursively expanded, thus causing GCC to sometimes use the
+# SRCDIR path and sometimes the RTE_OUT/include, making the builds not
+# reproducible.
+CFLAGS := -I$(SRCDIR) $(CFLAGS)
+CFLAGS += $(WERROR_FLAGS) -O3
 
 EXPORT_MAP := rte_ring_version.map
 
diff --git a/lib/librte_timer/Makefile b/lib/librte_timer/Makefile
index 03a15390e..0c57cc6d3 100644
--- a/lib/librte_timer/Makefile
+++ b/lib/librte_timer/Makefile
@@ -34,7 +34,14 @@ include $(RTE_SDK)/mk/rte.vars.mk
 # library name
 LIB = librte_timer.a
 
-CFLAGS += $(WERROR_FLAGS) -I$(SRCDIR) -O3
+# Include the source dir first, so that headers paths are always picked
+# from there. By including it last race conditions might happen during parallel
+# builds, and headers might be already installed in RTE_OUT/include when the
+# variable is recursively expanded, thus causing GCC to sometimes use the
+# SRCDIR path and sometimes the RTE_OUT/include, making the builds not
+# reproducible.
+CFLAGS := -I$(SRCDIR) $(CFLAGS)
+CFLAGS += $(WERROR_FLAGS) -O3
 
 EXPORT_MAP := rte_timer_version.map
 
diff --git a/lib/librte_vhost/Makefile b/lib/librte_vhost/Makefile
index 4a116fe31..ef0d86990 100644
--- a/lib/librte_vhost/Makefile
+++ b/lib/librte_vhost/Makefile
@@ -38,7 +38,14 @@ EXPORT_MAP := rte_vhost_version.map
 
 LIBABIVER := 4
 
-CFLAGS += $(WERROR_FLAGS) -I$(SRCDIR) -O3 -D_FILE_OFFSET_BITS=64
+# Include the source dir first, so that headers paths are always picked
+# from there. By including it last race conditions might happen during parallel
+# builds, and headers might be already installed in RTE_OUT/include when the
+# variable is recursively expanded, thus causing GCC to sometimes use the
+# SRCDIR path and sometimes the RTE_OUT/include, making the builds not
+# reproducible.
+CFLAGS := -I$(SRCDIR) $(CFLAGS)
+CFLAGS += $(WERROR_FLAGS) -O3 -D_FILE_OFFSET_BITS=64
 CFLAGS += -I vhost_user
 LDLIBS += -lpthread
 
-- 
2.11.0

Re: [dpdk-dev] [PATCH v3 7/8] mk: sort object files when building deps lists

[Luca Boccassi]

On Tue, 2017-06-27 at 18:14 +0200, Thomas Monjalon wrote:
> 27/06/2017 16:51, Luca Boccassi:
> > On Tue, 2017-06-27 at 15:52 +0200, Thomas Monjalon wrote:
> > > 27/06/2017 12:43, Luca Boccassi:
> > > > On Tue, 2017-06-27 at 01:20 +0200, Thomas Monjalon wrote:
> > > > > 23/06/2017 20:41, lboccass@brocade.com:
> > > > > > From: Luca Boccassi <luca.boccassi@gmail.com>
> > > > > > 
> > > > > > In order to achieve reproducible builds, always use the
> > > > > > same
> > > > > > order when listing object files to build dependencies
> > > > > > lists.
> > > > > > 
> > > > > > Signed-off-by: Luca Boccassi <luca.boccassi@gmail.com>
> > > > > > ---
> > > > > >  mk/rte.app.mk     | 4 ++--
> > > > > >  mk/rte.hostapp.mk | 4 ++--
> > > > > >  mk/rte.shared.mk  | 4 ++--
> > > > > >  3 files changed, 6 insertions(+), 6 deletions(-)
> > > > > > 
> > > > > > --- a/mk/rte.app.mk
> > > > > > +++ b/mk/rte.app.mk
> > > > > > @@ -263,8 +263,8 @@ LDLIBS_NAMES += $(patsubst -Wl$(comma)-
> > > > > > l%,lib%.a,$(filter -Wl$(comma)-l%,$(LDLIB
> > > > > >  
> > > > > >  # list of found libraries files (useful for deps). If not
> > > > > > found,
> > > > > > the
> > > > > >  # library is silently ignored and dep won't be checked
> > > > > > -LDLIBS_FILES := $(wildcard $(foreach dir,$(LDLIBS_PATH),\
> > > > > > -	$(addprefix $(dir)/,$(LDLIBS_NAMES))))
> > > > > > +LDLIBS_FILES := $(sort $(wildcard $(foreach
> > > > > > dir,$(LDLIBS_PATH),\
> > > > > > +	$(addprefix $(dir)/,$(LDLIBS_NAMES)))))
> > > > > 
> > > > > You cannot sort libraries.
> > > > > Check - for instance - this comment above in this file:
> > > > > 	# Eliminate duplicates without sorting, only keep the
> > > > > last
> > > > > occurrence
> > > > > 	filter-libs = \
> > > > 
> > > > Not sure I follow - what's the reason for avoiding to sort the
> > > > list
> > > > of
> > > > libs to link against?
> > > 
> > > Sorry, the ordering issue is with LDLIBS not LDLIBS_FILES.
> > > 
> > > > > Why sorting them?
> > > > > What is random in libraries list?
> > > > 
> > > > The issue is that the output of wildcard is not fully
> > > > deterministic. It
> > > > can depend on the filesystem, and even on the locale settings
> > > > [1].
> > > > Before GNU Make 3.82 (2009) it used to automatically sort the
> > > > output,
> > > > but that was removed (to make it faster... sigh). [2]
> > > 
> > > It is not a true wildcard here. It is just filtering files which
> > > do not exist.
> > > I think you do not need this patch for deterministic build.
> > 
> > But then those lists are passed down in the .SECONDEXPANSION rule
> > right?
> 
> I do not follow you.
> Please explain what is the benefit of the patch in the next version.

I thought that these lists are used to determine which files to
recompile - and incidentally, in which order as they are passed in this
snippet in rte.compile-pre.mk:

.SECONDEXPANSION:
%.o: %.c $$(wildcard $$(dep_$$@)) $$(DEP_$$(@)) FORCE
	@[ -d $(dir $@) ] || mkdir -p $(dir $@)
	$(if $(D),\
		@echo -n "$< -> $@ " ; \
		echo -n "file_missing=$(call boolean,$(file_missing)) " ; \
		echo -n "cmdline_changed=$(call boolean,$(call cmdline_changed,$(C_TO_O))) " ; \
		echo -n "depfile_missing=$(call boolean,$(depfile_missing)) " ; \
		echo "depfile_newer=$(call boolean,$(depfile_newer))")
	$(if $(or \
		$(file_missing),\
		$(call cmdline_changed,$(C_TO_O)),\
		$(depfile_missing),\
		$(depfile_newer)),\
		$(C_TO_O_DO))

Did I get that wrong? (It is a bit convoluted :-P )


But nevertheless, I've finally found the root cause for the "wandering
header" - when building the libraries, CFLAGS lists
-I$(RTE_OUTDIR)/include first and then -I$(SRCDIR) last. There is a
race, and sometimes when GCC is called the public header has already
been installed in RTE_OUTDIR and sometimes it has not. This causes the
instability, and causes the expansion of __FILE__ and the directory
listing in the DWARF objects to randomly list the full path to either
RTE_OUTDIR/include or SRCDIR for each library.

By always passing -I$(SRCDIR) first this instability is solved.

I've now dropped this patch and added this new fix in v4.
This might mean that partial rebuilds are not stable, if I understand
correctly how rte.compile-pre.mk works (eg: change 2 files, rebuild,
change them again, rebuild -> output final binary _might_ be
different), but a full rebuild from scratch now seems to be always
reproducible.

> > I am trying to find out an alternative solution. The problem to
> > solve
> > is that the build system picks the public headers path (which is
> > embedded in the object files as notation and in the debug symbol)
> > from
> > a seemingly random location between the make install path and the
> > source path (build/include/rte_foo.h vs lib/librte_foo/rte_foo.h)
> > and
> > this makes the build not reproducible.
> > 
> > Nonetheless, while I work more on the last 4 patches, could you
> > please
> > have a look and eventually take patch 3 and 4? They are needed to
> > respectively have a deterministic list of files in the libdpdk.so
> > linker script and a list of source files in one of the example
> > documentation files.
> 
> I think it's better to consider and apply the whole series making
> a reproducible build.
> The rule is to avoid splitting series without good reason,
> so tracking patches is easier.

Sure, fair enough.

-- 
Kind regards,
Luca Boccassi

Re: [dpdk-dev] [PATCH v3 7/8] mk: sort object files when building deps lists

[Thomas Monjalon]

28/06/2017 16:07, Luca Boccassi:
> On Tue, 2017-06-27 at 18:14 +0200, Thomas Monjalon wrote:
> > 27/06/2017 16:51, Luca Boccassi:
> > > On Tue, 2017-06-27 at 15:52 +0200, Thomas Monjalon wrote:
> > > > 27/06/2017 12:43, Luca Boccassi:
> > > > > On Tue, 2017-06-27 at 01:20 +0200, Thomas Monjalon wrote:
> > > > > > 23/06/2017 20:41, lboccass@brocade.com:
> > > > > > > From: Luca Boccassi <luca.boccassi@gmail.com>
> > > > > > > 
> > > > > > > In order to achieve reproducible builds, always use the
> > > > > > > same
> > > > > > > order when listing object files to build dependencies
> > > > > > > lists.
> > > > > > > 
> > > > > > > Signed-off-by: Luca Boccassi <luca.boccassi@gmail.com>
> > > > > > > ---
> > > > > > >  mk/rte.app.mk     | 4 ++--
> > > > > > >  mk/rte.hostapp.mk | 4 ++--
> > > > > > >  mk/rte.shared.mk  | 4 ++--
> > > > > > >  3 files changed, 6 insertions(+), 6 deletions(-)
> > > > > > > 
> > > > > > > --- a/mk/rte.app.mk
> > > > > > > +++ b/mk/rte.app.mk
> > > > > > > @@ -263,8 +263,8 @@ LDLIBS_NAMES += $(patsubst -Wl$(comma)-
> > > > > > > l%,lib%.a,$(filter -Wl$(comma)-l%,$(LDLIB
> > > > > > >  
> > > > > > >  # list of found libraries files (useful for deps). If not
> > > > > > > found,
> > > > > > > the
> > > > > > >  # library is silently ignored and dep won't be checked
> > > > > > > -LDLIBS_FILES := $(wildcard $(foreach dir,$(LDLIBS_PATH),\
> > > > > > > -	$(addprefix $(dir)/,$(LDLIBS_NAMES))))
> > > > > > > +LDLIBS_FILES := $(sort $(wildcard $(foreach
> > > > > > > dir,$(LDLIBS_PATH),\
> > > > > > > +	$(addprefix $(dir)/,$(LDLIBS_NAMES)))))
> > > > > > 
> > > > > > You cannot sort libraries.
> > > > > > Check - for instance - this comment above in this file:
> > > > > > 	# Eliminate duplicates without sorting, only keep the
> > > > > > last
> > > > > > occurrence
> > > > > > 	filter-libs = \
> > > > > 
> > > > > Not sure I follow - what's the reason for avoiding to sort the
> > > > > list
> > > > > of
> > > > > libs to link against?
> > > > 
> > > > Sorry, the ordering issue is with LDLIBS not LDLIBS_FILES.
> > > > 
> > > > > > Why sorting them?
> > > > > > What is random in libraries list?
> > > > > 
> > > > > The issue is that the output of wildcard is not fully
> > > > > deterministic. It
> > > > > can depend on the filesystem, and even on the locale settings
> > > > > [1].
> > > > > Before GNU Make 3.82 (2009) it used to automatically sort the
> > > > > output,
> > > > > but that was removed (to make it faster... sigh). [2]
> > > > 
> > > > It is not a true wildcard here. It is just filtering files which
> > > > do not exist.
> > > > I think you do not need this patch for deterministic build.
> > > 
> > > But then those lists are passed down in the .SECONDEXPANSION rule
> > > right?
> > 
> > I do not follow you.
> > Please explain what is the benefit of the patch in the next version.
> 
> I thought that these lists are used to determine which files to
> recompile - and incidentally, in which order as they are passed in this
> snippet in rte.compile-pre.mk:
> 
> .SECONDEXPANSION:
> %.o: %.c $$(wildcard $$(dep_$$@)) $$(DEP_$$(@)) FORCE
> 	@[ -d $(dir $@) ] || mkdir -p $(dir $@)
> 	$(if $(D),\
> 		@echo -n "$< -> $@ " ; \
> 		echo -n "file_missing=$(call boolean,$(file_missing)) " ; \
> 		echo -n "cmdline_changed=$(call boolean,$(call cmdline_changed,$(C_TO_O))) " ; \
> 		echo -n "depfile_missing=$(call boolean,$(depfile_missing)) " ; \
> 		echo "depfile_newer=$(call boolean,$(depfile_newer))")
> 	$(if $(or \
> 		$(file_missing),\
> 		$(call cmdline_changed,$(C_TO_O)),\
> 		$(depfile_missing),\
> 		$(depfile_newer)),\
> 		$(C_TO_O_DO))
> 
> Did I get that wrong? (It is a bit convoluted :-P )

LDLIBS_FILES are just used to link the executable AFAIK.
The libraries are built in separate recursive make calls.

> But nevertheless, I've finally found the root cause for the "wandering
> header" - when building the libraries, CFLAGS lists
> -I$(RTE_OUTDIR)/include first and then -I$(SRCDIR) last. There is a
> race, and sometimes when GCC is called the public header has already
> been installed in RTE_OUTDIR and sometimes it has not. This causes the
> instability, and causes the expansion of __FILE__ and the directory
> listing in the DWARF objects to randomly list the full path to either
> RTE_OUTDIR/include or SRCDIR for each library.
> 
> By always passing -I$(SRCDIR) first this instability is solved.

Good news.

Re: [dpdk-dev] [PATCH v3 7/8] mk: sort object files when building deps lists

[Luca Boccassi]

On Wed, 2017-06-28 at 16:37 +0200, Thomas Monjalon wrote:
> 28/06/2017 16:07, Luca Boccassi:
> > On Tue, 2017-06-27 at 18:14 +0200, Thomas Monjalon wrote:
> > > 27/06/2017 16:51, Luca Boccassi:
> > > > On Tue, 2017-06-27 at 15:52 +0200, Thomas Monjalon wrote:
> > > > > 27/06/2017 12:43, Luca Boccassi:
> > > > > > On Tue, 2017-06-27 at 01:20 +0200, Thomas Monjalon wrote:
> > > > > > > 23/06/2017 20:41, lboccass@brocade.com:
> > > > > > > > From: Luca Boccassi <luca.boccassi@gmail.com>
> > > > > > > > 
> > > > > > > > In order to achieve reproducible builds, always use the
> > > > > > > > same
> > > > > > > > order when listing object files to build dependencies
> > > > > > > > lists.
> > > > > > > > 
> > > > > > > > Signed-off-by: Luca Boccassi <luca.boccassi@gmail.com>
> > > > > > > > ---
> > > > > > > >  mk/rte.app.mk     | 4 ++--
> > > > > > > >  mk/rte.hostapp.mk | 4 ++--
> > > > > > > >  mk/rte.shared.mk  | 4 ++--
> > > > > > > >  3 files changed, 6 insertions(+), 6 deletions(-)
> > > > > > > > 
> > > > > > > > --- a/mk/rte.app.mk
> > > > > > > > +++ b/mk/rte.app.mk
> > > > > > > > @@ -263,8 +263,8 @@ LDLIBS_NAMES += $(patsubst
> > > > > > > > -Wl$(comma)-
> > > > > > > > l%,lib%.a,$(filter -Wl$(comma)-l%,$(LDLIB
> > > > > > > >  
> > > > > > > >  # list of found libraries files (useful for deps). If
> > > > > > > > not
> > > > > > > > found,
> > > > > > > > the
> > > > > > > >  # library is silently ignored and dep won't be checked
> > > > > > > > -LDLIBS_FILES := $(wildcard $(foreach
> > > > > > > > dir,$(LDLIBS_PATH),\
> > > > > > > > -	$(addprefix $(dir)/,$(LDLIBS_NAMES))))
> > > > > > > > +LDLIBS_FILES := $(sort $(wildcard $(foreach
> > > > > > > > dir,$(LDLIBS_PATH),\
> > > > > > > > +	$(addprefix $(dir)/,$(LDLIBS_NAMES)))))
> > > > > > > 
> > > > > > > You cannot sort libraries.
> > > > > > > Check - for instance - this comment above in this file:
> > > > > > > 	# Eliminate duplicates without sorting, only keep the
> > > > > > > last
> > > > > > > occurrence
> > > > > > > 	filter-libs = \
> > > > > > 
> > > > > > Not sure I follow - what's the reason for avoiding to sort
> > > > > > the
> > > > > > list
> > > > > > of
> > > > > > libs to link against?
> > > > > 
> > > > > Sorry, the ordering issue is with LDLIBS not LDLIBS_FILES.
> > > > > 
> > > > > > > Why sorting them?
> > > > > > > What is random in libraries list?
> > > > > > 
> > > > > > The issue is that the output of wildcard is not fully
> > > > > > deterministic. It
> > > > > > can depend on the filesystem, and even on the locale
> > > > > > settings
> > > > > > [1].
> > > > > > Before GNU Make 3.82 (2009) it used to automatically sort
> > > > > > the
> > > > > > output,
> > > > > > but that was removed (to make it faster... sigh). [2]
> > > > > 
> > > > > It is not a true wildcard here. It is just filtering files
> > > > > which
> > > > > do not exist.
> > > > > I think you do not need this patch for deterministic build.
> > > > 
> > > > But then those lists are passed down in the .SECONDEXPANSION
> > > > rule
> > > > right?
> > > 
> > > I do not follow you.
> > > Please explain what is the benefit of the patch in the next
> > > version.
> > 
> > I thought that these lists are used to determine which files to
> > recompile - and incidentally, in which order as they are passed in
> > this
> > snippet in rte.compile-pre.mk:
> > 
> > .SECONDEXPANSION:
> > %.o: %.c $$(wildcard $$(dep_$$@)) $$(DEP_$$(@)) FORCE
> > 	@[ -d $(dir $@) ] || mkdir -p $(dir $@)
> > 	$(if $(D),\
> > 		@echo -n "$< -> $@ " ; \
> > 		echo -n "file_missing=$(call boolean,$(file_missing)) "
> > ; \
> > 		echo -n "cmdline_changed=$(call boolean,$(call
> > cmdline_changed,$(C_TO_O))) " ; \
> > 		echo -n "depfile_missing=$(call
> > boolean,$(depfile_missing)) " ; \
> > 		echo "depfile_newer=$(call boolean,$(depfile_newer))")
> > 	$(if $(or \
> > 		$(file_missing),\
> > 		$(call cmdline_changed,$(C_TO_O)),\
> > 		$(depfile_missing),\
> > 		$(depfile_newer)),\
> > 		$(C_TO_O_DO))
> > 
> > Did I get that wrong? (It is a bit convoluted :-P )
> 
> LDLIBS_FILES are just used to link the executable AFAIK.
> The libraries are built in separate recursive make calls.
> 
> > But nevertheless, I've finally found the root cause for the
> > "wandering
> > header" - when building the libraries, CFLAGS lists
> > -I$(RTE_OUTDIR)/include first and then -I$(SRCDIR) last. There is a
> > race, and sometimes when GCC is called the public header has
> > already
> > been installed in RTE_OUTDIR and sometimes it has not. This causes
> > the
> > instability, and causes the expansion of __FILE__ and the directory
> > listing in the DWARF objects to randomly list the full path to
> > either
> > RTE_OUTDIR/include or SRCDIR for each library.
> > 
> > By always passing -I$(SRCDIR) first this instability is solved.
> 
> Good news.

BTW as an example here's the diff between 2 builds, if anyone is
interested:

https://paste.debian.net/973727/

-- 
Kind regards,
Luca Boccassi

Re: [dpdk-dev] [PATCH v4 0/6] Reproducible build

[Stephen Hemminger]

On Wed, 28 Jun 2017 14:56:56 +0100
<lboccass@brocade.com> wrote:

> From: Luca Boccassi <luca.boccassi@gmail.com>
> 
> In the past couple of years a concerted effort among almost all Linux
> distros has been striving toward achieving reproducible builds. [1]
> This involves changes to the toolchain, new tools and CI systems. [2]
> 
> v1 fixed the documentation, examples and linker script generation.
> v2 fixes all problems, which were caused by unstable order of headers
> inclusion, source files listing and object file listing when passing
> them to the compiler.
> DPDK's build, at least with the default configuration, is fully
> reproducible with this patch series as tested by the Reproducible
> Builds developers experimental toolchain. [3]
> 
> v3 restores the first patch, which was eaten by git send-email.
> 
> v4 drops the patch that reorders rebuilds, and adds a patch to make
> the inclusion of headers deterministic with regards to GCC embedding
> the full file path when expading __FILE__ and when writing the
> directory listing in the DWARF objects.
> It also drops the first 2 patches which have already been merged.
> 
> [1] https://reproducible-builds.org/
> [2] https://reproducible-builds.org/tools/
> [3] https://wiki.debian.org/ReproducibleBuilds/ExperimentalToolchain#Us

Looks good.

Looking ahead, how does this work with the proposed new build system?
Is there an automated way to check new submissions so that new features
don't undo this.


Acked-by: Stephen Hemminger <stephen@networkplumber.org>

Re: [dpdk-dev] [PATCH v4 0/6] Reproducible build

[Bruce Richardson]

On Wed, Jun 28, 2017 at 08:57:33AM -0700, Stephen Hemminger wrote:
> On Wed, 28 Jun 2017 14:56:56 +0100
> <lboccass@brocade.com> wrote:
> 
> > From: Luca Boccassi <luca.boccassi@gmail.com>
> > 
> > In the past couple of years a concerted effort among almost all Linux
> > distros has been striving toward achieving reproducible builds. [1]
> > This involves changes to the toolchain, new tools and CI systems. [2]
> > 
> > v1 fixed the documentation, examples and linker script generation.
> > v2 fixes all problems, which were caused by unstable order of headers
> > inclusion, source files listing and object file listing when passing
> > them to the compiler.
> > DPDK's build, at least with the default configuration, is fully
> > reproducible with this patch series as tested by the Reproducible
> > Builds developers experimental toolchain. [3]
> > 
> > v3 restores the first patch, which was eaten by git send-email.
> > 
> > v4 drops the patch that reorders rebuilds, and adds a patch to make
> > the inclusion of headers deterministic with regards to GCC embedding
> > the full file path when expading __FILE__ and when writing the
> > directory listing in the DWARF objects.
> > It also drops the first 2 patches which have already been merged.
> > 
> > [1] https://reproducible-builds.org/
> > [2] https://reproducible-builds.org/tools/
> > [3] https://wiki.debian.org/ReproducibleBuilds/ExperimentalToolchain#Us
> 
> Looks good.
> 
> Looking ahead, how does this work with the proposed new build system?
> Is there an automated way to check new submissions so that new features
> don't undo this.
> 
> 
> Acked-by: Stephen Hemminger <stephen@networkplumber.org>

http://mesonbuild.com/Reproducible-builds.html

I'd hope if we switch build system, this shouldn't be a problem. It's
definitely something to watch out for.

/Bruce

Re: [dpdk-dev] [PATCH v4 0/6] Reproducible build

[Luca Boccassi]

On Wed, 2017-06-28 at 17:04 +0100, Bruce Richardson wrote:
> On Wed, Jun 28, 2017 at 08:57:33AM -0700, Stephen Hemminger wrote:
> > On Wed, 28 Jun 2017 14:56:56 +0100
> > <lboccass@brocade.com> wrote:
> > 
> > > From: Luca Boccassi <luca.boccassi@gmail.com>
> > > 
> > > In the past couple of years a concerted effort among almost all
> > > Linux
> > > distros has been striving toward achieving reproducible builds.
> > > [1]
> > > This involves changes to the toolchain, new tools and CI systems.
> > > [2]
> > > 
> > > v1 fixed the documentation, examples and linker script
> > > generation.
> > > v2 fixes all problems, which were caused by unstable order of
> > > headers
> > > inclusion, source files listing and object file listing when
> > > passing
> > > them to the compiler.
> > > DPDK's build, at least with the default configuration, is fully
> > > reproducible with this patch series as tested by the Reproducible
> > > Builds developers experimental toolchain. [3]
> > > 
> > > v3 restores the first patch, which was eaten by git send-email.
> > > 
> > > v4 drops the patch that reorders rebuilds, and adds a patch to
> > > make
> > > the inclusion of headers deterministic with regards to GCC
> > > embedding
> > > the full file path when expading __FILE__ and when writing the
> > > directory listing in the DWARF objects.
> > > It also drops the first 2 patches which have already been merged.
> > > 
> > > [1] https://urldefense.proofpoint.com/v2/url?u=https-3A__reproduc
> > > ible-
> > > 2Dbuilds.org_&d=DwIBAg&c=IL_XqQWOjubgfqINi2jTzg&r=QTEM8ICX7t_SLgW
> > > P3qPWtKiwKMps487LPWQx-
> > > B9AqIc&m=HlHnLJOJRokDA9x9IHxyhvyPOnXIakHUFgofHP7STEQ&s=ibn1k95lSK
> > > CcbF9_9Gy4SpcLwLHaTD0aBr-Gm5uuCFQ&e= 
> > > [2] https://urldefense.proofpoint.com/v2/url?u=https-3A__reproduc
> > > ible-
> > > 2Dbuilds.org_tools_&d=DwIBAg&c=IL_XqQWOjubgfqINi2jTzg&r=QTEM8ICX7
> > > t_SLgWP3qPWtKiwKMps487LPWQx-
> > > B9AqIc&m=HlHnLJOJRokDA9x9IHxyhvyPOnXIakHUFgofHP7STEQ&s=u4leWgQouu
> > > DTOMv9EReHQMsopArSu9Xqc1uKe0cCqS0&e= 
> > > [3] https://urldefense.proofpoint.com/v2/url?u=https-3A__wiki.deb
> > > ian.org_ReproducibleBuilds_ExperimentalToolchain-
> > > 23Us&d=DwIBAg&c=IL_XqQWOjubgfqINi2jTzg&r=QTEM8ICX7t_SLgWP3qPWtKiw
> > > KMps487LPWQx-
> > > B9AqIc&m=HlHnLJOJRokDA9x9IHxyhvyPOnXIakHUFgofHP7STEQ&s=I1iNIvHcZX
> > > YecfQKFo5HtwyhF7jogVTb-lGzBf0wHFg&e= 
> > 
> > Looks good.
> > 
> > Looking ahead, how does this work with the proposed new build
> > system?
> > Is there an automated way to check new submissions so that new
> > features
> > don't undo this.
> > 
> > 
> > Acked-by: Stephen Hemminger <stephen@networkplumber.org>
> 
> https://urldefense.proofpoint.com/v2/url?u=http-3A__mesonbuild.com_Re
> producible-
> 2Dbuilds.html&d=DwIBAg&c=IL_XqQWOjubgfqINi2jTzg&r=QTEM8ICX7t_SLgWP3qP
> WtKiwKMps487LPWQx-
> B9AqIc&m=HlHnLJOJRokDA9x9IHxyhvyPOnXIakHUFgofHP7STEQ&s=0jXWXi5GtPcmXP
> RIXTNEoD1RhY54ig_799tJp4azboE&e= 
> 
> I'd hope if we switch build system, this shouldn't be a problem. It's
> definitely something to watch out for.
> 
> /Bruce

Meson claims support for reproducible builds by default:

http://mesonbuild.com/Reproducible-builds.html

I am not very familiar with it, but I assume it means stable
input/output to makefiles (list of sources/headers/objects etc).

The most common cause for unreproducibility is usually embedding build
timestamps - it was not the case for DPDK, but it might happen in the
future, and it's worth keeping in mind.

In general anything that uses find or wildcard to build lists (as
opposed as a boolean - if <wildcard> exists; then) might cause
problems.

In terms of testing, Debian has setup a pretty good automated CI
system:

https://tests.reproducible-builds.org/debian/reproducible.html

DPDK's page:

https://tests.reproducible-builds.org/debian/rb-pkg/unstable/amd64/dpdk
.html

Every time we do an upload it will be tested there.

-- 
Kind regards,
Luca Boccassi

[dpdk-dev] [PATCH v5 0/6] Reproducible build

[luca.boccassi]

From: Luca Boccassi <luca.boccassi@gmail.com>

In the past couple of years a concerted effort among almost all Linux
distros has been striving toward achieving reproducible builds. [1]
This involves changes to the toolchain, new tools and CI systems. [2]

v1 fixed the documentation, examples and linker script generation.
v2 fixes all problems, which were caused by unstable order of headers
inclusion, source files listing and object file listing when passing
them to the compiler.
DPDK's build, at least with the default configuration, is fully
reproducible with this patch series as tested by the Reproducible
Builds developers experimental toolchain. [3]

v3 restores the first patch, which was eaten by git send-email.

v4 drops the patch that reorders rebuilds, and adds a patch to make
the inclusion of headers deterministic with regards to GCC embedding
the full file path when expading __FILE__ and when writing the
directory listing in the DWARF objects.
It also drops the first 2 patches which have already been merged.

v5 adds the -I$(SRCDIR) workaround to librte_eal linuxapp's and
librte_gro's Makefiles.

[1] https://reproducible-builds.org/
[2] https://reproducible-builds.org/tools/
[3] https://wiki.debian.org/ReproducibleBuilds/ExperimentalToolchain#Us

Luca Boccassi (6):
  mk: sort list of shared objects in linker script
  mk: sort list of files in examples.dox
  mk: sort headers before wildcard inclusion
  mk: sort source files before passing them to the compiler
  mk: sort object files when building deps lists
  mk: set -ISCDIR before -IRTE_OUT/include in CFLAGS

 drivers/net/cxgbe/Makefile                                 |  2 +-
 drivers/net/e1000/Makefile                                 |  2 +-
 drivers/net/fm10k/Makefile                                 |  2 +-
 drivers/net/i40e/Makefile                                  |  2 +-
 drivers/net/ixgbe/Makefile                                 |  2 +-
 drivers/net/qede/Makefile                                  |  2 +-
 drivers/net/sfc/Makefile                                   |  2 +-
 drivers/net/thunderx/Makefile                              |  2 +-
 examples/ip_pipeline/Makefile                              |  2 +-
 examples/multi_process/client_server_mp/mp_server/Makefile |  2 +-
 examples/server_node_efd/server/Makefile                   |  2 +-
 lib/librte_acl/Makefile                                    | 10 ++++++++--
 lib/librte_bitratestats/Makefile                           |  9 ++++++++-
 lib/librte_cmdline/Makefile                                |  9 ++++++++-
 lib/librte_distributor/Makefile                            | 10 ++++++++--
 lib/librte_eal/common/Makefile                             |  2 +-
 lib/librte_eal/linuxapp/eal/Makefile                       |  8 +++++++-
 lib/librte_efd/Makefile                                    | 10 ++++++++--
 lib/librte_gro/Makefile                                    |  9 ++++++++-
 lib/librte_hash/Makefile                                   | 10 ++++++++--
 lib/librte_ip_frag/Makefile                                | 10 ++++++++--
 lib/librte_jobstats/Makefile                               | 10 ++++++++--
 lib/librte_kni/Makefile                                    |  9 ++++++++-
 lib/librte_kvargs/Makefile                                 |  9 ++++++++-
 lib/librte_latencystats/Makefile                           |  9 ++++++++-
 lib/librte_lpm/Makefile                                    | 10 ++++++++--
 lib/librte_mbuf/Makefile                                   |  9 ++++++++-
 lib/librte_mempool/Makefile                                |  9 ++++++++-
 lib/librte_metrics/Makefile                                |  9 ++++++++-
 lib/librte_net/Makefile                                    |  9 ++++++++-
 lib/librte_pdump/Makefile                                  | 10 ++++++++--
 lib/librte_power/Makefile                                  |  9 ++++++++-
 lib/librte_reorder/Makefile                                | 10 ++++++++--
 lib/librte_ring/Makefile                                   |  9 ++++++++-
 lib/librte_timer/Makefile                                  |  9 ++++++++-
 lib/librte_vhost/Makefile                                  |  9 ++++++++-
 mk/rte.app.mk                                              |  4 ++--
 mk/rte.combinedlib.mk                                      |  2 +-
 mk/rte.hostapp.mk                                          |  4 ++--
 mk/rte.sdkdoc.mk                                           |  2 +-
 mk/rte.shared.mk                                           |  4 ++--
 41 files changed, 211 insertions(+), 53 deletions(-)

-- 
2.11.0

[dpdk-dev] [PATCH v5 1/6] mk: sort list of shared objects in linker script

[luca.boccassi]

From: Luca Boccassi <luca.boccassi@gmail.com>

The output of wildcard might not be stable and depend on the
filesystem and other factors.
This means the content libdpdk.so linker script might change between
builds from the same sources.
Run the list through sort to ensure reproducibility.

Signed-off-by: Luca Boccassi <luca.boccassi@gmail.com>
---
 mk/rte.combinedlib.mk | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/mk/rte.combinedlib.mk b/mk/rte.combinedlib.mk
index 449358b33..2ab7ee8a1 100644
--- a/mk/rte.combinedlib.mk
+++ b/mk/rte.combinedlib.mk
@@ -42,7 +42,7 @@ endif
 RTE_LIBNAME := dpdk
 COMBINEDLIB := lib$(RTE_LIBNAME)$(EXT)
 
-LIBS := $(filter-out $(COMBINEDLIB), $(notdir $(wildcard $(RTE_OUTPUT)/lib/*$(EXT))))
+LIBS := $(filter-out $(COMBINEDLIB), $(sort $(notdir $(wildcard $(RTE_OUTPUT)/lib/*$(EXT)))))
 
 all: FORCE
 	$(Q)echo "GROUP ( $(LIBS) )" > $(RTE_OUTPUT)/lib/$(COMBINEDLIB)
-- 
2.11.0

[dpdk-dev] [PATCH v5 2/6] mk: sort list of files in examples.dox

[luca.boccassi]

From: Luca Boccassi <luca.boccassi@gmail.com>

The result of find might not be stable depending on external
conditions.
Pipe it through LC_ALL=C sort to ensure reproducible results when
generating examples.dox.

Signed-off-by: Luca Boccassi <luca.boccassi@gmail.com>
---
 mk/rte.sdkdoc.mk | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/mk/rte.sdkdoc.mk b/mk/rte.sdkdoc.mk
index c0eaa3502..de31b78cf 100644
--- a/mk/rte.sdkdoc.mk
+++ b/mk/rte.sdkdoc.mk
@@ -93,7 +93,7 @@ $(API_EXAMPLES): api-html-clean
 	$(Q)mkdir -p $(@D)
 	@printf '/**\n' > $(API_EXAMPLES)
 	@printf '@page examples DPDK Example Programs\n\n' >> $(API_EXAMPLES)
-	@find examples -type f -name '*.c' -printf '@example %p\n' >> $(API_EXAMPLES)
+	@find examples -type f -name '*.c' -printf '@example %p\n' | LC_ALL=C sort >> $(API_EXAMPLES)
 	@printf '*/\n' >> $(API_EXAMPLES)
 
 guides-pdf-clean: guides-pdf-img-clean
-- 
2.11.0

[dpdk-dev] [PATCH v5 3/6] mk: sort headers before wildcard inclusion

[luca.boccassi]

From: Luca Boccassi <luca.boccassi@gmail.com>

In order to achieve fully reproducible builds, always use the same
inclusion order for headers in the Makefiles.

Signed-off-by: Luca Boccassi <luca.boccassi@gmail.com>
---
 examples/ip_pipeline/Makefile                              | 2 +-
 examples/multi_process/client_server_mp/mp_server/Makefile | 2 +-
 examples/server_node_efd/server/Makefile                   | 2 +-
 lib/librte_eal/common/Makefile                             | 2 +-
 4 files changed, 4 insertions(+), 4 deletions(-)

diff --git a/examples/ip_pipeline/Makefile b/examples/ip_pipeline/Makefile
index dc7e0ddd7..12ce0a1d5 100644
--- a/examples/ip_pipeline/Makefile
+++ b/examples/ip_pipeline/Makefile
@@ -43,7 +43,7 @@ APP = ip_pipeline
 
 VPATH += $(SRCDIR)/pipeline
 
-INC += $(wildcard *.h) $(wildcard pipeline/*.h)
+INC += $(sort $(wildcard *.h)) $(sort $(wildcard pipeline/*.h))
 
 # all source are stored in SRCS-y
 SRCS-$(CONFIG_RTE_LIBRTE_PIPELINE) := main.c
diff --git a/examples/multi_process/client_server_mp/mp_server/Makefile b/examples/multi_process/client_server_mp/mp_server/Makefile
index 5552999b5..160c17b68 100644
--- a/examples/multi_process/client_server_mp/mp_server/Makefile
+++ b/examples/multi_process/client_server_mp/mp_server/Makefile
@@ -49,7 +49,7 @@ APP = mp_server
 # all source are stored in SRCS-y
 SRCS-y := main.c init.c args.c
 
-INC := $(wildcard *.h)
+INC := $(sort $(wildcard *.h))
 
 CFLAGS += $(WERROR_FLAGS) -O3
 CFLAGS += -I$(SRCDIR)/../shared
diff --git a/examples/server_node_efd/server/Makefile b/examples/server_node_efd/server/Makefile
index a2f2f361b..9f1fe2894 100644
--- a/examples/server_node_efd/server/Makefile
+++ b/examples/server_node_efd/server/Makefile
@@ -49,7 +49,7 @@ APP = server
 # all source are stored in SRCS-y
 SRCS-y := main.c init.c args.c
 
-INC := $(wildcard *.h)
+INC := $(sort $(wildcard *.h))
 
 CFLAGS += $(WERROR_FLAGS) -O3
 CFLAGS += -I$(SRCDIR)/../shared
diff --git a/lib/librte_eal/common/Makefile b/lib/librte_eal/common/Makefile
index e8fd67a27..4e6baaa72 100644
--- a/lib/librte_eal/common/Makefile
+++ b/lib/librte_eal/common/Makefile
@@ -49,7 +49,7 @@ GENERIC_INC += rte_vect.h rte_pause.h rte_io.h
 
 # defined in mk/arch/$(RTE_ARCH)/rte.vars.mk
 ARCH_DIR ?= $(RTE_ARCH)
-ARCH_INC := $(notdir $(wildcard $(RTE_SDK)/lib/librte_eal/common/include/arch/$(ARCH_DIR)/*.h))
+ARCH_INC := $(sort $(notdir $(wildcard $(RTE_SDK)/lib/librte_eal/common/include/arch/$(ARCH_DIR)/*.h)))
 
 SYMLINK-$(CONFIG_RTE_LIBRTE_EAL)-include := $(addprefix include/,$(INC))
 SYMLINK-$(CONFIG_RTE_LIBRTE_EAL)-include += \
-- 
2.11.0

[dpdk-dev] [PATCH v5 4/6] mk: sort source files before passing them to the compiler

[luca.boccassi]

From: Luca Boccassi <luca.boccassi@gmail.com>

In order to achieve reproducible builds, always use the same
order when listing files for compilation.

Signed-off-by: Luca Boccassi <luca.boccassi@gmail.com>
---
 drivers/net/cxgbe/Makefile    | 2 +-
 drivers/net/e1000/Makefile    | 2 +-
 drivers/net/fm10k/Makefile    | 2 +-
 drivers/net/i40e/Makefile     | 2 +-
 drivers/net/ixgbe/Makefile    | 2 +-
 drivers/net/qede/Makefile     | 2 +-
 drivers/net/sfc/Makefile      | 2 +-
 drivers/net/thunderx/Makefile | 2 +-
 8 files changed, 8 insertions(+), 8 deletions(-)

diff --git a/drivers/net/cxgbe/Makefile b/drivers/net/cxgbe/Makefile
index 7cef6279c..b4666b5af 100644
--- a/drivers/net/cxgbe/Makefile
+++ b/drivers/net/cxgbe/Makefile
@@ -67,7 +67,7 @@ endif
 # Add extra flags for base driver files (also known as shared code)
 # to disable warnings in them
 #
-BASE_DRIVER_OBJS=$(patsubst %.c,%.o,$(notdir $(wildcard $(SRCDIR)/base/*.c)))
+BASE_DRIVER_OBJS=$(sort $(patsubst %.c,%.o,$(notdir $(wildcard $(SRCDIR)/base/*.c))))
 $(foreach obj, $(BASE_DRIVER_OBJS), $(eval CFLAGS_$(obj)+=$(CFLAGS_BASE_DRIVER)))
 
 VPATH += $(SRCDIR)/base
diff --git a/drivers/net/e1000/Makefile b/drivers/net/e1000/Makefile
index ffdf36d37..59d96bca1 100644
--- a/drivers/net/e1000/Makefile
+++ b/drivers/net/e1000/Makefile
@@ -68,7 +68,7 @@ endif
 # Add extra flags for base driver files (also known as shared code)
 # to disable warnings in them
 #
-BASE_DRIVER_OBJS=$(patsubst %.c,%.o,$(notdir $(wildcard $(SRCDIR)/base/*.c)))
+BASE_DRIVER_OBJS=$(sort $(patsubst %.c,%.o,$(notdir $(wildcard $(SRCDIR)/base/*.c))))
 $(foreach obj, $(BASE_DRIVER_OBJS), $(eval CFLAGS_$(obj)+=$(CFLAGS_BASE_DRIVER)))
 
 VPATH += $(SRCDIR)/base
diff --git a/drivers/net/fm10k/Makefile b/drivers/net/fm10k/Makefile
index e0024f052..0bc124eb1 100644
--- a/drivers/net/fm10k/Makefile
+++ b/drivers/net/fm10k/Makefile
@@ -80,7 +80,7 @@ endif
 #
 # Add extra flags for base driver source files to disable warnings in them
 #
-BASE_DRIVER_OBJS=$(patsubst %.c,%.o,$(notdir $(wildcard $(SRCDIR)/base/*.c)))
+BASE_DRIVER_OBJS=$(sort $(patsubst %.c,%.o,$(notdir $(wildcard $(SRCDIR)/base/*.c))))
 $(foreach obj, $(BASE_DRIVER_OBJS), $(eval CFLAGS_$(obj)+=$(CFLAGS_BASE_DRIVER)))
 
 VPATH += $(SRCDIR)/base
diff --git a/drivers/net/i40e/Makefile b/drivers/net/i40e/Makefile
index 55c79a60a..8040a890e 100644
--- a/drivers/net/i40e/Makefile
+++ b/drivers/net/i40e/Makefile
@@ -78,7 +78,7 @@ endif
 
 CFLAGS_i40e_lan_hmc.o += -Wno-error
 endif
-OBJS_BASE_DRIVER=$(patsubst %.c,%.o,$(notdir $(wildcard $(SRCDIR)/base/*.c)))
+OBJS_BASE_DRIVER=$(sort $(patsubst %.c,%.o,$(notdir $(wildcard $(SRCDIR)/base/*.c))))
 $(foreach obj, $(OBJS_BASE_DRIVER), $(eval CFLAGS_$(obj)+=$(CFLAGS_BASE_DRIVER)))
 
 VPATH += $(SRCDIR)/base
diff --git a/drivers/net/ixgbe/Makefile b/drivers/net/ixgbe/Makefile
index 5e57cb353..d47ce24aa 100644
--- a/drivers/net/ixgbe/Makefile
+++ b/drivers/net/ixgbe/Makefile
@@ -87,7 +87,7 @@ endif
 # Add extra flags for base driver files (also known as shared code)
 # to disable warnings in them
 #
-BASE_DRIVER_OBJS=$(patsubst %.c,%.o,$(notdir $(wildcard $(SRCDIR)/base/*.c)))
+BASE_DRIVER_OBJS=$(sort $(patsubst %.c,%.o,$(notdir $(wildcard $(SRCDIR)/base/*.c))))
 $(foreach obj, $(BASE_DRIVER_OBJS), $(eval CFLAGS_$(obj)+=$(CFLAGS_BASE_DRIVER)))
 
 VPATH += $(SRCDIR)/base
diff --git a/drivers/net/qede/Makefile b/drivers/net/qede/Makefile
index f03441d9a..83ff95474 100644
--- a/drivers/net/qede/Makefile
+++ b/drivers/net/qede/Makefile
@@ -78,7 +78,7 @@ endif
 # to disable warnings in them
 #
 #
-BASE_DRIVER_OBJS=$(patsubst %.c,%.o,$(notdir $(wildcard $(SRCDIR)/base/*.c)))
+BASE_DRIVER_OBJS=$(sort $(patsubst %.c,%.o,$(notdir $(wildcard $(SRCDIR)/base/*.c))))
 $(foreach obj, $(BASE_DRIVER_OBJS), $(eval CFLAGS_$(obj)+=$(CFLAGS_BASE_DRIVER)))
 
 VPATH += $(SRCDIR)/base
diff --git a/drivers/net/sfc/Makefile b/drivers/net/sfc/Makefile
index 57aa963ba..8cfd14d45 100644
--- a/drivers/net/sfc/Makefile
+++ b/drivers/net/sfc/Makefile
@@ -71,7 +71,7 @@ endif
 # List of base driver object files for which
 # special CFLAGS above should be applied
 #
-BASE_DRIVER_OBJS=$(patsubst %.c,%.o,$(notdir $(wildcard $(SRCDIR)/base/*.c)))
+BASE_DRIVER_OBJS=$(sort $(patsubst %.c,%.o,$(notdir $(wildcard $(SRCDIR)/base/*.c))))
 $(foreach obj, $(BASE_DRIVER_OBJS), \
   $(eval CFLAGS_$(obj)+=$(CFLAGS_BASE_DRIVER)))
 
diff --git a/drivers/net/thunderx/Makefile b/drivers/net/thunderx/Makefile
index 915ae945a..a2b6caae2 100644
--- a/drivers/net/thunderx/Makefile
+++ b/drivers/net/thunderx/Makefile
@@ -45,7 +45,7 @@ EXPORT_MAP := rte_pmd_thunderx_nicvf_version.map
 
 LIBABIVER := 1
 
-OBJS_BASE_DRIVER=$(patsubst %.c,%.o,$(notdir $(wildcard $(SRCDIR)/base/*.c)))
+OBJS_BASE_DRIVER=$(sort $(patsubst %.c,%.o,$(notdir $(wildcard $(SRCDIR)/base/*.c))))
 $(foreach obj, $(OBJS_BASE_DRIVER), $(eval CFLAGS_$(obj)+=$(CFLAGS_BASE_DRIVER)))
 
 VPATH += $(SRCDIR)/base
-- 
2.11.0

[dpdk-dev] [PATCH v5 5/6] mk: sort object files when building deps lists

[luca.boccassi]

From: Luca Boccassi <luca.boccassi@gmail.com>

In order to achieve reproducible builds, always use the same
order when listing object files to build dependencies lists.

Signed-off-by: Luca Boccassi <luca.boccassi@gmail.com>
---
 mk/rte.app.mk     | 4 ++--
 mk/rte.hostapp.mk | 4 ++--
 mk/rte.shared.mk  | 4 ++--
 3 files changed, 6 insertions(+), 6 deletions(-)

diff --git a/mk/rte.app.mk b/mk/rte.app.mk
index c25fdd9f5..c5b8c1ead 100644
--- a/mk/rte.app.mk
+++ b/mk/rte.app.mk
@@ -270,8 +270,8 @@ LDLIBS_NAMES += $(patsubst -Wl$(comma)-l%,lib%.a,$(filter -Wl$(comma)-l%,$(LDLIB
 
 # list of found libraries files (useful for deps). If not found, the
 # library is silently ignored and dep won't be checked
-LDLIBS_FILES := $(wildcard $(foreach dir,$(LDLIBS_PATH),\
-	$(addprefix $(dir)/,$(LDLIBS_NAMES))))
+LDLIBS_FILES := $(sort $(wildcard $(foreach dir,$(LDLIBS_PATH),\
+	$(addprefix $(dir)/,$(LDLIBS_NAMES)))))
 
 #
 # Compile executable file if needed
diff --git a/mk/rte.hostapp.mk b/mk/rte.hostapp.mk
index 5cb4909cb..f58173c31 100644
--- a/mk/rte.hostapp.mk
+++ b/mk/rte.hostapp.mk
@@ -69,9 +69,9 @@ O_TO_EXE_DO = @set -e; \
 -include .$(HOSTAPP).cmd
 
 # list of .a files that are linked to this application
-LDLIBS_FILES := $(wildcard \
+LDLIBS_FILES := $(sort $(wildcard \
 	$(addprefix $(RTE_OUTPUT)/lib/, \
-	$(patsubst -l%,lib%.a,$(filter -l%,$(LDLIBS)))))
+	$(patsubst -l%,lib%.a,$(filter -l%,$(LDLIBS))))))
 
 #
 # Compile executable file if needed
diff --git a/mk/rte.shared.mk b/mk/rte.shared.mk
index 87ccf0ba4..4e680bc03 100644
--- a/mk/rte.shared.mk
+++ b/mk/rte.shared.mk
@@ -85,8 +85,8 @@ LDLIBS_NAMES += $(patsubst -Wl$(comma)-l%,lib%.a,$(filter -Wl$(comma)-l%,$(LDLIB
 
 # list of found libraries files (useful for deps). If not found, the
 # library is silently ignored and dep won't be checked
-LDLIBS_FILES := $(wildcard $(foreach dir,$(LDLIBS_PATH),\
-	$(addprefix $(dir)/,$(LDLIBS_NAMES))))
+LDLIBS_FILES := $(sort $(wildcard $(foreach dir,$(LDLIBS_PATH),\
+	$(addprefix $(dir)/,$(LDLIBS_NAMES)))))
 
 #
 # Archive objects in .so file if needed
-- 
2.11.0

[dpdk-dev] [PATCH v5 6/6] mk: set -ISCDIR before -IRTE_OUT/include in CFLAGS

[luca.boccassi]

From: Luca Boccassi <luca.boccassi@gmail.com>

A race condition can happen during parallel builds, where a header
might be installed in RTE_OUT/include before CFLAGS is recursively
expanded. This causes GCC to sometimes pick the header path as
SRCDIR/... and sometimes as RTE_OUT/include/... making the build
unreproducible, as the full path is used for the expansion of
__FILE__ and in the DWARF directory listing.
Always pass -ISRCDIR first to CFLAGS so that it's deterministic.

Signed-off-by: Luca Boccassi <luca.boccassi@gmail.com>
---
 lib/librte_acl/Makefile              | 10 ++++++++--
 lib/librte_bitratestats/Makefile     |  9 ++++++++-
 lib/librte_cmdline/Makefile          |  9 ++++++++-
 lib/librte_distributor/Makefile      | 10 ++++++++--
 lib/librte_eal/linuxapp/eal/Makefile |  8 +++++++-
 lib/librte_efd/Makefile              | 10 ++++++++--
 lib/librte_gro/Makefile              |  9 ++++++++-
 lib/librte_hash/Makefile             | 10 ++++++++--
 lib/librte_ip_frag/Makefile          | 10 ++++++++--
 lib/librte_jobstats/Makefile         | 10 ++++++++--
 lib/librte_kni/Makefile              |  9 ++++++++-
 lib/librte_kvargs/Makefile           |  9 ++++++++-
 lib/librte_latencystats/Makefile     |  9 ++++++++-
 lib/librte_lpm/Makefile              | 10 ++++++++--
 lib/librte_mbuf/Makefile             |  9 ++++++++-
 lib/librte_mempool/Makefile          |  9 ++++++++-
 lib/librte_metrics/Makefile          |  9 ++++++++-
 lib/librte_net/Makefile              |  9 ++++++++-
 lib/librte_pdump/Makefile            | 10 ++++++++--
 lib/librte_power/Makefile            |  9 ++++++++-
 lib/librte_reorder/Makefile          | 10 ++++++++--
 lib/librte_ring/Makefile             |  9 ++++++++-
 lib/librte_timer/Makefile            |  9 ++++++++-
 lib/librte_vhost/Makefile            |  9 ++++++++-
 24 files changed, 191 insertions(+), 33 deletions(-)

diff --git a/lib/librte_acl/Makefile b/lib/librte_acl/Makefile
index 59767920a..bce78813b 100644
--- a/lib/librte_acl/Makefile
+++ b/lib/librte_acl/Makefile
@@ -34,8 +34,14 @@ include $(RTE_SDK)/mk/rte.vars.mk
 # library name
 LIB = librte_acl.a
 
-CFLAGS += -O3
-CFLAGS += $(WERROR_FLAGS) -I$(SRCDIR)
+# Include the source dir first, so that headers paths are always picked
+# from there. By including it last race conditions might happen during paralle
+# builds, and headers might be already installed in RTE_OUT/include when the
+# variable is recursively expanded, thus causing GCC to sometimes use the
+# SRCDIR path and sometimes the RTE_OUT/include, making the builds not
+# reproducible.
+CFLAGS := -I$(SRCDIR) $(CFLAGS)
+CFLAGS += $(WERROR_FLAGS) -O3
 
 EXPORT_MAP := rte_acl_version.map
 
diff --git a/lib/librte_bitratestats/Makefile b/lib/librte_bitratestats/Makefile
index 58a20ea09..26a3f4908 100644
--- a/lib/librte_bitratestats/Makefile
+++ b/lib/librte_bitratestats/Makefile
@@ -34,7 +34,14 @@ include $(RTE_SDK)/mk/rte.vars.mk
 # library name
 LIB = librte_bitratestats.a
 
-CFLAGS += $(WERROR_FLAGS) -I$(SRCDIR) -O3
+# Include the source dir first, so that headers paths are always picked
+# from there. By including it last race conditions might happen during paralle
+# builds, and headers might be already installed in RTE_OUT/include when the
+# variable is recursively expanded, thus causing GCC to sometimes use the
+# SRCDIR path and sometimes the RTE_OUT/include, making the builds not
+# reproducible.
+CFLAGS := -I$(SRCDIR) $(CFLAGS)
+CFLAGS += $(WERROR_FLAGS) -O3
 
 EXPORT_MAP := rte_bitratestats_version.map
 
diff --git a/lib/librte_cmdline/Makefile b/lib/librte_cmdline/Makefile
index 644f68e47..9dd75f2d2 100644
--- a/lib/librte_cmdline/Makefile
+++ b/lib/librte_cmdline/Makefile
@@ -34,7 +34,14 @@ include $(RTE_SDK)/mk/rte.vars.mk
 # library name
 LIB = librte_cmdline.a
 
-CFLAGS += $(WERROR_FLAGS) -I$(SRCDIR) -O3
+# Include the source dir first, so that headers paths are always picked
+# from there. By including it last race conditions might happen during paralle
+# builds, and headers might be already installed in RTE_OUT/include when the
+# variable is recursively expanded, thus causing GCC to sometimes use the
+# SRCDIR path and sometimes the RTE_OUT/include, making the builds not
+# reproducible.
+CFLAGS := -I$(SRCDIR) $(CFLAGS)
+CFLAGS += $(WERROR_FLAGS) -O3
 
 EXPORT_MAP := rte_cmdline_version.map
 
diff --git a/lib/librte_distributor/Makefile b/lib/librte_distributor/Makefile
index b417ee7be..a51d7c8b2 100644
--- a/lib/librte_distributor/Makefile
+++ b/lib/librte_distributor/Makefile
@@ -34,8 +34,14 @@ include $(RTE_SDK)/mk/rte.vars.mk
 # library name
 LIB = librte_distributor.a
 
-CFLAGS += -O3
-CFLAGS += $(WERROR_FLAGS) -I$(SRCDIR)
+# Include the source dir first, so that headers paths are always picked
+# from there. By including it last race conditions might happen during paralle
+# builds, and headers might be already installed in RTE_OUT/include when the
+# variable is recursively expanded, thus causing GCC to sometimes use the
+# SRCDIR path and sometimes the RTE_OUT/include, making the builds not
+# reproducible.
+CFLAGS := -I$(SRCDIR) $(CFLAGS)
+CFLAGS += $(WERROR_FLAGS) -O3
 
 EXPORT_MAP := rte_distributor_version.map
 
diff --git a/lib/librte_eal/linuxapp/eal/Makefile b/lib/librte_eal/linuxapp/eal/Makefile
index 90bca4d68..f5eee1c6a 100644
--- a/lib/librte_eal/linuxapp/eal/Makefile
+++ b/lib/librte_eal/linuxapp/eal/Makefile
@@ -41,7 +41,13 @@ LIBABIVER := 5
 
 VPATH += $(RTE_SDK)/lib/librte_eal/common
 
-CFLAGS += -I$(SRCDIR)/include
+# Include the source dir first, so that headers paths are always picked
+# from there. By including it last race conditions might happen during paralle
+# builds, and headers might be already installed in RTE_OUT/include when the
+# variable is recursively expanded, thus causing GCC to sometimes use the
+# SRCDIR path and sometimes the RTE_OUT/include, making the builds not
+# reproducible.
+CFLAGS := -I$(SRCDIR)/include $(CFLAGS)
 CFLAGS += -I$(RTE_SDK)/lib/librte_eal/common
 CFLAGS += -I$(RTE_SDK)/lib/librte_eal/common/include
 CFLAGS += $(WERROR_FLAGS) -O3
diff --git a/lib/librte_efd/Makefile b/lib/librte_efd/Makefile
index b9277bc5d..b169e3240 100644
--- a/lib/librte_efd/Makefile
+++ b/lib/librte_efd/Makefile
@@ -34,8 +34,14 @@ include $(RTE_SDK)/mk/rte.vars.mk
 # library name
 LIB = librte_efd.a
 
-CFLAGS += -O3
-CFLAGS += $(WERROR_FLAGS) -I$(SRCDIR)
+# Include the source dir first, so that headers paths are always picked
+# from there. By including it last race conditions might happen during paralle
+# builds, and headers might be already installed in RTE_OUT/include when the
+# variable is recursively expanded, thus causing GCC to sometimes use the
+# SRCDIR path and sometimes the RTE_OUT/include, making the builds not
+# reproducible.
+CFLAGS := -I$(SRCDIR) $(CFLAGS)
+CFLAGS += $(WERROR_FLAGS) -O3
 
 EXPORT_MAP := rte_efd_version.map
 
diff --git a/lib/librte_gro/Makefile b/lib/librte_gro/Makefile
index 747eeec9e..367f78299 100644
--- a/lib/librte_gro/Makefile
+++ b/lib/librte_gro/Makefile
@@ -34,8 +34,15 @@ include $(RTE_SDK)/mk/rte.vars.mk
 # library name
 LIB = librte_gro.a
 
+# Include the source dir first, so that headers paths are always picked
+# from there. By including it last race conditions might happen during paralle
+# builds, and headers might be already installed in RTE_OUT/include when the
+# variable is recursively expanded, thus causing GCC to sometimes use the
+# SRCDIR path and sometimes the RTE_OUT/include, making the builds not
+# reproducible.
+CFLAGS := -I$(SRCDIR)/include $(CFLAGS)
 CFLAGS += -O3
-CFLAGS += $(WERROR_FLAGS) -I$(SRCDIR)
+CFLAGS += $(WERROR_FLAGS)
 
 EXPORT_MAP := rte_gro_version.map
 
diff --git a/lib/librte_hash/Makefile b/lib/librte_hash/Makefile
index 9cf13a045..677d494e7 100644
--- a/lib/librte_hash/Makefile
+++ b/lib/librte_hash/Makefile
@@ -34,8 +34,14 @@ include $(RTE_SDK)/mk/rte.vars.mk
 # library name
 LIB = librte_hash.a
 
-CFLAGS += -O3
-CFLAGS += $(WERROR_FLAGS) -I$(SRCDIR)
+# Include the source dir first, so that headers paths are always picked
+# from there. By including it last race conditions might happen during paralle
+# builds, and headers might be already installed in RTE_OUT/include when the
+# variable is recursively expanded, thus causing GCC to sometimes use the
+# SRCDIR path and sometimes the RTE_OUT/include, making the builds not
+# reproducible.
+CFLAGS := -I$(SRCDIR) $(CFLAGS)
+CFLAGS += $(WERROR_FLAGS) -O3
 
 EXPORT_MAP := rte_hash_version.map
 
diff --git a/lib/librte_ip_frag/Makefile b/lib/librte_ip_frag/Makefile
index 4e693bf8f..de45ec2d3 100644
--- a/lib/librte_ip_frag/Makefile
+++ b/lib/librte_ip_frag/Makefile
@@ -34,8 +34,14 @@ include $(RTE_SDK)/mk/rte.vars.mk
 # library name
 LIB = librte_ip_frag.a
 
-CFLAGS += -O3
-CFLAGS += $(WERROR_FLAGS) -I$(SRCDIR)
+# Include the source dir first, so that headers paths are always picked
+# from there. By including it last race conditions might happen during paralle
+# builds, and headers might be already installed in RTE_OUT/include when the
+# variable is recursively expanded, thus causing GCC to sometimes use the
+# SRCDIR path and sometimes the RTE_OUT/include, making the builds not
+# reproducible.
+CFLAGS := -I$(SRCDIR) $(CFLAGS)
+CFLAGS += $(WERROR_FLAGS) -O3
 
 EXPORT_MAP := rte_ipfrag_version.map
 
diff --git a/lib/librte_jobstats/Makefile b/lib/librte_jobstats/Makefile
index 561a0678c..57329b18e 100644
--- a/lib/librte_jobstats/Makefile
+++ b/lib/librte_jobstats/Makefile
@@ -34,8 +34,14 @@ include $(RTE_SDK)/mk/rte.vars.mk
 # library name
 LIB = librte_jobstats.a
 
-CFLAGS += -O3
-CFLAGS += $(WERROR_FLAGS) -I$(SRCDIR)
+# Include the source dir first, so that headers paths are always picked
+# from there. By including it last race conditions might happen during paralle
+# builds, and headers might be already installed in RTE_OUT/include when the
+# variable is recursively expanded, thus causing GCC to sometimes use the
+# SRCDIR path and sometimes the RTE_OUT/include, making the builds not
+# reproducible.
+CFLAGS := -I$(SRCDIR) $(CFLAGS)
+CFLAGS += $(WERROR_FLAGS) -O3
 
 EXPORT_MAP := rte_jobstats_version.map
 
diff --git a/lib/librte_kni/Makefile b/lib/librte_kni/Makefile
index 70f1ca8f6..130d6fd74 100644
--- a/lib/librte_kni/Makefile
+++ b/lib/librte_kni/Makefile
@@ -34,7 +34,14 @@ include $(RTE_SDK)/mk/rte.vars.mk
 # library name
 LIB = librte_kni.a
 
-CFLAGS += $(WERROR_FLAGS) -I$(SRCDIR) -O3 -fno-strict-aliasing
+# Include the source dir first, so that headers paths are always picked
+# from there. By including it last race conditions might happen during paralle
+# builds, and headers might be already installed in RTE_OUT/include when the
+# variable is recursively expanded, thus causing GCC to sometimes use the
+# SRCDIR path and sometimes the RTE_OUT/include, making the builds not
+# reproducible.
+CFLAGS := -I$(SRCDIR) $(CFLAGS)
+CFLAGS += $(WERROR_FLAGS) -O3 -fno-strict-aliasing
 
 EXPORT_MAP := rte_kni_version.map
 
diff --git a/lib/librte_kvargs/Makefile b/lib/librte_kvargs/Makefile
index 564dd3102..7c332c110 100644
--- a/lib/librte_kvargs/Makefile
+++ b/lib/librte_kvargs/Makefile
@@ -36,7 +36,14 @@ include $(RTE_SDK)/mk/rte.vars.mk
 # library name
 LIB = librte_kvargs.a
 
-CFLAGS += $(WERROR_FLAGS) -I$(SRCDIR) -O3
+# Include the source dir first, so that headers paths are always picked
+# from there. By including it last race conditions might happen during paralle
+# builds, and headers might be already installed in RTE_OUT/include when the
+# variable is recursively expanded, thus causing GCC to sometimes use the
+# SRCDIR path and sometimes the RTE_OUT/include, making the builds not
+# reproducible.
+CFLAGS := -I$(SRCDIR) $(CFLAGS)
+CFLAGS += $(WERROR_FLAGS) -O3
 
 EXPORT_MAP := rte_kvargs_version.map
 
diff --git a/lib/librte_latencystats/Makefile b/lib/librte_latencystats/Makefile
index eaacbb731..46a8ecd34 100644
--- a/lib/librte_latencystats/Makefile
+++ b/lib/librte_latencystats/Makefile
@@ -33,7 +33,14 @@ include $(RTE_SDK)/mk/rte.vars.mk
 # library name
 LIB = librte_latencystats.a
 
-CFLAGS += $(WERROR_FLAGS) -I$(SRCDIR) -O3
+# Include the source dir first, so that headers paths are always picked
+# from there. By including it last race conditions might happen during paralle
+# builds, and headers might be already installed in RTE_OUT/include when the
+# variable is recursively expanded, thus causing GCC to sometimes use the
+# SRCDIR path and sometimes the RTE_OUT/include, making the builds not
+# reproducible.
+CFLAGS := -I$(SRCDIR) $(CFLAGS)
+CFLAGS += $(WERROR_FLAGS) -O3
 LDLIBS += -lm
 LDLIBS += -lpthread
 
diff --git a/lib/librte_lpm/Makefile b/lib/librte_lpm/Makefile
index 32be46b3b..6a97fdc97 100644
--- a/lib/librte_lpm/Makefile
+++ b/lib/librte_lpm/Makefile
@@ -34,8 +34,14 @@ include $(RTE_SDK)/mk/rte.vars.mk
 # library name
 LIB = librte_lpm.a
 
-CFLAGS += -O3
-CFLAGS += $(WERROR_FLAGS) -I$(SRCDIR)
+# Include the source dir first, so that headers paths are always picked
+# from there. By including it last race conditions might happen during paralle
+# builds, and headers might be already installed in RTE_OUT/include when the
+# variable is recursively expanded, thus causing GCC to sometimes use the
+# SRCDIR path and sometimes the RTE_OUT/include, making the builds not
+# reproducible.
+CFLAGS := -I$(SRCDIR) $(CFLAGS)
+CFLAGS += $(WERROR_FLAGS) -O3
 
 EXPORT_MAP := rte_lpm_version.map
 
diff --git a/lib/librte_mbuf/Makefile b/lib/librte_mbuf/Makefile
index 548273054..4d51191ee 100644
--- a/lib/librte_mbuf/Makefile
+++ b/lib/librte_mbuf/Makefile
@@ -34,7 +34,14 @@ include $(RTE_SDK)/mk/rte.vars.mk
 # library name
 LIB = librte_mbuf.a
 
-CFLAGS += $(WERROR_FLAGS) -I$(SRCDIR) -O3
+# Include the source dir first, so that headers paths are always picked
+# from there. By including it last race conditions might happen during paralle
+# builds, and headers might be already installed in RTE_OUT/include when the
+# variable is recursively expanded, thus causing GCC to sometimes use the
+# SRCDIR path and sometimes the RTE_OUT/include, making the builds not
+# reproducible.
+CFLAGS := -I$(SRCDIR) $(CFLAGS)
+CFLAGS += $(WERROR_FLAGS) -O3
 
 EXPORT_MAP := rte_mbuf_version.map
 
diff --git a/lib/librte_mempool/Makefile b/lib/librte_mempool/Makefile
index 7b5bdfee7..33678192f 100644
--- a/lib/librte_mempool/Makefile
+++ b/lib/librte_mempool/Makefile
@@ -34,7 +34,14 @@ include $(RTE_SDK)/mk/rte.vars.mk
 # library name
 LIB = librte_mempool.a
 
-CFLAGS += $(WERROR_FLAGS) -I$(SRCDIR) -O3
+# Include the source dir first, so that headers paths are always picked
+# from there. By including it last race conditions might happen during paralle
+# builds, and headers might be already installed in RTE_OUT/include when the
+# variable is recursively expanded, thus causing GCC to sometimes use the
+# SRCDIR path and sometimes the RTE_OUT/include, making the builds not
+# reproducible.
+CFLAGS := -I$(SRCDIR) $(CFLAGS)
+CFLAGS += $(WERROR_FLAGS) -O3
 
 EXPORT_MAP := rte_mempool_version.map
 
diff --git a/lib/librte_metrics/Makefile b/lib/librte_metrics/Makefile
index d4990e839..195bd4d56 100644
--- a/lib/librte_metrics/Makefile
+++ b/lib/librte_metrics/Makefile
@@ -34,7 +34,14 @@ include $(RTE_SDK)/mk/rte.vars.mk
 # library name
 LIB = librte_metrics.a
 
-CFLAGS += $(WERROR_FLAGS) -I$(SRCDIR) -O3
+# Include the source dir first, so that headers paths are always picked
+# from there. By including it last race conditions might happen during paralle
+# builds, and headers might be already installed in RTE_OUT/include when the
+# variable is recursively expanded, thus causing GCC to sometimes use the
+# SRCDIR path and sometimes the RTE_OUT/include, making the builds not
+# reproducible.
+CFLAGS := -I$(SRCDIR) $(CFLAGS)
+CFLAGS += $(WERROR_FLAGS) -O3
 
 EXPORT_MAP := rte_metrics_version.map
 
diff --git a/lib/librte_net/Makefile b/lib/librte_net/Makefile
index 56727c4df..a08a7dd8b 100644
--- a/lib/librte_net/Makefile
+++ b/lib/librte_net/Makefile
@@ -33,7 +33,14 @@ include $(RTE_SDK)/mk/rte.vars.mk
 
 LIB = librte_net.a
 
-CFLAGS += $(WERROR_FLAGS) -I$(SRCDIR) -O3
+# Include the source dir first, so that headers paths are always picked
+# from there. By including it last race conditions might happen during paralle
+# builds, and headers might be already installed in RTE_OUT/include when the
+# variable is recursively expanded, thus causing GCC to sometimes use the
+# SRCDIR path and sometimes the RTE_OUT/include, making the builds not
+# reproducible.
+CFLAGS := -I$(SRCDIR) $(CFLAGS)
+CFLAGS += $(WERROR_FLAGS) -O3
 
 EXPORT_MAP := rte_net_version.map
 LIBABIVER := 1
diff --git a/lib/librte_pdump/Makefile b/lib/librte_pdump/Makefile
index 1c03bcbb7..8923d7499 100644
--- a/lib/librte_pdump/Makefile
+++ b/lib/librte_pdump/Makefile
@@ -34,8 +34,14 @@ include $(RTE_SDK)/mk/rte.vars.mk
 # library name
 LIB = librte_pdump.a
 
-CFLAGS += $(WERROR_FLAGS) -I$(SRCDIR) -O3
-CFLAGS += -D_GNU_SOURCE
+# Include the source dir first, so that headers paths are always picked
+# from there. By including it last race conditions might happen during paralle
+# builds, and headers might be already installed in RTE_OUT/include when the
+# variable is recursively expanded, thus causing GCC to sometimes use the
+# SRCDIR path and sometimes the RTE_OUT/include, making the builds not
+# reproducible.
+CFLAGS := -I$(SRCDIR) $(CFLAGS)
+CFLAGS += $(WERROR_FLAGS) -O3 -D_GNU_SOURCE
 LDLIBS += -lpthread
 
 EXPORT_MAP := rte_pdump_version.map
diff --git a/lib/librte_power/Makefile b/lib/librte_power/Makefile
index 06cd10e86..b76ad689e 100644
--- a/lib/librte_power/Makefile
+++ b/lib/librte_power/Makefile
@@ -34,7 +34,14 @@ include $(RTE_SDK)/mk/rte.vars.mk
 # library name
 LIB = librte_power.a
 
-CFLAGS += $(WERROR_FLAGS) -I$(SRCDIR) -O3 -fno-strict-aliasing
+# Include the source dir first, so that headers paths are always picked
+# from there. By including it last race conditions might happen during paralle
+# builds, and headers might be already installed in RTE_OUT/include when the
+# variable is recursively expanded, thus causing GCC to sometimes use the
+# SRCDIR path and sometimes the RTE_OUT/include, making the builds not
+# reproducible.
+CFLAGS := -I$(SRCDIR) $(CFLAGS)
+CFLAGS += $(WERROR_FLAGS) -O3 -fno-strict-aliasing
 
 EXPORT_MAP := rte_power_version.map
 
diff --git a/lib/librte_reorder/Makefile b/lib/librte_reorder/Makefile
index 4e44e72f0..51b5d490d 100644
--- a/lib/librte_reorder/Makefile
+++ b/lib/librte_reorder/Makefile
@@ -34,8 +34,14 @@ include $(RTE_SDK)/mk/rte.vars.mk
 # library name
 LIB = librte_reorder.a
 
-CFLAGS += -O3
-CFLAGS += $(WERROR_FLAGS) -I$(SRCDIR)
+# Include the source dir first, so that headers paths are always picked
+# from there. By including it last race conditions might happen during paralle
+# builds, and headers might be already installed in RTE_OUT/include when the
+# variable is recursively expanded, thus causing GCC to sometimes use the
+# SRCDIR path and sometimes the RTE_OUT/include, making the builds not
+# reproducible.
+CFLAGS := -I$(SRCDIR) $(CFLAGS)
+CFLAGS += $(WERROR_FLAGS) -O3
 
 EXPORT_MAP := rte_reorder_version.map
 
diff --git a/lib/librte_ring/Makefile b/lib/librte_ring/Makefile
index 3e2f4b873..e5248a2c1 100644
--- a/lib/librte_ring/Makefile
+++ b/lib/librte_ring/Makefile
@@ -34,7 +34,14 @@ include $(RTE_SDK)/mk/rte.vars.mk
 # library name
 LIB = librte_ring.a
 
-CFLAGS += $(WERROR_FLAGS) -I$(SRCDIR) -O3
+# Include the source dir first, so that headers paths are always picked
+# from there. By including it last race conditions might happen during paralle
+# builds, and headers might be already installed in RTE_OUT/include when the
+# variable is recursively expanded, thus causing GCC to sometimes use the
+# SRCDIR path and sometimes the RTE_OUT/include, making the builds not
+# reproducible.
+CFLAGS := -I$(SRCDIR) $(CFLAGS)
+CFLAGS += $(WERROR_FLAGS) -O3
 
 EXPORT_MAP := rte_ring_version.map
 
diff --git a/lib/librte_timer/Makefile b/lib/librte_timer/Makefile
index 03a15390e..0c57cc6d3 100644
--- a/lib/librte_timer/Makefile
+++ b/lib/librte_timer/Makefile
@@ -34,7 +34,14 @@ include $(RTE_SDK)/mk/rte.vars.mk
 # library name
 LIB = librte_timer.a
 
-CFLAGS += $(WERROR_FLAGS) -I$(SRCDIR) -O3
+# Include the source dir first, so that headers paths are always picked
+# from there. By including it last race conditions might happen during parallel
+# builds, and headers might be already installed in RTE_OUT/include when the
+# variable is recursively expanded, thus causing GCC to sometimes use the
+# SRCDIR path and sometimes the RTE_OUT/include, making the builds not
+# reproducible.
+CFLAGS := -I$(SRCDIR) $(CFLAGS)
+CFLAGS += $(WERROR_FLAGS) -O3
 
 EXPORT_MAP := rte_timer_version.map
 
diff --git a/lib/librte_vhost/Makefile b/lib/librte_vhost/Makefile
index 4a116fe31..ef0d86990 100644
--- a/lib/librte_vhost/Makefile
+++ b/lib/librte_vhost/Makefile
@@ -38,7 +38,14 @@ EXPORT_MAP := rte_vhost_version.map
 
 LIBABIVER := 4
 
-CFLAGS += $(WERROR_FLAGS) -I$(SRCDIR) -O3 -D_FILE_OFFSET_BITS=64
+# Include the source dir first, so that headers paths are always picked
+# from there. By including it last race conditions might happen during parallel
+# builds, and headers might be already installed in RTE_OUT/include when the
+# variable is recursively expanded, thus causing GCC to sometimes use the
+# SRCDIR path and sometimes the RTE_OUT/include, making the builds not
+# reproducible.
+CFLAGS := -I$(SRCDIR) $(CFLAGS)
+CFLAGS += $(WERROR_FLAGS) -O3 -D_FILE_OFFSET_BITS=64
 CFLAGS += -I vhost_user
 LDLIBS += -lpthread
 
-- 
2.11.0

Re: [dpdk-dev] [PATCH v4 0/6] Reproducible build

[Luca Boccassi]

On Wed, 2017-06-28 at 17:04 +0100, Bruce Richardson wrote:
> On Wed, Jun 28, 2017 at 08:57:33AM -0700, Stephen Hemminger wrote:
> > On Wed, 28 Jun 2017 14:56:56 +0100
> > <lboccass@brocade.com> wrote:
> > 
> > > From: Luca Boccassi <luca.boccassi@gmail.com>
> > > 
> > > In the past couple of years a concerted effort among almost all
> > > Linux
> > > distros has been striving toward achieving reproducible builds.
> > > [1]
> > > This involves changes to the toolchain, new tools and CI systems.
> > > [2]
> > > 
> > > v1 fixed the documentation, examples and linker script
> > > generation.
> > > v2 fixes all problems, which were caused by unstable order of
> > > headers
> > > inclusion, source files listing and object file listing when
> > > passing
> > > them to the compiler.
> > > DPDK's build, at least with the default configuration, is fully
> > > reproducible with this patch series as tested by the Reproducible
> > > Builds developers experimental toolchain. [3]
> > > 
> > > v3 restores the first patch, which was eaten by git send-email.
> > > 
> > > v4 drops the patch that reorders rebuilds, and adds a patch to
> > > make
> > > the inclusion of headers deterministic with regards to GCC
> > > embedding
> > > the full file path when expading __FILE__ and when writing the
> > > directory listing in the DWARF objects.
> > > It also drops the first 2 patches which have already been merged.
> > > 
> > > [1] https://reproducible-builds.org/
> > > [2] https://reproducible-builds.org/tools/
> > > [3] https://wiki.debian.org/ReproducibleBuilds/ExperimentalToolch
> > > ain#Us
> > 
> > Looks good.
> > 
> > Looking ahead, how does this work with the proposed new build
> > system?
> > Is there an automated way to check new submissions so that new
> > features
> > don't undo this.
> > 
> > 
> > Acked-by: Stephen Hemminger <stephen@networkplumber.org>
> 
> http://mesonbuild.com/Reproducible-builds.html
> 
> I'd hope if we switch build system, this shouldn't be a problem. It's
> definitely something to watch out for.
> 
> /Bruce

The one issue to look for, with the current build system, is the CFLAGS
include path order (the last patch) in the makefiles under lib/

The pattern seems to be always the same, would it be possible &
acceptable to add a check in checkpatch?

bad:
CFLAGS += -I$(SRCDIR)

good:
CFLAGS := -I$(SRCDIR) $(CFLAGS)

-- 
Kind regards,
Luca Boccassi

[dpdk-dev] [PATCH v6 0/6] Reproducible build

[luca.boccassi]

From: Luca Boccassi <luca.boccassi@gmail.com>

In the past couple of years a concerted effort among almost all Linux
distros has been striving toward achieving reproducible builds. [1]
This involves changes to the toolchain, new tools and CI systems. [2]

v1 fixed the documentation, examples and linker script generation.
v2 fixes all problems, which were caused by unstable order of headers
inclusion, source files listing and object file listing when passing
them to the compiler.
DPDK's build, at least with the default configuration, is fully
reproducible with this patch series as tested by the Reproducible
Builds developers experimental toolchain. [3]

v3 restores the first patch, which was eaten by git send-email.

v4 drops the patch that reorders rebuilds, and adds a patch to make
the inclusion of headers deterministic with regards to GCC embedding
the full file path when expading __FILE__ and when writing the
directory listing in the DWARF objects.
It also drops the first 2 patches which have already been merged.

v5 adds the -I$(SRCDIR) workaround to librte_eal linuxapp's and
librte_gro's Makefiles.

v6 fixes copypasta added in v5 - librte-gro's Makefile should have used
$(SRCDIR) rather than $(SRCDIR)/include.

[1] https://reproducible-builds.org/
[2] https://reproducible-builds.org/tools/
[3] https://wiki.debian.org/ReproducibleBuilds/ExperimentalToolchain#Us

Luca Boccassi (6):
  mk: sort list of shared objects in linker script
  mk: sort list of files in examples.dox
  mk: sort headers before wildcard inclusion
  mk: sort source files before passing them to the compiler
  mk: sort object files when building deps lists
  mk: set -ISCDIR before -IRTE_OUT/include in CFLAGS

 drivers/net/cxgbe/Makefile                                 |  2 +-
 drivers/net/e1000/Makefile                                 |  2 +-
 drivers/net/fm10k/Makefile                                 |  2 +-
 drivers/net/i40e/Makefile                                  |  2 +-
 drivers/net/ixgbe/Makefile                                 |  2 +-
 drivers/net/qede/Makefile                                  |  2 +-
 drivers/net/sfc/Makefile                                   |  2 +-
 drivers/net/thunderx/Makefile                              |  2 +-
 examples/ip_pipeline/Makefile                              |  2 +-
 examples/multi_process/client_server_mp/mp_server/Makefile |  2 +-
 examples/server_node_efd/server/Makefile                   |  2 +-
 lib/librte_acl/Makefile                                    | 10 ++++++++--
 lib/librte_bitratestats/Makefile                           |  9 ++++++++-
 lib/librte_cmdline/Makefile                                |  9 ++++++++-
 lib/librte_distributor/Makefile                            | 10 ++++++++--
 lib/librte_eal/common/Makefile                             |  2 +-
 lib/librte_eal/linuxapp/eal/Makefile                       |  8 +++++++-
 lib/librte_efd/Makefile                                    | 10 ++++++++--
 lib/librte_gro/Makefile                                    |  9 ++++++++-
 lib/librte_hash/Makefile                                   | 10 ++++++++--
 lib/librte_ip_frag/Makefile                                | 10 ++++++++--
 lib/librte_jobstats/Makefile                               | 10 ++++++++--
 lib/librte_kni/Makefile                                    |  9 ++++++++-
 lib/librte_kvargs/Makefile                                 |  9 ++++++++-
 lib/librte_latencystats/Makefile                           |  9 ++++++++-
 lib/librte_lpm/Makefile                                    | 10 ++++++++--
 lib/librte_mbuf/Makefile                                   |  9 ++++++++-
 lib/librte_mempool/Makefile                                |  9 ++++++++-
 lib/librte_metrics/Makefile                                |  9 ++++++++-
 lib/librte_net/Makefile                                    |  9 ++++++++-
 lib/librte_pdump/Makefile                                  | 10 ++++++++--
 lib/librte_power/Makefile                                  |  9 ++++++++-
 lib/librte_reorder/Makefile                                | 10 ++++++++--
 lib/librte_ring/Makefile                                   |  9 ++++++++-
 lib/librte_timer/Makefile                                  |  9 ++++++++-
 lib/librte_vhost/Makefile                                  |  9 ++++++++-
 mk/rte.app.mk                                              |  4 ++--
 mk/rte.combinedlib.mk                                      |  2 +-
 mk/rte.hostapp.mk                                          |  4 ++--
 mk/rte.sdkdoc.mk                                           |  2 +-
 mk/rte.shared.mk                                           |  4 ++--
 41 files changed, 211 insertions(+), 53 deletions(-)

-- 
2.11.0

[dpdk-dev] [PATCH v6 1/6] mk: sort list of shared objects in linker script

[luca.boccassi]

From: Luca Boccassi <luca.boccassi@gmail.com>

The output of wildcard might not be stable and depend on the
filesystem and other factors.
This means the content libdpdk.so linker script might change between
builds from the same sources.
Run the list through sort to ensure reproducibility.

Signed-off-by: Luca Boccassi <luca.boccassi@gmail.com>
---
 mk/rte.combinedlib.mk | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/mk/rte.combinedlib.mk b/mk/rte.combinedlib.mk
index 449358b33..2ab7ee8a1 100644
--- a/mk/rte.combinedlib.mk
+++ b/mk/rte.combinedlib.mk
@@ -42,7 +42,7 @@ endif
 RTE_LIBNAME := dpdk
 COMBINEDLIB := lib$(RTE_LIBNAME)$(EXT)
 
-LIBS := $(filter-out $(COMBINEDLIB), $(notdir $(wildcard $(RTE_OUTPUT)/lib/*$(EXT))))
+LIBS := $(filter-out $(COMBINEDLIB), $(sort $(notdir $(wildcard $(RTE_OUTPUT)/lib/*$(EXT)))))
 
 all: FORCE
 	$(Q)echo "GROUP ( $(LIBS) )" > $(RTE_OUTPUT)/lib/$(COMBINEDLIB)
-- 
2.11.0

[dpdk-dev] [PATCH v6 2/6] mk: sort list of files in examples.dox

[luca.boccassi]

From: Luca Boccassi <luca.boccassi@gmail.com>

The result of find might not be stable depending on external
conditions.
Pipe it through LC_ALL=C sort to ensure reproducible results when
generating examples.dox.

Signed-off-by: Luca Boccassi <luca.boccassi@gmail.com>
---
 mk/rte.sdkdoc.mk | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/mk/rte.sdkdoc.mk b/mk/rte.sdkdoc.mk
index c0eaa3502..de31b78cf 100644
--- a/mk/rte.sdkdoc.mk
+++ b/mk/rte.sdkdoc.mk
@@ -93,7 +93,7 @@ $(API_EXAMPLES): api-html-clean
 	$(Q)mkdir -p $(@D)
 	@printf '/**\n' > $(API_EXAMPLES)
 	@printf '@page examples DPDK Example Programs\n\n' >> $(API_EXAMPLES)
-	@find examples -type f -name '*.c' -printf '@example %p\n' >> $(API_EXAMPLES)
+	@find examples -type f -name '*.c' -printf '@example %p\n' | LC_ALL=C sort >> $(API_EXAMPLES)
 	@printf '*/\n' >> $(API_EXAMPLES)
 
 guides-pdf-clean: guides-pdf-img-clean
-- 
2.11.0

[dpdk-dev] [PATCH v6 3/6] mk: sort headers before wildcard inclusion

[luca.boccassi]

From: Luca Boccassi <luca.boccassi@gmail.com>

In order to achieve fully reproducible builds, always use the same
inclusion order for headers in the Makefiles.

Signed-off-by: Luca Boccassi <luca.boccassi@gmail.com>
---
 examples/ip_pipeline/Makefile                              | 2 +-
 examples/multi_process/client_server_mp/mp_server/Makefile | 2 +-
 examples/server_node_efd/server/Makefile                   | 2 +-
 lib/librte_eal/common/Makefile                             | 2 +-
 4 files changed, 4 insertions(+), 4 deletions(-)

diff --git a/examples/ip_pipeline/Makefile b/examples/ip_pipeline/Makefile
index dc7e0ddd7..12ce0a1d5 100644
--- a/examples/ip_pipeline/Makefile
+++ b/examples/ip_pipeline/Makefile
@@ -43,7 +43,7 @@ APP = ip_pipeline
 
 VPATH += $(SRCDIR)/pipeline
 
-INC += $(wildcard *.h) $(wildcard pipeline/*.h)
+INC += $(sort $(wildcard *.h)) $(sort $(wildcard pipeline/*.h))
 
 # all source are stored in SRCS-y
 SRCS-$(CONFIG_RTE_LIBRTE_PIPELINE) := main.c
diff --git a/examples/multi_process/client_server_mp/mp_server/Makefile b/examples/multi_process/client_server_mp/mp_server/Makefile
index 5552999b5..160c17b68 100644
--- a/examples/multi_process/client_server_mp/mp_server/Makefile
+++ b/examples/multi_process/client_server_mp/mp_server/Makefile
@@ -49,7 +49,7 @@ APP = mp_server
 # all source are stored in SRCS-y
 SRCS-y := main.c init.c args.c
 
-INC := $(wildcard *.h)
+INC := $(sort $(wildcard *.h))
 
 CFLAGS += $(WERROR_FLAGS) -O3
 CFLAGS += -I$(SRCDIR)/../shared
diff --git a/examples/server_node_efd/server/Makefile b/examples/server_node_efd/server/Makefile
index a2f2f361b..9f1fe2894 100644
--- a/examples/server_node_efd/server/Makefile
+++ b/examples/server_node_efd/server/Makefile
@@ -49,7 +49,7 @@ APP = server
 # all source are stored in SRCS-y
 SRCS-y := main.c init.c args.c
 
-INC := $(wildcard *.h)
+INC := $(sort $(wildcard *.h))
 
 CFLAGS += $(WERROR_FLAGS) -O3
 CFLAGS += -I$(SRCDIR)/../shared
diff --git a/lib/librte_eal/common/Makefile b/lib/librte_eal/common/Makefile
index e8fd67a27..4e6baaa72 100644
--- a/lib/librte_eal/common/Makefile
+++ b/lib/librte_eal/common/Makefile
@@ -49,7 +49,7 @@ GENERIC_INC += rte_vect.h rte_pause.h rte_io.h
 
 # defined in mk/arch/$(RTE_ARCH)/rte.vars.mk
 ARCH_DIR ?= $(RTE_ARCH)
-ARCH_INC := $(notdir $(wildcard $(RTE_SDK)/lib/librte_eal/common/include/arch/$(ARCH_DIR)/*.h))
+ARCH_INC := $(sort $(notdir $(wildcard $(RTE_SDK)/lib/librte_eal/common/include/arch/$(ARCH_DIR)/*.h)))
 
 SYMLINK-$(CONFIG_RTE_LIBRTE_EAL)-include := $(addprefix include/,$(INC))
 SYMLINK-$(CONFIG_RTE_LIBRTE_EAL)-include += \
-- 
2.11.0

[dpdk-dev] [PATCH v6 4/6] mk: sort source files before passing them to the compiler

[luca.boccassi]

From: Luca Boccassi <luca.boccassi@gmail.com>

In order to achieve reproducible builds, always use the same
order when listing files for compilation.

Signed-off-by: Luca Boccassi <luca.boccassi@gmail.com>
---
 drivers/net/cxgbe/Makefile    | 2 +-
 drivers/net/e1000/Makefile    | 2 +-
 drivers/net/fm10k/Makefile    | 2 +-
 drivers/net/i40e/Makefile     | 2 +-
 drivers/net/ixgbe/Makefile    | 2 +-
 drivers/net/qede/Makefile     | 2 +-
 drivers/net/sfc/Makefile      | 2 +-
 drivers/net/thunderx/Makefile | 2 +-
 8 files changed, 8 insertions(+), 8 deletions(-)

diff --git a/drivers/net/cxgbe/Makefile b/drivers/net/cxgbe/Makefile
index 7cef6279c..b4666b5af 100644
--- a/drivers/net/cxgbe/Makefile
+++ b/drivers/net/cxgbe/Makefile
@@ -67,7 +67,7 @@ endif
 # Add extra flags for base driver files (also known as shared code)
 # to disable warnings in them
 #
-BASE_DRIVER_OBJS=$(patsubst %.c,%.o,$(notdir $(wildcard $(SRCDIR)/base/*.c)))
+BASE_DRIVER_OBJS=$(sort $(patsubst %.c,%.o,$(notdir $(wildcard $(SRCDIR)/base/*.c))))
 $(foreach obj, $(BASE_DRIVER_OBJS), $(eval CFLAGS_$(obj)+=$(CFLAGS_BASE_DRIVER)))
 
 VPATH += $(SRCDIR)/base
diff --git a/drivers/net/e1000/Makefile b/drivers/net/e1000/Makefile
index ffdf36d37..59d96bca1 100644
--- a/drivers/net/e1000/Makefile
+++ b/drivers/net/e1000/Makefile
@@ -68,7 +68,7 @@ endif
 # Add extra flags for base driver files (also known as shared code)
 # to disable warnings in them
 #
-BASE_DRIVER_OBJS=$(patsubst %.c,%.o,$(notdir $(wildcard $(SRCDIR)/base/*.c)))
+BASE_DRIVER_OBJS=$(sort $(patsubst %.c,%.o,$(notdir $(wildcard $(SRCDIR)/base/*.c))))
 $(foreach obj, $(BASE_DRIVER_OBJS), $(eval CFLAGS_$(obj)+=$(CFLAGS_BASE_DRIVER)))
 
 VPATH += $(SRCDIR)/base
diff --git a/drivers/net/fm10k/Makefile b/drivers/net/fm10k/Makefile
index e0024f052..0bc124eb1 100644
--- a/drivers/net/fm10k/Makefile
+++ b/drivers/net/fm10k/Makefile
@@ -80,7 +80,7 @@ endif
 #
 # Add extra flags for base driver source files to disable warnings in them
 #
-BASE_DRIVER_OBJS=$(patsubst %.c,%.o,$(notdir $(wildcard $(SRCDIR)/base/*.c)))
+BASE_DRIVER_OBJS=$(sort $(patsubst %.c,%.o,$(notdir $(wildcard $(SRCDIR)/base/*.c))))
 $(foreach obj, $(BASE_DRIVER_OBJS), $(eval CFLAGS_$(obj)+=$(CFLAGS_BASE_DRIVER)))
 
 VPATH += $(SRCDIR)/base
diff --git a/drivers/net/i40e/Makefile b/drivers/net/i40e/Makefile
index 55c79a60a..8040a890e 100644
--- a/drivers/net/i40e/Makefile
+++ b/drivers/net/i40e/Makefile
@@ -78,7 +78,7 @@ endif
 
 CFLAGS_i40e_lan_hmc.o += -Wno-error
 endif
-OBJS_BASE_DRIVER=$(patsubst %.c,%.o,$(notdir $(wildcard $(SRCDIR)/base/*.c)))
+OBJS_BASE_DRIVER=$(sort $(patsubst %.c,%.o,$(notdir $(wildcard $(SRCDIR)/base/*.c))))
 $(foreach obj, $(OBJS_BASE_DRIVER), $(eval CFLAGS_$(obj)+=$(CFLAGS_BASE_DRIVER)))
 
 VPATH += $(SRCDIR)/base
diff --git a/drivers/net/ixgbe/Makefile b/drivers/net/ixgbe/Makefile
index 5e57cb353..d47ce24aa 100644
--- a/drivers/net/ixgbe/Makefile
+++ b/drivers/net/ixgbe/Makefile
@@ -87,7 +87,7 @@ endif
 # Add extra flags for base driver files (also known as shared code)
 # to disable warnings in them
 #
-BASE_DRIVER_OBJS=$(patsubst %.c,%.o,$(notdir $(wildcard $(SRCDIR)/base/*.c)))
+BASE_DRIVER_OBJS=$(sort $(patsubst %.c,%.o,$(notdir $(wildcard $(SRCDIR)/base/*.c))))
 $(foreach obj, $(BASE_DRIVER_OBJS), $(eval CFLAGS_$(obj)+=$(CFLAGS_BASE_DRIVER)))
 
 VPATH += $(SRCDIR)/base
diff --git a/drivers/net/qede/Makefile b/drivers/net/qede/Makefile
index f03441d9a..83ff95474 100644
--- a/drivers/net/qede/Makefile
+++ b/drivers/net/qede/Makefile
@@ -78,7 +78,7 @@ endif
 # to disable warnings in them
 #
 #
-BASE_DRIVER_OBJS=$(patsubst %.c,%.o,$(notdir $(wildcard $(SRCDIR)/base/*.c)))
+BASE_DRIVER_OBJS=$(sort $(patsubst %.c,%.o,$(notdir $(wildcard $(SRCDIR)/base/*.c))))
 $(foreach obj, $(BASE_DRIVER_OBJS), $(eval CFLAGS_$(obj)+=$(CFLAGS_BASE_DRIVER)))
 
 VPATH += $(SRCDIR)/base
diff --git a/drivers/net/sfc/Makefile b/drivers/net/sfc/Makefile
index 57aa963ba..8cfd14d45 100644
--- a/drivers/net/sfc/Makefile
+++ b/drivers/net/sfc/Makefile
@@ -71,7 +71,7 @@ endif
 # List of base driver object files for which
 # special CFLAGS above should be applied
 #
-BASE_DRIVER_OBJS=$(patsubst %.c,%.o,$(notdir $(wildcard $(SRCDIR)/base/*.c)))
+BASE_DRIVER_OBJS=$(sort $(patsubst %.c,%.o,$(notdir $(wildcard $(SRCDIR)/base/*.c))))
 $(foreach obj, $(BASE_DRIVER_OBJS), \
   $(eval CFLAGS_$(obj)+=$(CFLAGS_BASE_DRIVER)))
 
diff --git a/drivers/net/thunderx/Makefile b/drivers/net/thunderx/Makefile
index 915ae945a..a2b6caae2 100644
--- a/drivers/net/thunderx/Makefile
+++ b/drivers/net/thunderx/Makefile
@@ -45,7 +45,7 @@ EXPORT_MAP := rte_pmd_thunderx_nicvf_version.map
 
 LIBABIVER := 1
 
-OBJS_BASE_DRIVER=$(patsubst %.c,%.o,$(notdir $(wildcard $(SRCDIR)/base/*.c)))
+OBJS_BASE_DRIVER=$(sort $(patsubst %.c,%.o,$(notdir $(wildcard $(SRCDIR)/base/*.c))))
 $(foreach obj, $(OBJS_BASE_DRIVER), $(eval CFLAGS_$(obj)+=$(CFLAGS_BASE_DRIVER)))
 
 VPATH += $(SRCDIR)/base
-- 
2.11.0

[dpdk-dev] [PATCH v6 5/6] mk: sort object files when building deps lists

[luca.boccassi]

From: Luca Boccassi <luca.boccassi@gmail.com>

In order to achieve reproducible builds, always use the same
order when listing object files to build dependencies lists.

Signed-off-by: Luca Boccassi <luca.boccassi@gmail.com>
---
 mk/rte.app.mk     | 4 ++--
 mk/rte.hostapp.mk | 4 ++--
 mk/rte.shared.mk  | 4 ++--
 3 files changed, 6 insertions(+), 6 deletions(-)

diff --git a/mk/rte.app.mk b/mk/rte.app.mk
index c25fdd9f5..c5b8c1ead 100644
--- a/mk/rte.app.mk
+++ b/mk/rte.app.mk
@@ -270,8 +270,8 @@ LDLIBS_NAMES += $(patsubst -Wl$(comma)-l%,lib%.a,$(filter -Wl$(comma)-l%,$(LDLIB
 
 # list of found libraries files (useful for deps). If not found, the
 # library is silently ignored and dep won't be checked
-LDLIBS_FILES := $(wildcard $(foreach dir,$(LDLIBS_PATH),\
-	$(addprefix $(dir)/,$(LDLIBS_NAMES))))
+LDLIBS_FILES := $(sort $(wildcard $(foreach dir,$(LDLIBS_PATH),\
+	$(addprefix $(dir)/,$(LDLIBS_NAMES)))))
 
 #
 # Compile executable file if needed
diff --git a/mk/rte.hostapp.mk b/mk/rte.hostapp.mk
index 5cb4909cb..f58173c31 100644
--- a/mk/rte.hostapp.mk
+++ b/mk/rte.hostapp.mk
@@ -69,9 +69,9 @@ O_TO_EXE_DO = @set -e; \
 -include .$(HOSTAPP).cmd
 
 # list of .a files that are linked to this application
-LDLIBS_FILES := $(wildcard \
+LDLIBS_FILES := $(sort $(wildcard \
 	$(addprefix $(RTE_OUTPUT)/lib/, \
-	$(patsubst -l%,lib%.a,$(filter -l%,$(LDLIBS)))))
+	$(patsubst -l%,lib%.a,$(filter -l%,$(LDLIBS))))))
 
 #
 # Compile executable file if needed
diff --git a/mk/rte.shared.mk b/mk/rte.shared.mk
index 87ccf0ba4..4e680bc03 100644
--- a/mk/rte.shared.mk
+++ b/mk/rte.shared.mk
@@ -85,8 +85,8 @@ LDLIBS_NAMES += $(patsubst -Wl$(comma)-l%,lib%.a,$(filter -Wl$(comma)-l%,$(LDLIB
 
 # list of found libraries files (useful for deps). If not found, the
 # library is silently ignored and dep won't be checked
-LDLIBS_FILES := $(wildcard $(foreach dir,$(LDLIBS_PATH),\
-	$(addprefix $(dir)/,$(LDLIBS_NAMES))))
+LDLIBS_FILES := $(sort $(wildcard $(foreach dir,$(LDLIBS_PATH),\
+	$(addprefix $(dir)/,$(LDLIBS_NAMES)))))
 
 #
 # Archive objects in .so file if needed
-- 
2.11.0

[dpdk-dev] [PATCH v6 6/6] mk: set -ISCDIR before -IRTE_OUT/include in CFLAGS

[luca.boccassi]

From: Luca Boccassi <luca.boccassi@gmail.com>

A race condition can happen during parallel builds, where a header
might be installed in RTE_OUT/include before CFLAGS is recursively
expanded. This causes GCC to sometimes pick the header path as
SRCDIR/... and sometimes as RTE_OUT/include/... making the build
unreproducible, as the full path is used for the expansion of
__FILE__ and in the DWARF directory listing.
Always pass -ISRCDIR first to CFLAGS so that it's deterministic.

Signed-off-by: Luca Boccassi <luca.boccassi@gmail.com>
---
 lib/librte_acl/Makefile              | 10 ++++++++--
 lib/librte_bitratestats/Makefile     |  9 ++++++++-
 lib/librte_cmdline/Makefile          |  9 ++++++++-
 lib/librte_distributor/Makefile      | 10 ++++++++--
 lib/librte_eal/linuxapp/eal/Makefile |  8 +++++++-
 lib/librte_efd/Makefile              | 10 ++++++++--
 lib/librte_gro/Makefile              |  9 ++++++++-
 lib/librte_hash/Makefile             | 10 ++++++++--
 lib/librte_ip_frag/Makefile          | 10 ++++++++--
 lib/librte_jobstats/Makefile         | 10 ++++++++--
 lib/librte_kni/Makefile              |  9 ++++++++-
 lib/librte_kvargs/Makefile           |  9 ++++++++-
 lib/librte_latencystats/Makefile     |  9 ++++++++-
 lib/librte_lpm/Makefile              | 10 ++++++++--
 lib/librte_mbuf/Makefile             |  9 ++++++++-
 lib/librte_mempool/Makefile          |  9 ++++++++-
 lib/librte_metrics/Makefile          |  9 ++++++++-
 lib/librte_net/Makefile              |  9 ++++++++-
 lib/librte_pdump/Makefile            | 10 ++++++++--
 lib/librte_power/Makefile            |  9 ++++++++-
 lib/librte_reorder/Makefile          | 10 ++++++++--
 lib/librte_ring/Makefile             |  9 ++++++++-
 lib/librte_timer/Makefile            |  9 ++++++++-
 lib/librte_vhost/Makefile            |  9 ++++++++-
 24 files changed, 191 insertions(+), 33 deletions(-)

diff --git a/lib/librte_acl/Makefile b/lib/librte_acl/Makefile
index 59767920a..bce78813b 100644
--- a/lib/librte_acl/Makefile
+++ b/lib/librte_acl/Makefile
@@ -34,8 +34,14 @@ include $(RTE_SDK)/mk/rte.vars.mk
 # library name
 LIB = librte_acl.a
 
-CFLAGS += -O3
-CFLAGS += $(WERROR_FLAGS) -I$(SRCDIR)
+# Include the source dir first, so that headers paths are always picked
+# from there. By including it last race conditions might happen during paralle
+# builds, and headers might be already installed in RTE_OUT/include when the
+# variable is recursively expanded, thus causing GCC to sometimes use the
+# SRCDIR path and sometimes the RTE_OUT/include, making the builds not
+# reproducible.
+CFLAGS := -I$(SRCDIR) $(CFLAGS)
+CFLAGS += $(WERROR_FLAGS) -O3
 
 EXPORT_MAP := rte_acl_version.map
 
diff --git a/lib/librte_bitratestats/Makefile b/lib/librte_bitratestats/Makefile
index 58a20ea09..26a3f4908 100644
--- a/lib/librte_bitratestats/Makefile
+++ b/lib/librte_bitratestats/Makefile
@@ -34,7 +34,14 @@ include $(RTE_SDK)/mk/rte.vars.mk
 # library name
 LIB = librte_bitratestats.a
 
-CFLAGS += $(WERROR_FLAGS) -I$(SRCDIR) -O3
+# Include the source dir first, so that headers paths are always picked
+# from there. By including it last race conditions might happen during paralle
+# builds, and headers might be already installed in RTE_OUT/include when the
+# variable is recursively expanded, thus causing GCC to sometimes use the
+# SRCDIR path and sometimes the RTE_OUT/include, making the builds not
+# reproducible.
+CFLAGS := -I$(SRCDIR) $(CFLAGS)
+CFLAGS += $(WERROR_FLAGS) -O3
 
 EXPORT_MAP := rte_bitratestats_version.map
 
diff --git a/lib/librte_cmdline/Makefile b/lib/librte_cmdline/Makefile
index 644f68e47..9dd75f2d2 100644
--- a/lib/librte_cmdline/Makefile
+++ b/lib/librte_cmdline/Makefile
@@ -34,7 +34,14 @@ include $(RTE_SDK)/mk/rte.vars.mk
 # library name
 LIB = librte_cmdline.a
 
-CFLAGS += $(WERROR_FLAGS) -I$(SRCDIR) -O3
+# Include the source dir first, so that headers paths are always picked
+# from there. By including it last race conditions might happen during paralle
+# builds, and headers might be already installed in RTE_OUT/include when the
+# variable is recursively expanded, thus causing GCC to sometimes use the
+# SRCDIR path and sometimes the RTE_OUT/include, making the builds not
+# reproducible.
+CFLAGS := -I$(SRCDIR) $(CFLAGS)
+CFLAGS += $(WERROR_FLAGS) -O3
 
 EXPORT_MAP := rte_cmdline_version.map
 
diff --git a/lib/librte_distributor/Makefile b/lib/librte_distributor/Makefile
index b417ee7be..a51d7c8b2 100644
--- a/lib/librte_distributor/Makefile
+++ b/lib/librte_distributor/Makefile
@@ -34,8 +34,14 @@ include $(RTE_SDK)/mk/rte.vars.mk
 # library name
 LIB = librte_distributor.a
 
-CFLAGS += -O3
-CFLAGS += $(WERROR_FLAGS) -I$(SRCDIR)
+# Include the source dir first, so that headers paths are always picked
+# from there. By including it last race conditions might happen during paralle
+# builds, and headers might be already installed in RTE_OUT/include when the
+# variable is recursively expanded, thus causing GCC to sometimes use the
+# SRCDIR path and sometimes the RTE_OUT/include, making the builds not
+# reproducible.
+CFLAGS := -I$(SRCDIR) $(CFLAGS)
+CFLAGS += $(WERROR_FLAGS) -O3
 
 EXPORT_MAP := rte_distributor_version.map
 
diff --git a/lib/librte_eal/linuxapp/eal/Makefile b/lib/librte_eal/linuxapp/eal/Makefile
index 90bca4d68..f5eee1c6a 100644
--- a/lib/librte_eal/linuxapp/eal/Makefile
+++ b/lib/librte_eal/linuxapp/eal/Makefile
@@ -41,7 +41,13 @@ LIBABIVER := 5
 
 VPATH += $(RTE_SDK)/lib/librte_eal/common
 
-CFLAGS += -I$(SRCDIR)/include
+# Include the source dir first, so that headers paths are always picked
+# from there. By including it last race conditions might happen during paralle
+# builds, and headers might be already installed in RTE_OUT/include when the
+# variable is recursively expanded, thus causing GCC to sometimes use the
+# SRCDIR path and sometimes the RTE_OUT/include, making the builds not
+# reproducible.
+CFLAGS := -I$(SRCDIR)/include $(CFLAGS)
 CFLAGS += -I$(RTE_SDK)/lib/librte_eal/common
 CFLAGS += -I$(RTE_SDK)/lib/librte_eal/common/include
 CFLAGS += $(WERROR_FLAGS) -O3
diff --git a/lib/librte_efd/Makefile b/lib/librte_efd/Makefile
index b9277bc5d..b169e3240 100644
--- a/lib/librte_efd/Makefile
+++ b/lib/librte_efd/Makefile
@@ -34,8 +34,14 @@ include $(RTE_SDK)/mk/rte.vars.mk
 # library name
 LIB = librte_efd.a
 
-CFLAGS += -O3
-CFLAGS += $(WERROR_FLAGS) -I$(SRCDIR)
+# Include the source dir first, so that headers paths are always picked
+# from there. By including it last race conditions might happen during paralle
+# builds, and headers might be already installed in RTE_OUT/include when the
+# variable is recursively expanded, thus causing GCC to sometimes use the
+# SRCDIR path and sometimes the RTE_OUT/include, making the builds not
+# reproducible.
+CFLAGS := -I$(SRCDIR) $(CFLAGS)
+CFLAGS += $(WERROR_FLAGS) -O3
 
 EXPORT_MAP := rte_efd_version.map
 
diff --git a/lib/librte_gro/Makefile b/lib/librte_gro/Makefile
index 747eeec9e..4a9f12d85 100644
--- a/lib/librte_gro/Makefile
+++ b/lib/librte_gro/Makefile
@@ -34,8 +34,15 @@ include $(RTE_SDK)/mk/rte.vars.mk
 # library name
 LIB = librte_gro.a
 
+# Include the source dir first, so that headers paths are always picked
+# from there. By including it last race conditions might happen during paralle
+# builds, and headers might be already installed in RTE_OUT/include when the
+# variable is recursively expanded, thus causing GCC to sometimes use the
+# SRCDIR path and sometimes the RTE_OUT/include, making the builds not
+# reproducible.
+CFLAGS := -I$(SRCDIR) $(CFLAGS)
 CFLAGS += -O3
-CFLAGS += $(WERROR_FLAGS) -I$(SRCDIR)
+CFLAGS += $(WERROR_FLAGS)
 
 EXPORT_MAP := rte_gro_version.map
 
diff --git a/lib/librte_hash/Makefile b/lib/librte_hash/Makefile
index 9cf13a045..677d494e7 100644
--- a/lib/librte_hash/Makefile
+++ b/lib/librte_hash/Makefile
@@ -34,8 +34,14 @@ include $(RTE_SDK)/mk/rte.vars.mk
 # library name
 LIB = librte_hash.a
 
-CFLAGS += -O3
-CFLAGS += $(WERROR_FLAGS) -I$(SRCDIR)
+# Include the source dir first, so that headers paths are always picked
+# from there. By including it last race conditions might happen during paralle
+# builds, and headers might be already installed in RTE_OUT/include when the
+# variable is recursively expanded, thus causing GCC to sometimes use the
+# SRCDIR path and sometimes the RTE_OUT/include, making the builds not
+# reproducible.
+CFLAGS := -I$(SRCDIR) $(CFLAGS)
+CFLAGS += $(WERROR_FLAGS) -O3
 
 EXPORT_MAP := rte_hash_version.map
 
diff --git a/lib/librte_ip_frag/Makefile b/lib/librte_ip_frag/Makefile
index 4e693bf8f..de45ec2d3 100644
--- a/lib/librte_ip_frag/Makefile
+++ b/lib/librte_ip_frag/Makefile
@@ -34,8 +34,14 @@ include $(RTE_SDK)/mk/rte.vars.mk
 # library name
 LIB = librte_ip_frag.a
 
-CFLAGS += -O3
-CFLAGS += $(WERROR_FLAGS) -I$(SRCDIR)
+# Include the source dir first, so that headers paths are always picked
+# from there. By including it last race conditions might happen during paralle
+# builds, and headers might be already installed in RTE_OUT/include when the
+# variable is recursively expanded, thus causing GCC to sometimes use the
+# SRCDIR path and sometimes the RTE_OUT/include, making the builds not
+# reproducible.
+CFLAGS := -I$(SRCDIR) $(CFLAGS)
+CFLAGS += $(WERROR_FLAGS) -O3
 
 EXPORT_MAP := rte_ipfrag_version.map
 
diff --git a/lib/librte_jobstats/Makefile b/lib/librte_jobstats/Makefile
index 561a0678c..57329b18e 100644
--- a/lib/librte_jobstats/Makefile
+++ b/lib/librte_jobstats/Makefile
@@ -34,8 +34,14 @@ include $(RTE_SDK)/mk/rte.vars.mk
 # library name
 LIB = librte_jobstats.a
 
-CFLAGS += -O3
-CFLAGS += $(WERROR_FLAGS) -I$(SRCDIR)
+# Include the source dir first, so that headers paths are always picked
+# from there. By including it last race conditions might happen during paralle
+# builds, and headers might be already installed in RTE_OUT/include when the
+# variable is recursively expanded, thus causing GCC to sometimes use the
+# SRCDIR path and sometimes the RTE_OUT/include, making the builds not
+# reproducible.
+CFLAGS := -I$(SRCDIR) $(CFLAGS)
+CFLAGS += $(WERROR_FLAGS) -O3
 
 EXPORT_MAP := rte_jobstats_version.map
 
diff --git a/lib/librte_kni/Makefile b/lib/librte_kni/Makefile
index 70f1ca8f6..130d6fd74 100644
--- a/lib/librte_kni/Makefile
+++ b/lib/librte_kni/Makefile
@@ -34,7 +34,14 @@ include $(RTE_SDK)/mk/rte.vars.mk
 # library name
 LIB = librte_kni.a
 
-CFLAGS += $(WERROR_FLAGS) -I$(SRCDIR) -O3 -fno-strict-aliasing
+# Include the source dir first, so that headers paths are always picked
+# from there. By including it last race conditions might happen during paralle
+# builds, and headers might be already installed in RTE_OUT/include when the
+# variable is recursively expanded, thus causing GCC to sometimes use the
+# SRCDIR path and sometimes the RTE_OUT/include, making the builds not
+# reproducible.
+CFLAGS := -I$(SRCDIR) $(CFLAGS)
+CFLAGS += $(WERROR_FLAGS) -O3 -fno-strict-aliasing
 
 EXPORT_MAP := rte_kni_version.map
 
diff --git a/lib/librte_kvargs/Makefile b/lib/librte_kvargs/Makefile
index 564dd3102..7c332c110 100644
--- a/lib/librte_kvargs/Makefile
+++ b/lib/librte_kvargs/Makefile
@@ -36,7 +36,14 @@ include $(RTE_SDK)/mk/rte.vars.mk
 # library name
 LIB = librte_kvargs.a
 
-CFLAGS += $(WERROR_FLAGS) -I$(SRCDIR) -O3
+# Include the source dir first, so that headers paths are always picked
+# from there. By including it last race conditions might happen during paralle
+# builds, and headers might be already installed in RTE_OUT/include when the
+# variable is recursively expanded, thus causing GCC to sometimes use the
+# SRCDIR path and sometimes the RTE_OUT/include, making the builds not
+# reproducible.
+CFLAGS := -I$(SRCDIR) $(CFLAGS)
+CFLAGS += $(WERROR_FLAGS) -O3
 
 EXPORT_MAP := rte_kvargs_version.map
 
diff --git a/lib/librte_latencystats/Makefile b/lib/librte_latencystats/Makefile
index eaacbb731..46a8ecd34 100644
--- a/lib/librte_latencystats/Makefile
+++ b/lib/librte_latencystats/Makefile
@@ -33,7 +33,14 @@ include $(RTE_SDK)/mk/rte.vars.mk
 # library name
 LIB = librte_latencystats.a
 
-CFLAGS += $(WERROR_FLAGS) -I$(SRCDIR) -O3
+# Include the source dir first, so that headers paths are always picked
+# from there. By including it last race conditions might happen during paralle
+# builds, and headers might be already installed in RTE_OUT/include when the
+# variable is recursively expanded, thus causing GCC to sometimes use the
+# SRCDIR path and sometimes the RTE_OUT/include, making the builds not
+# reproducible.
+CFLAGS := -I$(SRCDIR) $(CFLAGS)
+CFLAGS += $(WERROR_FLAGS) -O3
 LDLIBS += -lm
 LDLIBS += -lpthread
 
diff --git a/lib/librte_lpm/Makefile b/lib/librte_lpm/Makefile
index 32be46b3b..6a97fdc97 100644
--- a/lib/librte_lpm/Makefile
+++ b/lib/librte_lpm/Makefile
@@ -34,8 +34,14 @@ include $(RTE_SDK)/mk/rte.vars.mk
 # library name
 LIB = librte_lpm.a
 
-CFLAGS += -O3
-CFLAGS += $(WERROR_FLAGS) -I$(SRCDIR)
+# Include the source dir first, so that headers paths are always picked
+# from there. By including it last race conditions might happen during paralle
+# builds, and headers might be already installed in RTE_OUT/include when the
+# variable is recursively expanded, thus causing GCC to sometimes use the
+# SRCDIR path and sometimes the RTE_OUT/include, making the builds not
+# reproducible.
+CFLAGS := -I$(SRCDIR) $(CFLAGS)
+CFLAGS += $(WERROR_FLAGS) -O3
 
 EXPORT_MAP := rte_lpm_version.map
 
diff --git a/lib/librte_mbuf/Makefile b/lib/librte_mbuf/Makefile
index 548273054..4d51191ee 100644
--- a/lib/librte_mbuf/Makefile
+++ b/lib/librte_mbuf/Makefile
@@ -34,7 +34,14 @@ include $(RTE_SDK)/mk/rte.vars.mk
 # library name
 LIB = librte_mbuf.a
 
-CFLAGS += $(WERROR_FLAGS) -I$(SRCDIR) -O3
+# Include the source dir first, so that headers paths are always picked
+# from there. By including it last race conditions might happen during paralle
+# builds, and headers might be already installed in RTE_OUT/include when the
+# variable is recursively expanded, thus causing GCC to sometimes use the
+# SRCDIR path and sometimes the RTE_OUT/include, making the builds not
+# reproducible.
+CFLAGS := -I$(SRCDIR) $(CFLAGS)
+CFLAGS += $(WERROR_FLAGS) -O3
 
 EXPORT_MAP := rte_mbuf_version.map
 
diff --git a/lib/librte_mempool/Makefile b/lib/librte_mempool/Makefile
index 7b5bdfee7..33678192f 100644
--- a/lib/librte_mempool/Makefile
+++ b/lib/librte_mempool/Makefile
@@ -34,7 +34,14 @@ include $(RTE_SDK)/mk/rte.vars.mk
 # library name
 LIB = librte_mempool.a
 
-CFLAGS += $(WERROR_FLAGS) -I$(SRCDIR) -O3
+# Include the source dir first, so that headers paths are always picked
+# from there. By including it last race conditions might happen during paralle
+# builds, and headers might be already installed in RTE_OUT/include when the
+# variable is recursively expanded, thus causing GCC to sometimes use the
+# SRCDIR path and sometimes the RTE_OUT/include, making the builds not
+# reproducible.
+CFLAGS := -I$(SRCDIR) $(CFLAGS)
+CFLAGS += $(WERROR_FLAGS) -O3
 
 EXPORT_MAP := rte_mempool_version.map
 
diff --git a/lib/librte_metrics/Makefile b/lib/librte_metrics/Makefile
index d4990e839..195bd4d56 100644
--- a/lib/librte_metrics/Makefile
+++ b/lib/librte_metrics/Makefile
@@ -34,7 +34,14 @@ include $(RTE_SDK)/mk/rte.vars.mk
 # library name
 LIB = librte_metrics.a
 
-CFLAGS += $(WERROR_FLAGS) -I$(SRCDIR) -O3
+# Include the source dir first, so that headers paths are always picked
+# from there. By including it last race conditions might happen during paralle
+# builds, and headers might be already installed in RTE_OUT/include when the
+# variable is recursively expanded, thus causing GCC to sometimes use the
+# SRCDIR path and sometimes the RTE_OUT/include, making the builds not
+# reproducible.
+CFLAGS := -I$(SRCDIR) $(CFLAGS)
+CFLAGS += $(WERROR_FLAGS) -O3
 
 EXPORT_MAP := rte_metrics_version.map
 
diff --git a/lib/librte_net/Makefile b/lib/librte_net/Makefile
index 56727c4df..a08a7dd8b 100644
--- a/lib/librte_net/Makefile
+++ b/lib/librte_net/Makefile
@@ -33,7 +33,14 @@ include $(RTE_SDK)/mk/rte.vars.mk
 
 LIB = librte_net.a
 
-CFLAGS += $(WERROR_FLAGS) -I$(SRCDIR) -O3
+# Include the source dir first, so that headers paths are always picked
+# from there. By including it last race conditions might happen during paralle
+# builds, and headers might be already installed in RTE_OUT/include when the
+# variable is recursively expanded, thus causing GCC to sometimes use the
+# SRCDIR path and sometimes the RTE_OUT/include, making the builds not
+# reproducible.
+CFLAGS := -I$(SRCDIR) $(CFLAGS)
+CFLAGS += $(WERROR_FLAGS) -O3
 
 EXPORT_MAP := rte_net_version.map
 LIBABIVER := 1
diff --git a/lib/librte_pdump/Makefile b/lib/librte_pdump/Makefile
index 1c03bcbb7..8923d7499 100644
--- a/lib/librte_pdump/Makefile
+++ b/lib/librte_pdump/Makefile
@@ -34,8 +34,14 @@ include $(RTE_SDK)/mk/rte.vars.mk
 # library name
 LIB = librte_pdump.a
 
-CFLAGS += $(WERROR_FLAGS) -I$(SRCDIR) -O3
-CFLAGS += -D_GNU_SOURCE
+# Include the source dir first, so that headers paths are always picked
+# from there. By including it last race conditions might happen during paralle
+# builds, and headers might be already installed in RTE_OUT/include when the
+# variable is recursively expanded, thus causing GCC to sometimes use the
+# SRCDIR path and sometimes the RTE_OUT/include, making the builds not
+# reproducible.
+CFLAGS := -I$(SRCDIR) $(CFLAGS)
+CFLAGS += $(WERROR_FLAGS) -O3 -D_GNU_SOURCE
 LDLIBS += -lpthread
 
 EXPORT_MAP := rte_pdump_version.map
diff --git a/lib/librte_power/Makefile b/lib/librte_power/Makefile
index 06cd10e86..b76ad689e 100644
--- a/lib/librte_power/Makefile
+++ b/lib/librte_power/Makefile
@@ -34,7 +34,14 @@ include $(RTE_SDK)/mk/rte.vars.mk
 # library name
 LIB = librte_power.a
 
-CFLAGS += $(WERROR_FLAGS) -I$(SRCDIR) -O3 -fno-strict-aliasing
+# Include the source dir first, so that headers paths are always picked
+# from there. By including it last race conditions might happen during paralle
+# builds, and headers might be already installed in RTE_OUT/include when the
+# variable is recursively expanded, thus causing GCC to sometimes use the
+# SRCDIR path and sometimes the RTE_OUT/include, making the builds not
+# reproducible.
+CFLAGS := -I$(SRCDIR) $(CFLAGS)
+CFLAGS += $(WERROR_FLAGS) -O3 -fno-strict-aliasing
 
 EXPORT_MAP := rte_power_version.map
 
diff --git a/lib/librte_reorder/Makefile b/lib/librte_reorder/Makefile
index 4e44e72f0..51b5d490d 100644
--- a/lib/librte_reorder/Makefile
+++ b/lib/librte_reorder/Makefile
@@ -34,8 +34,14 @@ include $(RTE_SDK)/mk/rte.vars.mk
 # library name
 LIB = librte_reorder.a
 
-CFLAGS += -O3
-CFLAGS += $(WERROR_FLAGS) -I$(SRCDIR)
+# Include the source dir first, so that headers paths are always picked
+# from there. By including it last race conditions might happen during paralle
+# builds, and headers might be already installed in RTE_OUT/include when the
+# variable is recursively expanded, thus causing GCC to sometimes use the
+# SRCDIR path and sometimes the RTE_OUT/include, making the builds not
+# reproducible.
+CFLAGS := -I$(SRCDIR) $(CFLAGS)
+CFLAGS += $(WERROR_FLAGS) -O3
 
 EXPORT_MAP := rte_reorder_version.map
 
diff --git a/lib/librte_ring/Makefile b/lib/librte_ring/Makefile
index 3e2f4b873..e5248a2c1 100644
--- a/lib/librte_ring/Makefile
+++ b/lib/librte_ring/Makefile
@@ -34,7 +34,14 @@ include $(RTE_SDK)/mk/rte.vars.mk
 # library name
 LIB = librte_ring.a
 
-CFLAGS += $(WERROR_FLAGS) -I$(SRCDIR) -O3
+# Include the source dir first, so that headers paths are always picked
+# from there. By including it last race conditions might happen during paralle
+# builds, and headers might be already installed in RTE_OUT/include when the
+# variable is recursively expanded, thus causing GCC to sometimes use the
+# SRCDIR path and sometimes the RTE_OUT/include, making the builds not
+# reproducible.
+CFLAGS := -I$(SRCDIR) $(CFLAGS)
+CFLAGS += $(WERROR_FLAGS) -O3
 
 EXPORT_MAP := rte_ring_version.map
 
diff --git a/lib/librte_timer/Makefile b/lib/librte_timer/Makefile
index 03a15390e..0c57cc6d3 100644
--- a/lib/librte_timer/Makefile
+++ b/lib/librte_timer/Makefile
@@ -34,7 +34,14 @@ include $(RTE_SDK)/mk/rte.vars.mk
 # library name
 LIB = librte_timer.a
 
-CFLAGS += $(WERROR_FLAGS) -I$(SRCDIR) -O3
+# Include the source dir first, so that headers paths are always picked
+# from there. By including it last race conditions might happen during parallel
+# builds, and headers might be already installed in RTE_OUT/include when the
+# variable is recursively expanded, thus causing GCC to sometimes use the
+# SRCDIR path and sometimes the RTE_OUT/include, making the builds not
+# reproducible.
+CFLAGS := -I$(SRCDIR) $(CFLAGS)
+CFLAGS += $(WERROR_FLAGS) -O3
 
 EXPORT_MAP := rte_timer_version.map
 
diff --git a/lib/librte_vhost/Makefile b/lib/librte_vhost/Makefile
index 4a116fe31..ef0d86990 100644
--- a/lib/librte_vhost/Makefile
+++ b/lib/librte_vhost/Makefile
@@ -38,7 +38,14 @@ EXPORT_MAP := rte_vhost_version.map
 
 LIBABIVER := 4
 
-CFLAGS += $(WERROR_FLAGS) -I$(SRCDIR) -O3 -D_FILE_OFFSET_BITS=64
+# Include the source dir first, so that headers paths are always picked
+# from there. By including it last race conditions might happen during parallel
+# builds, and headers might be already installed in RTE_OUT/include when the
+# variable is recursively expanded, thus causing GCC to sometimes use the
+# SRCDIR path and sometimes the RTE_OUT/include, making the builds not
+# reproducible.
+CFLAGS := -I$(SRCDIR) $(CFLAGS)
+CFLAGS += $(WERROR_FLAGS) -O3 -D_FILE_OFFSET_BITS=64
 CFLAGS += -I vhost_user
 LDLIBS += -lpthread
 
-- 
2.11.0