From: Luca Boccassi <lboccass@Brocade.com>
To: "thomas@monjalon.net" <thomas@monjalon.net>
Cc: "dev@dpdk.org" <dev@dpdk.org>
Subject: Re: [dpdk-dev] [PATCH v3 7/8] mk: sort object files when building deps lists
Date: Tue, 27 Jun 2017 14:51:38 +0000 [thread overview]
Message-ID: <1498575098.16925.5.camel@brocade.com> (raw)
In-Reply-To: <2009930.OQrImQK4oz@xps>
On Tue, 2017-06-27 at 15:52 +0200, Thomas Monjalon wrote:
> 27/06/2017 12:43, Luca Boccassi:
> > On Tue, 2017-06-27 at 01:20 +0200, Thomas Monjalon wrote:
> > > 23/06/2017 20:41, lboccass@brocade.com:
> > > > From: Luca Boccassi <luca.boccassi@gmail.com>
> > > >
> > > > In order to achieve reproducible builds, always use the same
> > > > order when listing object files to build dependencies lists.
> > > >
> > > > Signed-off-by: Luca Boccassi <luca.boccassi@gmail.com>
> > > > ---
> > > > mk/rte.app.mk | 4 ++--
> > > > mk/rte.hostapp.mk | 4 ++--
> > > > mk/rte.shared.mk | 4 ++--
> > > > 3 files changed, 6 insertions(+), 6 deletions(-)
> > > >
> > > > --- a/mk/rte.app.mk
> > > > +++ b/mk/rte.app.mk
> > > > @@ -263,8 +263,8 @@ LDLIBS_NAMES += $(patsubst -Wl$(comma)-
> > > > l%,lib%.a,$(filter -Wl$(comma)-l%,$(LDLIB
> > > >
> > > > # list of found libraries files (useful for deps). If not
> > > > found,
> > > > the
> > > > # library is silently ignored and dep won't be checked
> > > > -LDLIBS_FILES := $(wildcard $(foreach dir,$(LDLIBS_PATH),\
> > > > - $(addprefix $(dir)/,$(LDLIBS_NAMES))))
> > > > +LDLIBS_FILES := $(sort $(wildcard $(foreach
> > > > dir,$(LDLIBS_PATH),\
> > > > + $(addprefix $(dir)/,$(LDLIBS_NAMES)))))
> > >
> > > You cannot sort libraries.
> > > Check - for instance - this comment above in this file:
> > > # Eliminate duplicates without sorting, only keep the last
> > > occurrence
> > > filter-libs = \
> >
> > Not sure I follow - what's the reason for avoiding to sort the list
> > of
> > libs to link against?
>
> Sorry, the ordering issue is with LDLIBS not LDLIBS_FILES.
>
> > > Why sorting them?
> > > What is random in libraries list?
> >
> > The issue is that the output of wildcard is not fully
> > deterministic. It
> > can depend on the filesystem, and even on the locale settings [1].
> > Before GNU Make 3.82 (2009) it used to automatically sort the
> > output,
> > but that was removed (to make it faster... sigh). [2]
>
> It is not a true wildcard here. It is just filtering files which
> do not exist.
> I think you do not need this patch for deterministic build.
But then those lists are passed down in the .SECONDEXPANSION rule
right?
I am trying to find out an alternative solution. The problem to solve
is that the build system picks the public headers path (which is
embedded in the object files as notation and in the debug symbol) from
a seemingly random location between the make install path and the
source path (build/include/rte_foo.h vs lib/librte_foo/rte_foo.h) and
this makes the build not reproducible.
Nonetheless, while I work more on the last 4 patches, could you please
have a look and eventually take patch 3 and 4? They are needed to
respectively have a deterministic list of files in the libdpdk.so
linker script and a list of source files in one of the example
documentation files.
Thanks!
--
Kind regards,
Luca Boccassi
next prev parent reply other threads:[~2017-06-27 14:53 UTC|newest]
Thread overview: 65+ messages / expand[flat|nested] mbox.gz Atom feed top
2017-06-22 13:58 [dpdk-dev] [PATCH 0/4] reproducible builds - docs and linker script lboccass
2017-06-22 13:58 ` [dpdk-dev] [PATCH 1/4] mk: use make silent flag to print HTML doc version lboccass
2017-06-22 13:58 ` [dpdk-dev] [PATCH 2/4] mk: fix excluding .doctrees when installing docs lboccass
2017-06-22 13:58 ` [dpdk-dev] [PATCH 3/4] mk: sort list of shared objects in linker script lboccass
2017-06-22 13:58 ` [dpdk-dev] [PATCH 4/4] mk: sort list of files in examples.dox lboccass
2017-06-23 18:16 ` [dpdk-dev] [PATCH v2 0/7] Reproducible build lboccass
2017-06-23 18:16 ` [dpdk-dev] [PATCH v2 1/7] mk: fix excluding .doctrees when installing docs lboccass
2017-06-23 18:16 ` [dpdk-dev] [PATCH v2 2/7] mk: sort list of shared objects in linker script lboccass
2017-06-23 18:16 ` [dpdk-dev] [PATCH v2 3/7] mk: sort list of files in examples.dox lboccass
2017-06-23 18:16 ` [dpdk-dev] [PATCH v2 4/7] mk: sort headers before wildcard inclusion lboccass
2017-06-23 18:33 ` [dpdk-dev] [PATCH v2 5/7] mk: sort source files before passing them to the compiler lboccass
2017-06-23 18:33 ` [dpdk-dev] [PATCH v2 6/7] mk: sort object files when building deps lists lboccass
2017-06-23 18:33 ` [dpdk-dev] [PATCH v2 7/7] mk: always rebuild in the same order lboccass
2017-06-23 18:41 ` [dpdk-dev] [PATCH v2 0/7] Reproducible build Luca Boccassi
2017-06-23 18:41 ` [dpdk-dev] [PATCH v3 0/8] " lboccass
2017-06-23 18:41 ` [dpdk-dev] [PATCH v3 1/8] mk: use make silent flag to print HTML doc version lboccass
2017-06-26 14:45 ` Mcnamara, John
2017-06-23 18:41 ` [dpdk-dev] [PATCH v3 2/8] mk: fix excluding .doctrees when installing docs lboccass
2017-06-26 14:46 ` Mcnamara, John
2017-06-23 18:41 ` [dpdk-dev] [PATCH v3 3/8] mk: sort list of shared objects in linker script lboccass
2017-06-23 18:41 ` [dpdk-dev] [PATCH v3 4/8] mk: sort list of files in examples.dox lboccass
2017-06-26 14:48 ` Mcnamara, John
2017-06-26 14:56 ` Mcnamara, John
2017-06-23 18:41 ` [dpdk-dev] [PATCH v3 5/8] mk: sort headers before wildcard inclusion lboccass
2017-06-26 14:49 ` Mcnamara, John
2017-06-23 18:41 ` [dpdk-dev] [PATCH v3 6/8] mk: sort source files before passing them to the compiler lboccass
2017-06-23 18:41 ` [dpdk-dev] [PATCH v3 7/8] mk: sort object files when building deps lists lboccass
2017-06-26 23:20 ` Thomas Monjalon
2017-06-27 10:43 ` Luca Boccassi
2017-06-27 13:52 ` Thomas Monjalon
2017-06-27 14:51 ` Luca Boccassi [this message]
2017-06-27 16:14 ` Thomas Monjalon
2017-06-28 14:07 ` Luca Boccassi
2017-06-28 14:37 ` Thomas Monjalon
2017-06-28 14:49 ` Luca Boccassi
2017-06-23 18:41 ` [dpdk-dev] [PATCH v3 8/8] mk: always rebuild in the same order lboccass
2017-06-26 23:22 ` Thomas Monjalon
2017-06-27 10:36 ` Luca Boccassi
2017-06-26 22:11 ` [dpdk-dev] [PATCH v3 0/8] Reproducible build Thomas Monjalon
2017-06-26 23:15 ` Thomas Monjalon
2017-06-28 13:56 ` [dpdk-dev] [PATCH v4 0/6] " lboccass
2017-06-28 13:56 ` [dpdk-dev] [PATCH v4 1/6] mk: sort list of shared objects in linker script lboccass
2017-06-28 13:56 ` [dpdk-dev] [PATCH v4 2/6] mk: sort list of files in examples.dox lboccass
2017-06-28 13:56 ` [dpdk-dev] [PATCH v4 3/6] mk: sort headers before wildcard inclusion lboccass
2017-06-28 13:57 ` [dpdk-dev] [PATCH v4 4/6] mk: sort source files before passing them to the compiler lboccass
2017-06-28 13:57 ` [dpdk-dev] [PATCH v4 5/6] mk: sort object files when building deps lists lboccass
2017-06-28 13:57 ` [dpdk-dev] [PATCH v4 6/6] mk: set -ISCDIR before -IRTE_OUT/include in CFLAGS lboccass
2017-06-28 15:57 ` [dpdk-dev] [PATCH v4 0/6] Reproducible build Stephen Hemminger
2017-06-28 16:04 ` Bruce Richardson
2017-06-28 17:52 ` Luca Boccassi
2017-08-11 12:43 ` Luca Boccassi
2017-08-10 18:23 ` [dpdk-dev] [PATCH v5 " luca.boccassi
2017-08-10 18:23 ` [dpdk-dev] [PATCH v5 1/6] mk: sort list of shared objects in linker script luca.boccassi
2017-08-10 18:23 ` [dpdk-dev] [PATCH v5 2/6] mk: sort list of files in examples.dox luca.boccassi
2017-08-10 18:23 ` [dpdk-dev] [PATCH v5 3/6] mk: sort headers before wildcard inclusion luca.boccassi
2017-08-10 18:23 ` [dpdk-dev] [PATCH v5 4/6] mk: sort source files before passing them to the compiler luca.boccassi
2017-08-10 18:23 ` [dpdk-dev] [PATCH v5 5/6] mk: sort object files when building deps lists luca.boccassi
2017-08-10 18:23 ` [dpdk-dev] [PATCH v5 6/6] mk: set -ISCDIR before -IRTE_OUT/include in CFLAGS luca.boccassi
2017-08-18 11:03 ` [dpdk-dev] [PATCH v6 0/6] Reproducible build luca.boccassi
2017-08-18 11:03 ` [dpdk-dev] [PATCH v6 1/6] mk: sort list of shared objects in linker script luca.boccassi
2017-08-18 11:03 ` [dpdk-dev] [PATCH v6 2/6] mk: sort list of files in examples.dox luca.boccassi
2017-08-18 11:03 ` [dpdk-dev] [PATCH v6 3/6] mk: sort headers before wildcard inclusion luca.boccassi
2017-08-18 11:03 ` [dpdk-dev] [PATCH v6 4/6] mk: sort source files before passing them to the compiler luca.boccassi
2017-08-18 11:03 ` [dpdk-dev] [PATCH v6 5/6] mk: sort object files when building deps lists luca.boccassi
2017-08-18 11:03 ` [dpdk-dev] [PATCH v6 6/6] mk: set -ISCDIR before -IRTE_OUT/include in CFLAGS luca.boccassi
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=1498575098.16925.5.camel@brocade.com \
--to=lboccass@brocade.com \
--cc=dev@dpdk.org \
--cc=thomas@monjalon.net \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).