From: Luca Boccassi <lboccass@Brocade.com>
To: "stephen@networkplumber.org" <stephen@networkplumber.org>,
"bruce.richardson@intel.com" <bruce.richardson@intel.com>
Cc: "dev@dpdk.org" <dev@dpdk.org>
Subject: Re: [dpdk-dev] [PATCH v4 0/6] Reproducible build
Date: Wed, 28 Jun 2017 17:52:18 +0000 [thread overview]
Message-ID: <1498672339.5956.2.camel@brocade.com> (raw)
In-Reply-To: <20170628160409.GA7876@bricha3-MOBL3.ger.corp.intel.com>
On Wed, 2017-06-28 at 17:04 +0100, Bruce Richardson wrote:
> On Wed, Jun 28, 2017 at 08:57:33AM -0700, Stephen Hemminger wrote:
> > On Wed, 28 Jun 2017 14:56:56 +0100
> > <lboccass@brocade.com> wrote:
> >
> > > From: Luca Boccassi <luca.boccassi@gmail.com>
> > >
> > > In the past couple of years a concerted effort among almost all
> > > Linux
> > > distros has been striving toward achieving reproducible builds.
> > > [1]
> > > This involves changes to the toolchain, new tools and CI systems.
> > > [2]
> > >
> > > v1 fixed the documentation, examples and linker script
> > > generation.
> > > v2 fixes all problems, which were caused by unstable order of
> > > headers
> > > inclusion, source files listing and object file listing when
> > > passing
> > > them to the compiler.
> > > DPDK's build, at least with the default configuration, is fully
> > > reproducible with this patch series as tested by the Reproducible
> > > Builds developers experimental toolchain. [3]
> > >
> > > v3 restores the first patch, which was eaten by git send-email.
> > >
> > > v4 drops the patch that reorders rebuilds, and adds a patch to
> > > make
> > > the inclusion of headers deterministic with regards to GCC
> > > embedding
> > > the full file path when expading __FILE__ and when writing the
> > > directory listing in the DWARF objects.
> > > It also drops the first 2 patches which have already been merged.
> > >
> > > [1] https://urldefense.proofpoint.com/v2/url?u=https-3A__reproduc
> > > ible-
> > > 2Dbuilds.org_&d=DwIBAg&c=IL_XqQWOjubgfqINi2jTzg&r=QTEM8ICX7t_SLgW
> > > P3qPWtKiwKMps487LPWQx-
> > > B9AqIc&m=HlHnLJOJRokDA9x9IHxyhvyPOnXIakHUFgofHP7STEQ&s=ibn1k95lSK
> > > CcbF9_9Gy4SpcLwLHaTD0aBr-Gm5uuCFQ&e=
> > > [2] https://urldefense.proofpoint.com/v2/url?u=https-3A__reproduc
> > > ible-
> > > 2Dbuilds.org_tools_&d=DwIBAg&c=IL_XqQWOjubgfqINi2jTzg&r=QTEM8ICX7
> > > t_SLgWP3qPWtKiwKMps487LPWQx-
> > > B9AqIc&m=HlHnLJOJRokDA9x9IHxyhvyPOnXIakHUFgofHP7STEQ&s=u4leWgQouu
> > > DTOMv9EReHQMsopArSu9Xqc1uKe0cCqS0&e=
> > > [3] https://urldefense.proofpoint.com/v2/url?u=https-3A__wiki.deb
> > > ian.org_ReproducibleBuilds_ExperimentalToolchain-
> > > 23Us&d=DwIBAg&c=IL_XqQWOjubgfqINi2jTzg&r=QTEM8ICX7t_SLgWP3qPWtKiw
> > > KMps487LPWQx-
> > > B9AqIc&m=HlHnLJOJRokDA9x9IHxyhvyPOnXIakHUFgofHP7STEQ&s=I1iNIvHcZX
> > > YecfQKFo5HtwyhF7jogVTb-lGzBf0wHFg&e=
> >
> > Looks good.
> >
> > Looking ahead, how does this work with the proposed new build
> > system?
> > Is there an automated way to check new submissions so that new
> > features
> > don't undo this.
> >
> >
> > Acked-by: Stephen Hemminger <stephen@networkplumber.org>
>
> https://urldefense.proofpoint.com/v2/url?u=http-3A__mesonbuild.com_Re
> producible-
> 2Dbuilds.html&d=DwIBAg&c=IL_XqQWOjubgfqINi2jTzg&r=QTEM8ICX7t_SLgWP3qP
> WtKiwKMps487LPWQx-
> B9AqIc&m=HlHnLJOJRokDA9x9IHxyhvyPOnXIakHUFgofHP7STEQ&s=0jXWXi5GtPcmXP
> RIXTNEoD1RhY54ig_799tJp4azboE&e=
>
> I'd hope if we switch build system, this shouldn't be a problem. It's
> definitely something to watch out for.
>
> /Bruce
Meson claims support for reproducible builds by default:
http://mesonbuild.com/Reproducible-builds.html
I am not very familiar with it, but I assume it means stable
input/output to makefiles (list of sources/headers/objects etc).
The most common cause for unreproducibility is usually embedding build
timestamps - it was not the case for DPDK, but it might happen in the
future, and it's worth keeping in mind.
In general anything that uses find or wildcard to build lists (as
opposed as a boolean - if <wildcard> exists; then) might cause
problems.
In terms of testing, Debian has setup a pretty good automated CI
system:
https://tests.reproducible-builds.org/debian/reproducible.html
DPDK's page:
https://tests.reproducible-builds.org/debian/rb-pkg/unstable/amd64/dpdk
.html
Every time we do an upload it will be tested there.
--
Kind regards,
Luca Boccassi
next prev parent reply other threads:[~2017-06-28 18:04 UTC|newest]
Thread overview: 65+ messages / expand[flat|nested] mbox.gz Atom feed top
2017-06-22 13:58 [dpdk-dev] [PATCH 0/4] reproducible builds - docs and linker script lboccass
2017-06-22 13:58 ` [dpdk-dev] [PATCH 1/4] mk: use make silent flag to print HTML doc version lboccass
2017-06-22 13:58 ` [dpdk-dev] [PATCH 2/4] mk: fix excluding .doctrees when installing docs lboccass
2017-06-22 13:58 ` [dpdk-dev] [PATCH 3/4] mk: sort list of shared objects in linker script lboccass
2017-06-22 13:58 ` [dpdk-dev] [PATCH 4/4] mk: sort list of files in examples.dox lboccass
2017-06-23 18:16 ` [dpdk-dev] [PATCH v2 0/7] Reproducible build lboccass
2017-06-23 18:16 ` [dpdk-dev] [PATCH v2 1/7] mk: fix excluding .doctrees when installing docs lboccass
2017-06-23 18:16 ` [dpdk-dev] [PATCH v2 2/7] mk: sort list of shared objects in linker script lboccass
2017-06-23 18:16 ` [dpdk-dev] [PATCH v2 3/7] mk: sort list of files in examples.dox lboccass
2017-06-23 18:16 ` [dpdk-dev] [PATCH v2 4/7] mk: sort headers before wildcard inclusion lboccass
2017-06-23 18:33 ` [dpdk-dev] [PATCH v2 5/7] mk: sort source files before passing them to the compiler lboccass
2017-06-23 18:33 ` [dpdk-dev] [PATCH v2 6/7] mk: sort object files when building deps lists lboccass
2017-06-23 18:33 ` [dpdk-dev] [PATCH v2 7/7] mk: always rebuild in the same order lboccass
2017-06-23 18:41 ` [dpdk-dev] [PATCH v2 0/7] Reproducible build Luca Boccassi
2017-06-23 18:41 ` [dpdk-dev] [PATCH v3 0/8] " lboccass
2017-06-23 18:41 ` [dpdk-dev] [PATCH v3 1/8] mk: use make silent flag to print HTML doc version lboccass
2017-06-26 14:45 ` Mcnamara, John
2017-06-23 18:41 ` [dpdk-dev] [PATCH v3 2/8] mk: fix excluding .doctrees when installing docs lboccass
2017-06-26 14:46 ` Mcnamara, John
2017-06-23 18:41 ` [dpdk-dev] [PATCH v3 3/8] mk: sort list of shared objects in linker script lboccass
2017-06-23 18:41 ` [dpdk-dev] [PATCH v3 4/8] mk: sort list of files in examples.dox lboccass
2017-06-26 14:48 ` Mcnamara, John
2017-06-26 14:56 ` Mcnamara, John
2017-06-23 18:41 ` [dpdk-dev] [PATCH v3 5/8] mk: sort headers before wildcard inclusion lboccass
2017-06-26 14:49 ` Mcnamara, John
2017-06-23 18:41 ` [dpdk-dev] [PATCH v3 6/8] mk: sort source files before passing them to the compiler lboccass
2017-06-23 18:41 ` [dpdk-dev] [PATCH v3 7/8] mk: sort object files when building deps lists lboccass
2017-06-26 23:20 ` Thomas Monjalon
2017-06-27 10:43 ` Luca Boccassi
2017-06-27 13:52 ` Thomas Monjalon
2017-06-27 14:51 ` Luca Boccassi
2017-06-27 16:14 ` Thomas Monjalon
2017-06-28 14:07 ` Luca Boccassi
2017-06-28 14:37 ` Thomas Monjalon
2017-06-28 14:49 ` Luca Boccassi
2017-06-23 18:41 ` [dpdk-dev] [PATCH v3 8/8] mk: always rebuild in the same order lboccass
2017-06-26 23:22 ` Thomas Monjalon
2017-06-27 10:36 ` Luca Boccassi
2017-06-26 22:11 ` [dpdk-dev] [PATCH v3 0/8] Reproducible build Thomas Monjalon
2017-06-26 23:15 ` Thomas Monjalon
2017-06-28 13:56 ` [dpdk-dev] [PATCH v4 0/6] " lboccass
2017-06-28 13:56 ` [dpdk-dev] [PATCH v4 1/6] mk: sort list of shared objects in linker script lboccass
2017-06-28 13:56 ` [dpdk-dev] [PATCH v4 2/6] mk: sort list of files in examples.dox lboccass
2017-06-28 13:56 ` [dpdk-dev] [PATCH v4 3/6] mk: sort headers before wildcard inclusion lboccass
2017-06-28 13:57 ` [dpdk-dev] [PATCH v4 4/6] mk: sort source files before passing them to the compiler lboccass
2017-06-28 13:57 ` [dpdk-dev] [PATCH v4 5/6] mk: sort object files when building deps lists lboccass
2017-06-28 13:57 ` [dpdk-dev] [PATCH v4 6/6] mk: set -ISCDIR before -IRTE_OUT/include in CFLAGS lboccass
2017-06-28 15:57 ` [dpdk-dev] [PATCH v4 0/6] Reproducible build Stephen Hemminger
2017-06-28 16:04 ` Bruce Richardson
2017-06-28 17:52 ` Luca Boccassi [this message]
2017-08-11 12:43 ` Luca Boccassi
2017-08-10 18:23 ` [dpdk-dev] [PATCH v5 " luca.boccassi
2017-08-10 18:23 ` [dpdk-dev] [PATCH v5 1/6] mk: sort list of shared objects in linker script luca.boccassi
2017-08-10 18:23 ` [dpdk-dev] [PATCH v5 2/6] mk: sort list of files in examples.dox luca.boccassi
2017-08-10 18:23 ` [dpdk-dev] [PATCH v5 3/6] mk: sort headers before wildcard inclusion luca.boccassi
2017-08-10 18:23 ` [dpdk-dev] [PATCH v5 4/6] mk: sort source files before passing them to the compiler luca.boccassi
2017-08-10 18:23 ` [dpdk-dev] [PATCH v5 5/6] mk: sort object files when building deps lists luca.boccassi
2017-08-10 18:23 ` [dpdk-dev] [PATCH v5 6/6] mk: set -ISCDIR before -IRTE_OUT/include in CFLAGS luca.boccassi
2017-08-18 11:03 ` [dpdk-dev] [PATCH v6 0/6] Reproducible build luca.boccassi
2017-08-18 11:03 ` [dpdk-dev] [PATCH v6 1/6] mk: sort list of shared objects in linker script luca.boccassi
2017-08-18 11:03 ` [dpdk-dev] [PATCH v6 2/6] mk: sort list of files in examples.dox luca.boccassi
2017-08-18 11:03 ` [dpdk-dev] [PATCH v6 3/6] mk: sort headers before wildcard inclusion luca.boccassi
2017-08-18 11:03 ` [dpdk-dev] [PATCH v6 4/6] mk: sort source files before passing them to the compiler luca.boccassi
2017-08-18 11:03 ` [dpdk-dev] [PATCH v6 5/6] mk: sort object files when building deps lists luca.boccassi
2017-08-18 11:03 ` [dpdk-dev] [PATCH v6 6/6] mk: set -ISCDIR before -IRTE_OUT/include in CFLAGS luca.boccassi
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=1498672339.5956.2.camel@brocade.com \
--to=lboccass@brocade.com \
--cc=bruce.richardson@intel.com \
--cc=dev@dpdk.org \
--cc=stephen@networkplumber.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).