From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mga07.intel.com (mga07.intel.com [134.134.136.100]) by dpdk.org (Postfix) with ESMTP id 43B3CFE5 for ; Wed, 6 Sep 2017 12:45:12 +0200 (CEST) Received: from fmsmga006.fm.intel.com ([10.253.24.20]) by orsmga105.jf.intel.com with ESMTP; 06 Sep 2017 03:45:11 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.41,483,1498546800"; d="scan'208";a="148704602" Received: from bricha3-mobl3.ger.corp.intel.com ([10.237.221.24]) by fmsmga006.fm.intel.com with SMTP; 06 Sep 2017 03:45:09 -0700 Received: by (sSMTP sendmail emulation); Wed, 06 Sep 2017 11:45:07 +0100 Date: Wed, 6 Sep 2017 11:45:07 +0100 From: Bruce Richardson To: Luca Boccassi Cc: Stephen Hemminger , dev@dpdk.org, ferruh.yigit@intel.com Message-ID: <20170906104507.GA27220@bricha3-MOBL3.ger.corp.intel.com> References: <20170905205159.8606-1-luca.boccassi@gmail.com> <20170905140055.0ae224f1@xeon-e3> <1504646129.20044.2.camel@gmail.com> <20170906084658.GA24204@bricha3-MOBL3.ger.corp.intel.com> <1504689034.2725.0.camel@gmail.com> MIME-Version: 1.0 Content-Type: text/plain; charset=iso-8859-1 Content-Disposition: inline Content-Transfer-Encoding: 8bit In-Reply-To: <1504689034.2725.0.camel@gmail.com> Organization: Intel Research and =?iso-8859-1?Q?De=ACvel?= =?iso-8859-1?Q?opment?= Ireland Ltd. User-Agent: Mutt/1.8.3 (2017-05-23) Subject: Re: [dpdk-dev] [PATCH v7 0/6] Reproducible build X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 06 Sep 2017 10:45:12 -0000 On Wed, Sep 06, 2017 at 10:10:34AM +0100, Luca Boccassi wrote: > On Wed, 2017-09-06 at 09:46 +0100, Bruce Richardson wrote: > > On Tue, Sep 05, 2017 at 10:15:29PM +0100, Luca Boccassi wrote: > > > On Tue, 2017-09-05 at 14:00 -0700, Stephen Hemminger wrote: > > > > On Tue,  5 Sep 2017 21:51:53 +0100 > > > > luca.boccassi@gmail.com wrote: > > > > > > > > > From: Luca Boccassi > > > > > > > > > > In the past couple of years a concerted effort among almost all > > > > > Linux > > > > > distros has been striving toward achieving reproducible builds. > > > > > [1] > > > > > This involves changes to the toolchain, new tools and CI > > > > > systems. > > > > > [2] > > > > > > > > > > v1 fixed the documentation, examples and linker script > > > > > generation. > > > > > v2 fixes all problems, which were caused by unstable order of > > > > > headers > > > > > inclusion, source files listing and object file listing when > > > > > passing > > > > > them to the compiler. > > > > > DPDK's build, at least with the default configuration, is fully > > > > > reproducible with this patch series as tested by the > > > > > Reproducible > > > > > Builds developers experimental toolchain. [3] > > > > > > > > > > v3 restores the first patch, which was eaten by git send-email. > > > > > > > > > > v4 drops the patch that reorders rebuilds, and adds a patch to > > > > > make > > > > > the inclusion of headers deterministic with regards to GCC > > > > > embedding > > > > > the full file path when expading __FILE__ and when writing the > > > > > directory listing in the DWARF objects. > > > > > It also drops the first 2 patches which have already been > > > > > merged. > > > > > > > > > > v5 adds the -I$(SRCDIR) workaround to librte_eal linuxapp's and > > > > > librte_gro's Makefiles. > > > > > > > > > > v6 fixes copypasta added in v5 - librte-gro's Makefile should > > > > > have > > > > > used > > > > > $(SRCDIR) rather than $(SRCDIR)/include. > > > > > > > > > > v7 dropped the big patch to change the CFLAGS in all libraries > > > > > Makefiles > > > > > and instead uses a much simpler approach suggested by Ferruh > > > > > (thanks!), > > > > > which consists in installing the library public headers > > > > > symlinks > > > > > before > > > > > building. Although the drawback is that the DWARF directory > > > > > listing > > > > > will always include the user configured build output directory, > > > > > it > > > > > is > > > > > a much smaller patch that will be rendered obsolete by the move > > > > > to > > > > > Meson > > > > > anyway. > > > > > > > > > > [1] https://reproducible-builds.org/ > > > > > [2] https://reproducible-builds.org/tools/ > > > > > [3] https://wiki.debian.org/ReproducibleBuilds/ExperimentalTool > > > > > chai > > > > > n#Us > > > > > > > > > > Luca Boccassi (6): > > > > >   mk: sort list of shared objects in linker script > > > > >   mk: sort list of files in examples.dox > > > > >   mk: sort headers before wildcard inclusion > > > > >   mk: sort source files before passing them to the compiler > > > > >   mk: sort object files when building deps lists > > > > >   mk: install symlinks before build step > > > > > > > > > >  drivers/net/cxgbe/Makefile                                 | 2 > > > > > +- > > > > >  drivers/net/e1000/Makefile                                 | 2 > > > > > +- > > > > >  drivers/net/fm10k/Makefile                                 | 2 > > > > > +- > > > > >  drivers/net/i40e/Makefile                                  | 2 > > > > > +- > > > > >  drivers/net/ixgbe/Makefile                                 | 2 > > > > > +- > > > > >  drivers/net/qede/Makefile                                  | 2 > > > > > +- > > > > >  drivers/net/sfc/Makefile                                   | 2 > > > > > +- > > > > >  drivers/net/thunderx/Makefile                              | 2 > > > > > +- > > > > >  examples/ip_pipeline/Makefile                              | 2 > > > > > +- > > > > >  examples/multi_process/client_server_mp/mp_server/Makefile | 2 > > > > > +- > > > > >  examples/server_node_efd/server/Makefile                   | 2 > > > > > +- > > > > >  lib/librte_eal/common/Makefile                             | 2 > > > > > +- > > > > >  mk/rte.app.mk                                              | 4 > > > > > ++- > > > > > - > > > > >  mk/rte.combinedlib.mk                                      | 2 > > > > > +- > > > > >  mk/rte.hostapp.mk                                          | 4 > > > > > ++- > > > > > - > > > > >  mk/rte.lib.mk                                              | 7 > > > > > ++++++- > > > > >  mk/rte.sdkdoc.mk                                           | 2 > > > > > +- > > > > >  mk/rte.shared.mk                                           | 4 > > > > > ++- > > > > > - > > > > >  18 files changed, 26 insertions(+), 21 deletions(-) > > > > > > > > > > > > > How does this interact with propose new build process. > > > > Hate to have do everything twice. > > > > > > Most of these patches, mainly for the custom bits that discover the > > > source/headers files on the system (all the wildcards, etc) will > > > not be > > > necessary any longer. > > > > > > The documentation build is not there yet in the patchset, so I'll > > > check > > > once that's available. > > > > > > But I'm still testing the Meson patches, I had issues integrating > > > it in > > > the package builds as Meson support is still quite new in > > > debhelper, I > > > hope to finish that up next week and give feedback to Bruce. > > > > > > The reproducible build testing infra I've got is picky and eats > > > only > > > deb packages so until I get that up and running I cannot test the > > > reproducibility using Meson unfortunately. > > > > > > --  > > > > It's also safer to assume that any replacement build system is going > > to > > take a while longer to complete, so it's as well to consider these > > changes for integration, since they are not that large of patches. > > I agree :-) > > > Also, looking forward to your feedback on the meson build patches. I > > was > > going to do a v2 with minor updates, but I'll hold off until next > > week > > when I get your comments. > > > > /Bruce > > Ok, but please bear in mind that I'll be travelling from today to > Monday, so it will probably be end of the week. So if needed feel free > to send v2 already. Sorry for the delay! > Ok, thanks for the info. I'll see how it goes and how many other changes I might have to make. The only major issue right now is the one that you already flagged about removing the check for a Kconfig file in the kernel. /Bruce