Re: [dpdk-dev] [RFC PATCH] replace DPDK config and build system

[Neil Horman <nhorman@tuxdriver.com>]

On Wed, Jun 07, 2017 at 11:47:42AM +0100, Bruce Richardson wrote:
> Hi all,
> 
> following on from the pressing need to add support in DPDK for detecting
> and managing external dependencies, I undertook to see what options we had.
> However, unrelated to this, over time, I have become increasingly
> frustrated by the complexity of the DPDK configuration and build system. As
> such, I feel that looking at some of the newer build tools that are out
> there might give us the additional functionality we want, along with other
> benefits. As such, I undertook a prototype using "meson"[1] for
> configuration, which uses "ninja" as a backend for doing the actual build.
> 
> With these tools we get the following benefits (not a complete list):
> * support for detecting dependencies on the system
> * support for detecting compiler features, including functions, defines
> * improved maintainability through a high-level language, which gives
>   decent error messages including line numbers (!!)
> * co-existence with the existing makefile system without making any changes
>   to it
> * faster builds using ninja - on my many-core system, the builds seem
>   significantly faster than with our existing system. Especially in the
>   nothing-has-changed case, builds with my prototype return instantly,
>   compared to taking a few seconds to recursively check each directory with
>   the current build system
> * the ability to switch to using a standard "ninja" + "ninja install" setup
> * the chance to rework our existing build-config files, and hopefully
>   pretty much remove them.
> * pkg-config support.
> * we get to move away from our bespoke build system
> * dependencies in each lib can be moved back to being tracked in the libs
>   files themselves, not up a level
> 
> 
> Of course, it's not a panacea, but having spent hours on the prototype thus
> far, I find working with meson and ninja far more user-friendly than
> working on our makefiles, and again the build speed is a really nice
> improvment too.
> 
> The prototype is incomplete, but it does build a reasonable number of our
> libraries, some unit tests, the i40e PMD and the testpmd binary, and I have
> successfully passed traffic using testpmd from the build. Some things are
> not fully correct, e.g. static builds aren't working right now, as I haven't
> correctly done all the dependency tracking, I think, and the cpu flag
> detection has issues. It also has only been tried on x86_64 linux, on a
> couple of systems, so YMMV. However, I feel it's a reasonable enough start
> point to show what we might be able to achieve.
> 
> Please take the prototype and test it out. I think it's a better
> alternative to trying to bolt on additional functionality to our existing
> config and build system.
> 
> [1] http://mesonbuild.com/

A few notes:

1) It is quick for a native build, which is nice

2) Its seemingly much more clear to understand than the current Makefile system
(which is horribly opaque).  That said, its still not a particularly common
build system.  I wonder if there wouldn't be better reception to just cleaning
up the existing Makefiles, with something like autoconf.   That likely still
won't be as fast as ninja, but it will be common and well understood, and still
reasonably fast.

3) The tools that are needed here, aren't universally available.  I can speak to
RHEL specifically which doesn't have ninja currently, nor meson (they are
available in EPEL, but that can't be used for official builds). Its not an
insurmountable problem, but its something to consider when considering product
reach, and the impact this will have on distributions.

4) I'm a bit lost as to how inheritance works in meson (or if it exists at all).
All the examples you provide here seem to duplicate work.  For instance, several
libraries independently check  if dpdk_conf.has('RTE_MACHINE_CPUFLAG_SSE4_1')
and set proper cflags accordingly, rather than inheriting the flag from a higher
level.  While that in and of itself isn't a singular problem, it could well be a
problem at scale for more complex operations (see below)

5) I know you said this was just an incomplete prototype, which is fine, but I
noticed that the build it produces doesn't yield any pmd exported info (of the
type you would get using dpdk-pmdinfo.py).  Thats because the building of that
info requires a multi-stage build process, wherein .o files are post-processed
to generate new .c files, which are then compiled and linked into the final .so
or .a file.  While I think I see how you might use the gen_src and custom_target
rules to make that happen, the (supposed) lack of inheritance noted above, could
lead to the duplication of alot of fairly complex rules that would have to be
distributed to every pmd in the system.  Thats something to consider.

6) Integration with other build systems.  We build some kernel modules in dpdk
that likely need to remain as part of the make build system (they don't have to,
I suppose, but it would seem like alot of effort to change that).  Is using
something non-make going to fit there?

Just a few thoughts.  All in all the speed is nice, but I think theres alot to
consider before dumping Make.  Not so much because Make is great, but because
its common, and well understood, and integrated with other projects.

Neil