From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mga02.intel.com (mga02.intel.com [134.134.136.20]) by dpdk.org (Postfix) with ESMTP id 9A05F2BD8 for ; Wed, 7 Jun 2017 15:09:29 +0200 (CEST) Received: from orsmga003.jf.intel.com ([10.7.209.27]) by orsmga101.jf.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 07 Jun 2017 06:09:02 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.39,311,1493708400"; d="scan'208";a="977869055" Received: from irsmsx102.ger.corp.intel.com ([163.33.3.155]) by orsmga003.jf.intel.com with ESMTP; 07 Jun 2017 06:09:01 -0700 Received: from irsmsx101.ger.corp.intel.com ([169.254.1.242]) by IRSMSX102.ger.corp.intel.com ([169.254.2.87]) with mapi id 14.03.0319.002; Wed, 7 Jun 2017 14:09:00 +0100 From: "Van Haaren, Harry" To: "dev@dpdk.org" CC: "Richardson, Bruce" , "christian.ehrhardt@canonical.com" , "luca.boccassi@gmail.com" Thread-Topic: [dpdk-dev] [RFC PATCH] replace DPDK config and build system Thread-Index: AQHS33+RXTxb/uKeL0eIqKQp0BOgzaIZU3+A Date: Wed, 7 Jun 2017 13:08:59 +0000 Message-ID: References: <20170607104743.275149-1-bruce.richardson@intel.com> In-Reply-To: <20170607104743.275149-1-bruce.richardson@intel.com> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-titus-metadata-40: eyJDYXRlZ29yeUxhYmVscyI6IiIsIk1ldGFkYXRhIjp7Im5zIjoiaHR0cDpcL1wvd3d3LnRpdHVzLmNvbVwvbnNcL0ludGVsMyIsImlkIjoiNGU1NzhkMGQtOGI4ZC00ZWVkLWFhMGYtMDFlNDVjNDczNDhhIiwicHJvcHMiOlt7Im4iOiJDVFBDbGFzc2lmaWNhdGlvbiIsInZhbHMiOlt7InZhbHVlIjoiQ1RQX1BVQkxJQyJ9XX1dfSwiU3ViamVjdExhYmVscyI6W10sIlRNQ1ZlcnNpb24iOiIxNS45LjYuNiIsIlRydXN0ZWRMYWJlbEhhc2giOiJVYnlvYUtPcFArNG5sTVwvUTgwa3h6UDlkNTM3Vk5ncG5qcHF5dzJ3MkZhZz0ifQ== x-ctpclassification: CTP_PUBLIC dlp-product: dlpe-windows dlp-version: 10.0.102.7 dlp-reaction: no-action x-originating-ip: [163.33.239.182] Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Subject: Re: [dpdk-dev] [RFC PATCH] replace DPDK config and build system X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 07 Jun 2017 13:09:30 -0000 > From: dev [mailto:dev-bounces@dpdk.org] On Behalf Of Bruce Richardson > Sent: Wednesday, June 7, 2017 11:48 AM > To: dev@dpdk.org > Cc: Richardson, Bruce > Subject: [dpdk-dev] [RFC PATCH] replace DPDK config and build system >=20 > Hi all, >=20 > following on from the pressing need to add support in DPDK for detecting > and managing external dependencies, I undertook to see what options we ha= d. > However, unrelated to this, over time, I have become increasingly > frustrated by the complexity of the DPDK configuration and build system. = As > such, I feel that looking at some of the newer build tools that are out > there might give us the additional functionality we want, along with othe= r > benefits. As such, I undertook a prototype using "meson"[1] for > configuration, which uses "ninja" as a backend for doing the actual build= . >=20 > With these tools we get the following benefits (not a complete list): > * support for detecting dependencies on the system > * support for detecting compiler features, including functions, defines > * improved maintainability through a high-level language, which gives > decent error messages including line numbers (!!) > * co-existence with the existing makefile system without making any chang= es > to it > * faster builds using ninja - on my many-core system, the builds seem > significantly faster than with our existing system. Especially in the > nothing-has-changed case, builds with my prototype return instantly, > compared to taking a few seconds to recursively check each directory wi= th > the current build system > * the ability to switch to using a standard "ninja" + "ninja install" set= up > * the chance to rework our existing build-config files, and hopefully > pretty much remove them. > * pkg-config support. > * we get to move away from our bespoke build system > * dependencies in each lib can be moved back to being tracked in the libs > files themselves, not up a level >=20 >=20 > Of course, it's not a panacea, but having spent hours on the prototype th= us > far, I find working with meson and ninja far more user-friendly than > working on our makefiles, and again the build speed is a really nice > improvment too. >=20 > The prototype is incomplete, but it does build a reasonable number of our > libraries, some unit tests, the i40e PMD and the testpmd binary, and I ha= ve > successfully passed traffic using testpmd from the build. Some things are > not fully correct, e.g. static builds aren't working right now, as I have= n't > correctly done all the dependency tracking, I think, and the cpu flag > detection has issues. It also has only been tried on x86_64 linux, on a > couple of systems, so YMMV. However, I feel it's a reasonable enough star= t > point to show what we might be able to achieve. >=20 > Please take the prototype and test it out. I think it's a better > alternative to trying to bolt on additional functionality to our existing > config and build system. Test drive done; here are my experiences / thoughts: 1) Understanding the Meson build files is much easier for me than the curre= nt build-files. I'll admit that bash scripting is not my forte, with the ca= veat that if my basic bash scripting doesn't suffice, others in the communi= ty are probably in a similar position. 2) I see huge value in pkg-config integration - linking made easy. From a d= eveloper usability POV, a project should provide a pkg-config file, and con= figure itself based on the pkg-config files available on the system.=20 3) The speed of ninja is impressive - particularly in the small-amounts-of-= work case. Try it on your own machine if you don't believe me :) 4) "build variants" are super easy, meson build && meson build_variant wil= l create two directories, with .o files contained under it (same as one pos= sible usage of the current system - just calling out that that feature rema= ins). 5) Vim users, there is a "mesonic" plugin[1] which provides syntax highligh= ting for Meson files, the Meson Options file, and Meson integration to the = Vim compiler hooks. Aka typing :make will cause Ninja to be invoked and = compile. There are more fancy features, check the script page. 6) Ninja uninstall will reverse a previous installation, and returns a clea= n system. Nice for testing and integrating DPDK with applications that expe= ct DPDK to be installed system-wide. Of course no build system is perfect - and I'd guess Meson also has a few g= ritty details that are not ideal - but to me this test-drive has shown prom= ise. Opinions from non-developers may be of particular interest.. @DPDK Packagers, do you have experience with Meson? Does it integrate well = with (your) existing infrastructure? Hope you don't mind me adding you to C= C :) -Harry [1] http://www.vim.org/scripts/script.php?script_id=3D5378