From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from dpdk.org (dpdk.org [92.243.14.124]) by dpdk.space (Postfix) with ESMTP id 82F09A045E for ; Thu, 30 May 2019 12:03:12 +0200 (CEST) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id D7F585680; Thu, 30 May 2019 12:03:10 +0200 (CEST) Received: from mailout1.w1.samsung.com (mailout1.w1.samsung.com [210.118.77.11]) by dpdk.org (Postfix) with ESMTP id EF205375B for ; Thu, 30 May 2019 12:03:08 +0200 (CEST) Received: from eucas1p2.samsung.com (unknown [182.198.249.207]) by mailout1.w1.samsung.com (KnoxPortal) with ESMTP id 20190530100307euoutp019c882a2595db1444b7dbbe8c7b77f4a7~jbcTFXL_10488304883euoutp01J for ; Thu, 30 May 2019 10:03:07 +0000 (GMT) DKIM-Filter: OpenDKIM Filter v2.11.0 mailout1.w1.samsung.com 20190530100307euoutp019c882a2595db1444b7dbbe8c7b77f4a7~jbcTFXL_10488304883euoutp01J DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=samsung.com; s=mail20170921; t=1559210587; bh=h9Vj3fMr4G7P3LUTJzcavt88VhVyt+3TevFVIQtERNY=; h=Subject:To:Cc:From:Date:In-Reply-To:References:From; b=uvEtUQL0PgEnRAXh1DC7GwB9u94anmlDzYFp5jUxIQVCb7TUskMz0KKHZAok3B3cl Qg95JkO9N/w5wIPrDzkuy7xeQnE3wRhCueGT5g5ck0rBJGq7+jOVKaiOOy4/42pSKw PZC+r/kQAQ6AyDlZPWV/3j0aDi9LVmh1TUW442yA= Received: from eusmges2new.samsung.com (unknown [203.254.199.244]) by eucas1p2.samsung.com (KnoxPortal) with ESMTP id 20190530100307eucas1p2a2e7eedaaf4b3d7d2dccec7bfb30cccb~jbcSlXFzz1122811228eucas1p2j; Thu, 30 May 2019 10:03:07 +0000 (GMT) Received: from eucas1p2.samsung.com ( [182.198.249.207]) by eusmges2new.samsung.com (EUCPMTA) with SMTP id 77.30.04377.B5AAFEC5; Thu, 30 May 2019 11:03:07 +0100 (BST) Received: from eusmtrp2.samsung.com (unknown [182.198.249.139]) by eucas1p1.samsung.com (KnoxPortal) with ESMTPA id 20190530100306eucas1p1c471a62c059bc2b005e199c1cfd1e8f7~jbcRvfcX42260622606eucas1p1M; Thu, 30 May 2019 10:03:06 +0000 (GMT) Received: from eusmgms1.samsung.com (unknown [182.198.249.179]) by eusmtrp2.samsung.com (KnoxPortal) with ESMTP id 20190530100306eusmtrp25f836c1478082862e0036b1def2af675~jbcRf2qBc3003330033eusmtrp2G; Thu, 30 May 2019 10:03:06 +0000 (GMT) X-AuditID: cbfec7f4-12dff70000001119-8b-5cefaa5be9de Received: from eusmtip2.samsung.com ( [203.254.199.222]) by eusmgms1.samsung.com (EUCPMTA) with SMTP id 09.49.04146.A5AAFEC5; Thu, 30 May 2019 11:03:06 +0100 (BST) Received: from [106.109.129.180] (unknown [106.109.129.180]) by eusmtip2.samsung.com (KnoxPortal) with ESMTPA id 20190530100305eusmtip237e0c2da88bea18dd2bc4e91c2985181~jbcQ_Yjf_0453204532eusmtip2s; Thu, 30 May 2019 10:03:05 +0000 (GMT) To: Luca Boccassi , dev@dpdk.org, Thomas Monjalon Cc: Bruce Richardson , Aaron Conole , Kevin Traynor From: Ilya Maximets Message-ID: <0cc060dc-36e6-88ea-491e-e3b9c552bd7a@samsung.com> Date: Thu, 30 May 2019 13:03:04 +0300 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:60.0) Gecko/20100101 Thunderbird/60.7.0 MIME-Version: 1.0 In-Reply-To: <1a07d1cd59d84dce84e56c10fdabf5e5504560a6.camel@debian.org> Content-Language: en-GB Content-Transfer-Encoding: 7bit X-Brightmail-Tracker: H4sIAAAAAAAAA+NgFlrPKsWRmVeSWpSXmKPExsWy7djP87rRq97HGPy/z2Tx680Ddos5TevY LW6ssrd492k7k8WV9p/sFmsPfWC3+PTgBIsDu8evtrnMHr8WLGX1WLznJZPHsZvT2D3e77vK 5tG3ZRVjAFsUl01Kak5mWWqRvl0CV8a1+bfYC97pV/TO283UwHhKpYuRk0NCwETi159vzF2M XBxCAisYJf4cusoG4XxhlHjc84MFwvnMKPHm6Xt2mJbNj0+yQiSWM0ocenWSCcL5yCjxbuk1 sCphASeJh5NABnNyiAhES7S8OwoWZxaolrg8fwqYzSagI3Fq9RFGEJtXwE5i9Y8DLCA2i4Cq RMuUBiYQW1QgQuLLzk1QNYISJ2c+AavhFPCQ2LO6gRViprhE05eVULa8xPa3c8AekhDYxi5x cOEiRoizXSS+nljPBmELS7w6vgXqHRmJ/zvnM0HY9RL3W14yQjR3MEpMP/QPKmEvseX1OaAG DqANmhLrd+mDmBICjhIzu90hTD6JG28FIU7gk5i0bTozRJhXoqNNCGKGisTvg8uZIWwpiZvv PrNPYFSaheSxWUiemYXkmVkIaxcwsqxiFE8tLc5NTy02ykst1ytOzC0uzUvXS87P3cQITEan /x3/soNx15+kQ4wCHIxKPLwT8t/FCLEmlhVX5h5ilOBgVhLh/bkcKMSbklhZlVqUH19UmpNa fIhRmoNFSZy3muFBtJBAemJJanZqakFqEUyWiYNTqoGRMfjhscJDV5e7bt6o8d787650J2PP 43mTeycWfudo+8jxmb8vNy1OdTufKU/99EtyF54L39m74Kr/thNL5rmErj6Xt+zKzcDfp4wZ kvVFTYsjHNvZglzjD/1iLFZ/8nr7Gy3ez4VM/D5zLL0uzTp5bcq3IxJeASvYMkPYt+w/lfbi 3f9+A9ZLSizFGYmGWsxFxYkA/WXTAkIDAAA= X-Brightmail-Tracker: H4sIAAAAAAAAA+NgFjrMIsWRmVeSWpSXmKPExsVy+t/xe7pRq97HGOxZxWrx680Ddos5TevY LW6ssrd492k7k8WV9p/sFmsPfWC3+PTgBIsDu8evtrnMHr8WLGX1WLznJZPHsZvT2D3e77vK 5tG3ZRVjAFuUnk1RfmlJqkJGfnGJrVK0oYWRnqGlhZ6RiaWeobF5rJWRqZK+nU1Kak5mWWqR vl2CXsa1+bfYC97pV/TO283UwHhKpYuRk0NCwERi8+OTrF2MXBxCAksZJfqv3meFSEhJ/Ph1 AcoWlvhzrYsNoug9o8Sx9e9ZQBLCAk4SDyd9YwaxRQSiJb42XQdrYBaolvh1uZ8FouE/o8Sp 7/fYQBJsAjoSp1YfYQSxeQXsJFb/OAA2iEVAVaJlSgMTiC0qECExe1cDC0SNoMTJmU/AbE4B D4k9qxugFqhL/Jl3iRnCFpdo+rISKi4vsf3tHOYJjEKzkLTPQtIyC0nLLCQtCxhZVjGKpJYW 56bnFhvqFSfmFpfmpesl5+duYgRG4LZjPzfvYLy0MfgQowAHoxIP74T8dzFCrIllxZW5hxgl OJiVRHh/LgcK8aYkVlalFuXHF5XmpBYfYjQFem4is5Rocj4wOeSVxBuaGppbWBqaG5sbm1ko ifN2CByMERJITyxJzU5NLUgtgulj4uCUamAU+naht7RE+dwezrM6h9NWSZ//P7+/RlBOctOG 424eYddnNvq+fhT4fqYIr6lwr90SJ+vrfxmSt8UEfllUyBqnF66m5Tnnx5ra6rkCyom6e5qd 7pjK/dCoe7V++nr26QZKFw4pJqT5T2qNntk/P36X2oG82vpN+4JutUhWJW49FuCSaBu7+akS S3FGoqEWc1FxIgDGQU2r1gIAAA== X-CMS-MailID: 20190530100306eucas1p1c471a62c059bc2b005e199c1cfd1e8f7 X-Msg-Generator: CA Content-Type: text/plain; charset="utf-8" X-RootMTR: 20190529164009eucas1p289f1dcf87012ecf049efc8eee2c2ea9d X-EPHeader: CA CMS-TYPE: 201P X-CMS-RootMailID: 20190529164009eucas1p289f1dcf87012ecf049efc8eee2c2ea9d References: <20190529163958.30796-1-i.maximets@samsung.com> <20190529163958.30796-3-i.maximets@samsung.com> <1a07d1cd59d84dce84e56c10fdabf5e5504560a6.camel@debian.org> Subject: Re: [dpdk-dev] [PATCH 2/2] meson: make build configurable X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" On 29.05.2019 23:37, Luca Boccassi wrote: > On Wed, 2019-05-29 at 19:39 +0300, Ilya Maximets wrote: >> The first thing many developers do before start building DPDK is >> disabling all the not needed divers and libraries. This happens >> just because more than a half of DPDK dirvers and libraries are not >> needed for the particular reason. For example, you don't need >> dpaa*, octeon*, various croypto devices, eventdev, etc. if you're >> only want to build OVS for x86_64 with static linking. >> >> By disabling everything you don't need, build speeds up literally 10x >> times. This is important for CI systems. For example, TravisCI wastes >> 10 minutes for the default DPDK build just to check linking with OVS. >> >> Another thing is the binary size. Number of DPDK libraries and, >> as a result, size of resulted statically linked application decreases >> significantly. >> >> Important thing also that you're able to not install some >> dependencies >> if you don't have them on a target platform. Just disable >> libs/drivers >> that depends on it. Similar thing for the glibc version mismatch >> between build and target platforms. >> >> Also, I have to note that less code means less probability of >> failures and less number of attack vectors. >> >> This patch gives 'meson' the power of configurability that we >> have with 'make'. Using new options it's possible to enable just >> what you need and nothing more. >> >> For example, following cmdline could be used to build almost minimal >> set of DPDK libs and drivers to check OVS build: >> >> $ meson build -Dexamples='' -Dtests=false -Denable_kmods=false \ >> -Ddrivers_bus=pci,vdev \ >> -Ddrivers_mempool=ring \ >> -Ddrivers_net=null,virtio,ring \ >> -Ddrivers_crypto=virtio \ >> -Ddrivers_compress=none \ >> -Ddrivers_event=none \ >> -Ddrivers_baseband=none \ >> -Ddrivers_raw=none \ >> -Ddrivers_common=none \ >> >> -Dlibs=kvargs,eal,cmdline,ring,mempool,mbuf,net,meter,\ >> ethdev,pci,hash,cryptodev,pdump,vhost \ >> -Dapps=none >> >> Adding a few real net drivers will give configuration that can be >> used >> in production environment. >> >> Looks not very pretty, but this could be moved to a script. >> >> Build details: >> >> Build targets in project: 57 >> >> $ time ninja >> real 0m11,528s >> user 1m4,137s >> sys 0m4,935s >> >> $ du -sh ../dpdk_meson_install/ >> 3,5M ../dpdk_meson_install/ >> >> To compare with what we have without these options: >> >> $ meson build -Dexamples='' -Dtests=false -Denable_kmods=false >> Build targets in project: 434 >> >> $ time ninja >> real 1m38,963s >> user 10m18,624s >> sys 0m45,478s >> >> $ du -sh ../dpdk_meson_install/ >> 27M ../dpdk_meson_install/ >> >> 10x speed up for the user time. >> 7.7 times size decrease. >> >> This is probably not much user-friendly because it's not a Kconfig >> and dependency tracking in meson is really poor, so it requires >> usually few iterations to pick correct set of libraries to satisfy >> all dependencies. However, it's not a big deal. Options intended >> for a proficient users who knows what they need. > > Hi, > > We talked about this a few times in the past, and it was actually one > of the design goals to _avoid_ replicating the octopus-like config > system of the makefiles. That's because it makes the test matrix > insanely complicated, not to mention the harm to user friendliness, > among other things. > > If someone doesn't want to use a PMD, they can just avoid installing it > - it's simple enough. So how can I do this? I don't think 'ninja install' has such option. Also, if you think that it is safe to skip some libs/drivers in installation process, it must be safe to not build them at all. It's just a waste of time and computational resources to build something known to be not used. And if you're going to ship DPDK libraries separately in distros, you'll have to test their different combinations anyway. If they're so independent that you don't need to test them in various combinations, than your point about test matrix is not valid. > > Sorry, but from me it's a very strong NACK. Sorry, but let me disagree with you. For me, meson configurability is the essential thing to have in terms of deprecating the 'make' build system. DPDK was and keeps being (in most cases) the library that users statically linking to a single application built for particular platform and not using for anything else. This means that user in most cases knows which parts needed and which parts will never be used. Current meson build system doesn't allow to disable anything forcing users to link with the whole bunch of unused code. One major case is that you have to have build environment equal to your target platform in terms of availability of external libraries. So, if I have some external library on build system, meson will build all the modules it depends from and will link them to my application. As a result I'll not be able to run my application on a target platform without installing additional dependencies which is not acceptable. This patch will allow to specifically disable all the libs that has unsatisfiable dependencies on target. Without the patch it's required to manually remove resulted libs and fix pkg-config and stuff before building apps. This is far less user-friendly than options I proposed. And yes, I still have to waste time for building libraries I'll remove right after. While testing OVS on TravisCI, DPDK was built far more than 30K times which is more than half of a year of a wasted computational resources (if we'll count 10 minutes per build). I think this time could be used more wisely. Best regards, Ilya Maximets.