From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from dpdk.org (dpdk.org [92.243.14.124]) by dpdk.space (Postfix) with ESMTP id 24823A045E for ; Wed, 29 May 2019 22:15:18 +0200 (CEST) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id DF0121B953; Wed, 29 May 2019 22:15:13 +0200 (CEST) Received: from mx1.redhat.com (mx1.redhat.com [209.132.183.28]) by dpdk.org (Postfix) with ESMTP id 38F601B951 for ; Wed, 29 May 2019 22:15:12 +0200 (CEST) Received: from smtp.corp.redhat.com (int-mx05.intmail.prod.int.phx2.redhat.com [10.5.11.15]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 746CDC0AEE40; Wed, 29 May 2019 20:15:06 +0000 (UTC) Received: from localhost.localdomain (unknown [10.18.25.8]) by smtp.corp.redhat.com (Postfix) with ESMTP id 457E87A409; Wed, 29 May 2019 20:15:03 +0000 (UTC) To: Ilya Maximets , dev@dpdk.org, Thomas Monjalon Cc: Bruce Richardson , Aaron Conole , Kevin Traynor References: <20190529163958.30796-1-i.maximets@samsung.com> <20190529163958.30796-3-i.maximets@samsung.com> From: Michael Santana Francisco Organization: Red Hat Message-ID: Date: Wed, 29 May 2019 16:15:02 -0400 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:60.0) Gecko/20100101 Thunderbird/60.4.0 MIME-Version: 1.0 In-Reply-To: <20190529163958.30796-3-i.maximets@samsung.com> Content-Language: en-US X-Scanned-By: MIMEDefang 2.79 on 10.5.11.15 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.32]); Wed, 29 May 2019 20:15:11 +0000 (UTC) Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: 7bit X-Content-Filtered-By: Mailman/MimeDel 2.1.15 Subject: Re: [dpdk-dev] [PATCH 2/2] meson: make build configurable X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list Reply-To: msantana@redhat.com List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" On 5/29/19 12:39 PM, Ilya Maximets wrote: > The first thing many developers do before start building DPDK is > disabling all the not needed divers and libraries. This happens > just because more than a half of DPDK dirvers and libraries are not > needed for the particular reason. For example, you don't need > dpaa*, octeon*, various croypto devices, eventdev, etc. if you're > only want to build OVS for x86_64 with static linking. > > By disabling everything you don't need, build speeds up literally 10x > times. This is important for CI systems. For example, TravisCI wastes > 10 minutes for the default DPDK build just to check linking with OVS. > > Another thing is the binary size. Number of DPDK libraries and, > as a result, size of resulted statically linked application decreases > significantly. > > Important thing also that you're able to not install some dependencies > if you don't have them on a target platform. Just disable libs/drivers > that depends on it. Similar thing for the glibc version mismatch > between build and target platforms. > > Also, I have to note that less code means less probability of > failures and less number of attack vectors. > > This patch gives 'meson' the power of configurability that we > have with 'make'. Using new options it's possible to enable just > what you need and nothing more. > > For example, following cmdline could be used to build almost minimal > set of DPDK libs and drivers to check OVS build: > > $ meson build -Dexamples='' -Dtests=false -Denable_kmods=false \ > -Ddrivers_bus=pci,vdev \ > -Ddrivers_mempool=ring \ > -Ddrivers_net=null,virtio,ring \ > -Ddrivers_crypto=virtio \ > -Ddrivers_compress=none \ > -Ddrivers_event=none \ > -Ddrivers_baseband=none \ > -Ddrivers_raw=none \ > -Ddrivers_common=none \ > -Dlibs=kvargs,eal,cmdline,ring,mempool,mbuf,net,meter,\ > ethdev,pci,hash,cryptodev,pdump,vhost \ > -Dapps=none > > Adding a few real net drivers will give configuration that can be used > in production environment. > > Looks not very pretty, but this could be moved to a script. > > Build details: > > Build targets in project: 57 > > $ time ninja > real 0m11,528s > user 1m4,137s > sys 0m4,935s > > $ du -sh ../dpdk_meson_install/ > 3,5M ../dpdk_meson_install/ > > To compare with what we have without these options: > > $ meson build -Dexamples='' -Dtests=false -Denable_kmods=false > Build targets in project: 434 > > $ time ninja > real 1m38,963s > user 10m18,624s > sys 0m45,478s > > $ du -sh ../dpdk_meson_install/ > 27M ../dpdk_meson_install/ > > 10x speed up for the user time. > 7.7 times size decrease. > > This is probably not much user-friendly because it's not a Kconfig > and dependency tracking in meson is really poor, so it requires > usually few iterations to pick correct set of libraries to satisfy > all dependencies. However, it's not a big deal. Options intended > for a proficient users who knows what they need. > > Signed-off-by: Ilya Maximets > --- > app/meson.build | 5 +++++ > drivers/baseband/meson.build | 5 +++++ > drivers/bus/meson.build | 6 ++++++ > drivers/common/meson.build | 6 ++++++ > drivers/compress/meson.build | 5 +++++ > drivers/crypto/meson.build | 5 +++++ > drivers/event/meson.build | 6 ++++++ > drivers/mempool/meson.build | 6 ++++++ > drivers/net/meson.build | 6 ++++++ > drivers/raw/meson.build | 6 ++++++ > lib/meson.build | 5 +++++ > meson_options.txt | 22 ++++++++++++++++++++++ > 12 files changed, 83 insertions(+) > > diff --git a/app/meson.build b/app/meson.build > index 2b9fdef74..48972954c 100644 > --- a/app/meson.build > +++ b/app/meson.build > @@ -17,6 +17,11 @@ apps = [ > 'test-pipeline', > 'test-pmd'] > > +enabled_apps = get_option('apps') > +if enabled_apps != 'all' > + apps = (enabled_apps == 'none') ? [] : enabled_apps.split(',') > +endif > + > # for BSD only > lib_execinfo = cc.find_library('execinfo', required: false) > > diff --git a/drivers/baseband/meson.build b/drivers/baseband/meson.build > index 52489df35..fabc80fc2 100644 > --- a/drivers/baseband/meson.build > +++ b/drivers/baseband/meson.build > @@ -3,5 +3,10 @@ > > drivers = ['null'] > > +enabled_drivers = get_option('drivers_baseband') > +if enabled_drivers != 'all' > + drivers = (enabled_drivers == 'none') ? [] : enabled_drivers.split(',') > +endif > + > config_flag_fmt = 'RTE_LIBRTE_@0@_PMD' > driver_name_fmt = 'rte_pmd_@0@' > diff --git a/drivers/bus/meson.build b/drivers/bus/meson.build > index 80de2d91d..d1400ff28 100644 > --- a/drivers/bus/meson.build > +++ b/drivers/bus/meson.build > @@ -2,6 +2,12 @@ > # Copyright(c) 2017 Intel Corporation > > drivers = ['dpaa', 'fslmc', 'ifpga', 'pci', 'vdev', 'vmbus'] > + > +enabled_drivers = get_option('drivers_bus') > +if enabled_drivers != 'all' > + drivers = (enabled_drivers == 'none') ? [] : enabled_drivers.split(',') > +endif > + > std_deps = ['eal'] > config_flag_fmt = 'RTE_LIBRTE_@0@_BUS' > driver_name_fmt = 'rte_bus_@0@' > diff --git a/drivers/common/meson.build b/drivers/common/meson.build > index a50934108..311511365 100644 > --- a/drivers/common/meson.build > +++ b/drivers/common/meson.build > @@ -3,5 +3,11 @@ > > std_deps = ['eal'] > drivers = ['cpt', 'dpaax', 'mvep', 'octeontx', 'qat'] > + > +enabled_drivers = get_option('drivers_common') > +if enabled_drivers != 'all' > + drivers = (enabled_drivers == 'none') ? [] : enabled_drivers.split(',') > +endif > + > config_flag_fmt = 'RTE_LIBRTE_@0@_COMMON' > driver_name_fmt = 'rte_common_@0@' > diff --git a/drivers/compress/meson.build b/drivers/compress/meson.build > index 817ef3be4..5585ec537 100644 > --- a/drivers/compress/meson.build > +++ b/drivers/compress/meson.build > @@ -3,6 +3,11 @@ > > drivers = ['isal', 'octeontx', 'qat', 'zlib'] > > +enabled_drivers = get_option('drivers_compress') > +if enabled_drivers != 'all' > + drivers = (enabled_drivers == 'none') ? [] : enabled_drivers.split(',') > +endif > + > std_deps = ['compressdev'] # compressdev pulls in all other needed deps > config_flag_fmt = 'RTE_LIBRTE_@0@_PMD' > driver_name_fmt = 'rte_pmd_@0@' > diff --git a/drivers/crypto/meson.build b/drivers/crypto/meson.build > index 83e78860e..7b1363f23 100644 > --- a/drivers/crypto/meson.build > +++ b/drivers/crypto/meson.build > @@ -5,6 +5,11 @@ drivers = ['aesni_gcm', 'aesni_mb', 'caam_jr', 'ccp', 'dpaa_sec', 'dpaa2_sec', > 'kasumi', 'mvsam', 'null', 'octeontx', 'openssl', 'qat', 'scheduler', > 'snow3g', 'virtio', 'zuc'] > > +enabled_drivers = get_option('drivers_crypto') > +if enabled_drivers != 'all' > + drivers = (enabled_drivers == 'none') ? [] : enabled_drivers.split(',') > +endif > + > std_deps = ['cryptodev'] # cryptodev pulls in all other needed deps > config_flag_fmt = 'RTE_LIBRTE_@0@_PMD' > driver_name_fmt = 'rte_pmd_@0@' > diff --git a/drivers/event/meson.build b/drivers/event/meson.build > index fb723f727..14e0c68ac 100644 > --- a/drivers/event/meson.build > +++ b/drivers/event/meson.build > @@ -6,6 +6,12 @@ if not (toolchain == 'gcc' and cc.version().version_compare('<4.8.6') and > dpdk_conf.has('RTE_ARCH_ARM64')) > drivers += 'octeontx' > endif > + > +enabled_drivers = get_option('drivers_event') > +if enabled_drivers != 'all' > + drivers = (enabled_drivers == 'none') ? [] : enabled_drivers.split(',') > +endif > + > std_deps = ['eventdev', 'kvargs'] > config_flag_fmt = 'RTE_LIBRTE_@0@_EVENTDEV_PMD' > driver_name_fmt = 'rte_pmd_@0@_event' > diff --git a/drivers/mempool/meson.build b/drivers/mempool/meson.build > index 4527d9806..cfacedd1b 100644 > --- a/drivers/mempool/meson.build > +++ b/drivers/mempool/meson.build > @@ -2,6 +2,12 @@ > # Copyright(c) 2017 Intel Corporation > > drivers = ['bucket', 'dpaa', 'dpaa2', 'octeontx', 'ring', 'stack'] > + > +enabled_drivers = get_option('drivers_mempool') > +if enabled_drivers != 'all' > + drivers = (enabled_drivers == 'none') ? [] : enabled_drivers.split(',') > +endif > + > std_deps = ['mempool'] > config_flag_fmt = 'RTE_LIBRTE_@0@_MEMPOOL' > driver_name_fmt = 'rte_mempool_@0@' > diff --git a/drivers/net/meson.build b/drivers/net/meson.build > index ed99896c3..901be5eac 100644 > --- a/drivers/net/meson.build > +++ b/drivers/net/meson.build > @@ -42,6 +42,12 @@ drivers = ['af_packet', > 'virtio', > 'vmxnet3', > ] > + > +enabled_drivers = get_option('drivers_net') > +if enabled_drivers != 'all' > + drivers = (enabled_drivers == 'none') ? [] : enabled_drivers.split(',') > +endif > + > std_deps = ['ethdev', 'kvargs'] # 'ethdev' also pulls in mbuf, net, eal etc > std_deps += ['bus_pci'] # very many PMDs depend on PCI, so make std > std_deps += ['bus_vdev'] # same with vdev bus > diff --git a/drivers/raw/meson.build b/drivers/raw/meson.build > index a61cdccef..ca98e92f0 100644 > --- a/drivers/raw/meson.build > +++ b/drivers/raw/meson.build > @@ -2,6 +2,12 @@ > # Copyright 2018 NXP > > drivers = ['skeleton_rawdev', 'dpaa2_cmdif', 'dpaa2_qdma', 'ifpga_rawdev'] > + > +enabled_drivers = get_option('drivers_raw') > +if enabled_drivers != 'all' > + drivers = (enabled_drivers == 'none') ? [] : enabled_drivers.split(',') > +endif > + > std_deps = ['rawdev'] > config_flag_fmt = 'RTE_LIBRTE_PMD_@0@_RAWDEV' > driver_name_fmt = 'rte_pmd_@0@' > diff --git a/lib/meson.build b/lib/meson.build > index e067ce5ea..384867926 100644 > --- a/lib/meson.build > +++ b/lib/meson.build > @@ -34,6 +34,11 @@ if is_windows > libraries = ['kvargs','eal'] # only supported libraries for windows > endif > > +enabled_libs = get_option('libs') > +if enabled_libs != 'all' > + libraries = (enabled_libs == 'none') ? [] : enabled_libs.split(',') > +endif > + > default_cflags = machine_args > if cc.has_argument('-Wno-format-truncation') > default_cflags += '-Wno-format-truncation' > diff --git a/meson_options.txt b/meson_options.txt > index 16d9f92c6..5184917b0 100644 > --- a/meson_options.txt > +++ b/meson_options.txt > @@ -2,6 +2,26 @@ > > option('allow_invalid_socket_id', type: 'boolean', value: false, > description: 'allow out-of-range NUMA socket id\'s for platforms that don\'t report the value correctly') > +option('apps', type: 'string', value: 'all', > + description: 'List of applications to build. Defaults to all') > +option('drivers_baseband', type: 'string', value: 'all', > + description: 'List of baseband drivers to build. Defaults to all') > +option('drivers_bus', type: 'string', value: 'all', > + description: 'List of bus drivers to build. Defaults to all') > +option('drivers_common', type: 'string', value: 'all', > + description: 'List of common drivers to build. Defaults to all') > +option('drivers_compress', type: 'string', value: 'all', > + description: 'List of compress drivers to build. Defaults to all') > +option('drivers_crypto', type: 'string', value: 'all', > + description: 'List of crypto drivers to build. Defaults to all') > +option('drivers_event', type: 'string', value: 'all', > + description: 'List of event drivers to build. Defaults to all') > +option('drivers_mempool', type: 'string', value: 'all', > + description: 'List of mempool drivers to build. Defaults to all') > +option('drivers_net', type: 'string', value: 'all', > + description: 'List of net drivers to build. Defaults to all') > +option('drivers_raw', type: 'string', value: 'all', > + description: 'List of rawdev drivers to build. Defaults to all') > option('drivers_install_subdir', type: 'string', value: 'dpdk/pmds-', > description: 'Subdirectory of libdir where to install PMDs. Defaults to using a versioned subdirectory.') > option('enable_docs', type: 'boolean', value: false, > @@ -16,6 +36,8 @@ option('include_subdir_arch', type: 'string', value: '', > description: 'subdirectory where to install arch-dependent headers') > option('kernel_dir', type: 'string', value: '', > description: 'path to the kernel for building kernel modules, they will be installed in $DEST_DIR/$kernel_dir/../extra/dpdk') > +option('libs', type: 'string', value: 'all', > + description: 'List of libraries to build. Defaults to all') > option('lib_musdk_dir', type: 'string', value: '', > description: 'path to the MUSDK library installation directory') > option('machine', type: 'string', value: 'native', I think this patch is the right step forward. This patch will speed up travis builds which is a big plus Just one small comment, could you document/add some build examples to doc/build-sdk-meson.txt and other relevant doc files? Thanks Acked-by: Michael Santana