DPDK patches and discussions
 help / color / mirror / Atom feed
From: Bruce Richardson <bruce.richardson@intel.com>
To: Luca Boccassi <bluca@debian.org>
Cc: Ilya Maximets <i.maximets@samsung.com>,
	dev@dpdk.org, Thomas Monjalon <thomas@monjalon.net>,
	Aaron Conole <aconole@redhat.com>,
	Kevin Traynor <ktraynor@redhat.com>
Subject: Re: [dpdk-dev] [PATCH 2/2] meson: make build configurable
Date: Thu, 30 May 2019 11:22:29 +0100	[thread overview]
Message-ID: <20190530102229.GC1107@bricha3-MOBL.ger.corp.intel.com> (raw)
In-Reply-To: <1a07d1cd59d84dce84e56c10fdabf5e5504560a6.camel@debian.org>

On Wed, May 29, 2019 at 09:37:20PM +0100, Luca Boccassi wrote:
> On Wed, 2019-05-29 at 19:39 +0300, Ilya Maximets wrote:
> > The first thing many developers do before start building DPDK is
> > disabling all the not needed divers and libraries. This happens
> > just because more than a half of DPDK dirvers and libraries are not
> > needed for the particular reason. For example, you don't need
> > dpaa*, octeon*, various croypto devices, eventdev, etc. if you're
> > only want to build OVS for x86_64 with static linking.
> > 
> > By disabling everything you don't need, build speeds up literally 10x
> > times. This is important for CI systems. For example, TravisCI wastes
> > 10 minutes for the default DPDK build just to check linking with OVS.
> > 
> > Another thing is the binary size. Number of DPDK libraries and,
> > as a result, size of resulted statically linked application decreases
> > significantly.
> > 
> > Important thing also that you're able to not install some
> > dependencies
> > if you don't have them on a target platform. Just disable
> > libs/drivers
> > that depends on it. Similar thing for the glibc version mismatch
> > between build and target platforms.
> > 
> > Also, I have to note that less code means less probability of
> > failures and less number of attack vectors.
> > 
> > This patch gives 'meson' the power of configurability that we
> > have with 'make'. Using new options it's possible to enable just
> > what you need and nothing more.
> > 
> > For example, following cmdline could be used to build almost minimal
> > set of DPDK libs and drivers to check OVS build:
> > 
> >   $ meson build -Dexamples='' -Dtests=false -Denable_kmods=false \
> >                 -Ddrivers_bus=pci,vdev          \
> >                 -Ddrivers_mempool=ring          \
> >                 -Ddrivers_net=null,virtio,ring  \
> >                 -Ddrivers_crypto=virtio         \
> >                 -Ddrivers_compress=none         \
> >                 -Ddrivers_event=none            \
> >                 -Ddrivers_baseband=none         \
> >                 -Ddrivers_raw=none              \
> >                 -Ddrivers_common=none           \
> >                
> > -Dlibs=kvargs,eal,cmdline,ring,mempool,mbuf,net,meter,\
> >                        ethdev,pci,hash,cryptodev,pdump,vhost \
> >                 -Dapps=none
> > 
> > Adding a few real net drivers will give configuration that can be
> > used
> > in production environment.
> > 
> > Looks not very pretty, but this could be moved to a script.
> > 
> > Build details:
> > 
> >   Build targets in project: 57
> > 
> >   $ time ninja
> >   real    0m11,528s
> >   user    1m4,137s
> >   sys     0m4,935s
> > 
> >   $ du -sh ../dpdk_meson_install/
> >   3,5M    ../dpdk_meson_install/
> > 
> > To compare with what we have without these options:
> > 
> >   $ meson build -Dexamples='' -Dtests=false -Denable_kmods=false
> >   Build targets in project: 434
> > 
> >   $ time ninja
> >   real    1m38,963s
> >   user    10m18,624s
> >   sys     0m45,478s
> > 
> >   $ du -sh ../dpdk_meson_install/
> >   27M     ../dpdk_meson_install/
> > 
> > 10x speed up for the user time.
> > 7.7 times size decrease.
> > 
> > This is probably not much user-friendly because it's not a Kconfig
> > and dependency tracking in meson is really poor, so it requires
> > usually few iterations to pick correct set of libraries to satisfy
> > all dependencies. However, it's not a big deal. Options intended
> > for a proficient users who knows what they need.
> 
> Hi,
> 
> We talked about this a few times in the past, and it was actually one
> of the design goals to _avoid_ replicating the octopus-like config
> system of the makefiles. That's because it makes the test matrix
> insanely complicated, not to mention the harm to user friendliness,
> among other things.
> 
> If someone doesn't want to use a PMD, they can just avoid installing it
> - it's simple enough.
> 
> Sorry, but from me it's a very strong NACK.
> 
I would agree with this position - tracking the dependencies of the
libraries etc. is a nightmare, and requires lots of ifdef'ery in the code
for handling cases where libraries don't exist.

However, I might be ok with limiting the drivers somewhat, since they don't
tend to depend on each other so much, though ideally I'd still prefer to
have one build of DPDK that has minimal configuration. If we say that we
can disable some drivers, though,  issue then becomes whether e.g. the bus
drivers could selectively be disabled, and the knock-on effects of that.
I'd hate to see the case where we end up having the meson.build files for
drivers becoming a massive list of conditional checks for a bunch of
internal dependencies. If someone is wanting to do a custom build of DPDK,
they can always patch out the subdirectories they don't want in the
meson.build files - but because of testing matrixes for such
configurations, I don't think its something we want to explicitly support.

/Bruce

  parent reply	other threads:[~2019-05-30 10:22 UTC|newest]

Thread overview: 20+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
     [not found] <CGME20190529164004eucas1p154674b298c0b906f663ae04ebfcbc33c@eucas1p1.samsung.com>
2019-05-29 16:39 ` [dpdk-dev] [PATCH 0/2] Make meson configurable Ilya Maximets
     [not found]   ` <CGME20190529164006eucas1p296e902a46cb3ef7ac436d619f9e55d5b@eucas1p2.samsung.com>
2019-05-29 16:39     ` [dpdk-dev] [PATCH 1/2] meson: don't check dependencies for tests if not required Ilya Maximets
2019-05-30 11:55       ` Bruce Richardson
2019-05-30 12:06         ` Ilya Maximets
2019-05-30 12:20           ` Bruce Richardson
     [not found]       ` <CGME20190530123840eucas1p216e6df3737209db26e2b0ce678ba9da4@eucas1p2.samsung.com>
2019-05-30 12:38         ` [dpdk-dev] [PATCH v2] " Ilya Maximets
2019-05-30 12:48           ` Bruce Richardson
2019-06-05 16:48             ` Thomas Monjalon
2019-05-30 13:08           ` Aaron Conole
2019-05-30 13:34           ` Luca Boccassi
     [not found]   ` <CGME20190529164009eucas1p289f1dcf87012ecf049efc8eee2c2ea9d@eucas1p2.samsung.com>
2019-05-29 16:39     ` [dpdk-dev] [PATCH 2/2] meson: make build configurable Ilya Maximets
2019-05-29 20:15       ` Michael Santana Francisco
2019-05-30  9:22         ` Ilya Maximets
2019-05-29 20:37       ` Luca Boccassi
2019-05-30 10:03         ` Ilya Maximets
2019-05-30 11:06           ` Luca Boccassi
2019-05-30 11:59             ` Ilya Maximets
2019-05-30 13:30               ` Luca Boccassi
2019-05-30 10:22         ` Bruce Richardson [this message]
2019-05-30 11:46           ` Ilya Maximets

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20190530102229.GC1107@bricha3-MOBL.ger.corp.intel.com \
    --to=bruce.richardson@intel.com \
    --cc=aconole@redhat.com \
    --cc=bluca@debian.org \
    --cc=dev@dpdk.org \
    --cc=i.maximets@samsung.com \
    --cc=ktraynor@redhat.com \
    --cc=thomas@monjalon.net \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).