Re: [PATCH 24.03 v2] build: track mandatory rather than optional libs

[Bruce Richardson <bruce.richardson@intel.com>]

On Fri, Nov 03, 2023 at 09:19:53PM +0100, Morten Brørup wrote:
> > From: Bruce Richardson [mailto:bruce.richardson@intel.com]
> > Sent: Friday, 3 November 2023 19.09
> > 
> > On Fri, Nov 03, 2023 at 06:31:30PM +0100, Morten Brørup wrote:
> > > > From: Bruce Richardson [mailto:bruce.richardson@intel.com]
> > > > Sent: Friday, 3 November 2023 17.52
> > > >
> > > > DPDK now has more optional libraries than mandatory ones, so invert
> > the
> > > > list stored in the meson.build file from the optional ones to the
> > > > "always_enable" ones. As well as being a shorter list:
> > > >
> > > > * we can remove the loop building up the "always_enable" list
> > > >   dynamically from the optional list
> > > > * it better aligns with the drivers/meson.build file which
> > maintains an
> > > >   always_enable list.
> > > >
> > > > Signed-off-by: Bruce Richardson <bruce.richardson@intel.com>
> > >
> > > Excellent!
> > >
> > > It really shows how bloated DPDK CORE still is. I would like to see
> > these go optional:
> > >
> > 
> > For some I agree, but we need to decide what optional really means. :-)
> > 
> > For my mind, there are 3 (maybe 4) key components that need to be built
> > for
> > me to consider a build to be a valid DPDK one:
> > * EAL obviously,
> > * testpmd, because everyone seems to use it
> > * l3fwd, becaues it's the most commonly referenced example and used for
> >   benchmarking, and build testing in test-meson-builds. (There are
> > others,
> >   but they are pretty likely to build if l3fwd does!)
> > * dpdk-test - I feel this should always be buildable, but for me it's
> > the
> >   optional 4th component.
> > 
> > Now, the obviously one to relax here is l3fwd, since it is just an
> > example,
> > but I wonder if that may cause some heartache.
> 
> I don't consider any DPDK lib CORE just because the lib is used by testpmd and/or l3fwd. I agree that all libs should be included by default, so you can run testpmd, l3fwd, and other apps and examples.
> 
> However, many libs are not needed for *all* DPDK applications, so I would like other apps to be able to build DPDK without superfluous libs.
> 
> E.g. our StraightShaper CSP appliance is deployed at Layer 2, and doesn't use any of DPDK's L3 libs, so why should the DPDK L3 libs be considered CORE and thus included in our application? I suppose other companies are also using DPDK for other purposes than L3 routing, and don't need the DPDK L3 libs.
> 
> Furthermore, I suppose that some Layer 3 applications use their own RIB/FIB/LPM libraries. Does OVS use DPDK's rib/fib/lpm libraries?
> 

<snip for brevity>

> > Overall, if we want to make more libs optional, I would start looking
> > at
> > l3fwd and making it a bit more modular. I previously made its support
> > for
> > eventdev optional, we should do the same for lpm and fib. Beyond that,
> > we
> > need to decide what core really means.
> 
> Yes - defining CORE is the key to setting the goal here.
> 
> In my mind, CORE is the minimum requirement to running an absolutely minimal DPDK application.
> 
> A primary DPDK application would probably need to do some packet I/O; but it might be a simple layer two bridge, not using any of the L3 libs.
> 
> And a secondary DPDK application might attach to a primary DPDK application only to work on its data structures, e.g. to collect statistics, but not do any packet processing, so that application doesn't need any of those libs (not even the ethdev lib).
> 
> In reality, DPDK applications would probably need to build more libs than just CORE. But some application might need CORE + lib A, and some other application might need CORE + lib B. In essence, I don't want application A to drag around some unused lib B, and application B to drag around some unused lib A.
> 
> It's an optimization only available a build time. Distros should continue providing all DPDK libs.
> 
> There's also system testing and system attack surface to consider... all that bloat makes production systems more fragile and vulnerable.
> 

I largely agree, though I do think that trying to split primary-secondary
as having different builds could lead to some headaches, so I'd push any
work around that further down the road.

Some thoughts on next steps:
* From looks of my original list above, it appears the low-hanging fruit is
  largely gone, in terms of being able to turn off libs that have few
  dependencies, timer being one possible exception
* I think it's worth looking into making l3fwd more modular so it can be
  build only with backend x or y or z in it. However, if agreeable, we can
  just start marking lpm and rib/fib libs as optional directly and have
  l3fwd not buildable in those cases.
* For libs that depend on other libs for bits of functionality, we are
  getting into the realm of using ifdefs to start selectively removing
  bits. This is the not-so-nice bit as:

  - it makes it a lot harder to do proper build testing, as we now have to
    test with individual bits on and off. So long as we were just enabling/
    disabling whole components, the build-minimal target was good enough to
    test we had a working build. With some libs partially depending on
    others - both of which may be disablable independently - our build test
    matrix just explodes.
  - #ifdefs are just really, really ugly in the code, and make it far
    harder to maintain and manage.

Therefore, I'm wondering if we can come up with some sort of neater
solution here.

For example, can we add support to the build system that allows some form
of stubbing out of libraries when they are disabled? That would save the
putting of ifdefs throughout other parts of DPDK and keep the management of
the disabling of the library someway inside the library itself. One way to
do this might be to have a "stub" folder inside a library folder, where
that contains a minimal header file to be used to provide empty functions
in case where the lib itself is disabled.  Other, more interesting schemes,
might involve the automatic creation - from the version.map file - of dummy
functions for dependent libs to link against on build.

/Bruce