RE: [PATCH 24.03 v2] build: track mandatory rather than optional libs

["Morten Brørup" <mb@smartsharesystems.com>]

> From: Bruce Richardson [mailto:bruce.richardson@intel.com]
> Sent: Friday, 3 November 2023 19.09
> 
> On Fri, Nov 03, 2023 at 06:31:30PM +0100, Morten Brørup wrote:
> > > From: Bruce Richardson [mailto:bruce.richardson@intel.com]
> > > Sent: Friday, 3 November 2023 17.52
> > >
> > > DPDK now has more optional libraries than mandatory ones, so invert
> the
> > > list stored in the meson.build file from the optional ones to the
> > > "always_enable" ones. As well as being a shorter list:
> > >
> > > * we can remove the loop building up the "always_enable" list
> > >   dynamically from the optional list
> > > * it better aligns with the drivers/meson.build file which
> maintains an
> > >   always_enable list.
> > >
> > > Signed-off-by: Bruce Richardson <bruce.richardson@intel.com>
> >
> > Excellent!
> >
> > It really shows how bloated DPDK CORE still is. I would like to see
> these go optional:
> >
> 
> For some I agree, but we need to decide what optional really means. :-)
> 
> For my mind, there are 3 (maybe 4) key components that need to be built
> for
> me to consider a build to be a valid DPDK one:
> * EAL obviously,
> * testpmd, because everyone seems to use it
> * l3fwd, becaues it's the most commonly referenced example and used for
>   benchmarking, and build testing in test-meson-builds. (There are
> others,
>   but they are pretty likely to build if l3fwd does!)
> * dpdk-test - I feel this should always be buildable, but for me it's
> the
>   optional 4th component.
> 
> Now, the obviously one to relax here is l3fwd, since it is just an
> example,
> but I wonder if that may cause some heartache.

I don't consider any DPDK lib CORE just because the lib is used by testpmd and/or l3fwd. I agree that all libs should be included by default, so you can run testpmd, l3fwd, and other apps and examples.

However, many libs are not needed for *all* DPDK applications, so I would like other apps to be able to build DPDK without superfluous libs.

E.g. our StraightShaper CSP appliance is deployed at Layer 2, and doesn't use any of DPDK's L3 libs, so why should the DPDK L3 libs be considered CORE and thus included in our application? I suppose other companies are also using DPDK for other purposes than L3 routing, and don't need the DPDK L3 libs.

Furthermore, I suppose that some Layer 3 applications use their own RIB/FIB/LPM libraries. Does OVS use DPDK's rib/fib/lpm libraries?

> 
> Anyway some notes:
> 
> > acl - OPTIONAL
> Used by l3fwd

But not *all* apps.

> 
> > cmdline - OPTIONAL (if not used by EAL)
> I'd consider this core - used by testpmd and the auto-tests. I'd push
> this
> down the list to consider making optional.
> 
> > eal - CORE
> > ethdev - CORE, or OPTIONAL for secondary process
> Is there really much use of DPDK without ethdev? Let's just keep core
> for
> now.

Here, I am imagining a secondary process attaching to DPDK shared memory, but not doing any device I/O.

For a primary process, I agree 100 % that it is not much of a DPDK application without ethdev.

> 
> > fib - OPTIONAL
> Used by l3fwd
> 
> > hash - OPTIONAL (if not used by CORE libs)
> Needed by l3fwd.  Also listed as a dependency by a number of drivers,
> e.g.
> i40e, mlx5, nfp, tap. I think they use it for holding filters and
> things.
> This I'd be wary about allowing disabling without some work, as it
> could
> cause users surprise when some drivers unexpectedly stop building.

It is fair enough if some drivers depend on the hash lib; then they should register that dependency in the build system.

In principle, if someone builds DPDK without any of those drivers, so this lib is unused, why should it be included?

This lib has a small footprint, so I don't feel strongly about it.

> 
> > kvargs - OPTIONAL (if not used by EAL)
> Used by EAL
> 
> > log - CORE
> > lpm - OPTIONAL
> Used by l3fwd
> 
> > mbuf - CORE
> > mempool - CORE
> > meter - OPTIONAL
> Used by ethdev.

I wonder how this became a mandatory part of ethdev... It should be optional if not used by the application.

This seems to be EAL all over again... bloat is integrated deeply into EAL, and cannot be removed/disabled. Now, this other bloat is deeply integrated into ethdev, and cannot be removed/disabled.

> 
> > net - CORE
> > pci - CORE, or OPTIONAL for secondary process
> > rcu - OPTIONAL (if not used by CORE libs)
> Used by lpm and hash, which are used by l3fwd.
> 
> > rib - OPTIONAL
> Used by fib, used by l3fwd.
> 
> > ring - CORE
> Actually, this is one I'd look to maybe say optional. :-)

The ring is also used for pipelined applications. (Which in itself doesn't make it mandatory; run-to-completion applications might not need it.)

> 
> I think we should switch our default mempool backend from ring to
> stack,
> and make that mandatory. The reason being that run-to-completion apps
> should be running entirely out of mempool cache and not care about the
> underlying driver, while pipelined apps pushing buffers across cores
> would
> be better with a LIFO-based rather than FIFO-based mempool driver.
> Therefore, I think overall switching to stack from ring would be an
> overall
> win for performance.

As a mempool backend, you are probably right that a stack would provide better performance for pipelined applications (due to improved cache utilization).

But isn't the stack backend using locks, so it would run into problems if used by an unregistered non-EAL thread?

Although I haven't looked into the details, I am under the impression that some of the newer ring backends (Relaxed Tail Sync or Head/Tail Sync) don't have this problem.

> 
> > stack - OPTIONAL (if not used by CORE libs)
> For reasons above, I'd make this core.

We need at least one mempool backend, so either ring or stack must be mandatory.

> 
> > telemetry - OPTIONAL
> Used by EAL

This is mega-bloat, and I am surprised that it is not optional!

Not all apps use telemetry, and previous DPDK releases were able to provide an *environment abstraction layer* without considering telemetry a CORE component of making applications run on some underlying hardware/environment.

An excellent example of non-mandatory stuff added to EAL, instead of keeping EAL lean and mean. This also raises the barrier to implementing an EAL for some other hardware/environment.

Long term, I am hoping for a barebone DPDK be able to run on low-end CPE hardware. I we keep moving down the path of adding bloat to DPDK EAL and CORE, it will eventually require 32 GB RAM and 4 CPU cores to fire up DPDK.

> 
> > timer - OPTIONAL (if not used by CORE libs)
> Only lib that seems to depend on this is eventdev, so maybe this can be
> an
> easy removal. Though I do see it listed as a dependency of ENA driver
> too.

Excellent. If dependencies are set up correctly, it can be omitted from CORE.

> 
> >
> > Anyway,
> >
> > Acked-by: Morten Brørup <mb@smartsharesystems.com>
> >
> 
> Overall, if we want to make more libs optional, I would start looking
> at
> l3fwd and making it a bit more modular. I previously made its support
> for
> eventdev optional, we should do the same for lpm and fib. Beyond that,
> we
> need to decide what core really means.

Yes - defining CORE is the key to setting the goal here.

In my mind, CORE is the minimum requirement to running an absolutely minimal DPDK application.

A primary DPDK application would probably need to do some packet I/O; but it might be a simple layer two bridge, not using any of the L3 libs.

And a secondary DPDK application might attach to a primary DPDK application only to work on its data structures, e.g. to collect statistics, but not do any packet processing, so that application doesn't need any of those libs (not even the ethdev lib).

In reality, DPDK applications would probably need to build more libs than just CORE. But some application might need CORE + lib A, and some other application might need CORE + lib B. In essence, I don't want application A to drag around some unused lib B, and application B to drag around some unused lib A.

It's an optimization only available a build time. Distros should continue providing all DPDK libs.

There's also system testing and system attack surface to consider... all that bloat makes production systems more fragile and vulnerable.

-Morten