Re: DPDK configuration options

[Bruce Richardson <bruce.richardson@intel.com>]

On Wed, Aug 14, 2024 at 11:27:58AM +0200, Morten Brørup wrote:
> DPDK has many configuration options.
> 
> There are four levels of visibility:
> 
> 1. Some are changed by passing command line options to meson.
> 2. Some are changed by modifying their values in config/rte_config.h.
> 3. Some are changed by adding them to config/rte_config.h, but you have to magically know of their existence; e.g. RTE_ENABLE_ASSERT, RTE_MALLOC_DEBUG and RTE_LIBRTE_MBUF_DEBUG.
> 4. Some are hidden away in drivers, typically driver specific options.
> 
> And many of the configuration options are not even documented anywhere in the code; they are just used by the code.
> 
> It seems the level of visibility is currently determined by how "exotic" the option is considered to be. I think this is the wrong criteria.
> 
> There's also a expectation that a person building DPDK doesn't have to modify config/rte_config.h. I think this is a false expectation; if you are qualified to build DPDK and tweak it along the way, you certainly understand how to modify a header file, and there is no good reason to pass simple configuration values (e.g. max_ethports, mbuf_refcnt_atomic and pkt_mbuf_headroom) 1:1 through meson.
> 
> Furthermore, configuration options should not be hidden away or spread all over the place. It makes them difficult to find and modify.
> 
> Optimally, we would have the same way of configuring DPDK as the Linux kernel.
> But I don't see that happening anytime soon.
> So, in the interim, we could use one big configuration file, as follows:
> 
> Options that are not candidates for automatic detection at build time should not be level 1, but level 2. (Automatic detection makes sense for e.g. max_lcores, so that should remain at level 1.)
> 
> All level 3 options should be moved to level 2. If there's a configuration option, it should be presented (and documented), not hidden away.

Agreed on this.

> 
> Similarly, level 4 options should be moved to level 2; perhaps except options in drivers' "base" directories (code shared by DPDK, Linux and/or other systems).

I'm not sure how visible these need to be. If they are driver specific,
having them just documented in the specific driver docs is probably good
enough.

> 
> Each option should have a comment briefly describing what it does.
> 
Agreed.

Taking a step back from the specifics of what options go where, we do need
to decide overall how we want to manage build options. For example:

* In the past, we had loads of build options in a flag config file, but
  this turned out to have major issues around validation and didn't seem
  well liked.
* Back when the build system was changed from make to meson/ninja, the
  general consensus was that we wanted to - as far as possible - move away
  from build options, because it was impossible to validate all build
  combinations, and it was very easy to have broken code inside ifdefs that
  was never even compile-tested. Also, build options didn't work for distro
  targets, where one build was all that was done.
* Since then, though, even though we have had more runtime configurability
  - we have seen a constant increase in build options too.
* Within build options, not all options are equal. For example, numeric
  values which just affect e.g. array sizes such as number ethdevs, are
  probably pretty harmless from a testing viewpoint, and may need to be
  treated differently from build options, e.g. debug ones, which
  enable/disable code blocks and can therefore introduce subtle issues or
  hide problems in disabled code.
* Within the mechanisms of build options, the main issue I have with using
  rte_config.h is that it is a file shipped with dpdk and included in the
  repository. That means that any local changes to it get overwritten with
  any new DPDK release or update. If we want to have such a file-based
  approach, I think we need to change things so that we have support for e.g.
  an rte_config_local.h file which, if present, is used to provide local
  overrides for the rte_config settings. The exact mechanism by which such
  a scheme might work I'm not too clear on yet, though!

Just my 2c. at this point.

/Bruce