Re: [dpdk-dev] [PATCH v16 1/3] build: disable/enable drivers in Arm builds

["Juraj Linkeš" <juraj.linkes@pantheon.tech>]

> -----Original Message-----
> From: Honnappa Nagarahalli <Honnappa.Nagarahalli@arm.com>
> Sent: Tuesday, March 30, 2021 2:39 AM
> To: Jerin Jacob <jerinjacobk@gmail.com>; Juraj Linkeš
> <juraj.linkes@pantheon.tech>
> Cc: Bruce Richardson <bruce.richardson@intel.com>; Ruifeng Wang
> <Ruifeng.Wang@arm.com>; vcchunga@amazon.com; Dharmik Thakkar
> <Dharmik.Thakkar@arm.com>; hemant.agrawal@nxp.com; Ajit Khaparde
> (ajit.khaparde@broadcom.com) <ajit.khaparde@broadcom.com>;
> ferruh.yigit@intel.com; aboyer@pensando.io; lironh@marvell.com;
> dev@dpdk.org; nd <nd@arm.com>; Honnappa Nagarahalli
> <Honnappa.Nagarahalli@arm.com>; nd <nd@arm.com>
> Subject: RE: [PATCH v16 1/3] build: disable/enable drivers in Arm builds
> 
> <snip>
> 
> > > <removed parts which I think are not that relevant>
> > >
> > > > > > > > > > > The blocklist is, I think, agreed upon by everyone.
> > > > > > > > > > > The question is whether we want to support the
> > > > > > > > > > > allowlist alongside it and there seem to be enough
> > > > > > > > > > > reasons to do
> > that.
> > > > > > > > > > Sorry, may be this is answered already, but, what
> > > > > > > > > > additional benefit does allowlist provide over the blocklist?
> > > > > > > > > >
> > > > > > > > >
> > > > > > > > > VPP could use it:
> > > > > > > > > https://gerrit.fd.io/r/gitweb?p=vpp.git;a=blob;f=build/e
> > > > > > > > > xt
> > > > > > > > > erna
> > > > > > > > > l/pa
> > > > > > > > > ckag es/dpdk
> > > > > > > > > .mk;h=c35ac84c27b19411a0cfdf9a3524fdf36024762c;hb=HEAD
> > > > > > > > >
> > > > > > > > > They're disabling almost everything so an allowlist would fit there.
> > > > > > > > > And they won't need to update the list when a new driver
> > > > > > > > > is added (which they won't need).
> > > > > > > > This is different from how we started this discussion. The
> > > > > > > > current discussion was for DPDK internal use. But the one
> > > > > > > > you are referencing above is for users of DPDK. I am fine
> > > > > > > > for providing the allow list for the users of DPDK. But
> > > > > > > > for DPDK internal, I think block list is
> > > > > > enough.
> > > > > > > >
> > > > > > >
> > > > > > > That's an interesting suggestion. Jerin, what do you think?
> > > > > > > Why did you
> > > > > > want to have an allowlist? Would this work?
> > > > > >
> > > > > > # The very reason why VPP chooses to have allow list so that
> > > > > > they can control what needs to include.
> > > > > > # Another use case is like, in SoCs have fixed internal
> > > > > > devices, we can have optimized build for that can have only
> > > > > > allow list of the drivers that care about # For server market,
> > > > > > block list makes sense # For embedded SoC, allow list makes sense.
> > > > > For the embedded SoC, IMO, the upstream project could allow the
> > > > > compilation
> > > > for wider set of PMDs/libs. May be the end customer can use the
> > > > allow list to compile/use what is required?
> > > >
> > > > Just to understand, how end customer can enable allow list, if
> > > > DPDK build system does not support it?
> > > > Also to understand, If we are supporting blocklist, why not have
> > > > allowlist (I mean, both of them) as both are required as it caters
> > > > different use case as mention above. We can not emulate allowlist
> > > > with blocklist as each version of DPDK will have new libraries and
> > > > PMD's
> > which end user has no clue. Right?
> > >
> > > > I think, that is the reason why VPP is doing the allow list.
> > >
> > > I'm not sure what you mean by this, but to clarify, VPP likely would
> > > be using
> > the allowlist in this fashion, but that is not an arm specific
> > usecase. I think what Honnappa wanted to see was how the allowlist
> > could be used in an arm usecase (such as using it in an SoC configuration).
> >
> > There is nothing arm-specific here. Right? allowlist will be common
> > and will be used by all architecture. Right?
> Nothing Arm specific. I think for generic Arm servers platforms we can make
> sure that we allow for compilation of all the DPDK code. We can go ahead with
> implementing the allow list.
> 

I tried to actually use an allowlist and there are some problems with building apps and tests. When I tried to enable a random driver, such as common/sfc_efx, I've ran into dependency issues with apps:
app/meson.build:53:3: ERROR:  Tried to get unknown variable "static_rte_bus_vdev".

This is because bus/vdev is not enabled (the allowlist only enabled net/sfc_efx). When I implemented dependency discovery in apps similar to which exists for drivers/libs, I was then unable to build tests (which is a special case app):
app/test/meson.build:427:1: ERROR:  Tried to get unknown variable "static_rte_bus_pci".

This is becasue bus/pci is not enabled. If I understand the code correctly, the test dependencies are not matched to each test, meaning we can't disable the tests for which we don't have dependencies - we can only disable all tests.

In general, this problem also affects the blocklist, i.e. meson build -Ddisable_drivers=bus/pci produces
drivers/net/hinic/base/meson.build:34:0: ERROR:  Unknown variable "static_rte_bus_pci".
For blocklists, this is seldom going to be a problem, since most users won't disable "core" drivers.

I'm not sure what is the best course of action. We can implement the allowlist and then we'll leave it up to implementers to implement allowlists that produce a working build. We could then optionally address the dependency issues brought by disabled drivers in a separate patch.

Maybe we can just have a list of "core" drivers that can never be disabled (either using blocklists or allowlists)?

Bruce, what do you think?

> >
> >
> > >
> > >