[dpdk-dev] [PATCH 0/3] RFC: Support disabling DPDK drivers in meson builds

[dpdk-dev] [PATCH 0/3] RFC: Support disabling DPDK drivers in meson builds

[Bruce Richardson]

Following on from the discussion about meson at userspace 2019, here is
a patchset which allows a user to disable drivers in a DPDK build. This
set also includes the changes necessary to recursively disable any
drivers that depend upon the disabled one, e.g. disabling mempool/dpaa
also disables net/dpaa and event/dpaa with suitable reasons provided in
the output summary at the end of the meson configuration phase.

A number of changes are made to the lib/meson.build file too, in order
to keep the general logic flow between drivers/meson.build and
lib/meson.build as similar as possible.

Example of use:

	meson -Ddisable_drivers=mempool/dpaa,net/i40e build

Bruce Richardson (3):
  build: align variable names between drivers and libs
  build: process dependencies before main build check
  build: support disabling drivers with meson

 drivers/meson.build | 45 +++++++++++++++++++++++++++------------------
 lib/meson.build     | 23 +++++++++++++----------
 meson_options.txt   |  2 ++
 3 files changed, 42 insertions(+), 28 deletions(-)

-- 
2.21.0

[dpdk-dev] [PATCH 1/3] build: align variable names between drivers and libs

[Bruce Richardson]

The variable names in the library and drivers meson.build files are slighty
different with "static_deps" in one and "static_objs" in the other. Rename
to use "static_deps" in both for consistency.

Signed-off-by: Bruce Richardson <bruce.richardson@intel.com>
---
 drivers/meson.build | 22 +++++++++++-----------
 1 file changed, 11 insertions(+), 11 deletions(-)

diff --git a/drivers/meson.build b/drivers/meson.build
index 2ed2e9541..b54601697 100644
--- a/drivers/meson.build
+++ b/drivers/meson.build
@@ -79,18 +79,18 @@ foreach class:dpdk_driver_classes
 			endif
 
 			# get dependency objs from strings
-			shared_objs = []
-			static_objs = []
+			shared_deps = []
+			static_deps = []
 			foreach d:deps
 				if not is_variable('shared_rte_' + d)
 					error('Missing dependency ' + d +
 						' for driver ' + lib_name)
 				endif
-				shared_objs += [get_variable('shared_rte_' + d)]
-				static_objs += [get_variable('static_rte_' + d)]
+				shared_deps += [get_variable('shared_rte_' + d)]
+				static_deps += [get_variable('static_rte_' + d)]
 			endforeach
-			shared_objs += ext_deps
-			static_objs += ext_deps
+			shared_deps += ext_deps
+			static_deps += ext_deps
 			dpdk_extra_ldflags += pkgconfig_extra_libs
 
 			# generate pmdinfo sources by building a temporary
@@ -101,7 +101,7 @@ foreach class:dpdk_driver_classes
 			tmp_lib = static_library('tmp_' + lib_name,
 					sources,
 					include_directories: includes,
-					dependencies: static_objs,
+					dependencies: static_deps,
 					c_args: cflags)
 			objs += tmp_lib.extract_all_objects()
 			sources = custom_target(out_filename,
@@ -123,7 +123,7 @@ foreach class:dpdk_driver_classes
 				sources,
 				objects: objs,
 				include_directories: includes,
-				dependencies: static_objs,
+				dependencies: static_deps,
 				c_args: cflags,
 				install: true)
 
@@ -135,7 +135,7 @@ foreach class:dpdk_driver_classes
 				sources,
 				objects: objs,
 				include_directories: includes,
-				dependencies: shared_objs,
+				dependencies: shared_deps,
 				c_args: cflags,
 				link_args: '-Wl,--version-script=' + version_map,
 				link_depends: version_map,
@@ -148,10 +148,10 @@ foreach class:dpdk_driver_classes
 			# testpmd or other built-in apps can find it if necessary
 			shared_dep = declare_dependency(link_with: shared_lib,
 					include_directories: includes,
-					dependencies: shared_objs)
+					dependencies: shared_deps)
 			static_dep = declare_dependency(link_with: static_lib,
 					include_directories: includes,
-					dependencies: static_objs)
+					dependencies: static_deps)
 
 			dpdk_drivers += static_lib
 
-- 
2.21.0

[dpdk-dev] [PATCH 2/3] build: process dependencies before main build check

[Bruce Richardson]

If we want to add support for turning off components because of missing
dependencies, then we need to check for those dependencies before we
make a determination as to whether a component should be built or not,
assuming that the component says it should be built.

Signed-off-by: Bruce Richardson <bruce.richardson@intel.com>
---
 drivers/meson.build | 27 ++++++++++++++-------------
 lib/meson.build     | 23 +++++++++++++----------
 2 files changed, 27 insertions(+), 23 deletions(-)

diff --git a/drivers/meson.build b/drivers/meson.build
index b54601697..ee5db4157 100644
--- a/drivers/meson.build
+++ b/drivers/meson.build
@@ -59,6 +59,20 @@ foreach class:dpdk_driver_classes
 		# pull in driver directory which should assign to each of the above
 		subdir(drv_path)
 
+		if build
+			# get dependency objs from strings
+			shared_deps = ext_deps
+			static_deps = ext_deps
+			foreach d:deps
+				if not is_variable('shared_rte_' + d)
+					error('Missing internal dependency "@0@" for @1@ [@2@]'
+							.format(d, name, 'drivers/' + drv_path))
+				endif
+				shared_deps += [get_variable('shared_rte_' + d)]
+				static_deps += [get_variable('static_rte_' + d)]
+			endforeach
+		endif
+
 		if not build
 			# some driver directories are placeholders which
 			# are never built, so we allow suppression of the
@@ -78,19 +92,6 @@ foreach class:dpdk_driver_classes
 				cflags += '-DALLOW_EXPERIMENTAL_API'
 			endif
 
-			# get dependency objs from strings
-			shared_deps = []
-			static_deps = []
-			foreach d:deps
-				if not is_variable('shared_rte_' + d)
-					error('Missing dependency ' + d +
-						' for driver ' + lib_name)
-				endif
-				shared_deps += [get_variable('shared_rte_' + d)]
-				static_deps += [get_variable('static_rte_' + d)]
-			endforeach
-			shared_deps += ext_deps
-			static_deps += ext_deps
 			dpdk_extra_ldflags += pkgconfig_extra_libs
 
 			# generate pmdinfo sources by building a temporary
diff --git a/lib/meson.build b/lib/meson.build
index e5ff83893..fde2f5a02 100644
--- a/lib/meson.build
+++ b/lib/meson.build
@@ -66,6 +66,19 @@ foreach l:libraries
 	dir_name = 'librte_' + l
 	subdir(dir_name)
 
+	if build
+		shared_deps = ext_deps
+		static_deps = ext_deps
+		foreach d:deps
+			if not is_variable('shared_rte_' + d)
+				error('Missing internal dependency "@0@" for @1@ [@2@]'
+						.format(d, name, 'lib/' + dir_name))
+			endif
+			shared_deps += [get_variable('shared_rte_' + d)]
+			static_deps += [get_variable('static_rte_' + d)]
+		endforeach
+	endif
+
 	if not build
 		dpdk_libs_disabled += name
 		set_variable(name.underscorify() + '_disable_reason', reason)
@@ -82,16 +95,6 @@ foreach l:libraries
 			shared_dep = declare_dependency(include_directories: includes)
 			static_dep = shared_dep
 		else
-			shared_deps = ext_deps
-			static_deps = ext_deps
-			foreach d:deps
-				if not is_variable('shared_rte_' + d)
-					error('Missing dependency ' + d +
-						' for library ' + libname)
-				endif
-				shared_deps += [get_variable('shared_rte_' + d)]
-				static_deps += [get_variable('static_rte_' + d)]
-			endforeach
 
 			if allow_experimental_apis
 				cflags += '-DALLOW_EXPERIMENTAL_API'
-- 
2.21.0

[dpdk-dev] [PATCH 3/3] build: support disabling drivers with meson

[Bruce Richardson]

Add support for a new build option to turn off certain drivers. Any other
drivers which depend on the one being disabled will also be disabled with a
suitable debug message.

Signed-off-by: Bruce Richardson <bruce.richardson@intel.com>
---
 drivers/meson.build | 16 ++++++++++++----
 meson_options.txt   |  2 ++
 2 files changed, 14 insertions(+), 4 deletions(-)

diff --git a/drivers/meson.build b/drivers/meson.build
index ee5db4157..e7a696d52 100644
--- a/drivers/meson.build
+++ b/drivers/meson.build
@@ -16,6 +16,8 @@ dpdk_driver_classes = ['common',
 	       'baseband', # depends on common and bus.
 	       'raw']     # depends on common, bus, mempool, net and event.
 
+disabled_drivers = get_option('disable_drivers').split(',')
+
 default_cflags = machine_args
 if cc.has_argument('-Wno-format-truncation')
 	default_cflags += '-Wno-format-truncation'
@@ -59,17 +61,23 @@ foreach class:dpdk_driver_classes
 		# pull in driver directory which should assign to each of the above
 		subdir(drv_path)
 
-		if build
+		if drv_path in disabled_drivers
+			build = false
+			reason = 'Explicitly disabled via build config'
+		elif build
 			# get dependency objs from strings
 			shared_deps = ext_deps
 			static_deps = ext_deps
 			foreach d:deps
 				if not is_variable('shared_rte_' + d)
-					error('Missing internal dependency "@0@" for @1@ [@2@]'
+					build = false
+					reason = 'Missing internal dependency, "@0@"'.format(d)
+					message('Disabling @1@ [@2@]: missing internal dependency "@0@"'
 							.format(d, name, 'drivers/' + drv_path))
+				else
+					shared_deps += [get_variable('shared_rte_' + d)]
+					static_deps += [get_variable('static_rte_' + d)]
 				endif
-				shared_deps += [get_variable('shared_rte_' + d)]
-				static_deps += [get_variable('static_rte_' + d)]
 			endforeach
 		endif
 
diff --git a/meson_options.txt b/meson_options.txt
index 448f3e63d..89650b0e9 100644
--- a/meson_options.txt
+++ b/meson_options.txt
@@ -2,6 +2,8 @@
 
 option('allow_invalid_socket_id', type: 'boolean', value: false,
 	description: 'allow out-of-range NUMA socket id\'s for platforms that don\'t report the value correctly')
+option('disable_drivers', type: 'string', value: '',
+	description: 'Comma-separated list of drivers to explicitly disable.')
 option('drivers_install_subdir', type: 'string', value: 'dpdk/pmds-<VERSION>',
 	description: 'Subdirectory of libdir where to install PMDs. Defaults to using a versioned subdirectory.')
 option('enable_docs', type: 'boolean', value: false,
-- 
2.21.0

Re: [dpdk-dev] [PATCH 0/3] RFC: Support disabling DPDK drivers in meson builds

[Luca Boccassi]

On Wed, 2019-09-25 at 15:55 +0100, Bruce Richardson wrote:
> Following on from the discussion about meson at userspace 2019, here
> is
> a patchset which allows a user to disable drivers in a DPDK build.
> This
> set also includes the changes necessary to recursively disable any
> drivers that depend upon the disabled one, e.g. disabling
> mempool/dpaa
> also disables net/dpaa and event/dpaa with suitable reasons provided
> in
> the output summary at the end of the meson configuration phase.
> 
> A number of changes are made to the lib/meson.build file too, in
> order
> to keep the general logic flow between drivers/meson.build and
> lib/meson.build as similar as possible.
> 
> Example of use:
> 
> 	meson -Ddisable_drivers=mempool/dpaa,net/i40e build
> 
> Bruce Richardson (3):
>   build: align variable names between drivers and libs
>   build: process dependencies before main build check
>   build: support disabling drivers with meson
> 
>  drivers/meson.build | 45 +++++++++++++++++++++++++++--------------
> ----
>  lib/meson.build     | 23 +++++++++++++----------
>  meson_options.txt   |  2 ++
>  3 files changed, 42 insertions(+), 28 deletions(-)

Series-grumbly-acked-by: Luca Boccassi <bluca@debian.org>

-- 
Kind regards,
Luca Boccassi

Re: [dpdk-dev] [PATCH 0/3] RFC: Support disabling DPDK drivers in meson builds

[Bruce Richardson]

On Wed, Sep 25, 2019 at 07:01:10PM +0100, Luca Boccassi wrote:
> On Wed, 2019-09-25 at 15:55 +0100, Bruce Richardson wrote:
> > Following on from the discussion about meson at userspace 2019, here
> > is
> > a patchset which allows a user to disable drivers in a DPDK build.
> > This
> > set also includes the changes necessary to recursively disable any
> > drivers that depend upon the disabled one, e.g. disabling
> > mempool/dpaa
> > also disables net/dpaa and event/dpaa with suitable reasons provided
> > in
> > the output summary at the end of the meson configuration phase.
> > 
> > A number of changes are made to the lib/meson.build file too, in
> > order
> > to keep the general logic flow between drivers/meson.build and
> > lib/meson.build as similar as possible.
> > 
> > Example of use:
> > 
> > 	meson -Ddisable_drivers=mempool/dpaa,net/i40e build
> > 
> > Bruce Richardson (3):
> >   build: align variable names between drivers and libs
> >   build: process dependencies before main build check
> >   build: support disabling drivers with meson
> > 
> >  drivers/meson.build | 45 +++++++++++++++++++++++++++--------------
> > ----
> >  lib/meson.build     | 23 +++++++++++++----------
> >  meson_options.txt   |  2 ++
> >  3 files changed, 42 insertions(+), 28 deletions(-)
> 
> Series-grumbly-acked-by: Luca Boccassi <bluca@debian.org>
>
Not exactly doing my happy-dance here myself about this set, but I think
it's something those building their own DPDK will appreciate. 

Re: [dpdk-dev] [PATCH 0/3] RFC: Support disabling DPDK drivers in meson builds

[Morten Brørup]

> -----Original Message-----
> From: dev [mailto:dev-bounces@dpdk.org] On Behalf Of Bruce Richardson
> Sent: Thursday, September 26, 2019 10:34 AM
> To: Luca Boccassi
> Cc: dev@dpdk.org; thomas@monjalon.net; Morten Brørup; techboard@dpdk.org
> Subject: Re: [dpdk-dev] [PATCH 0/3] RFC: Support disabling DPDK drivers in
> meson builds
> 
> On Wed, Sep 25, 2019 at 07:01:10PM +0100, Luca Boccassi wrote:
> > On Wed, 2019-09-25 at 15:55 +0100, Bruce Richardson wrote:
> > > Following on from the discussion about meson at userspace 2019, here
> > > is
> > > a patchset which allows a user to disable drivers in a DPDK build.
> > > This
> > > set also includes the changes necessary to recursively disable any
> > > drivers that depend upon the disabled one, e.g. disabling
> > > mempool/dpaa
> > > also disables net/dpaa and event/dpaa with suitable reasons provided
> > > in
> > > the output summary at the end of the meson configuration phase.
> > >
> > > A number of changes are made to the lib/meson.build file too, in
> > > order
> > > to keep the general logic flow between drivers/meson.build and
> > > lib/meson.build as similar as possible.
> > >
> > > Example of use:
> > >
> > > 	meson -Ddisable_drivers=mempool/dpaa,net/i40e build
> > >
> > > Bruce Richardson (3):
> > >   build: align variable names between drivers and libs
> > >   build: process dependencies before main build check
> > >   build: support disabling drivers with meson
> > >
> > >  drivers/meson.build | 45 +++++++++++++++++++++++++++--------------
> > > ----
> > >  lib/meson.build     | 23 +++++++++++++----------
> > >  meson_options.txt   |  2 ++
> > >  3 files changed, 42 insertions(+), 28 deletions(-)
> >
> > Series-grumbly-acked-by: Luca Boccassi <bluca@debian.org>
> >
> Not exactly doing my happy-dance here myself about this set, but I think
> it's something those building their own DPDK will appreciate.

Yes, we will.


Med venlig hilsen / kind regards
- Morten Brørup

Re: [dpdk-dev] [dpdk-techboard] [PATCH 0/3] RFC: Support disabling DPDK drivers in meson builds

[Maxime Coquelin]

On 9/25/19 4:55 PM, Bruce Richardson wrote:
> Following on from the discussion about meson at userspace 2019, here is
> a patchset which allows a user to disable drivers in a DPDK build. This
> set also includes the changes necessary to recursively disable any
> drivers that depend upon the disabled one, e.g. disabling mempool/dpaa
> also disables net/dpaa and event/dpaa with suitable reasons provided in
> the output summary at the end of the meson configuration phase.
> 
> A number of changes are made to the lib/meson.build file too, in order
> to keep the general logic flow between drivers/meson.build and
> lib/meson.build as similar as possible.
> 
> Example of use:
> 
> 	meson -Ddisable_drivers=mempool/dpaa,net/i40e build
> 
> Bruce Richardson (3):
>   build: align variable names between drivers and libs
>   build: process dependencies before main build check
>   build: support disabling drivers with meson
> 
>  drivers/meson.build | 45 +++++++++++++++++++++++++++------------------
>  lib/meson.build     | 23 +++++++++++++----------
>  meson_options.txt   |  2 ++
>  3 files changed, 42 insertions(+), 28 deletions(-)
> 

Acked-by: Maxime Coquelin <maxime.coquelin@redhat.com>

Thanks,
Maxime

Re: [dpdk-dev] [PATCH 0/3] RFC: Support disabling DPDK drivers in meson builds

[Thomas Monjalon]

25/09/2019 20:01, Luca Boccassi:
> On Wed, 2019-09-25 at 15:55 +0100, Bruce Richardson wrote:
> > Bruce Richardson (3):
> >   build: align variable names between drivers and libs
> >   build: process dependencies before main build check
> >   build: support disabling drivers with meson
> 
> Series-grumbly-acked-by: Luca Boccassi <bluca@debian.org>

Applied, thanks for implementing a community request

RE: [PATCH 3/3] build: support disabling drivers with meson

[Morten Brørup]

> From: Bruce Richardson [mailto:bruce.richardson@intel.com]
> Sent: Wednesday, 25 September 2019 16.56
> 
> Add support for a new build option to turn off certain drivers. Any
> other
> drivers which depend on the one being disabled will also be disabled
> with a
> suitable debug message.
> 
> Signed-off-by: Bruce Richardson <bruce.richardson@intel.com>

Hi Bruce,

I was trying something along the lines of:

meson configure -Ddisable_drivers=net/* -Denable_drivers=net/i40e

But it seems that enable_drivers does not override disable_drivers. Ninja says:

[...]
net/hinic:      not in enabled drivers build config
net/hns3:       not in enabled drivers build config
net/i40e:       explicitly disabled via build config
net/iavf:       not in enabled drivers build config
[...]

Could you please fix that?

Also, since all sorts of exotic libraries are continuously being added to DPDK, it would be nice to have a similar Meson option for libs, i.e. enable_libs to override disable_libs=*.


Med venlig hilsen / Kind regards,
-Morten Brørup

Re: [PATCH 3/3] build: support disabling drivers with meson

[Thomas Monjalon]

06/01/2022 09:23, Morten Brørup:
> > From: Bruce Richardson [mailto:bruce.richardson@intel.com]
> > Sent: Wednesday, 25 September 2019 16.56
> > 
> > Add support for a new build option to turn off certain drivers. Any
> > other
> > drivers which depend on the one being disabled will also be disabled
> > with a
> > suitable debug message.
> > 
> > Signed-off-by: Bruce Richardson <bruce.richardson@intel.com>
> 
> Hi Bruce,
> 
> I was trying something along the lines of:
> 
> meson configure -Ddisable_drivers=net/* -Denable_drivers=net/i40e
> 
> But it seems that enable_drivers does not override disable_drivers. Ninja says:

It is the opposite logic. "disable" overrides "enable":

        if not enable_drivers.contains(drv_path)
            build = false
            reason = 'not in enabled drivers build config'
        elif disable_drivers.contains(drv_path)
            if always_enable.contains(drv_path)
                message('Driver @0@ cannot be disabled, not disabling.'.format(drv_path))
            else
                build = false
                reason = 'explicitly disabled via build config'
            endif
        endif

In this case, you don't need to disable everything because of this:

if enable_drivers.length() == 0
    enable_drivers = run_command(list_dir_globs, '*/*', check: true).stdout().split()
endif

RE: [PATCH 3/3] build: support disabling drivers with meson

[Morten Brørup]

> From: Thomas Monjalon [mailto:thomas@monjalon.net]
> Sent: Thursday, 6 January 2022 10.49
> 
> 06/01/2022 09:23, Morten Brørup:
> > > From: Bruce Richardson [mailto:bruce.richardson@intel.com]
> > > Sent: Wednesday, 25 September 2019 16.56
> > >
> > > Add support for a new build option to turn off certain drivers. Any
> > > other
> > > drivers which depend on the one being disabled will also be
> disabled
> > > with a
> > > suitable debug message.
> > >
> > > Signed-off-by: Bruce Richardson <bruce.richardson@intel.com>
> >
> > Hi Bruce,
> >
> > I was trying something along the lines of:
> >
> > meson configure -Ddisable_drivers=net/* -Denable_drivers=net/i40e
> >
> > But it seems that enable_drivers does not override disable_drivers.
> Ninja says:
> 
> It is the opposite logic. "disable" overrides "enable":
> 
>         if not enable_drivers.contains(drv_path)
>             build = false
>             reason = 'not in enabled drivers build config'
>         elif disable_drivers.contains(drv_path)
>             if always_enable.contains(drv_path)
>                 message('Driver @0@ cannot be disabled, not
> disabling.'.format(drv_path))
>             else
>                 build = false
>                 reason = 'explicitly disabled via build config'
>             endif
>         endif
> 
> In this case, you don't need to disable everything because of this:
> 
> if enable_drivers.length() == 0
>     enable_drivers = run_command(list_dir_globs, '*/*', check:
> true).stdout().split()
> endif
> 

Thank you, @Thomas! That solved the drivers part of my question.

@Bruce:

If you have too much time on your hands, and want to experiment with set theory on Meson, you could implement a "longest prefix match" for this, making the narrower selection take precedence. Just kidding! Enabling/disabling drivers as Thomas described works great.

Still, a similar Meson option for selecting libraries would be nice.

Re: [PATCH 3/3] build: support disabling drivers with meson

[Bruce Richardson]

On Thu, Jan 06, 2022 at 11:10:27AM +0100, Morten Brørup wrote:
> > From: Thomas Monjalon [mailto:thomas@monjalon.net]
> > Sent: Thursday, 6 January 2022 10.49
> > 
> > 06/01/2022 09:23, Morten Brørup:
> > > > From: Bruce Richardson [mailto:bruce.richardson@intel.com]
> > > > Sent: Wednesday, 25 September 2019 16.56
> > > >
> > > > Add support for a new build option to turn off certain drivers. Any
> > > > other
> > > > drivers which depend on the one being disabled will also be
> > disabled
> > > > with a
> > > > suitable debug message.
> > > >
> > > > Signed-off-by: Bruce Richardson <bruce.richardson@intel.com>
> > >
> > > Hi Bruce,
> > >
> > > I was trying something along the lines of:
> > >
> > > meson configure -Ddisable_drivers=net/* -Denable_drivers=net/i40e
> > >
> > > But it seems that enable_drivers does not override disable_drivers.
> > Ninja says:
> > 
> > It is the opposite logic. "disable" overrides "enable":
> > 
> >         if not enable_drivers.contains(drv_path)
> >             build = false
> >             reason = 'not in enabled drivers build config'
> >         elif disable_drivers.contains(drv_path)
> >             if always_enable.contains(drv_path)
> >                 message('Driver @0@ cannot be disabled, not
> > disabling.'.format(drv_path))
> >             else
> >                 build = false
> >                 reason = 'explicitly disabled via build config'
> >             endif
> >         endif
> > 
> > In this case, you don't need to disable everything because of this:
> > 
> > if enable_drivers.length() == 0
> >     enable_drivers = run_command(list_dir_globs, '*/*', check:
> > true).stdout().split()
> > endif
> > 
> 
> Thank you, @Thomas! That solved the drivers part of my question.
> 
> @Bruce:
> 
> If you have too much time on your hands, and want to experiment with set theory on Meson, you could implement a "longest prefix match" for this, making the narrower selection take precedence. Just kidding! Enabling/disabling drivers as Thomas described works great.
> 
> Still, a similar Meson option for selecting libraries would be nice.
>
Yes, I understand. I have a few concerns about implementing this just now
though. David previously posted a patch[1] to do what you are looking for, and
please see the discussion in that thread [2].

[1] http://inbox.dpdk.org/dev/20211110164814.5231-6-david.marchand@redhat.com/
[2] http://inbox.dpdk.org/dev/YYwCj0voYVUUYVMf@bricha3-MOBL.ger.corp.intel.com/