[dpdk-dev] [PATCH 0/6] meson build improvements

[dpdk-dev] [PATCH 0/6] meson build improvements

[Bruce Richardson]

These patches make some improvements to the meson build, particularly
for documentation. They also remove many, but not all warnings issued by
meson e.g. warnings about newer features unsupported in baseline.

The biggest change is to improve the handling of the guide html docs.
The change here is more significant, and the doc build now uses a
wrapper script around sphinx. This wrapper script allows us to output
correct dependency information for the sphinx build in a .d file. This
.d file is processed by ninja (not meson) on build, so that any changes
to doc files trigger a rebuild to the guides using sphinx.

For now, the two patches which remove the meson version warnings are
CC'ed to stable for backport, theoretically this who set could be
backported if so desired, as all changes could be considered fixes to
some degree or other, and nothing introduces a whole new feature.

Note: for completeness and simplicity, previously submitted patch
http://patches.dpdk.org/patch/64189/ is included in this set, and will
be marked superceded in patchwork.

Bruce Richardson (6):
  kernel/linux/kni: fix meson warning about console keyword
  build: skip processing docs folder if docs disabled
  doc/api: fix warning with meson build
  doc/guides: reduce whitespace in meson build file
  doc/guides: rebuild with meson whenever a file changes
  doc/api: reduce indentation in meson build file

 buildtools/call-sphinx-build.py | 29 ++++++++++
 buildtools/meson.build          |  6 +-
 doc/api/meson.build             | 99 +++++++++++++++++----------------
 doc/guides/meson.build          | 38 ++++++-------
 doc/meson.build                 |  4 ++
 kernel/linux/kni/meson.build    |  1 -
 6 files changed, 104 insertions(+), 73 deletions(-)
 create mode 100755 buildtools/call-sphinx-build.py

-- 
2.24.1

[dpdk-dev] [PATCH 1/6] kernel/linux/kni: fix meson warning about console keyword

[Bruce Richardson]

Since kni no longer includes the ethtool code and so is faster to build, we
no longer need the console parameter to have incremental screen updates as
it builds. Therefore, we drop the keyword which removes the warning.

Fixes: b78f32cff94d ("kni: support meson build")
Cc: bluca@debian.org

Signed-off-by: Bruce Richardson <bruce.richardson@intel.com>
---
 kernel/linux/kni/meson.build | 1 -
 1 file changed, 1 deletion(-)

diff --git a/kernel/linux/kni/meson.build b/kernel/linux/kni/meson.build
index 955eec949..f93e97fa0 100644
--- a/kernel/linux/kni/meson.build
+++ b/kernel/linux/kni/meson.build
@@ -23,7 +23,6 @@ custom_target('rte_kni',
 		' -I' + meson.current_source_dir(),
 		'modules'],
 	depends: kni_mkfile,
-	console: true,
 	install: true,
 	install_dir: kernel_dir + '/extra/dpdk',
 	build_by_default: get_option('enable_kmods'))
-- 
2.24.1

[dpdk-dev] [PATCH 2/6] build: skip processing docs folder if docs disabled

[Bruce Richardson]

While each target is set to be ignored if the docs are disabled in the
meson build, there is little reason to process the docs folder at all.

Signed-off-by: Bruce Richardson <bruce.richardson@intel.com>
---
 doc/meson.build | 4 ++++
 1 file changed, 4 insertions(+)

diff --git a/doc/meson.build b/doc/meson.build
index c5410d85d..c49ec8476 100644
--- a/doc/meson.build
+++ b/doc/meson.build
@@ -1,6 +1,10 @@
 # SPDX-License-Identifier: BSD-3-Clause
 # Copyright(c) 2018 Luca Boccassi <bluca@debian.org>
 
+if not get_option('enable_docs')
+	subdir_done()
+endif
+
 doc_targets = []
 doc_target_names = []
 subdir('api')
-- 
2.24.1

[dpdk-dev] [PATCH 3/6] doc/api: fix warning with meson build

[Bruce Richardson]

The install parameter to configure_file is new in 0.50 and generates a
warning since it is newer than our minimum version of 0.47.1. The
parameter, however, is unneeded as the documentation states:

"When omitted it defaults to true when install_dir is set and not empty,
false otherwise."

Given that install_dir is not set for this file, install defaults to false
so no need to explicitly specify it.

Fixes: 720b14db3ae2 ("build: generate API documentation with meson")
Cc: bluca@debian.org

Signed-off-by: Bruce Richardson <bruce.richardson@intel.com>
---
 doc/api/meson.build | 3 +--
 1 file changed, 1 insertion(+), 2 deletions(-)

diff --git a/doc/api/meson.build b/doc/api/meson.build
index 1c48b7672..23a7dfc75 100644
--- a/doc/api/meson.build
+++ b/doc/api/meson.build
@@ -38,8 +38,7 @@ if doxygen.found()
 
 	doxy_conf = configure_file(input: 'doxy-api.conf.in',
 		output: 'doxy-api.conf',
-		configuration: cdata,
-		install: false)
+		configuration: cdata)
 
 	doxy_build = custom_target('doxygen',
 		depends: example,
-- 
2.24.1

[dpdk-dev] [PATCH 4/6] doc/guides: reduce whitespace in meson build file

[Bruce Richardson]

For building the guides, we can make the meson.build easier to read by
using the subdir_done function to quit early.

Signed-off-by: Bruce Richardson <bruce.richardson@intel.com>
---
 doc/guides/meson.build | 44 ++++++++++++++++++++++--------------------
 1 file changed, 23 insertions(+), 21 deletions(-)

diff --git a/doc/guides/meson.build b/doc/guides/meson.build
index 7931ef3bb..80c21d168 100644
--- a/doc/guides/meson.build
+++ b/doc/guides/meson.build
@@ -3,26 +3,28 @@
 
 sphinx = find_program('sphinx-build', required: get_option('enable_docs'))
 
-if sphinx.found()
-	htmldir = join_paths('share', 'doc', 'dpdk')
-	html_guides_build = custom_target('html_guides_build',
-		input: meson.current_source_dir(),
-		output: 'guides',
-		command: [sphinx, '-b', 'html',
-			'-d', meson.current_build_dir() + '/.doctrees',
-			'@INPUT@', meson.current_build_dir() + '/guides'],
-		build_by_default: get_option('enable_docs'),
-		install: get_option('enable_docs'),
-		install_dir: htmldir)
+if not sphinx.found()
+	subdir_done()
+endif
 
-	doc_targets += html_guides_build
-	doc_target_names += 'HTML_Guides'
+htmldir = join_paths('share', 'doc', 'dpdk')
+html_guides = custom_target('html_guides',
+	input: meson.current_source_dir(),
+	output: 'guides',
+	command: [sphinx, '-b', 'html',
+		'-d', meson.current_build_dir() + '/.doctrees',
+		'@INPUT@', meson.current_build_dir() + '/guides'],
+	build_by_default: get_option('enable_docs'),
+	install: get_option('enable_docs'),
+	install_dir: htmldir)
 
-	# sphinx leaves a .buildinfo in the target directory, which we don't
-	# want to install. Note that sh -c has to be used, otherwise the
-	# env var does not get expanded if calling rm/install directly.
-	meson.add_install_script('sh', '-c',
-		'rm -f $MESON_INSTALL_DESTDIR_PREFIX/share/doc/dpdk/guides/.buildinfo')
-	meson.add_install_script('sh', '-c',
-		'install -D -m0644 $MESON_SOURCE_ROOT/doc/guides/custom.css $MESON_INSTALL_DESTDIR_PREFIX/share/doc/dpdk/guides/_static/css/custom.css')
-endif
+doc_targets += html_guides
+doc_target_names += 'HTML_Guides'
+
+# sphinx leaves a .buildinfo in the target directory, which we don't
+# want to install. Note that sh -c has to be used, otherwise the
+# env var does not get expanded if calling rm/install directly.
+meson.add_install_script('sh', '-c',
+	'rm -f $MESON_INSTALL_DESTDIR_PREFIX/share/doc/dpdk/guides/.buildinfo')
+meson.add_install_script('sh', '-c',
+	'install -D -m0644 $MESON_SOURCE_ROOT/doc/guides/custom.css $MESON_INSTALL_DESTDIR_PREFIX/share/doc/dpdk/guides/_static/css/custom.css')
-- 
2.24.1

[dpdk-dev] [PATCH 5/6] doc/guides: rebuild with meson whenever a file changes

[Bruce Richardson]

Add proper support for calling sphinx whenever a file in the doc
directory changes. This is accomplished by using a wrapper script
for sphinx, which runs sphinx but also emits a gcc-format dependency
file listing all the doc files. This is used by ninja so that any
change to the doc files triggers a rebuild of the docs.

Signed-off-by: Bruce Richardson <bruce.richardson@intel.com>
---
 buildtools/call-sphinx-build.py | 29 +++++++++++++++++++++++++++++
 buildtools/meson.build          |  6 ++++--
 doc/guides/meson.build          | 22 ++++++++--------------
 3 files changed, 41 insertions(+), 16 deletions(-)
 create mode 100755 buildtools/call-sphinx-build.py

diff --git a/buildtools/call-sphinx-build.py b/buildtools/call-sphinx-build.py
new file mode 100755
index 000000000..027317b9b
--- /dev/null
+++ b/buildtools/call-sphinx-build.py
@@ -0,0 +1,29 @@
+#! /usr/bin/env python3
+# SPDX-License-Identifier: BSD-3-Clause
+# Copyright(c) 2019 Intel Corporation
+#
+
+import sys
+import os
+import os.path
+import subprocess
+
+sphinx = sys.argv[1]
+src = sys.argv[2]
+dst = sys.argv[3]
+depfile = os.path.join(dst,'.html.d')
+
+# find all the files sphinx will process so we can write them as dependencies
+srcfiles = []
+for root, dirs, files in os.walk(src):
+    for f in files:
+        srcfiles.append(os.path.join(root, f))
+
+# run sphinx, putting the html output in a "html" directory
+subprocess.run([sphinx, '-j', 'auto', '-b', 'html', src,
+                os.path.join(dst, 'html')], check = True)
+
+# create a gcc format .d file giving all the dependencies of this doc build
+with open(depfile, 'w') as d:
+    d.write('html: ' + ' '.join(srcfiles) + '\n')
+subprocess.run(['cp', '-f', depfile, '/tmp'])
diff --git a/buildtools/meson.build b/buildtools/meson.build
index 6ef2c5721..cd1d05403 100644
--- a/buildtools/meson.build
+++ b/buildtools/meson.build
@@ -10,10 +10,12 @@ check_experimental_syms = find_program('check-experimental-syms.sh')
 # set up map-to-def script using python, either built-in or external
 python3 = import('python').find_installation(required: false)
 if python3.found()
-	map_to_def_cmd = [python3, files('map_to_def.py')]
+	py3 = [python3]
 else
-	map_to_def_cmd = ['meson', 'runpython', files('map_to_def.py')]
+	py3 = ['meson', 'runpython']
 endif
+map_to_def_cmd = py3 + files('map_to_def.py')
+sphinx_wrapper = py3 + files('call-sphinx-build.py')
 
 # stable ABI always starts with "DPDK_"
 is_experimental_cmd = [find_program('grep', 'findstr'), '^DPDK_']
diff --git a/doc/guides/meson.build b/doc/guides/meson.build
index 80c21d168..732e7ad3a 100644
--- a/doc/guides/meson.build
+++ b/doc/guides/meson.build
@@ -7,24 +7,18 @@ if not sphinx.found()
 	subdir_done()
 endif
 
-htmldir = join_paths('share', 'doc', 'dpdk')
+htmldir = join_paths(get_option('datadir'), 'doc', 'dpdk')
 html_guides = custom_target('html_guides',
-	input: meson.current_source_dir(),
-	output: 'guides',
-	command: [sphinx, '-b', 'html',
-		'-d', meson.current_build_dir() + '/.doctrees',
-		'@INPUT@', meson.current_build_dir() + '/guides'],
+	input: files('index.rst'),
+	output: 'html',
+	command: [sphinx_wrapper, sphinx, meson.current_source_dir(), meson.current_build_dir()],
+	depfile: '.html.d',
 	build_by_default: get_option('enable_docs'),
 	install: get_option('enable_docs'),
 	install_dir: htmldir)
 
+install_data(files('custom.css'),
+			install_dir: join_paths(htmldir,'_static', 'css'))
+
 doc_targets += html_guides
 doc_target_names += 'HTML_Guides'
-
-# sphinx leaves a .buildinfo in the target directory, which we don't
-# want to install. Note that sh -c has to be used, otherwise the
-# env var does not get expanded if calling rm/install directly.
-meson.add_install_script('sh', '-c',
-	'rm -f $MESON_INSTALL_DESTDIR_PREFIX/share/doc/dpdk/guides/.buildinfo')
-meson.add_install_script('sh', '-c',
-	'install -D -m0644 $MESON_SOURCE_ROOT/doc/guides/custom.css $MESON_INSTALL_DESTDIR_PREFIX/share/doc/dpdk/guides/_static/css/custom.css')
-- 
2.24.1

[dpdk-dev] [PATCH 6/6] doc/api: reduce indentation in meson build file

[Bruce Richardson]

When building the API docs, we can make the meson.build file easier to
read, and allow more code per line, by using subdir_done() to quit early.

Signed-off-by: Bruce Richardson <bruce.richardson@intel.com>
---
 doc/api/meson.build | 98 +++++++++++++++++++++++----------------------
 1 file changed, 50 insertions(+), 48 deletions(-)

diff --git a/doc/api/meson.build b/doc/api/meson.build
index 23a7dfc75..c72b880e1 100644
--- a/doc/api/meson.build
+++ b/doc/api/meson.build
@@ -3,52 +3,54 @@
 
 doxygen = find_program('doxygen', required: get_option('enable_docs'))
 
-if doxygen.found()
-	# due to the CSS customisation script, which needs to run on a file that
-	# is in a subdirectory that is created at build time and thus it cannot
-	# be an individual custom_target, we need to wrap the doxygen call in a
-	# script to run the CSS modification afterwards
-	generate_doxygen = find_program('generate_doxygen.sh')
-	generate_examples = find_program('generate_examples.sh')
-	generate_css = find_program('doxy-html-custom.sh')
-
-	inputdir = join_paths(meson.source_root(), 'examples')
-	htmldir = join_paths('share', 'doc', 'dpdk')
-
-	# due to the following bug: https://github.com/mesonbuild/meson/issues/4107
-	# if install is set to true it will override build_by_default and it will
-	# cause the target to always be built. If install were to be always set to
-	# false it would be impossible to install the docs.
-	# So use a configure option for now.
-	example = custom_target('examples.dox',
-		input: inputdir,
-		output: 'examples.dox',
-		command: [generate_examples, '@INPUT@', '@OUTPUT@'],
-		install: get_option('enable_docs'),
-		install_dir: htmldir,
-		build_by_default: get_option('enable_docs'))
-
-	cdata = configuration_data()
-	cdata.set('VERSION', meson.project_version())
-	cdata.set('API_EXAMPLES', join_paths(meson.build_root(), 'doc', 'api', 'examples.dox'))
-	cdata.set('OUTPUT', join_paths(meson.build_root(), 'doc', 'api'))
-	cdata.set('HTML_OUTPUT', 'api')
-	cdata.set('TOPDIR', meson.source_root())
-	cdata.set('STRIP_FROM_PATH', meson.source_root())
-
-	doxy_conf = configure_file(input: 'doxy-api.conf.in',
-		output: 'doxy-api.conf',
-		configuration: cdata)
-
-	doxy_build = custom_target('doxygen',
-		depends: example,
-		input: doxy_conf,
-		output: 'api',
-		command: [generate_doxygen, '@INPUT@', '@OUTPUT@', generate_css],
-		install: get_option('enable_docs'),
-		install_dir: htmldir,
-		build_by_default: get_option('enable_docs'))
-
-	doc_targets += doxy_build
-	doc_target_names += 'Doxygen_API'
+if not doxygen.found()
+  subdir_done()
 endif
+
+# due to the CSS customisation script, which needs to run on a file that
+# is in a subdirectory that is created at build time and thus it cannot
+# be an individual custom_target, we need to wrap the doxygen call in a
+# script to run the CSS modification afterwards
+generate_doxygen = find_program('generate_doxygen.sh')
+generate_examples = find_program('generate_examples.sh')
+generate_css = find_program('doxy-html-custom.sh')
+
+inputdir = join_paths(meson.source_root(), 'examples')
+htmldir = join_paths('share', 'doc', 'dpdk')
+
+# due to the following bug: https://github.com/mesonbuild/meson/issues/4107
+# if install is set to true it will override build_by_default and it will
+# cause the target to always be built. If install were to be always set to
+# false it would be impossible to install the docs.
+# So use a configure option for now.
+example = custom_target('examples.dox',
+	input: inputdir,
+	output: 'examples.dox',
+	command: [generate_examples, '@INPUT@', '@OUTPUT@'],
+	install: get_option('enable_docs'),
+	install_dir: htmldir,
+	build_by_default: get_option('enable_docs'))
+
+cdata = configuration_data()
+cdata.set('VERSION', meson.project_version())
+cdata.set('API_EXAMPLES', join_paths(meson.build_root(), 'doc', 'api', 'examples.dox'))
+cdata.set('OUTPUT', join_paths(meson.build_root(), 'doc', 'api'))
+cdata.set('HTML_OUTPUT', 'api')
+cdata.set('TOPDIR', meson.source_root())
+cdata.set('STRIP_FROM_PATH', meson.source_root())
+
+doxy_conf = configure_file(input: 'doxy-api.conf.in',
+	output: 'doxy-api.conf',
+	configuration: cdata)
+
+doxy_build = custom_target('doxygen',
+	depends: example,
+	input: doxy_conf,
+	output: 'api',
+	command: [generate_doxygen, '@INPUT@', '@OUTPUT@', generate_css],
+	install: get_option('enable_docs'),
+	install_dir: htmldir,
+	build_by_default: get_option('enable_docs'))
+
+doc_targets += doxy_build
+doc_target_names += 'Doxygen_API'
-- 
2.24.1

[dpdk-dev] [PATCH v2 0/6] meson build improvements

[Bruce Richardson]

These patches make some improvements to the meson build, particularly
for documentation. They also remove many, but not all warnings issued by
meson e.g. warnings about newer features unsupported in baseline.

The biggest change is to improve the handling of the guide html docs.
The change here is more significant, and the doc build now uses a
wrapper script around sphinx. This wrapper script allows us to output
correct dependency information for the sphinx build in a .d file. This
.d file is processed by ninja (not meson) on build, so that any changes
to doc files trigger a rebuild to the guides using sphinx.

For now, the two patches which remove the meson version warnings are
CC'ed to stable for backport, theoretically this who set could be
backported if so desired, as all changes could be considered fixes to
some degree or other, and nothing introduces a whole new feature.

Note: for completeness and simplicity, previously submitted patch
http://patches.dpdk.org/patch/64189/ is included in this set, and will
be marked superceded in patchwork.

V2: resend to correct email addresses

Bruce Richardson (6):
  kernel/linux/kni: fix meson warning about console keyword
  build: skip processing docs folder if docs disabled
  doc/api: fix warning with meson build
  doc/guides: reduce whitespace in meson build file
  doc/guides: rebuild with meson whenever a file changes
  doc/api: reduce indentation in meson build file

 buildtools/call-sphinx-build.py | 29 ++++++++++
 buildtools/meson.build          |  6 +-
 doc/api/meson.build             | 99 +++++++++++++++++----------------
 doc/guides/meson.build          | 38 ++++++-------
 doc/meson.build                 |  4 ++
 kernel/linux/kni/meson.build    |  1 -
 6 files changed, 104 insertions(+), 73 deletions(-)
 create mode 100755 buildtools/call-sphinx-build.py

-- 
2.24.1

[dpdk-dev] [PATCH v2 0/6] meson build improvements

[Bruce Richardson]

These patches make some improvements to the meson build, particularly
for documentation. They also remove many, but not all warnings issued by
meson e.g. warnings about newer features unsupported in baseline.

The biggest change is to improve the handling of the guide html docs.
The change here is more significant, and the doc build now uses a
wrapper script around sphinx. This wrapper script allows us to output
correct dependency information for the sphinx build in a .d file. This
.d file is processed by ninja (not meson) on build, so that any changes
to doc files trigger a rebuild to the guides using sphinx.

For now, the two patches which remove the meson version warnings are
CC'ed to stable for backport, theoretically this who set could be
backported if so desired, as all changes could be considered fixes to
some degree or other, and nothing introduces a whole new feature.

Note: for completeness and simplicity, previously submitted patch
http://patches.dpdk.org/patch/64189/ is included in this set, and will
be marked superceded in patchwork.

V2: resend to correct email addresses

Bruce Richardson (6):
  kernel/linux/kni: fix meson warning about console keyword
  build: skip processing docs folder if docs disabled
  doc/api: fix warning with meson build
  doc/guides: reduce whitespace in meson build file
  doc/guides: rebuild with meson whenever a file changes
  doc/api: reduce indentation in meson build file

 buildtools/call-sphinx-build.py | 29 ++++++++++
 buildtools/meson.build          |  6 +-
 doc/api/meson.build             | 99 +++++++++++++++++----------------
 doc/guides/meson.build          | 38 ++++++-------
 doc/meson.build                 |  4 ++
 kernel/linux/kni/meson.build    |  1 -
 6 files changed, 104 insertions(+), 73 deletions(-)
 create mode 100755 buildtools/call-sphinx-build.py

-- 
2.24.1

[dpdk-dev] [PATCH v2 1/6] kernel/linux/kni: fix meson warning about console keyword

[Bruce Richardson]

Since kni no longer includes the ethtool code and so is faster to build, we
no longer need the console parameter to have incremental screen updates as
it builds. Therefore, we drop the keyword which removes the warning.

Fixes: b78f32cff94d ("kni: support meson build")
Cc: bluca@debian.org

Signed-off-by: Bruce Richardson <bruce.richardson@intel.com>
---
 kernel/linux/kni/meson.build | 1 -
 1 file changed, 1 deletion(-)

diff --git a/kernel/linux/kni/meson.build b/kernel/linux/kni/meson.build
index 955eec949..f93e97fa0 100644
--- a/kernel/linux/kni/meson.build
+++ b/kernel/linux/kni/meson.build
@@ -23,7 +23,6 @@ custom_target('rte_kni',
 		' -I' + meson.current_source_dir(),
 		'modules'],
 	depends: kni_mkfile,
-	console: true,
 	install: true,
 	install_dir: kernel_dir + '/extra/dpdk',
 	build_by_default: get_option('enable_kmods'))
-- 
2.24.1

[dpdk-dev] [PATCH v2 2/6] build: skip processing docs folder if docs disabled

[Bruce Richardson]

While each target is set to be ignored if the docs are disabled in the
meson build, there is little reason to process the docs folder at all.

Signed-off-by: Bruce Richardson <bruce.richardson@intel.com>
---
 doc/meson.build | 4 ++++
 1 file changed, 4 insertions(+)

diff --git a/doc/meson.build b/doc/meson.build
index c5410d85d..c49ec8476 100644
--- a/doc/meson.build
+++ b/doc/meson.build
@@ -1,6 +1,10 @@
 # SPDX-License-Identifier: BSD-3-Clause
 # Copyright(c) 2018 Luca Boccassi <bluca@debian.org>
 
+if not get_option('enable_docs')
+	subdir_done()
+endif
+
 doc_targets = []
 doc_target_names = []
 subdir('api')
-- 
2.24.1

[dpdk-dev] [PATCH v2 3/6] doc/api: fix warning with meson build

[Bruce Richardson]

The install parameter to configure_file is new in 0.50 and generates a
warning since it is newer than our minimum version of 0.47.1. The
parameter, however, is unneeded as the documentation states:

"When omitted it defaults to true when install_dir is set and not empty,
false otherwise."

Given that install_dir is not set for this file, install defaults to false
so no need to explicitly specify it.

Fixes: 720b14db3ae2 ("build: generate API documentation with meson")
Cc: bluca@debian.org

Signed-off-by: Bruce Richardson <bruce.richardson@intel.com>
---
 doc/api/meson.build | 3 +--
 1 file changed, 1 insertion(+), 2 deletions(-)

diff --git a/doc/api/meson.build b/doc/api/meson.build
index 1c48b7672..23a7dfc75 100644
--- a/doc/api/meson.build
+++ b/doc/api/meson.build
@@ -38,8 +38,7 @@ if doxygen.found()
 
 	doxy_conf = configure_file(input: 'doxy-api.conf.in',
 		output: 'doxy-api.conf',
-		configuration: cdata,
-		install: false)
+		configuration: cdata)
 
 	doxy_build = custom_target('doxygen',
 		depends: example,
-- 
2.24.1

[dpdk-dev] [PATCH v2 4/6] doc/guides: reduce whitespace in meson build file

[Bruce Richardson]

For building the guides, we can make the meson.build easier to read by
using the subdir_done function to quit early.

Signed-off-by: Bruce Richardson <bruce.richardson@intel.com>
---
 doc/guides/meson.build | 44 ++++++++++++++++++++++--------------------
 1 file changed, 23 insertions(+), 21 deletions(-)

diff --git a/doc/guides/meson.build b/doc/guides/meson.build
index 7931ef3bb..80c21d168 100644
--- a/doc/guides/meson.build
+++ b/doc/guides/meson.build
@@ -3,26 +3,28 @@
 
 sphinx = find_program('sphinx-build', required: get_option('enable_docs'))
 
-if sphinx.found()
-	htmldir = join_paths('share', 'doc', 'dpdk')
-	html_guides_build = custom_target('html_guides_build',
-		input: meson.current_source_dir(),
-		output: 'guides',
-		command: [sphinx, '-b', 'html',
-			'-d', meson.current_build_dir() + '/.doctrees',
-			'@INPUT@', meson.current_build_dir() + '/guides'],
-		build_by_default: get_option('enable_docs'),
-		install: get_option('enable_docs'),
-		install_dir: htmldir)
+if not sphinx.found()
+	subdir_done()
+endif
 
-	doc_targets += html_guides_build
-	doc_target_names += 'HTML_Guides'
+htmldir = join_paths('share', 'doc', 'dpdk')
+html_guides = custom_target('html_guides',
+	input: meson.current_source_dir(),
+	output: 'guides',
+	command: [sphinx, '-b', 'html',
+		'-d', meson.current_build_dir() + '/.doctrees',
+		'@INPUT@', meson.current_build_dir() + '/guides'],
+	build_by_default: get_option('enable_docs'),
+	install: get_option('enable_docs'),
+	install_dir: htmldir)
 
-	# sphinx leaves a .buildinfo in the target directory, which we don't
-	# want to install. Note that sh -c has to be used, otherwise the
-	# env var does not get expanded if calling rm/install directly.
-	meson.add_install_script('sh', '-c',
-		'rm -f $MESON_INSTALL_DESTDIR_PREFIX/share/doc/dpdk/guides/.buildinfo')
-	meson.add_install_script('sh', '-c',
-		'install -D -m0644 $MESON_SOURCE_ROOT/doc/guides/custom.css $MESON_INSTALL_DESTDIR_PREFIX/share/doc/dpdk/guides/_static/css/custom.css')
-endif
+doc_targets += html_guides
+doc_target_names += 'HTML_Guides'
+
+# sphinx leaves a .buildinfo in the target directory, which we don't
+# want to install. Note that sh -c has to be used, otherwise the
+# env var does not get expanded if calling rm/install directly.
+meson.add_install_script('sh', '-c',
+	'rm -f $MESON_INSTALL_DESTDIR_PREFIX/share/doc/dpdk/guides/.buildinfo')
+meson.add_install_script('sh', '-c',
+	'install -D -m0644 $MESON_SOURCE_ROOT/doc/guides/custom.css $MESON_INSTALL_DESTDIR_PREFIX/share/doc/dpdk/guides/_static/css/custom.css')
-- 
2.24.1

[dpdk-dev] [PATCH v2 5/6] doc/guides: rebuild with meson whenever a file changes

[Bruce Richardson]

Add proper support for calling sphinx whenever a file in the doc
directory changes. This is accomplished by using a wrapper script
for sphinx, which runs sphinx but also emits a gcc-format dependency
file listing all the doc files. This is used by ninja so that any
change to the doc files triggers a rebuild of the docs.

Signed-off-by: Bruce Richardson <bruce.richardson@intel.com>
---
 buildtools/call-sphinx-build.py | 29 +++++++++++++++++++++++++++++
 buildtools/meson.build          |  6 ++++--
 doc/guides/meson.build          | 22 ++++++++--------------
 3 files changed, 41 insertions(+), 16 deletions(-)
 create mode 100755 buildtools/call-sphinx-build.py

diff --git a/buildtools/call-sphinx-build.py b/buildtools/call-sphinx-build.py
new file mode 100755
index 000000000..027317b9b
--- /dev/null
+++ b/buildtools/call-sphinx-build.py
@@ -0,0 +1,29 @@
+#! /usr/bin/env python3
+# SPDX-License-Identifier: BSD-3-Clause
+# Copyright(c) 2019 Intel Corporation
+#
+
+import sys
+import os
+import os.path
+import subprocess
+
+sphinx = sys.argv[1]
+src = sys.argv[2]
+dst = sys.argv[3]
+depfile = os.path.join(dst,'.html.d')
+
+# find all the files sphinx will process so we can write them as dependencies
+srcfiles = []
+for root, dirs, files in os.walk(src):
+    for f in files:
+        srcfiles.append(os.path.join(root, f))
+
+# run sphinx, putting the html output in a "html" directory
+subprocess.run([sphinx, '-j', 'auto', '-b', 'html', src,
+                os.path.join(dst, 'html')], check = True)
+
+# create a gcc format .d file giving all the dependencies of this doc build
+with open(depfile, 'w') as d:
+    d.write('html: ' + ' '.join(srcfiles) + '\n')
+subprocess.run(['cp', '-f', depfile, '/tmp'])
diff --git a/buildtools/meson.build b/buildtools/meson.build
index 6ef2c5721..cd1d05403 100644
--- a/buildtools/meson.build
+++ b/buildtools/meson.build
@@ -10,10 +10,12 @@ check_experimental_syms = find_program('check-experimental-syms.sh')
 # set up map-to-def script using python, either built-in or external
 python3 = import('python').find_installation(required: false)
 if python3.found()
-	map_to_def_cmd = [python3, files('map_to_def.py')]
+	py3 = [python3]
 else
-	map_to_def_cmd = ['meson', 'runpython', files('map_to_def.py')]
+	py3 = ['meson', 'runpython']
 endif
+map_to_def_cmd = py3 + files('map_to_def.py')
+sphinx_wrapper = py3 + files('call-sphinx-build.py')
 
 # stable ABI always starts with "DPDK_"
 is_experimental_cmd = [find_program('grep', 'findstr'), '^DPDK_']
diff --git a/doc/guides/meson.build b/doc/guides/meson.build
index 80c21d168..732e7ad3a 100644
--- a/doc/guides/meson.build
+++ b/doc/guides/meson.build
@@ -7,24 +7,18 @@ if not sphinx.found()
 	subdir_done()
 endif
 
-htmldir = join_paths('share', 'doc', 'dpdk')
+htmldir = join_paths(get_option('datadir'), 'doc', 'dpdk')
 html_guides = custom_target('html_guides',
-	input: meson.current_source_dir(),
-	output: 'guides',
-	command: [sphinx, '-b', 'html',
-		'-d', meson.current_build_dir() + '/.doctrees',
-		'@INPUT@', meson.current_build_dir() + '/guides'],
+	input: files('index.rst'),
+	output: 'html',
+	command: [sphinx_wrapper, sphinx, meson.current_source_dir(), meson.current_build_dir()],
+	depfile: '.html.d',
 	build_by_default: get_option('enable_docs'),
 	install: get_option('enable_docs'),
 	install_dir: htmldir)
 
+install_data(files('custom.css'),
+			install_dir: join_paths(htmldir,'_static', 'css'))
+
 doc_targets += html_guides
 doc_target_names += 'HTML_Guides'
-
-# sphinx leaves a .buildinfo in the target directory, which we don't
-# want to install. Note that sh -c has to be used, otherwise the
-# env var does not get expanded if calling rm/install directly.
-meson.add_install_script('sh', '-c',
-	'rm -f $MESON_INSTALL_DESTDIR_PREFIX/share/doc/dpdk/guides/.buildinfo')
-meson.add_install_script('sh', '-c',
-	'install -D -m0644 $MESON_SOURCE_ROOT/doc/guides/custom.css $MESON_INSTALL_DESTDIR_PREFIX/share/doc/dpdk/guides/_static/css/custom.css')
-- 
2.24.1

[dpdk-dev] [PATCH v2 6/6] doc/api: reduce indentation in meson build file

[Bruce Richardson]

When building the API docs, we can make the meson.build file easier to
read, and allow more code per line, by using subdir_done() to quit early.

Signed-off-by: Bruce Richardson <bruce.richardson@intel.com>
---
 doc/api/meson.build | 98 +++++++++++++++++++++++----------------------
 1 file changed, 50 insertions(+), 48 deletions(-)

diff --git a/doc/api/meson.build b/doc/api/meson.build
index 23a7dfc75..c72b880e1 100644
--- a/doc/api/meson.build
+++ b/doc/api/meson.build
@@ -3,52 +3,54 @@
 
 doxygen = find_program('doxygen', required: get_option('enable_docs'))
 
-if doxygen.found()
-	# due to the CSS customisation script, which needs to run on a file that
-	# is in a subdirectory that is created at build time and thus it cannot
-	# be an individual custom_target, we need to wrap the doxygen call in a
-	# script to run the CSS modification afterwards
-	generate_doxygen = find_program('generate_doxygen.sh')
-	generate_examples = find_program('generate_examples.sh')
-	generate_css = find_program('doxy-html-custom.sh')
-
-	inputdir = join_paths(meson.source_root(), 'examples')
-	htmldir = join_paths('share', 'doc', 'dpdk')
-
-	# due to the following bug: https://github.com/mesonbuild/meson/issues/4107
-	# if install is set to true it will override build_by_default and it will
-	# cause the target to always be built. If install were to be always set to
-	# false it would be impossible to install the docs.
-	# So use a configure option for now.
-	example = custom_target('examples.dox',
-		input: inputdir,
-		output: 'examples.dox',
-		command: [generate_examples, '@INPUT@', '@OUTPUT@'],
-		install: get_option('enable_docs'),
-		install_dir: htmldir,
-		build_by_default: get_option('enable_docs'))
-
-	cdata = configuration_data()
-	cdata.set('VERSION', meson.project_version())
-	cdata.set('API_EXAMPLES', join_paths(meson.build_root(), 'doc', 'api', 'examples.dox'))
-	cdata.set('OUTPUT', join_paths(meson.build_root(), 'doc', 'api'))
-	cdata.set('HTML_OUTPUT', 'api')
-	cdata.set('TOPDIR', meson.source_root())
-	cdata.set('STRIP_FROM_PATH', meson.source_root())
-
-	doxy_conf = configure_file(input: 'doxy-api.conf.in',
-		output: 'doxy-api.conf',
-		configuration: cdata)
-
-	doxy_build = custom_target('doxygen',
-		depends: example,
-		input: doxy_conf,
-		output: 'api',
-		command: [generate_doxygen, '@INPUT@', '@OUTPUT@', generate_css],
-		install: get_option('enable_docs'),
-		install_dir: htmldir,
-		build_by_default: get_option('enable_docs'))
-
-	doc_targets += doxy_build
-	doc_target_names += 'Doxygen_API'
+if not doxygen.found()
+  subdir_done()
 endif
+
+# due to the CSS customisation script, which needs to run on a file that
+# is in a subdirectory that is created at build time and thus it cannot
+# be an individual custom_target, we need to wrap the doxygen call in a
+# script to run the CSS modification afterwards
+generate_doxygen = find_program('generate_doxygen.sh')
+generate_examples = find_program('generate_examples.sh')
+generate_css = find_program('doxy-html-custom.sh')
+
+inputdir = join_paths(meson.source_root(), 'examples')
+htmldir = join_paths('share', 'doc', 'dpdk')
+
+# due to the following bug: https://github.com/mesonbuild/meson/issues/4107
+# if install is set to true it will override build_by_default and it will
+# cause the target to always be built. If install were to be always set to
+# false it would be impossible to install the docs.
+# So use a configure option for now.
+example = custom_target('examples.dox',
+	input: inputdir,
+	output: 'examples.dox',
+	command: [generate_examples, '@INPUT@', '@OUTPUT@'],
+	install: get_option('enable_docs'),
+	install_dir: htmldir,
+	build_by_default: get_option('enable_docs'))
+
+cdata = configuration_data()
+cdata.set('VERSION', meson.project_version())
+cdata.set('API_EXAMPLES', join_paths(meson.build_root(), 'doc', 'api', 'examples.dox'))
+cdata.set('OUTPUT', join_paths(meson.build_root(), 'doc', 'api'))
+cdata.set('HTML_OUTPUT', 'api')
+cdata.set('TOPDIR', meson.source_root())
+cdata.set('STRIP_FROM_PATH', meson.source_root())
+
+doxy_conf = configure_file(input: 'doxy-api.conf.in',
+	output: 'doxy-api.conf',
+	configuration: cdata)
+
+doxy_build = custom_target('doxygen',
+	depends: example,
+	input: doxy_conf,
+	output: 'api',
+	command: [generate_doxygen, '@INPUT@', '@OUTPUT@', generate_css],
+	install: get_option('enable_docs'),
+	install_dir: htmldir,
+	build_by_default: get_option('enable_docs'))
+
+doc_targets += doxy_build
+doc_target_names += 'Doxygen_API'
-- 
2.24.1

Re: [dpdk-dev] [PATCH v2 1/6] kernel/linux/kni: fix meson warning about console keyword

[Aaron Conole]

Bruce Richardson <bruce.richardson@intel.com> writes:

> Since kni no longer includes the ethtool code and so is faster to build, we
> no longer need the console parameter to have incremental screen updates as
> it builds. Therefore, we drop the keyword which removes the warning.
>
> Fixes: b78f32cff94d ("kni: support meson build")
> Cc: bluca@debian.org
>
> Signed-off-by: Bruce Richardson <bruce.richardson@intel.com>
> ---

Acked-by: Aaron Conole <aconole@redhat.com>

Re: [dpdk-dev] [PATCH v2 2/6] build: skip processing docs folder if docs disabled

[Aaron Conole]

Bruce Richardson <bruce.richardson@intel.com> writes:

> While each target is set to be ignored if the docs are disabled in the
> meson build, there is little reason to process the docs folder at all.
>
> Signed-off-by: Bruce Richardson <bruce.richardson@intel.com>
> ---

Makes sense.

Acked-by: Aaron Conole <aconole@redhat.com>

Re: [dpdk-dev] [PATCH v2 3/6] doc/api: fix warning with meson build

[Aaron Conole]

Bruce Richardson <bruce.richardson@intel.com> writes:

> The install parameter to configure_file is new in 0.50 and generates a
> warning since it is newer than our minimum version of 0.47.1. The
> parameter, however, is unneeded as the documentation states:
>
> "When omitted it defaults to true when install_dir is set and not empty,
> false otherwise."
>
> Given that install_dir is not set for this file, install defaults to false
> so no need to explicitly specify it.
>
> Fixes: 720b14db3ae2 ("build: generate API documentation with meson")
> Cc: bluca@debian.org
>
> Signed-off-by: Bruce Richardson <bruce.richardson@intel.com>
> ---

Acked-by: Aaron Conole <aconole@redhat.com>

Re: [dpdk-dev] [PATCH v2 4/6] doc/guides: reduce whitespace in meson build file

[Aaron Conole]

Bruce Richardson <bruce.richardson@intel.com> writes:

> For building the guides, we can make the meson.build easier to read by
> using the subdir_done function to quit early.
>
> Signed-off-by: Bruce Richardson <bruce.richardson@intel.com>
> ---

Nice.

Acked-by: Aaron Conole <aconole@redhat.com>

Re: [dpdk-dev] [PATCH v2 5/6] doc/guides: rebuild with meson whenever a file changes

[Aaron Conole]

Bruce Richardson <bruce.richardson@intel.com> writes:

> Add proper support for calling sphinx whenever a file in the doc
> directory changes. This is accomplished by using a wrapper script
> for sphinx, which runs sphinx but also emits a gcc-format dependency
> file listing all the doc files. This is used by ninja so that any
> change to the doc files triggers a rebuild of the docs.
>
> Signed-off-by: Bruce Richardson <bruce.richardson@intel.com>
> ---
>  buildtools/call-sphinx-build.py | 29 +++++++++++++++++++++++++++++
>  buildtools/meson.build          |  6 ++++--
>  doc/guides/meson.build          | 22 ++++++++--------------
>  3 files changed, 41 insertions(+), 16 deletions(-)
>  create mode 100755 buildtools/call-sphinx-build.py
>

Acked-by: Aaron Conole <aconole@redhat.com>

Some nits follow - it would be good to clean them up before applying but
they are fairly inconsequential.

> diff --git a/buildtools/call-sphinx-build.py b/buildtools/call-sphinx-build.py
> new file mode 100755
> index 000000000..027317b9b
> --- /dev/null
> +++ b/buildtools/call-sphinx-build.py
> @@ -0,0 +1,29 @@
> +#! /usr/bin/env python3
> +# SPDX-License-Identifier: BSD-3-Clause
> +# Copyright(c) 2019 Intel Corporation
> +#
> +
> +import sys
> +import os
> +import os.path
> +import subprocess
> +
> +sphinx = sys.argv[1]
> +src = sys.argv[2]
> +dst = sys.argv[3]
> +depfile = os.path.join(dst,'.html.d')
                              ^ whitespace here
> +
> +# find all the files sphinx will process so we can write them as dependencies
> +srcfiles = []
> +for root, dirs, files in os.walk(src):
> +    for f in files:
> +        srcfiles.append(os.path.join(root, f))
> +
> +# run sphinx, putting the html output in a "html" directory
> +subprocess.run([sphinx, '-j', 'auto', '-b', 'html', src,
> +                os.path.join(dst, 'html')], check = True)
                                                    ^ no whitespace
                                                      around =
> +
> +# create a gcc format .d file giving all the dependencies of this doc build
> +with open(depfile, 'w') as d:
> +    d.write('html: ' + ' '.join(srcfiles) + '\n')
> +subprocess.run(['cp', '-f', depfile, '/tmp'])
> diff --git a/buildtools/meson.build b/buildtools/meson.build
> index 6ef2c5721..cd1d05403 100644
> --- a/buildtools/meson.build
> +++ b/buildtools/meson.build
> @@ -10,10 +10,12 @@ check_experimental_syms = find_program('check-experimental-syms.sh')
>  # set up map-to-def script using python, either built-in or external
>  python3 = import('python').find_installation(required: false)
>  if python3.found()
> -	map_to_def_cmd = [python3, files('map_to_def.py')]
> +	py3 = [python3]
>  else
> -	map_to_def_cmd = ['meson', 'runpython', files('map_to_def.py')]
> +	py3 = ['meson', 'runpython']
>  endif
> +map_to_def_cmd = py3 + files('map_to_def.py')
> +sphinx_wrapper = py3 + files('call-sphinx-build.py')
>  
>  # stable ABI always starts with "DPDK_"
>  is_experimental_cmd = [find_program('grep', 'findstr'), '^DPDK_']
> diff --git a/doc/guides/meson.build b/doc/guides/meson.build
> index 80c21d168..732e7ad3a 100644
> --- a/doc/guides/meson.build
> +++ b/doc/guides/meson.build
> @@ -7,24 +7,18 @@ if not sphinx.found()
>  	subdir_done()
>  endif
>  
> -htmldir = join_paths('share', 'doc', 'dpdk')
> +htmldir = join_paths(get_option('datadir'), 'doc', 'dpdk')
>  html_guides = custom_target('html_guides',
> -	input: meson.current_source_dir(),
> -	output: 'guides',
> -	command: [sphinx, '-b', 'html',
> -		'-d', meson.current_build_dir() + '/.doctrees',
> -		'@INPUT@', meson.current_build_dir() + '/guides'],
> +	input: files('index.rst'),
> +	output: 'html',
> +	command: [sphinx_wrapper, sphinx, meson.current_source_dir(), meson.current_build_dir()],
> +	depfile: '.html.d',
>  	build_by_default: get_option('enable_docs'),
>  	install: get_option('enable_docs'),
>  	install_dir: htmldir)
>  
> +install_data(files('custom.css'),
> +			install_dir: join_paths(htmldir,'_static', 'css'))
> +
>  doc_targets += html_guides
>  doc_target_names += 'HTML_Guides'
> -
> -# sphinx leaves a .buildinfo in the target directory, which we don't
> -# want to install. Note that sh -c has to be used, otherwise the
> -# env var does not get expanded if calling rm/install directly.
> -meson.add_install_script('sh', '-c',
> -	'rm -f $MESON_INSTALL_DESTDIR_PREFIX/share/doc/dpdk/guides/.buildinfo')
> -meson.add_install_script('sh', '-c',
> -	'install -D -m0644 $MESON_SOURCE_ROOT/doc/guides/custom.css $MESON_INSTALL_DESTDIR_PREFIX/share/doc/dpdk/guides/_static/css/custom.css')

Re: [dpdk-dev] [PATCH v2 6/6] doc/api: reduce indentation in meson build file

[Aaron Conole]

Bruce Richardson <bruce.richardson@intel.com> writes:

> When building the API docs, we can make the meson.build file easier to
> read, and allow more code per line, by using subdir_done() to quit early.
>
> Signed-off-by: Bruce Richardson <bruce.richardson@intel.com>
> ---

Acked-by: Aaron Conole <aconole@redhat.com>

Re: [dpdk-dev] [PATCH v2 5/6] doc/guides: rebuild with meson whenever a file changes

[Bruce Richardson]

On Thu, Jan 09, 2020 at 09:27:42AM -0500, Aaron Conole wrote:
> Bruce Richardson <bruce.richardson@intel.com> writes:
> 
> > Add proper support for calling sphinx whenever a file in the doc
> > directory changes. This is accomplished by using a wrapper script
> > for sphinx, which runs sphinx but also emits a gcc-format dependency
> > file listing all the doc files. This is used by ninja so that any
> > change to the doc files triggers a rebuild of the docs.
> >
> > Signed-off-by: Bruce Richardson <bruce.richardson@intel.com>
> > ---
> >  buildtools/call-sphinx-build.py | 29 +++++++++++++++++++++++++++++
> >  buildtools/meson.build          |  6 ++++--
> >  doc/guides/meson.build          | 22 ++++++++--------------
> >  3 files changed, 41 insertions(+), 16 deletions(-)
> >  create mode 100755 buildtools/call-sphinx-build.py
> >
> 
> Acked-by: Aaron Conole <aconole@redhat.com>
> 
> Some nits follow - it would be good to clean them up before applying but
> they are fairly inconsequential.
> 
> > diff --git a/buildtools/call-sphinx-build.py b/buildtools/call-sphinx-build.py
> > new file mode 100755
> > index 000000000..027317b9b
> > --- /dev/null
> > +++ b/buildtools/call-sphinx-build.py
> > @@ -0,0 +1,29 @@
> > +#! /usr/bin/env python3
> > +# SPDX-License-Identifier: BSD-3-Clause
> > +# Copyright(c) 2019 Intel Corporation
> > +#
> > +
> > +import sys
> > +import os
> > +import os.path
> > +import subprocess
> > +
> > +sphinx = sys.argv[1]
> > +src = sys.argv[2]
> > +dst = sys.argv[3]
> > +depfile = os.path.join(dst,'.html.d')
>                               ^ whitespace here
> > +
> > +# find all the files sphinx will process so we can write them as dependencies
> > +srcfiles = []
> > +for root, dirs, files in os.walk(src):
> > +    for f in files:
> > +        srcfiles.append(os.path.join(root, f))
> > +
> > +# run sphinx, putting the html output in a "html" directory
> > +subprocess.run([sphinx, '-j', 'auto', '-b', 'html', src,
> > +                os.path.join(dst, 'html')], check = True)
>                                                     ^ no whitespace
>                                                       around =
> > +
> > +# create a gcc format .d file giving all the dependencies of this doc build
> > +with open(depfile, 'w') as d:
> > +    d.write('html: ' + ' '.join(srcfiles) + '\n')
> > +subprocess.run(['cp', '-f', depfile, '/tmp'])

Ok, will fix. I also see I left in an unnecessary subprocess.run at the end
to copy the file to /tmp. This was for debugging only since ninja deletes
the .d files after processing them.
[https://ninja-build.org/manual.html#_deps, see behaviour for deps=gcc]

/Bruce

Re: [dpdk-dev] [PATCH v2 0/6] meson build improvements

[Luca Boccassi]

On Thu, 2020-01-09 at 12:07 +0000, Bruce Richardson wrote:
> These patches make some improvements to the meson build, particularly
> for documentation. They also remove many, but not all warnings issued
> by
> meson e.g. warnings about newer features unsupported in baseline.
> 
> The biggest change is to improve the handling of the guide html docs.
> The change here is more significant, and the doc build now uses a
> wrapper script around sphinx. This wrapper script allows us to output
> correct dependency information for the sphinx build in a .d file.
> This
> .d file is processed by ninja (not meson) on build, so that any
> changes
> to doc files trigger a rebuild to the guides using sphinx.
> 
> For now, the two patches which remove the meson version warnings are
> CC'ed to stable for backport, theoretically this who set could be
> backported if so desired, as all changes could be considered fixes to
> some degree or other, and nothing introduces a whole new feature.
> 
> Note: for completeness and simplicity, previously submitted patch
> http://patches.dpdk.org/patch/64189/
>  is included in this set, and will
> be marked superceded in patchwork.
> 
> V2: resend to correct email addresses
> 
> Bruce Richardson (6):
>   kernel/linux/kni: fix meson warning about console keyword
>   build: skip processing docs folder if docs disabled
>   doc/api: fix warning with meson build
>   doc/guides: reduce whitespace in meson build file
>   doc/guides: rebuild with meson whenever a file changes
>   doc/api: reduce indentation in meson build file
> 
>  buildtools/call-sphinx-build.py | 29 ++++++++++
>  buildtools/meson.build          |  6 +-
>  doc/api/meson.build             | 99 +++++++++++++++++------------
> ----
>  doc/guides/meson.build          | 38 ++++++-------
>  doc/meson.build                 |  4 ++
>  kernel/linux/kni/meson.build    |  1 -
>  6 files changed, 104 insertions(+), 73 deletions(-)
>  create mode 100755 buildtools/call-sphinx-build.py

Series-acked-by: Luca Boccassi <bluca@debian.org>

-- 
Kind regards,
Luca Boccassi

[dpdk-dev] [PATCH v3 0/6] meson build improvements

[Bruce Richardson]

These patches make some improvements to the meson build, particularly
for documentation. They also remove many, but not all warnings issued by
meson e.g. warnings about newer features unsupported in baseline.

The biggest change is to improve the handling of the guide html docs.
The change here is more significant, and the doc build now uses a
wrapper script around sphinx. This wrapper script allows us to output
correct dependency information for the sphinx build in a .d file. This
.d file is processed by ninja (not meson) on build, so that any changes
to doc files trigger a rebuild to the guides using sphinx.

For now, the two patches which remove the meson version warnings are
CC'ed to stable for backport, theoretically this who set could be
backported if so desired, as all changes could be considered fixes to
some degree or other, and nothing introduces a whole new feature.

Note: for completeness and simplicity, previously submitted patch
http://patches.dpdk.org/patch/64189/ is included in this set, and will
be marked superceded in patchwork.

V3: Shorten the python code in patch 5, and ensure style compliance as
    flagged by Aaron.

V2: resend to correct email addresses

Bruce Richardson (6):
  kernel/linux/kni: fix meson warning about console keyword
  build: skip processing docs folder if docs disabled
  doc/api: fix warning with meson build
  doc/guides: reduce whitespace in meson build file
  doc/guides: rebuild with meson whenever a file changes
  doc/api: reduce indentation in meson build file

 buildtools/call-sphinx-build.py | 23 ++++++++
 buildtools/meson.build          |  6 +-
 doc/api/meson.build             | 99 +++++++++++++++++----------------
 doc/guides/meson.build          | 38 ++++++-------
 doc/meson.build                 |  4 ++
 kernel/linux/kni/meson.build    |  1 -
 6 files changed, 98 insertions(+), 73 deletions(-)
 create mode 100755 buildtools/call-sphinx-build.py

-- 
2.24.1

[dpdk-dev] [PATCH v3 1/6] kernel/linux/kni: fix meson warning about console keyword

[Bruce Richardson]

Since kni no longer includes the ethtool code and so is faster to build, we
no longer need the console parameter to have incremental screen updates as
it builds. Therefore, we drop the keyword which removes the warning.

Fixes: b78f32cff94d ("kni: support meson build")
Cc: bluca@debian.org

Signed-off-by: Bruce Richardson <bruce.richardson@intel.com>
Acked-by: Aaron Conole <aconole@redhat.com>
---
 kernel/linux/kni/meson.build | 1 -
 1 file changed, 1 deletion(-)

diff --git a/kernel/linux/kni/meson.build b/kernel/linux/kni/meson.build
index 955eec949..f93e97fa0 100644
--- a/kernel/linux/kni/meson.build
+++ b/kernel/linux/kni/meson.build
@@ -23,7 +23,6 @@ custom_target('rte_kni',
 		' -I' + meson.current_source_dir(),
 		'modules'],
 	depends: kni_mkfile,
-	console: true,
 	install: true,
 	install_dir: kernel_dir + '/extra/dpdk',
 	build_by_default: get_option('enable_kmods'))
-- 
2.24.1

[dpdk-dev] [PATCH v3 2/6] build: skip processing docs folder if docs disabled

[Bruce Richardson]

While each target is set to be ignored if the docs are disabled in the
meson build, there is little reason to process the docs folder at all.

Signed-off-by: Bruce Richardson <bruce.richardson@intel.com>
Acked-by: Aaron Conole <aconole@redhat.com>
---
 doc/meson.build | 4 ++++
 1 file changed, 4 insertions(+)

diff --git a/doc/meson.build b/doc/meson.build
index c5410d85d..c49ec8476 100644
--- a/doc/meson.build
+++ b/doc/meson.build
@@ -1,6 +1,10 @@
 # SPDX-License-Identifier: BSD-3-Clause
 # Copyright(c) 2018 Luca Boccassi <bluca@debian.org>
 
+if not get_option('enable_docs')
+	subdir_done()
+endif
+
 doc_targets = []
 doc_target_names = []
 subdir('api')
-- 
2.24.1

[dpdk-dev] [PATCH v3 3/6] doc/api: fix warning with meson build

[Bruce Richardson]

The install parameter to configure_file is new in 0.50 and generates a
warning since it is newer than our minimum version of 0.47.1. The
parameter, however, is unneeded as the documentation states:

"When omitted it defaults to true when install_dir is set and not empty,
false otherwise."

Given that install_dir is not set for this file, install defaults to false
so no need to explicitly specify it.

Fixes: 720b14db3ae2 ("build: generate API documentation with meson")
Cc: bluca@debian.org

Signed-off-by: Bruce Richardson <bruce.richardson@intel.com>
Acked-by: Aaron Conole <aconole@redhat.com>
---
 doc/api/meson.build | 3 +--
 1 file changed, 1 insertion(+), 2 deletions(-)

diff --git a/doc/api/meson.build b/doc/api/meson.build
index 1c48b7672..23a7dfc75 100644
--- a/doc/api/meson.build
+++ b/doc/api/meson.build
@@ -38,8 +38,7 @@ if doxygen.found()
 
 	doxy_conf = configure_file(input: 'doxy-api.conf.in',
 		output: 'doxy-api.conf',
-		configuration: cdata,
-		install: false)
+		configuration: cdata)
 
 	doxy_build = custom_target('doxygen',
 		depends: example,
-- 
2.24.1

[dpdk-dev] [PATCH v3 4/6] doc/guides: reduce whitespace in meson build file

[Bruce Richardson]

For building the guides, we can make the meson.build easier to read by
using the subdir_done function to quit early.

Signed-off-by: Bruce Richardson <bruce.richardson@intel.com>
Acked-by: Aaron Conole <aconole@redhat.com>
---
 doc/guides/meson.build | 44 ++++++++++++++++++++++--------------------
 1 file changed, 23 insertions(+), 21 deletions(-)

diff --git a/doc/guides/meson.build b/doc/guides/meson.build
index 7931ef3bb..80c21d168 100644
--- a/doc/guides/meson.build
+++ b/doc/guides/meson.build
@@ -3,26 +3,28 @@
 
 sphinx = find_program('sphinx-build', required: get_option('enable_docs'))
 
-if sphinx.found()
-	htmldir = join_paths('share', 'doc', 'dpdk')
-	html_guides_build = custom_target('html_guides_build',
-		input: meson.current_source_dir(),
-		output: 'guides',
-		command: [sphinx, '-b', 'html',
-			'-d', meson.current_build_dir() + '/.doctrees',
-			'@INPUT@', meson.current_build_dir() + '/guides'],
-		build_by_default: get_option('enable_docs'),
-		install: get_option('enable_docs'),
-		install_dir: htmldir)
+if not sphinx.found()
+	subdir_done()
+endif
 
-	doc_targets += html_guides_build
-	doc_target_names += 'HTML_Guides'
+htmldir = join_paths('share', 'doc', 'dpdk')
+html_guides = custom_target('html_guides',
+	input: meson.current_source_dir(),
+	output: 'guides',
+	command: [sphinx, '-b', 'html',
+		'-d', meson.current_build_dir() + '/.doctrees',
+		'@INPUT@', meson.current_build_dir() + '/guides'],
+	build_by_default: get_option('enable_docs'),
+	install: get_option('enable_docs'),
+	install_dir: htmldir)
 
-	# sphinx leaves a .buildinfo in the target directory, which we don't
-	# want to install. Note that sh -c has to be used, otherwise the
-	# env var does not get expanded if calling rm/install directly.
-	meson.add_install_script('sh', '-c',
-		'rm -f $MESON_INSTALL_DESTDIR_PREFIX/share/doc/dpdk/guides/.buildinfo')
-	meson.add_install_script('sh', '-c',
-		'install -D -m0644 $MESON_SOURCE_ROOT/doc/guides/custom.css $MESON_INSTALL_DESTDIR_PREFIX/share/doc/dpdk/guides/_static/css/custom.css')
-endif
+doc_targets += html_guides
+doc_target_names += 'HTML_Guides'
+
+# sphinx leaves a .buildinfo in the target directory, which we don't
+# want to install. Note that sh -c has to be used, otherwise the
+# env var does not get expanded if calling rm/install directly.
+meson.add_install_script('sh', '-c',
+	'rm -f $MESON_INSTALL_DESTDIR_PREFIX/share/doc/dpdk/guides/.buildinfo')
+meson.add_install_script('sh', '-c',
+	'install -D -m0644 $MESON_SOURCE_ROOT/doc/guides/custom.css $MESON_INSTALL_DESTDIR_PREFIX/share/doc/dpdk/guides/_static/css/custom.css')
-- 
2.24.1

[dpdk-dev] [PATCH v3 5/6] doc/guides: rebuild with meson whenever a file changes

[Bruce Richardson]

Add proper support for calling sphinx whenever a file in the doc
directory changes. This is accomplished by using a wrapper script
for sphinx, which runs sphinx but also emits a gcc-format dependency
file listing all the doc files. This is used by ninja so that any
change to the doc files triggers a rebuild of the docs.

Signed-off-by: Bruce Richardson <bruce.richardson@intel.com>
Acked-by: Aaron Conole <aconole@redhat.com>
---
 buildtools/call-sphinx-build.py | 23 +++++++++++++++++++++++
 buildtools/meson.build          |  6 ++++--
 doc/guides/meson.build          | 22 ++++++++--------------
 3 files changed, 35 insertions(+), 16 deletions(-)
 create mode 100755 buildtools/call-sphinx-build.py

diff --git a/buildtools/call-sphinx-build.py b/buildtools/call-sphinx-build.py
new file mode 100755
index 000000000..80bda2661
--- /dev/null
+++ b/buildtools/call-sphinx-build.py
@@ -0,0 +1,23 @@
+#! /usr/bin/env python3
+# SPDX-License-Identifier: BSD-3-Clause
+# Copyright(c) 2019 Intel Corporation
+#
+
+import sys
+import os
+from os.path import join
+from subprocess import run
+
+(sphinx, src, dst) = sys.argv[1:]  # assign parameters to variables
+
+# find all the files sphinx will process so we can write them as dependencies
+srcfiles = []
+for root, dirs, files in os.walk(src):
+    srcfiles.extend([join(root, f) for f in files])
+
+# run sphinx, putting the html output in a "html" directory
+run([sphinx, '-j', 'auto', '-b', 'html', src, join(dst, 'html')], check=True)
+
+# create a gcc format .d file giving all the dependencies of this doc build
+with open(join(dst, '.html.d'), 'w') as d:
+    d.write('html: ' + ' '.join(srcfiles) + '\n')
diff --git a/buildtools/meson.build b/buildtools/meson.build
index 6ef2c5721..cd1d05403 100644
--- a/buildtools/meson.build
+++ b/buildtools/meson.build
@@ -10,10 +10,12 @@ check_experimental_syms = find_program('check-experimental-syms.sh')
 # set up map-to-def script using python, either built-in or external
 python3 = import('python').find_installation(required: false)
 if python3.found()
-	map_to_def_cmd = [python3, files('map_to_def.py')]
+	py3 = [python3]
 else
-	map_to_def_cmd = ['meson', 'runpython', files('map_to_def.py')]
+	py3 = ['meson', 'runpython']
 endif
+map_to_def_cmd = py3 + files('map_to_def.py')
+sphinx_wrapper = py3 + files('call-sphinx-build.py')
 
 # stable ABI always starts with "DPDK_"
 is_experimental_cmd = [find_program('grep', 'findstr'), '^DPDK_']
diff --git a/doc/guides/meson.build b/doc/guides/meson.build
index 80c21d168..732e7ad3a 100644
--- a/doc/guides/meson.build
+++ b/doc/guides/meson.build
@@ -7,24 +7,18 @@ if not sphinx.found()
 	subdir_done()
 endif
 
-htmldir = join_paths('share', 'doc', 'dpdk')
+htmldir = join_paths(get_option('datadir'), 'doc', 'dpdk')
 html_guides = custom_target('html_guides',
-	input: meson.current_source_dir(),
-	output: 'guides',
-	command: [sphinx, '-b', 'html',
-		'-d', meson.current_build_dir() + '/.doctrees',
-		'@INPUT@', meson.current_build_dir() + '/guides'],
+	input: files('index.rst'),
+	output: 'html',
+	command: [sphinx_wrapper, sphinx, meson.current_source_dir(), meson.current_build_dir()],
+	depfile: '.html.d',
 	build_by_default: get_option('enable_docs'),
 	install: get_option('enable_docs'),
 	install_dir: htmldir)
 
+install_data(files('custom.css'),
+			install_dir: join_paths(htmldir,'_static', 'css'))
+
 doc_targets += html_guides
 doc_target_names += 'HTML_Guides'
-
-# sphinx leaves a .buildinfo in the target directory, which we don't
-# want to install. Note that sh -c has to be used, otherwise the
-# env var does not get expanded if calling rm/install directly.
-meson.add_install_script('sh', '-c',
-	'rm -f $MESON_INSTALL_DESTDIR_PREFIX/share/doc/dpdk/guides/.buildinfo')
-meson.add_install_script('sh', '-c',
-	'install -D -m0644 $MESON_SOURCE_ROOT/doc/guides/custom.css $MESON_INSTALL_DESTDIR_PREFIX/share/doc/dpdk/guides/_static/css/custom.css')
-- 
2.24.1

[dpdk-dev] [PATCH v3 6/6] doc/api: reduce indentation in meson build file

[Bruce Richardson]

When building the API docs, we can make the meson.build file easier to
read, and allow more code per line, by using subdir_done() to quit early.

Signed-off-by: Bruce Richardson <bruce.richardson@intel.com>
Acked-by: Aaron Conole <aconole@redhat.com>
---
 doc/api/meson.build | 98 +++++++++++++++++++++++----------------------
 1 file changed, 50 insertions(+), 48 deletions(-)

diff --git a/doc/api/meson.build b/doc/api/meson.build
index 23a7dfc75..c72b880e1 100644
--- a/doc/api/meson.build
+++ b/doc/api/meson.build
@@ -3,52 +3,54 @@
 
 doxygen = find_program('doxygen', required: get_option('enable_docs'))
 
-if doxygen.found()
-	# due to the CSS customisation script, which needs to run on a file that
-	# is in a subdirectory that is created at build time and thus it cannot
-	# be an individual custom_target, we need to wrap the doxygen call in a
-	# script to run the CSS modification afterwards
-	generate_doxygen = find_program('generate_doxygen.sh')
-	generate_examples = find_program('generate_examples.sh')
-	generate_css = find_program('doxy-html-custom.sh')
-
-	inputdir = join_paths(meson.source_root(), 'examples')
-	htmldir = join_paths('share', 'doc', 'dpdk')
-
-	# due to the following bug: https://github.com/mesonbuild/meson/issues/4107
-	# if install is set to true it will override build_by_default and it will
-	# cause the target to always be built. If install were to be always set to
-	# false it would be impossible to install the docs.
-	# So use a configure option for now.
-	example = custom_target('examples.dox',
-		input: inputdir,
-		output: 'examples.dox',
-		command: [generate_examples, '@INPUT@', '@OUTPUT@'],
-		install: get_option('enable_docs'),
-		install_dir: htmldir,
-		build_by_default: get_option('enable_docs'))
-
-	cdata = configuration_data()
-	cdata.set('VERSION', meson.project_version())
-	cdata.set('API_EXAMPLES', join_paths(meson.build_root(), 'doc', 'api', 'examples.dox'))
-	cdata.set('OUTPUT', join_paths(meson.build_root(), 'doc', 'api'))
-	cdata.set('HTML_OUTPUT', 'api')
-	cdata.set('TOPDIR', meson.source_root())
-	cdata.set('STRIP_FROM_PATH', meson.source_root())
-
-	doxy_conf = configure_file(input: 'doxy-api.conf.in',
-		output: 'doxy-api.conf',
-		configuration: cdata)
-
-	doxy_build = custom_target('doxygen',
-		depends: example,
-		input: doxy_conf,
-		output: 'api',
-		command: [generate_doxygen, '@INPUT@', '@OUTPUT@', generate_css],
-		install: get_option('enable_docs'),
-		install_dir: htmldir,
-		build_by_default: get_option('enable_docs'))
-
-	doc_targets += doxy_build
-	doc_target_names += 'Doxygen_API'
+if not doxygen.found()
+  subdir_done()
 endif
+
+# due to the CSS customisation script, which needs to run on a file that
+# is in a subdirectory that is created at build time and thus it cannot
+# be an individual custom_target, we need to wrap the doxygen call in a
+# script to run the CSS modification afterwards
+generate_doxygen = find_program('generate_doxygen.sh')
+generate_examples = find_program('generate_examples.sh')
+generate_css = find_program('doxy-html-custom.sh')
+
+inputdir = join_paths(meson.source_root(), 'examples')
+htmldir = join_paths('share', 'doc', 'dpdk')
+
+# due to the following bug: https://github.com/mesonbuild/meson/issues/4107
+# if install is set to true it will override build_by_default and it will
+# cause the target to always be built. If install were to be always set to
+# false it would be impossible to install the docs.
+# So use a configure option for now.
+example = custom_target('examples.dox',
+	input: inputdir,
+	output: 'examples.dox',
+	command: [generate_examples, '@INPUT@', '@OUTPUT@'],
+	install: get_option('enable_docs'),
+	install_dir: htmldir,
+	build_by_default: get_option('enable_docs'))
+
+cdata = configuration_data()
+cdata.set('VERSION', meson.project_version())
+cdata.set('API_EXAMPLES', join_paths(meson.build_root(), 'doc', 'api', 'examples.dox'))
+cdata.set('OUTPUT', join_paths(meson.build_root(), 'doc', 'api'))
+cdata.set('HTML_OUTPUT', 'api')
+cdata.set('TOPDIR', meson.source_root())
+cdata.set('STRIP_FROM_PATH', meson.source_root())
+
+doxy_conf = configure_file(input: 'doxy-api.conf.in',
+	output: 'doxy-api.conf',
+	configuration: cdata)
+
+doxy_build = custom_target('doxygen',
+	depends: example,
+	input: doxy_conf,
+	output: 'api',
+	command: [generate_doxygen, '@INPUT@', '@OUTPUT@', generate_css],
+	install: get_option('enable_docs'),
+	install_dir: htmldir,
+	build_by_default: get_option('enable_docs'))
+
+doc_targets += doxy_build
+doc_target_names += 'Doxygen_API'
-- 
2.24.1

Re: [dpdk-dev] [PATCH v3 1/6] kernel/linux/kni: fix meson warning about console keyword

[David Marchand]

On Thu, Jan 9, 2020 at 4:31 PM Bruce Richardson
<bruce.richardson@intel.com> wrote:
>
> Since kni no longer includes the ethtool code and so is faster to build, we
> no longer need the console parameter to have incremental screen updates as
> it builds. Therefore, we drop the keyword which removes the warning.
>
> Fixes: b78f32cff94d ("kni: support meson build")
> Cc: bluca@debian.org

Since this is linked to kni ethtool stuff removed from 19.11, I would
go with a Cc: stable@dpdk.org.
Then stable maintainers can ignore for everything but 19.11 ?


--
David Marchand

Re: [dpdk-dev] [PATCH v3 3/6] doc/api: fix warning with meson build

[David Marchand]

On Thu, Jan 9, 2020 at 4:31 PM Bruce Richardson
<bruce.richardson@intel.com> wrote:
>
> The install parameter to configure_file is new in 0.50 and generates a
> warning since it is newer than our minimum version of 0.47.1. The
> parameter, however, is unneeded as the documentation states:
>
> "When omitted it defaults to true when install_dir is set and not empty,
> false otherwise."
>
> Given that install_dir is not set for this file, install defaults to false
> so no need to explicitly specify it.
>
> Fixes: 720b14db3ae2 ("build: generate API documentation with meson")

Cc: stable@dpdk.org ? or does it depend on other changes?



--
David Marchand

Re: [dpdk-dev] [PATCH v3 5/6] doc/guides: rebuild with meson whenever a file changes

[David Marchand]

On Thu, Jan 9, 2020 at 4:31 PM Bruce Richardson
<bruce.richardson@intel.com> wrote:
>
> Add proper support for calling sphinx whenever a file in the doc
> directory changes. This is accomplished by using a wrapper script
> for sphinx, which runs sphinx but also emits a gcc-format dependency
> file listing all the doc files. This is used by ninja so that any
> change to the doc files triggers a rebuild of the docs.
>
> Signed-off-by: Bruce Richardson <bruce.richardson@intel.com>
> Acked-by: Aaron Conole <aconole@redhat.com>
> ---
>  buildtools/call-sphinx-build.py | 23 +++++++++++++++++++++++
>  buildtools/meson.build          |  6 ++++--
>  doc/guides/meson.build          | 22 ++++++++--------------
>  3 files changed, 35 insertions(+), 16 deletions(-)
>  create mode 100755 buildtools/call-sphinx-build.py

This new file is not added to MAINTAINERS.



--
David Marchand

Re: [dpdk-dev] [PATCH v3 5/6] doc/guides: rebuild with meson whenever a file changes

[David Marchand]

On Thu, Jan 9, 2020 at 7:03 PM David Marchand <david.marchand@redhat.com> wrote:
>
> On Thu, Jan 9, 2020 at 4:31 PM Bruce Richardson
> <bruce.richardson@intel.com> wrote:
> >
> > Add proper support for calling sphinx whenever a file in the doc
> > directory changes. This is accomplished by using a wrapper script
> > for sphinx, which runs sphinx but also emits a gcc-format dependency
> > file listing all the doc files. This is used by ninja so that any
> > change to the doc files triggers a rebuild of the docs.
> >
> > Signed-off-by: Bruce Richardson <bruce.richardson@intel.com>
> > Acked-by: Aaron Conole <aconole@redhat.com>
> > ---
> >  buildtools/call-sphinx-build.py | 23 +++++++++++++++++++++++
> >  buildtools/meson.build          |  6 ++++--
> >  doc/guides/meson.build          | 22 ++++++++--------------
> >  3 files changed, 35 insertions(+), 16 deletions(-)
> >  create mode 100755 buildtools/call-sphinx-build.py
>
> This new file is not added to MAINTAINERS.

I can fix this when applying.
I would add this to the section:

Meson build
M: Bruce Richardson <bruce.richardson@intel.com>


-- 
David Marchand

Re: [dpdk-dev] [PATCH v3 5/6] doc/guides: rebuild with meson whenever a file changes

[Bruce Richardson]

On Thu, Jan 09, 2020 at 07:06:31PM +0100, David Marchand wrote:
> On Thu, Jan 9, 2020 at 7:03 PM David Marchand <david.marchand@redhat.com> wrote:
> >
> > On Thu, Jan 9, 2020 at 4:31 PM Bruce Richardson
> > <bruce.richardson@intel.com> wrote:
> > >
> > > Add proper support for calling sphinx whenever a file in the doc
> > > directory changes. This is accomplished by using a wrapper script
> > > for sphinx, which runs sphinx but also emits a gcc-format dependency
> > > file listing all the doc files. This is used by ninja so that any
> > > change to the doc files triggers a rebuild of the docs.
> > >
> > > Signed-off-by: Bruce Richardson <bruce.richardson@intel.com>
> > > Acked-by: Aaron Conole <aconole@redhat.com>
> > > ---
> > >  buildtools/call-sphinx-build.py | 23 +++++++++++++++++++++++
> > >  buildtools/meson.build          |  6 ++++--
> > >  doc/guides/meson.build          | 22 ++++++++--------------
> > >  3 files changed, 35 insertions(+), 16 deletions(-)
> > >  create mode 100755 buildtools/call-sphinx-build.py
> >
> > This new file is not added to MAINTAINERS.
> 
> I can fix this when applying.
> I would add this to the section:
> 
> Meson build
> M: Bruce Richardson <bruce.richardson@intel.com>
>
Agreed. 

Re: [dpdk-dev] [PATCH v3 3/6] doc/api: fix warning with meson build

[Bruce Richardson]

On Thu, Jan 09, 2020 at 07:03:05PM +0100, David Marchand wrote:
> On Thu, Jan 9, 2020 at 4:31 PM Bruce Richardson
> <bruce.richardson@intel.com> wrote:
> >
> > The install parameter to configure_file is new in 0.50 and generates a
> > warning since it is newer than our minimum version of 0.47.1. The
> > parameter, however, is unneeded as the documentation states:
> >
> > "When omitted it defaults to true when install_dir is set and not empty,
> > false otherwise."
> >
> > Given that install_dir is not set for this file, install defaults to false
> > so no need to explicitly specify it.
> >
> > Fixes: 720b14db3ae2 ("build: generate API documentation with meson")
> 
> Cc: stable@dpdk.org ? or does it depend on other changes?
>
I don't think it requires any other changes. Should probably go to 19.11
anyway - the 18.11 tree can skip it, I think. 

Re: [dpdk-dev] [PATCH v3 5/6] doc/guides: rebuild with meson whenever a file changes

[Bruce Richardson]

On Thu, Jan 09, 2020 at 03:31:11PM +0000, Bruce Richardson wrote:
> Add proper support for calling sphinx whenever a file in the doc
> directory changes. This is accomplished by using a wrapper script
> for sphinx, which runs sphinx but also emits a gcc-format dependency
> file listing all the doc files. This is used by ninja so that any
> change to the doc files triggers a rebuild of the docs.
> 
> Signed-off-by: Bruce Richardson <bruce.richardson@intel.com>
> Acked-by: Aaron Conole <aconole@redhat.com>
> ---
>  buildtools/call-sphinx-build.py | 23 +++++++++++++++++++++++
>  buildtools/meson.build          |  6 ++++--
>  doc/guides/meson.build          | 22 ++++++++--------------
>  3 files changed, 35 insertions(+), 16 deletions(-)
>  create mode 100755 buildtools/call-sphinx-build.py
> 
> diff --git a/buildtools/call-sphinx-build.py b/buildtools/call-sphinx-build.py
> new file mode 100755
> index 000000000..80bda2661
> --- /dev/null
> +++ b/buildtools/call-sphinx-build.py
> @@ -0,0 +1,23 @@
> +#! /usr/bin/env python3
> +# SPDX-License-Identifier: BSD-3-Clause
> +# Copyright(c) 2019 Intel Corporation
> +#
> +
> +import sys
> +import os
> +from os.path import join
> +from subprocess import run
> +
> +(sphinx, src, dst) = sys.argv[1:]  # assign parameters to variables
> +
> +# find all the files sphinx will process so we can write them as dependencies
> +srcfiles = []
> +for root, dirs, files in os.walk(src):
> +    srcfiles.extend([join(root, f) for f in files])
> +
> +# run sphinx, putting the html output in a "html" directory
> +run([sphinx, '-j', 'auto', '-b', 'html', src, join(dst, 'html')], check=True)
> +

Looking in travis, it appears that sphinx there is too old to support the
"-j auto" flag, so I'll do a new revision to fix that.

/Bruce

Re: [dpdk-dev] [PATCH v3 1/6] kernel/linux/kni: fix meson warning about console keyword

[Kevin Traynor]

On 09/01/2020 18:02, David Marchand wrote:
> On Thu, Jan 9, 2020 at 4:31 PM Bruce Richardson
> <bruce.richardson@intel.com> wrote:
>>
>> Since kni no longer includes the ethtool code and so is faster to build, we
>> no longer need the console parameter to have incremental screen updates as
>> it builds. Therefore, we drop the keyword which removes the warning.
>>
>> Fixes: b78f32cff94d ("kni: support meson build")
>> Cc: bluca@debian.org
> 
> Since this is linked to kni ethtool stuff removed from 19.11, I would
> go with a Cc: stable@dpdk.org.
> Then stable maintainers can ignore for everything but 19.11 ?
> 

+1

> 
> --
> David Marchand
> 

[dpdk-dev] [PATCH v4 0/6] meson build improvements

[Bruce Richardson]

These patches make some improvements to the meson build, particularly
for documentation. They also remove many, but not all warnings issued by
meson e.g. warnings about newer features unsupported in baseline.

The biggest change is to improve the handling of the guide html docs.
The change here is more significant, and the doc build now uses a
wrapper script around sphinx. This wrapper script allows us to output
correct dependency information for the sphinx build in a .d file. This
.d file is processed by ninja (not meson) on build, so that any changes
to doc files trigger a rebuild to the guides using sphinx.

For now, the two patches which remove the meson version warnings are
CC'ed to stable for backport, theoretically this who set could be
backported if so desired, as all changes could be considered fixes to
some degree or other, and nothing introduces a whole new feature.

Note: for completeness and simplicity, previously submitted patch
http://patches.dpdk.org/patch/64189/ is included in this set, and will
be marked superceded in patchwork.

V4: fix build with older sphinx

V3: Shorten the python code in patch 5, and ensure style compliance as
    flagged by Aaron.

V2: resend to correct email addresses


Bruce Richardson (6):
  kernel/linux/kni: fix meson warning about console keyword
  build: skip processing docs folder if docs disabled
  doc/api: fix warning with meson build
  doc/guides: reduce whitespace in meson build file
  doc/guides: rebuild with meson whenever a file changes
  doc/api: reduce indentation in meson build file

 MAINTAINERS                     |  1 +
 buildtools/call-sphinx-build.py | 31 +++++++++++
 buildtools/meson.build          |  6 +-
 doc/api/meson.build             | 99 +++++++++++++++++----------------
 doc/guides/meson.build          | 38 ++++++-------
 doc/meson.build                 |  4 ++
 kernel/linux/kni/meson.build    |  1 -
 7 files changed, 107 insertions(+), 73 deletions(-)
 create mode 100755 buildtools/call-sphinx-build.py

-- 
2.24.1

[dpdk-dev] [PATCH v4 1/6] kernel/linux/kni: fix meson warning about console keyword

[Bruce Richardson]

Since kni no longer includes the ethtool code and so is faster to build, we
no longer need the console parameter to have incremental screen updates as
it builds. Therefore, we drop the keyword which removes the warning.

Fixes: b78f32cff94d ("kni: support meson build")
Cc: stable@dpdk.org

Signed-off-by: Bruce Richardson <bruce.richardson@intel.com>
Acked-by: Aaron Conole <aconole@redhat.com>
Acked-by: Luca Boccassi <bluca@debian.org>
---
 kernel/linux/kni/meson.build | 1 -
 1 file changed, 1 deletion(-)

diff --git a/kernel/linux/kni/meson.build b/kernel/linux/kni/meson.build
index 955eec949..f93e97fa0 100644
--- a/kernel/linux/kni/meson.build
+++ b/kernel/linux/kni/meson.build
@@ -23,7 +23,6 @@ custom_target('rte_kni',
 		' -I' + meson.current_source_dir(),
 		'modules'],
 	depends: kni_mkfile,
-	console: true,
 	install: true,
 	install_dir: kernel_dir + '/extra/dpdk',
 	build_by_default: get_option('enable_kmods'))
-- 
2.24.1

[dpdk-dev] [PATCH v4 2/6] build: skip processing docs folder if docs disabled

[Bruce Richardson]

While each target is set to be ignored if the docs are disabled in the
meson build, there is little reason to process the docs folder at all.

Signed-off-by: Bruce Richardson <bruce.richardson@intel.com>
Acked-by: Aaron Conole <aconole@redhat.com>
Acked-by: Luca Boccassi <bluca@debian.org>
---
 doc/meson.build | 4 ++++
 1 file changed, 4 insertions(+)

diff --git a/doc/meson.build b/doc/meson.build
index c5410d85d..c49ec8476 100644
--- a/doc/meson.build
+++ b/doc/meson.build
@@ -1,6 +1,10 @@
 # SPDX-License-Identifier: BSD-3-Clause
 # Copyright(c) 2018 Luca Boccassi <bluca@debian.org>
 
+if not get_option('enable_docs')
+	subdir_done()
+endif
+
 doc_targets = []
 doc_target_names = []
 subdir('api')
-- 
2.24.1

[dpdk-dev] [PATCH v4 3/6] doc/api: fix warning with meson build

[Bruce Richardson]

The install parameter to configure_file is new in 0.50 and generates a
warning since it is newer than our minimum version of 0.47.1. The
parameter, however, is unneeded as the documentation states:

"When omitted it defaults to true when install_dir is set and not empty,
false otherwise."

Given that install_dir is not set for this file, install defaults to false
so no need to explicitly specify it.

Fixes: 720b14db3ae2 ("build: generate API documentation with meson")
Cc: stable@dpdk.org

Signed-off-by: Bruce Richardson <bruce.richardson@intel.com>
Acked-by: Aaron Conole <aconole@redhat.com>
Acked-by: Luca Boccassi <bluca@debian.org>
---
 doc/api/meson.build | 3 +--
 1 file changed, 1 insertion(+), 2 deletions(-)

diff --git a/doc/api/meson.build b/doc/api/meson.build
index 1c48b7672..23a7dfc75 100644
--- a/doc/api/meson.build
+++ b/doc/api/meson.build
@@ -38,8 +38,7 @@ if doxygen.found()
 
 	doxy_conf = configure_file(input: 'doxy-api.conf.in',
 		output: 'doxy-api.conf',
-		configuration: cdata,
-		install: false)
+		configuration: cdata)
 
 	doxy_build = custom_target('doxygen',
 		depends: example,
-- 
2.24.1

[dpdk-dev] [PATCH v4 4/6] doc/guides: reduce whitespace in meson build file

[Bruce Richardson]

For building the guides, we can make the meson.build easier to read by
using the subdir_done function to quit early.

Cc: stable@dpdk.org

Signed-off-by: Bruce Richardson <bruce.richardson@intel.com>
Acked-by: Aaron Conole <aconole@redhat.com>
Acked-by: Luca Boccassi <bluca@debian.org>

---
@stable, this may be considered for 19.11 backport to make other
backporting to this file easier.
---
 doc/guides/meson.build | 44 ++++++++++++++++++++++--------------------
 1 file changed, 23 insertions(+), 21 deletions(-)

diff --git a/doc/guides/meson.build b/doc/guides/meson.build
index 7931ef3bb..80c21d168 100644
--- a/doc/guides/meson.build
+++ b/doc/guides/meson.build
@@ -3,26 +3,28 @@
 
 sphinx = find_program('sphinx-build', required: get_option('enable_docs'))
 
-if sphinx.found()
-	htmldir = join_paths('share', 'doc', 'dpdk')
-	html_guides_build = custom_target('html_guides_build',
-		input: meson.current_source_dir(),
-		output: 'guides',
-		command: [sphinx, '-b', 'html',
-			'-d', meson.current_build_dir() + '/.doctrees',
-			'@INPUT@', meson.current_build_dir() + '/guides'],
-		build_by_default: get_option('enable_docs'),
-		install: get_option('enable_docs'),
-		install_dir: htmldir)
+if not sphinx.found()
+	subdir_done()
+endif
 
-	doc_targets += html_guides_build
-	doc_target_names += 'HTML_Guides'
+htmldir = join_paths('share', 'doc', 'dpdk')
+html_guides = custom_target('html_guides',
+	input: meson.current_source_dir(),
+	output: 'guides',
+	command: [sphinx, '-b', 'html',
+		'-d', meson.current_build_dir() + '/.doctrees',
+		'@INPUT@', meson.current_build_dir() + '/guides'],
+	build_by_default: get_option('enable_docs'),
+	install: get_option('enable_docs'),
+	install_dir: htmldir)
 
-	# sphinx leaves a .buildinfo in the target directory, which we don't
-	# want to install. Note that sh -c has to be used, otherwise the
-	# env var does not get expanded if calling rm/install directly.
-	meson.add_install_script('sh', '-c',
-		'rm -f $MESON_INSTALL_DESTDIR_PREFIX/share/doc/dpdk/guides/.buildinfo')
-	meson.add_install_script('sh', '-c',
-		'install -D -m0644 $MESON_SOURCE_ROOT/doc/guides/custom.css $MESON_INSTALL_DESTDIR_PREFIX/share/doc/dpdk/guides/_static/css/custom.css')
-endif
+doc_targets += html_guides
+doc_target_names += 'HTML_Guides'
+
+# sphinx leaves a .buildinfo in the target directory, which we don't
+# want to install. Note that sh -c has to be used, otherwise the
+# env var does not get expanded if calling rm/install directly.
+meson.add_install_script('sh', '-c',
+	'rm -f $MESON_INSTALL_DESTDIR_PREFIX/share/doc/dpdk/guides/.buildinfo')
+meson.add_install_script('sh', '-c',
+	'install -D -m0644 $MESON_SOURCE_ROOT/doc/guides/custom.css $MESON_INSTALL_DESTDIR_PREFIX/share/doc/dpdk/guides/_static/css/custom.css')
-- 
2.24.1

[dpdk-dev] [PATCH v4 5/6] doc/guides: rebuild with meson whenever a file changes

[Bruce Richardson]

Add proper support for calling sphinx whenever a file in the doc
directory changes. This is accomplished by using a wrapper script
for sphinx, which runs sphinx but also emits a gcc-format dependency
file listing all the doc files. This is used by ninja so that any
change to the doc files triggers a rebuild of the docs.

Signed-off-by: Bruce Richardson <bruce.richardson@intel.com>
Acked-by: Aaron Conole <aconole@redhat.com>
Acked-by: Luca Boccassi <bluca@debian.org>
---
 MAINTAINERS                     |  1 +
 buildtools/call-sphinx-build.py | 31 +++++++++++++++++++++++++++++++
 buildtools/meson.build          |  6 ++++--
 doc/guides/meson.build          | 22 ++++++++--------------
 4 files changed, 44 insertions(+), 16 deletions(-)
 create mode 100755 buildtools/call-sphinx-build.py

diff --git a/MAINTAINERS b/MAINTAINERS
index 4395d8df1..aa32bfc59 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -128,6 +128,7 @@ F: meson.build
 F: lib/librte_eal/freebsd/BSDmakefile.meson
 F: meson_options.txt
 F: config/rte_config.h
+F: buildtools/call-sphinx-build.py
 F: buildtools/gen-pmdinfo-cfile.sh
 F: buildtools/map_to_def.py
 F: buildtools/symlink-drivers-solibs.sh
diff --git a/buildtools/call-sphinx-build.py b/buildtools/call-sphinx-build.py
new file mode 100755
index 000000000..b9a3994e1
--- /dev/null
+++ b/buildtools/call-sphinx-build.py
@@ -0,0 +1,31 @@
+#! /usr/bin/env python3
+# SPDX-License-Identifier: BSD-3-Clause
+# Copyright(c) 2019 Intel Corporation
+#
+
+import sys
+import os
+from os.path import join
+from subprocess import run, PIPE
+from distutils.version import StrictVersion
+
+(sphinx, src, dst) = sys.argv[1:]  # assign parameters to variables
+
+# for sphinx version >= 1.7 add parallelism using "-j auto"
+ver = run([sphinx, '--version'], stdout=PIPE).stdout.decode().split()[-1]
+sphinx_cmd = [sphinx]
+if StrictVersion(ver) >= StrictVersion('1.7'):
+    sphinx_cmd += ['-j', 'auto']
+
+# find all the files sphinx will process so we can write them as dependencies
+srcfiles = []
+for root, dirs, files in os.walk(src):
+    srcfiles.extend([join(root, f) for f in files])
+
+# run sphinx, putting the html output in a "html" directory
+process = run(sphinx_cmd + ['-b', 'html', src, join(dst, 'html')], check=True)
+print(str(process.args) + ' Done OK')
+
+# create a gcc format .d file giving all the dependencies of this doc build
+with open(join(dst, '.html.d'), 'w') as d:
+    d.write('html: ' + ' '.join(srcfiles) + '\n')
diff --git a/buildtools/meson.build b/buildtools/meson.build
index 6ef2c5721..cd1d05403 100644
--- a/buildtools/meson.build
+++ b/buildtools/meson.build
@@ -10,10 +10,12 @@ check_experimental_syms = find_program('check-experimental-syms.sh')
 # set up map-to-def script using python, either built-in or external
 python3 = import('python').find_installation(required: false)
 if python3.found()
-	map_to_def_cmd = [python3, files('map_to_def.py')]
+	py3 = [python3]
 else
-	map_to_def_cmd = ['meson', 'runpython', files('map_to_def.py')]
+	py3 = ['meson', 'runpython']
 endif
+map_to_def_cmd = py3 + files('map_to_def.py')
+sphinx_wrapper = py3 + files('call-sphinx-build.py')
 
 # stable ABI always starts with "DPDK_"
 is_experimental_cmd = [find_program('grep', 'findstr'), '^DPDK_']
diff --git a/doc/guides/meson.build b/doc/guides/meson.build
index 80c21d168..732e7ad3a 100644
--- a/doc/guides/meson.build
+++ b/doc/guides/meson.build
@@ -7,24 +7,18 @@ if not sphinx.found()
 	subdir_done()
 endif
 
-htmldir = join_paths('share', 'doc', 'dpdk')
+htmldir = join_paths(get_option('datadir'), 'doc', 'dpdk')
 html_guides = custom_target('html_guides',
-	input: meson.current_source_dir(),
-	output: 'guides',
-	command: [sphinx, '-b', 'html',
-		'-d', meson.current_build_dir() + '/.doctrees',
-		'@INPUT@', meson.current_build_dir() + '/guides'],
+	input: files('index.rst'),
+	output: 'html',
+	command: [sphinx_wrapper, sphinx, meson.current_source_dir(), meson.current_build_dir()],
+	depfile: '.html.d',
 	build_by_default: get_option('enable_docs'),
 	install: get_option('enable_docs'),
 	install_dir: htmldir)
 
+install_data(files('custom.css'),
+			install_dir: join_paths(htmldir,'_static', 'css'))
+
 doc_targets += html_guides
 doc_target_names += 'HTML_Guides'
-
-# sphinx leaves a .buildinfo in the target directory, which we don't
-# want to install. Note that sh -c has to be used, otherwise the
-# env var does not get expanded if calling rm/install directly.
-meson.add_install_script('sh', '-c',
-	'rm -f $MESON_INSTALL_DESTDIR_PREFIX/share/doc/dpdk/guides/.buildinfo')
-meson.add_install_script('sh', '-c',
-	'install -D -m0644 $MESON_SOURCE_ROOT/doc/guides/custom.css $MESON_INSTALL_DESTDIR_PREFIX/share/doc/dpdk/guides/_static/css/custom.css')
-- 
2.24.1

[dpdk-dev] [PATCH v4 6/6] doc/api: reduce indentation in meson build file

[Bruce Richardson]

When building the API docs, we can make the meson.build file easier to
read, and allow more code per line, by using subdir_done() to quit early.

Cc: stable@dpdk.org
Signed-off-by: Bruce Richardson <bruce.richardson@intel.com>
Acked-by: Aaron Conole <aconole@redhat.com>
Acked-by: Luca Boccassi <bluca@debian.org>

---
@stable, this patch may be considered for backporting to 19.11 to allow
easier merging of other patches to the affected file.
---
 doc/api/meson.build | 98 +++++++++++++++++++++++----------------------
 1 file changed, 50 insertions(+), 48 deletions(-)

diff --git a/doc/api/meson.build b/doc/api/meson.build
index 23a7dfc75..c72b880e1 100644
--- a/doc/api/meson.build
+++ b/doc/api/meson.build
@@ -3,52 +3,54 @@
 
 doxygen = find_program('doxygen', required: get_option('enable_docs'))
 
-if doxygen.found()
-	# due to the CSS customisation script, which needs to run on a file that
-	# is in a subdirectory that is created at build time and thus it cannot
-	# be an individual custom_target, we need to wrap the doxygen call in a
-	# script to run the CSS modification afterwards
-	generate_doxygen = find_program('generate_doxygen.sh')
-	generate_examples = find_program('generate_examples.sh')
-	generate_css = find_program('doxy-html-custom.sh')
-
-	inputdir = join_paths(meson.source_root(), 'examples')
-	htmldir = join_paths('share', 'doc', 'dpdk')
-
-	# due to the following bug: https://github.com/mesonbuild/meson/issues/4107
-	# if install is set to true it will override build_by_default and it will
-	# cause the target to always be built. If install were to be always set to
-	# false it would be impossible to install the docs.
-	# So use a configure option for now.
-	example = custom_target('examples.dox',
-		input: inputdir,
-		output: 'examples.dox',
-		command: [generate_examples, '@INPUT@', '@OUTPUT@'],
-		install: get_option('enable_docs'),
-		install_dir: htmldir,
-		build_by_default: get_option('enable_docs'))
-
-	cdata = configuration_data()
-	cdata.set('VERSION', meson.project_version())
-	cdata.set('API_EXAMPLES', join_paths(meson.build_root(), 'doc', 'api', 'examples.dox'))
-	cdata.set('OUTPUT', join_paths(meson.build_root(), 'doc', 'api'))
-	cdata.set('HTML_OUTPUT', 'api')
-	cdata.set('TOPDIR', meson.source_root())
-	cdata.set('STRIP_FROM_PATH', meson.source_root())
-
-	doxy_conf = configure_file(input: 'doxy-api.conf.in',
-		output: 'doxy-api.conf',
-		configuration: cdata)
-
-	doxy_build = custom_target('doxygen',
-		depends: example,
-		input: doxy_conf,
-		output: 'api',
-		command: [generate_doxygen, '@INPUT@', '@OUTPUT@', generate_css],
-		install: get_option('enable_docs'),
-		install_dir: htmldir,
-		build_by_default: get_option('enable_docs'))
-
-	doc_targets += doxy_build
-	doc_target_names += 'Doxygen_API'
+if not doxygen.found()
+  subdir_done()
 endif
+
+# due to the CSS customisation script, which needs to run on a file that
+# is in a subdirectory that is created at build time and thus it cannot
+# be an individual custom_target, we need to wrap the doxygen call in a
+# script to run the CSS modification afterwards
+generate_doxygen = find_program('generate_doxygen.sh')
+generate_examples = find_program('generate_examples.sh')
+generate_css = find_program('doxy-html-custom.sh')
+
+inputdir = join_paths(meson.source_root(), 'examples')
+htmldir = join_paths('share', 'doc', 'dpdk')
+
+# due to the following bug: https://github.com/mesonbuild/meson/issues/4107
+# if install is set to true it will override build_by_default and it will
+# cause the target to always be built. If install were to be always set to
+# false it would be impossible to install the docs.
+# So use a configure option for now.
+example = custom_target('examples.dox',
+	input: inputdir,
+	output: 'examples.dox',
+	command: [generate_examples, '@INPUT@', '@OUTPUT@'],
+	install: get_option('enable_docs'),
+	install_dir: htmldir,
+	build_by_default: get_option('enable_docs'))
+
+cdata = configuration_data()
+cdata.set('VERSION', meson.project_version())
+cdata.set('API_EXAMPLES', join_paths(meson.build_root(), 'doc', 'api', 'examples.dox'))
+cdata.set('OUTPUT', join_paths(meson.build_root(), 'doc', 'api'))
+cdata.set('HTML_OUTPUT', 'api')
+cdata.set('TOPDIR', meson.source_root())
+cdata.set('STRIP_FROM_PATH', meson.source_root())
+
+doxy_conf = configure_file(input: 'doxy-api.conf.in',
+	output: 'doxy-api.conf',
+	configuration: cdata)
+
+doxy_build = custom_target('doxygen',
+	depends: example,
+	input: doxy_conf,
+	output: 'api',
+	command: [generate_doxygen, '@INPUT@', '@OUTPUT@', generate_css],
+	install: get_option('enable_docs'),
+	install_dir: htmldir,
+	build_by_default: get_option('enable_docs'))
+
+doc_targets += doxy_build
+doc_target_names += 'Doxygen_API'
-- 
2.24.1

Re: [dpdk-dev] [PATCH v4 5/6] doc/guides: rebuild with meson whenever a file changes

[David Marchand]

On Fri, Jan 10, 2020 at 10:52 PM Bruce Richardson
<bruce.richardson@intel.com> wrote:
>
> Add proper support for calling sphinx whenever a file in the doc
> directory changes. This is accomplished by using a wrapper script
> for sphinx, which runs sphinx but also emits a gcc-format dependency
> file listing all the doc files. This is used by ninja so that any
> change to the doc files triggers a rebuild of the docs.
>
> Signed-off-by: Bruce Richardson <bruce.richardson@intel.com>
> Acked-by: Aaron Conole <aconole@redhat.com>
> Acked-by: Luca Boccassi <bluca@debian.org>
> ---
>  MAINTAINERS                     |  1 +
>  buildtools/call-sphinx-build.py | 31 +++++++++++++++++++++++++++++++
>  buildtools/meson.build          |  6 ++++--
>  doc/guides/meson.build          | 22 ++++++++--------------
>  4 files changed, 44 insertions(+), 16 deletions(-)
>  create mode 100755 buildtools/call-sphinx-build.py

Caught an issue on fc30:

[2094/2338] Generating html_guides with a custom command.
FAILED: doc/guides/html
/usr/bin/python3 ../../dpdk/buildtools/call-sphinx-build.py
/usr/libexec/python2-sphinx/sphinx-build
/home/dmarchan/dpdk/doc/guides
/home/dmarchan/builds/build-x86-default/doc/guides
sphinx-build 1.8.4
Traceback (most recent call last):
  File "../../dpdk/buildtools/call-sphinx-build.py", line 15, in <module>
    ver = run([sphinx, '--version'], stdout=PIPE).stdout.decode().split()[-1]
IndexError: list index out of range

Double checked, the version went to stderr:

$ sphinx-build --version >/dev/null
sphinx-build 1.8.4
$ sphinx-build --version 2>/dev/null


-- 
David Marchand

Re: [dpdk-dev] [PATCH v4 0/6] meson build improvements

[David Marchand]

On Fri, Jan 10, 2020 at 10:52 PM Bruce Richardson
<bruce.richardson@intel.com> wrote:
>
> These patches make some improvements to the meson build, particularly
> for documentation. They also remove many, but not all warnings issued by
> meson e.g. warnings about newer features unsupported in baseline.
>
> The biggest change is to improve the handling of the guide html docs.
> The change here is more significant, and the doc build now uses a
> wrapper script around sphinx. This wrapper script allows us to output
> correct dependency information for the sphinx build in a .d file. This
> .d file is processed by ninja (not meson) on build, so that any changes
> to doc files trigger a rebuild to the guides using sphinx.
>
> For now, the two patches which remove the meson version warnings are
> CC'ed to stable for backport, theoretically this who set could be
> backported if so desired, as all changes could be considered fixes to
> some degree or other, and nothing introduces a whole new feature.
>
> Note: for completeness and simplicity, previously submitted patch
> http://patches.dpdk.org/patch/64189/ is included in this set, and will
> be marked superceded in patchwork.
>
> V4: fix build with older sphinx
>
> V3: Shorten the python code in patch 5, and ensure style compliance as
>     flagged by Aaron.
>
> V2: resend to correct email addresses

Tried this series in Travis.
It went fine through my own checks too.

But OBS is not happy with Debian 10, Debian Next and Ubuntu >= 18.04.
The packaging fails when ninja can't find the 'doc' target.


--
David Marchand

Re: [dpdk-dev] [PATCH v4 5/6] doc/guides: rebuild with meson whenever a file changes

[Bruce Richardson]

On Tue, Jan 14, 2020 at 04:59:24PM +0100, David Marchand wrote:
> On Fri, Jan 10, 2020 at 10:52 PM Bruce Richardson
> <bruce.richardson@intel.com> wrote:
> >
> > Add proper support for calling sphinx whenever a file in the doc
> > directory changes. This is accomplished by using a wrapper script
> > for sphinx, which runs sphinx but also emits a gcc-format dependency
> > file listing all the doc files. This is used by ninja so that any
> > change to the doc files triggers a rebuild of the docs.
> >
> > Signed-off-by: Bruce Richardson <bruce.richardson@intel.com>
> > Acked-by: Aaron Conole <aconole@redhat.com>
> > Acked-by: Luca Boccassi <bluca@debian.org>
> > ---
> >  MAINTAINERS                     |  1 +
> >  buildtools/call-sphinx-build.py | 31 +++++++++++++++++++++++++++++++
> >  buildtools/meson.build          |  6 ++++--
> >  doc/guides/meson.build          | 22 ++++++++--------------
> >  4 files changed, 44 insertions(+), 16 deletions(-)
> >  create mode 100755 buildtools/call-sphinx-build.py
> 
> Caught an issue on fc30:
> 
> [2094/2338] Generating html_guides with a custom command.
> FAILED: doc/guides/html
> /usr/bin/python3 ../../dpdk/buildtools/call-sphinx-build.py
> /usr/libexec/python2-sphinx/sphinx-build
> /home/dmarchan/dpdk/doc/guides
> /home/dmarchan/builds/build-x86-default/doc/guides
> sphinx-build 1.8.4
> Traceback (most recent call last):
>   File "../../dpdk/buildtools/call-sphinx-build.py", line 15, in <module>
>     ver = run([sphinx, '--version'], stdout=PIPE).stdout.decode().split()[-1]
> IndexError: list index out of range
> 
> Double checked, the version went to stderr:
> 
> $ sphinx-build --version >/dev/null
> sphinx-build 1.8.4
> $ sphinx-build --version 2>/dev/null
> 
That is strange. Didn't hit any issues on fedora 31 which I am using here.
I'll see if I can make it work with both stderr and stdout reporting.

/Bruce

Re: [dpdk-dev] [PATCH v4 0/6] meson build improvements

[Bruce Richardson]

On Tue, Jan 14, 2020 at 05:00:17PM +0100, David Marchand wrote:
> On Fri, Jan 10, 2020 at 10:52 PM Bruce Richardson
> <bruce.richardson@intel.com> wrote:
> >
> > These patches make some improvements to the meson build, particularly
> > for documentation. They also remove many, but not all warnings issued
> > by meson e.g. warnings about newer features unsupported in baseline.
> >
> > The biggest change is to improve the handling of the guide html docs.
> > The change here is more significant, and the doc build now uses a
> > wrapper script around sphinx. This wrapper script allows us to output
> > correct dependency information for the sphinx build in a .d file. This
> > .d file is processed by ninja (not meson) on build, so that any changes
> > to doc files trigger a rebuild to the guides using sphinx.
> >
> > For now, the two patches which remove the meson version warnings are
> > CC'ed to stable for backport, theoretically this who set could be
> > backported if so desired, as all changes could be considered fixes to
> > some degree or other, and nothing introduces a whole new feature.
> >
> > Note: for completeness and simplicity, previously submitted patch
> > http://patches.dpdk.org/patch/64189/ is included in this set, and will
> > be marked superceded in patchwork.
> >
> > V4: fix build with older sphinx
> >
> > V3: Shorten the python code in patch 5, and ensure style compliance as
> > flagged by Aaron.
> >
> > V2: resend to correct email addresses
> 
> Tried this series in Travis.  It went fine through my own checks too.
> 
> But OBS is not happy with Debian 10, Debian Next and Ubuntu >= 18.04.
> The packaging fails when ninja can't find the 'doc' target.
> 
Can you send on a link to the failure logs?

I suspect this may be due to returning early from the doc folder if the
document building is disabled (but need to see logs to confirm). However,
I'm not sure it makes sense to have a doc target defined if doc building is
disabled in build config.

Re: [dpdk-dev] [PATCH v4 0/6] meson build improvements

[David Marchand]

On Tue, Jan 14, 2020 at 5:19 PM Bruce Richardson
<bruce.richardson@intel.com> wrote:
>
> On Tue, Jan 14, 2020 at 05:00:17PM +0100, David Marchand wrote:
> > On Fri, Jan 10, 2020 at 10:52 PM Bruce Richardson
> > <bruce.richardson@intel.com> wrote:
> > >
> > > These patches make some improvements to the meson build, particularly
> > > for documentation. They also remove many, but not all warnings issued
> > > by meson e.g. warnings about newer features unsupported in baseline.
> > >
> > > The biggest change is to improve the handling of the guide html docs.
> > > The change here is more significant, and the doc build now uses a
> > > wrapper script around sphinx. This wrapper script allows us to output
> > > correct dependency information for the sphinx build in a .d file. This
> > > .d file is processed by ninja (not meson) on build, so that any changes
> > > to doc files trigger a rebuild to the guides using sphinx.
> > >
> > > For now, the two patches which remove the meson version warnings are
> > > CC'ed to stable for backport, theoretically this who set could be
> > > backported if so desired, as all changes could be considered fixes to
> > > some degree or other, and nothing introduces a whole new feature.
> > >
> > > Note: for completeness and simplicity, previously submitted patch
> > > http://patches.dpdk.org/patch/64189/ is included in this set, and will
> > > be marked superceded in patchwork.
> > >
> > > V4: fix build with older sphinx
> > >
> > > V3: Shorten the python code in patch 5, and ensure style compliance as
> > > flagged by Aaron.
> > >
> > > V2: resend to correct email addresses
> >
> > Tried this series in Travis.  It went fine through my own checks too.
> >
> > But OBS is not happy with Debian 10, Debian Next and Ubuntu >= 18.04.
> > The packaging fails when ninja can't find the 'doc' target.
> >
> Can you send on a link to the failure logs?
>
> I suspect this may be due to returning early from the doc folder if the
> document building is disabled (but need to see logs to confirm). However,
> I'm not sure it makes sense to have a doc target defined if doc building is
> disabled in build config.
>

Currently restarting my jobs and I might lose the logs, so I saved it here:
https://paste.centos.org/view/7d6e0545

-- 
David Marchand

Re: [dpdk-dev] [PATCH v4 0/6] meson build improvements

[Luca Boccassi]

On Tue, 2020-01-14 at 16:19 +0000, Bruce Richardson wrote:
> On Tue, Jan 14, 2020 at 05:00:17PM +0100, David Marchand wrote:
> > On Fri, Jan 10, 2020 at 10:52 PM Bruce Richardson
> > <
> > bruce.richardson@intel.com
> > > wrote:
> > > These patches make some improvements to the meson build,
> > > particularly
> > > for documentation. They also remove many, but not all warnings
> > > issued
> > > by meson e.g. warnings about newer features unsupported in
> > > baseline.
> > > 
> > > The biggest change is to improve the handling of the guide html
> > > docs.
> > > The change here is more significant, and the doc build now uses a
> > > wrapper script around sphinx. This wrapper script allows us to
> > > output
> > > correct dependency information for the sphinx build in a .d file.
> > > This
> > > .d file is processed by ninja (not meson) on build, so that any
> > > changes
> > > to doc files trigger a rebuild to the guides using sphinx.
> > > 
> > > For now, the two patches which remove the meson version warnings
> > > are
> > > CC'ed to stable for backport, theoretically this who set could be
> > > backported if so desired, as all changes could be considered
> > > fixes to
> > > some degree or other, and nothing introduces a whole new feature.
> > > 
> > > Note: for completeness and simplicity, previously submitted patch
> > > http://patches.dpdk.org/patch/64189/
> > >  is included in this set, and will
> > > be marked superceded in patchwork.
> > > 
> > > V4: fix build with older sphinx
> > > 
> > > V3: Shorten the python code in patch 5, and ensure style
> > > compliance as
> > > flagged by Aaron.
> > > 
> > > V2: resend to correct email addresses
> > 
> > Tried this series in Travis.  It went fine through my own checks
> > too.
> > 
> > But OBS is not happy with Debian 10, Debian Next and Ubuntu >=
> > 18.04.
> > The packaging fails when ninja can't find the 'doc' target.
> > 
> 
> Can you send on a link to the failure logs?
> 
> I suspect this may be due to returning early from the doc folder if
> the
> document building is disabled (but need to see logs to confirm).
> However,
> I'm not sure it makes sense to have a doc target defined if doc
> building is
> disabled in build config.

Sorry, should have thought about this when reviewing: the usefulness of
having the target work without the option set is that the docs will
only be built when called explicitly, without having to change the
configuration.

Given the doc build is very slow and it's arch-independent, in
Debian/Ubuntu we only do it once, in the arch-independent target (arch:
all).

-- 
Kind regards,
Luca Boccassi

Re: [dpdk-dev] [PATCH v4 0/6] meson build improvements

[Bruce Richardson]

On Tue, Jan 14, 2020 at 04:35:09PM +0000, Luca Boccassi wrote:
> On Tue, 2020-01-14 at 16:19 +0000, Bruce Richardson wrote:
> > On Tue, Jan 14, 2020 at 05:00:17PM +0100, David Marchand wrote:
> > > On Fri, Jan 10, 2020 at 10:52 PM Bruce Richardson
> > > <
> > > bruce.richardson@intel.com
> > > > wrote:
> > > > These patches make some improvements to the meson build,
> > > > particularly
> > > > for documentation. They also remove many, but not all warnings
> > > > issued
> > > > by meson e.g. warnings about newer features unsupported in
> > > > baseline.
> > > > 
> > > > The biggest change is to improve the handling of the guide html
> > > > docs.
> > > > The change here is more significant, and the doc build now uses a
> > > > wrapper script around sphinx. This wrapper script allows us to
> > > > output
> > > > correct dependency information for the sphinx build in a .d file.
> > > > This
> > > > .d file is processed by ninja (not meson) on build, so that any
> > > > changes
> > > > to doc files trigger a rebuild to the guides using sphinx.
> > > > 
> > > > For now, the two patches which remove the meson version warnings
> > > > are
> > > > CC'ed to stable for backport, theoretically this who set could be
> > > > backported if so desired, as all changes could be considered
> > > > fixes to
> > > > some degree or other, and nothing introduces a whole new feature.
> > > > 
> > > > Note: for completeness and simplicity, previously submitted patch
> > > > http://patches.dpdk.org/patch/64189/
> > > >  is included in this set, and will
> > > > be marked superceded in patchwork.
> > > > 
> > > > V4: fix build with older sphinx
> > > > 
> > > > V3: Shorten the python code in patch 5, and ensure style
> > > > compliance as
> > > > flagged by Aaron.
> > > > 
> > > > V2: resend to correct email addresses
> > > 
> > > Tried this series in Travis.  It went fine through my own checks
> > > too.
> > > 
> > > But OBS is not happy with Debian 10, Debian Next and Ubuntu >=
> > > 18.04.
> > > The packaging fails when ninja can't find the 'doc' target.
> > > 
> > 
> > Can you send on a link to the failure logs?
> > 
> > I suspect this may be due to returning early from the doc folder if
> > the
> > document building is disabled (but need to see logs to confirm).
> > However,
> > I'm not sure it makes sense to have a doc target defined if doc
> > building is
> > disabled in build config.
> 
> Sorry, should have thought about this when reviewing: the usefulness of
> having the target work without the option set is that the docs will
> only be built when called explicitly, without having to change the
> configuration.
> 
> Given the doc build is very slow and it's arch-independent, in
> Debian/Ubuntu we only do it once, in the arch-independent target (arch:
> all).
> 
Right. I think the offending patch #2, can just be dropped from the set.

/Bruce

Re: [dpdk-dev] [PATCH v4 0/6] meson build improvements

[David Marchand]

On Tue, Jan 14, 2020 at 6:36 PM Bruce Richardson
<bruce.richardson@intel.com> wrote:
>
> On Tue, Jan 14, 2020 at 04:35:09PM +0000, Luca Boccassi wrote:
> > On Tue, 2020-01-14 at 16:19 +0000, Bruce Richardson wrote:
> > > On Tue, Jan 14, 2020 at 05:00:17PM +0100, David Marchand wrote:
> > > > On Fri, Jan 10, 2020 at 10:52 PM Bruce Richardson
> > > > <
> > > > bruce.richardson@intel.com
> > > > > wrote:
> > > > > These patches make some improvements to the meson build,
> > > > > particularly
> > > > > for documentation. They also remove many, but not all warnings
> > > > > issued
> > > > > by meson e.g. warnings about newer features unsupported in
> > > > > baseline.
> > > > >
> > > > > The biggest change is to improve the handling of the guide html
> > > > > docs.
> > > > > The change here is more significant, and the doc build now uses a
> > > > > wrapper script around sphinx. This wrapper script allows us to
> > > > > output
> > > > > correct dependency information for the sphinx build in a .d file.
> > > > > This
> > > > > .d file is processed by ninja (not meson) on build, so that any
> > > > > changes
> > > > > to doc files trigger a rebuild to the guides using sphinx.
> > > > >
> > > > > For now, the two patches which remove the meson version warnings
> > > > > are
> > > > > CC'ed to stable for backport, theoretically this who set could be
> > > > > backported if so desired, as all changes could be considered
> > > > > fixes to
> > > > > some degree or other, and nothing introduces a whole new feature.
> > > > >
> > > > > Note: for completeness and simplicity, previously submitted patch
> > > > > http://patches.dpdk.org/patch/64189/
> > > > >  is included in this set, and will
> > > > > be marked superceded in patchwork.
> > > > >
> > > > > V4: fix build with older sphinx
> > > > >
> > > > > V3: Shorten the python code in patch 5, and ensure style
> > > > > compliance as
> > > > > flagged by Aaron.
> > > > >
> > > > > V2: resend to correct email addresses
> > > >
> > > > Tried this series in Travis.  It went fine through my own checks
> > > > too.
> > > >
> > > > But OBS is not happy with Debian 10, Debian Next and Ubuntu >=
> > > > 18.04.
> > > > The packaging fails when ninja can't find the 'doc' target.
> > > >
> > >
> > > Can you send on a link to the failure logs?
> > >
> > > I suspect this may be due to returning early from the doc folder if
> > > the
> > > document building is disabled (but need to see logs to confirm).
> > > However,
> > > I'm not sure it makes sense to have a doc target defined if doc
> > > building is
> > > disabled in build config.
> >
> > Sorry, should have thought about this when reviewing: the usefulness of
> > having the target work without the option set is that the docs will
> > only be built when called explicitly, without having to change the
> > configuration.
> >
> > Given the doc build is very slow and it's arch-independent, in
> > Debian/Ubuntu we only do it once, in the arch-independent target (arch:
> > all).
> >
> Right. I think the offending patch #2, can just be dropped from the set.

Ok, I dropped this patch.
Packages for debian distributions in OBS build fine now.

I am ok with a followup patch for fc30, I disabled the doc check in my
env for now.

Series & ~patch2 applied.
Thanks.


--
David Marchand

Re: [dpdk-dev] [PATCH v4 5/6] doc/guides: rebuild with meson whenever a file changes

[Richardson, Bruce]

> -----Original Message-----
> From: David Marchand <david.marchand@redhat.com>
> Sent: Tuesday, January 14, 2020 3:59 PM
> To: Richardson, Bruce <bruce.richardson@intel.com>
> Cc: Mcnamara, John <john.mcnamara@intel.com>; Luca Boccassi
> <bluca@debian.org>; dev <dev@dpdk.org>; Aaron Conole <aconole@redhat.com>;
> Thomas Monjalon <thomas@monjalon.net>
> Subject: Re: [PATCH v4 5/6] doc/guides: rebuild with meson whenever a file
> changes
> 
> On Fri, Jan 10, 2020 at 10:52 PM Bruce Richardson
> <bruce.richardson@intel.com> wrote:
> >
> > Add proper support for calling sphinx whenever a file in the doc
> > directory changes. This is accomplished by using a wrapper script for
> > sphinx, which runs sphinx but also emits a gcc-format dependency file
> > listing all the doc files. This is used by ninja so that any change to
> > the doc files triggers a rebuild of the docs.
> >
> > Signed-off-by: Bruce Richardson <bruce.richardson@intel.com>
> > Acked-by: Aaron Conole <aconole@redhat.com>
> > Acked-by: Luca Boccassi <bluca@debian.org>
> > ---
> >  MAINTAINERS                     |  1 +
> >  buildtools/call-sphinx-build.py | 31 +++++++++++++++++++++++++++++++
> >  buildtools/meson.build          |  6 ++++--
> >  doc/guides/meson.build          | 22 ++++++++--------------
> >  4 files changed, 44 insertions(+), 16 deletions(-)  create mode
> > 100755 buildtools/call-sphinx-build.py
> 
> Caught an issue on fc30:
> 
> [2094/2338] Generating html_guides with a custom command.
> FAILED: doc/guides/html
> /usr/bin/python3 ../../dpdk/buildtools/call-sphinx-build.py
> /usr/libexec/python2-sphinx/sphinx-build
> /home/dmarchan/dpdk/doc/guides
> /home/dmarchan/builds/build-x86-default/doc/guides
> sphinx-build 1.8.4
> Traceback (most recent call last):
>   File "../../dpdk/buildtools/call-sphinx-build.py", line 15, in <module>
>     ver = run([sphinx, '--version'],
> stdout=PIPE).stdout.decode().split()[-1]
> IndexError: list index out of range
> 
> Double checked, the version went to stderr:
> 
> $ sphinx-build --version >/dev/null
> sphinx-build 1.8.4
> $ sphinx-build --version 2>/dev/null
> 

The series http://patches.dpdk.org/project/dpdk/list/?series=8190 should fix
this issue and also adds support for the "werror" option for building the guides.

/Bruce