[dpdk-dev] [PATCH 1/2] doc: fix doc build when sphinx reports version to stderr

[dpdk-dev] [PATCH 1/2] doc: fix doc build when sphinx reports version to stderr

[Bruce Richardson]

When sphinx-build reports its version information to stderr rather
than stdout, the wrapper script misses it, and then fails to run.
We can fix this by redirecting stderr to stdout for the version
query call.

Fixes: f5ab2074cfba ("doc: rebuild with meson whenever a file changes")

Signed-off-by: Bruce Richardson <bruce.richardson@intel.com>
---
 buildtools/call-sphinx-build.py | 5 +++--
 1 file changed, 3 insertions(+), 2 deletions(-)

diff --git a/buildtools/call-sphinx-build.py b/buildtools/call-sphinx-build.py
index b9a3994e1..85c9e0156 100755
--- a/buildtools/call-sphinx-build.py
+++ b/buildtools/call-sphinx-build.py
@@ -6,13 +6,14 @@
 import sys
 import os
 from os.path import join
-from subprocess import run, PIPE
+from subprocess import run, PIPE, STDOUT
 from distutils.version import StrictVersion
 
 (sphinx, src, dst) = sys.argv[1:]  # assign parameters to variables
 
 # for sphinx version >= 1.7 add parallelism using "-j auto"
-ver = run([sphinx, '--version'], stdout=PIPE).stdout.decode().split()[-1]
+ver = run([sphinx, '--version'], stdout=PIPE,
+          stderr=STDOUT).stdout.decode().split()[-1]
 sphinx_cmd = [sphinx]
 if StrictVersion(ver) >= StrictVersion('1.7'):
     sphinx_cmd += ['-j', 'auto']
-- 
2.20.1

[dpdk-dev] [PATCH 2/2] doc: pass "werror" setting through to doc build

[Bruce Richardson]

When werror is set for the build, we should pass that flag through to
sphinx so that it can flag warnings as errors too.

Signed-off-by: Bruce Richardson <bruce.richardson@intel.com>
---
 buildtools/call-sphinx-build.py | 5 ++---
 doc/guides/meson.build          | 4 ++++
 2 files changed, 6 insertions(+), 3 deletions(-)

diff --git a/buildtools/call-sphinx-build.py b/buildtools/call-sphinx-build.py
index 85c9e0156..fc4e9d040 100755
--- a/buildtools/call-sphinx-build.py
+++ b/buildtools/call-sphinx-build.py
@@ -9,12 +9,11 @@
 from subprocess import run, PIPE, STDOUT
 from distutils.version import StrictVersion
 
-(sphinx, src, dst) = sys.argv[1:]  # assign parameters to variables
+(*sphinx_cmd, src, dst) = sys.argv[1:]  # assign parameters to variables
 
 # for sphinx version >= 1.7 add parallelism using "-j auto"
-ver = run([sphinx, '--version'], stdout=PIPE,
+ver = run(sphinx_cmd + ['--version'], stdout=PIPE,
           stderr=STDOUT).stdout.decode().split()[-1]
-sphinx_cmd = [sphinx]
 if StrictVersion(ver) >= StrictVersion('1.7'):
     sphinx_cmd += ['-j', 'auto']
 
diff --git a/doc/guides/meson.build b/doc/guides/meson.build
index 732e7ad3a..5a2b854e8 100644
--- a/doc/guides/meson.build
+++ b/doc/guides/meson.build
@@ -7,6 +7,10 @@ if not sphinx.found()
 	subdir_done()
 endif
 
+if get_option('werror')
+	sphinx = [sphinx, '-W']
+endif
+
 htmldir = join_paths(get_option('datadir'), 'doc', 'dpdk')
 html_guides = custom_target('html_guides',
 	input: files('index.rst'),
-- 
2.20.1

Re: [dpdk-dev] [PATCH 2/2] doc: pass "werror" setting through to doc build

[Aaron Conole]

Bruce Richardson <bruce.richardson@intel.com> writes:

> When werror is set for the build, we should pass that flag through to
> sphinx so that it can flag warnings as errors too.
>
> Signed-off-by: Bruce Richardson <bruce.richardson@intel.com>
> ---

I see that this actually works to generate the errors... BUT

if we merge this it will break the build.  Can you also insert a patch
to address the warning so that the series could be merged?

>  buildtools/call-sphinx-build.py | 5 ++---
>  doc/guides/meson.build          | 4 ++++
>  2 files changed, 6 insertions(+), 3 deletions(-)
>
> diff --git a/buildtools/call-sphinx-build.py b/buildtools/call-sphinx-build.py
> index 85c9e0156..fc4e9d040 100755
> --- a/buildtools/call-sphinx-build.py
> +++ b/buildtools/call-sphinx-build.py
> @@ -9,12 +9,11 @@
>  from subprocess import run, PIPE, STDOUT
>  from distutils.version import StrictVersion
>  
> -(sphinx, src, dst) = sys.argv[1:]  # assign parameters to variables
> +(*sphinx_cmd, src, dst) = sys.argv[1:]  # assign parameters to variables
>  
>  # for sphinx version >= 1.7 add parallelism using "-j auto"
> -ver = run([sphinx, '--version'], stdout=PIPE,
> +ver = run(sphinx_cmd + ['--version'], stdout=PIPE,
>            stderr=STDOUT).stdout.decode().split()[-1]
> -sphinx_cmd = [sphinx]
>  if StrictVersion(ver) >= StrictVersion('1.7'):
>      sphinx_cmd += ['-j', 'auto']
>  
> diff --git a/doc/guides/meson.build b/doc/guides/meson.build
> index 732e7ad3a..5a2b854e8 100644
> --- a/doc/guides/meson.build
> +++ b/doc/guides/meson.build
> @@ -7,6 +7,10 @@ if not sphinx.found()
>  	subdir_done()
>  endif
>  
> +if get_option('werror')
> +	sphinx = [sphinx, '-W']
> +endif
> +
>  htmldir = join_paths(get_option('datadir'), 'doc', 'dpdk')
>  html_guides = custom_target('html_guides',
>  	input: files('index.rst'),

Re: [dpdk-dev] [PATCH 2/2] doc: pass "werror" setting through to doc build

[Bruce Richardson]

On Fri, Jan 17, 2020 at 08:16:55AM -0500, Aaron Conole wrote:
> Bruce Richardson <bruce.richardson@intel.com> writes:
> 
> > When werror is set for the build, we should pass that flag through to
> > sphinx so that it can flag warnings as errors too.
> >
> > Signed-off-by: Bruce Richardson <bruce.richardson@intel.com>
> > ---
> 
> I see that this actually works to generate the errors... BUT
> 
> if we merge this it will break the build.  Can you also insert a patch
> to address the warning so that the series could be merged?
> 

Ok, I didn't have any warnings in my setup, which is why I didn't see any
problems. I assume that the warnings are showing up in travis? Anywhere
else?

/Bruce

Re: [dpdk-dev] [PATCH 2/2] doc: pass "werror" setting through to doc build

[Aaron Conole]

Bruce Richardson <bruce.richardson@intel.com> writes:

> On Fri, Jan 17, 2020 at 08:16:55AM -0500, Aaron Conole wrote:
>> Bruce Richardson <bruce.richardson@intel.com> writes:
>> 
>> > When werror is set for the build, we should pass that flag through to
>> > sphinx so that it can flag warnings as errors too.
>> >
>> > Signed-off-by: Bruce Richardson <bruce.richardson@intel.com>
>> > ---
>> 
>> I see that this actually works to generate the errors... BUT
>> 
>> if we merge this it will break the build.  Can you also insert a patch
>> to address the warning so that the series could be merged?
>> 
>
> Ok, I didn't have any warnings in my setup, which is why I didn't see any
> problems. I assume that the warnings are showing up in travis? Anywhere
> else?

I only saw them on Travis.

> /Bruce

Re: [dpdk-dev] [PATCH 2/2] doc: pass "werror" setting through to doc build

[Bruce Richardson]

On Fri, Jan 17, 2020 at 08:34:01AM -0500, Aaron Conole wrote:
> Bruce Richardson <bruce.richardson@intel.com> writes:
> 
> > On Fri, Jan 17, 2020 at 08:16:55AM -0500, Aaron Conole wrote:
> >> Bruce Richardson <bruce.richardson@intel.com> writes:
> >> 
> >> > When werror is set for the build, we should pass that flag through to
> >> > sphinx so that it can flag warnings as errors too.
> >> >
> >> > Signed-off-by: Bruce Richardson <bruce.richardson@intel.com>
> >> > ---
> >> 
> >> I see that this actually works to generate the errors... BUT
> >> 
> >> if we merge this it will break the build.  Can you also insert a patch
> >> to address the warning so that the series could be merged?
> >> 
> >
> > Ok, I didn't have any warnings in my setup, which is why I didn't see any
> > problems. I assume that the warnings are showing up in travis? Anywhere
> > else?
> 
> I only saw them on Travis.
> 
The error from sphinx in travis looks like a false positive that is fixed
in later versions of sphinx. The error I see is:

/home/travis/build/bruce-richardson/dpdk/doc/guides/linux_gsg/eal_args.include.rst:: WARNING: document isn't included in any toctree

However, that file is an include one that is included in both the linux and
freebsd eal parameters docs, and so is not missing though not included in
the index. What is the best approach to deal with this, do you think?

* rework so it has a toctree entry e.g. by creating a new section for
  common parameters
* other workaround in the code, e.g. rename the file to not end in .rst
* can we update sphinx in the travis build to avoid the warning altogether?

/Bruce

Re: [dpdk-dev] [PATCH 2/2] doc: pass "werror" setting through to doc build

[Aaron Conole]

Bruce Richardson <bruce.richardson@intel.com> writes:

> On Fri, Jan 17, 2020 at 08:34:01AM -0500, Aaron Conole wrote:
>> Bruce Richardson <bruce.richardson@intel.com> writes:
>> 
>> > On Fri, Jan 17, 2020 at 08:16:55AM -0500, Aaron Conole wrote:
>> >> Bruce Richardson <bruce.richardson@intel.com> writes:
>> >> 
>> >> > When werror is set for the build, we should pass that flag through to
>> >> > sphinx so that it can flag warnings as errors too.
>> >> >
>> >> > Signed-off-by: Bruce Richardson <bruce.richardson@intel.com>
>> >> > ---
>> >> 
>> >> I see that this actually works to generate the errors... BUT
>> >> 
>> >> if we merge this it will break the build.  Can you also insert a patch
>> >> to address the warning so that the series could be merged?
>> >> 
>> >
>> > Ok, I didn't have any warnings in my setup, which is why I didn't see any
>> > problems. I assume that the warnings are showing up in travis? Anywhere
>> > else?
>> 
>> I only saw them on Travis.
>> 
> The error from sphinx in travis looks like a false positive that is fixed
> in later versions of sphinx. The error I see is:
>
> /home/travis/build/bruce-richardson/dpdk/doc/guides/linux_gsg/eal_args.include.rst::
> WARNING: document isn't included in any toctree
>
> However, that file is an include one that is included in both the linux and
> freebsd eal parameters docs, and so is not missing though not included in
> the index. What is the best approach to deal with this, do you think?
>
> * rework so it has a toctree entry e.g. by creating a new section for
>   common parameters
> * other workaround in the code, e.g. rename the file to not end in .rst
> * can we update sphinx in the travis build to avoid the warning altogether?

I think the third option is best.  Distributions can always disable
werror on doc builds or upgrade their own sphinx packages.

If that isn't acceptable to anyone else, then the 1st option is my
next choice.  I dislike the approach of renaming the file.

> /Bruce

[dpdk-dev] [PATCH v2] doc: fix doc build when sphinx reports version to stderr

[Bruce Richardson]

When sphinx-build reports its version information to stderr rather
than stdout, the wrapper script misses it, and then fails to run.
We can fix this by redirecting stderr to stdout for the version
query call.

Fixes: f5ab2074cfba ("doc: rebuild with meson whenever a file changes")

Signed-off-by: Bruce Richardson <bruce.richardson@intel.com>
---
V2: no changes to this patch, dropped patch 2 from series.
---
 buildtools/call-sphinx-build.py | 5 +++--
 1 file changed, 3 insertions(+), 2 deletions(-)

diff --git a/buildtools/call-sphinx-build.py b/buildtools/call-sphinx-build.py
index b9a3994e1..85c9e0156 100755
--- a/buildtools/call-sphinx-build.py
+++ b/buildtools/call-sphinx-build.py
@@ -6,13 +6,14 @@
 import sys
 import os
 from os.path import join
-from subprocess import run, PIPE
+from subprocess import run, PIPE, STDOUT
 from distutils.version import StrictVersion
 
 (sphinx, src, dst) = sys.argv[1:]  # assign parameters to variables
 
 # for sphinx version >= 1.7 add parallelism using "-j auto"
-ver = run([sphinx, '--version'], stdout=PIPE).stdout.decode().split()[-1]
+ver = run([sphinx, '--version'], stdout=PIPE,
+          stderr=STDOUT).stdout.decode().split()[-1]
 sphinx_cmd = [sphinx]
 if StrictVersion(ver) >= StrictVersion('1.7'):
     sphinx_cmd += ['-j', 'auto']
-- 
2.25.1

Re: [dpdk-dev] [PATCH v2] doc: fix doc build when sphinx reports version to stderr

[Power, Ciara]

>-----Original Message-----
>From: dev <dev-bounces@dpdk.org> On Behalf Of Bruce Richardson
>Sent: Monday 22 June 2020 15:00
>To: david.marchand@redhat.com
>Cc: dev@dpdk.org; Richardson, Bruce <bruce.richardson@intel.com>
>Subject: [dpdk-dev] [PATCH v2] doc: fix doc build when sphinx reports version
>to stderr
>
>When sphinx-build reports its version information to stderr rather than
>stdout, the wrapper script misses it, and then fails to run.
>We can fix this by redirecting stderr to stdout for the version query call.
>
>Fixes: f5ab2074cfba ("doc: rebuild with meson whenever a file changes")
>
>Signed-off-by: Bruce Richardson <bruce.richardson@intel.com>
>---
>V2: no changes to this patch, dropped patch 2 from series.
>---
> buildtools/call-sphinx-build.py | 5 +++--
> 1 file changed, 3 insertions(+), 2 deletions(-)
>
>diff --git a/buildtools/call-sphinx-build.py b/buildtools/call-sphinx-build.py
>index b9a3994e1..85c9e0156 100755
>--- a/buildtools/call-sphinx-build.py
>+++ b/buildtools/call-sphinx-build.py
>@@ -6,13 +6,14 @@
> import sys
> import os
> from os.path import join
>-from subprocess import run, PIPE
>+from subprocess import run, PIPE, STDOUT
> from distutils.version import StrictVersion
>
> (sphinx, src, dst) = sys.argv[1:]  # assign parameters to variables
>
> # for sphinx version >= 1.7 add parallelism using "-j auto"
>-ver = run([sphinx, '--version'], stdout=PIPE).stdout.decode().split()[-1]
>+ver = run([sphinx, '--version'], stdout=PIPE,
>+          stderr=STDOUT).stdout.decode().split()[-1]
> sphinx_cmd = [sphinx]
> if StrictVersion(ver) >= StrictVersion('1.7'):
>     sphinx_cmd += ['-j', 'auto']
>--
>2.25.1


Hi Bruce,

I hit the issue described here, and this patch fixed it for me.
Thanks!

Tested-by: Ciara Power <ciara.power@intel.com>

Re: [dpdk-dev] [PATCH v2] doc: fix doc build when sphinx reports version to stderr

[David Marchand]

On Mon, Jun 22, 2020 at 4:00 PM Bruce Richardson
<bruce.richardson@intel.com> wrote:
>
> When sphinx-build reports its version information to stderr rather
> than stdout, the wrapper script misses it, and then fails to run.
> We can fix this by redirecting stderr to stdout for the version
> query call.
>
> Fixes: f5ab2074cfba ("doc: rebuild with meson whenever a file changes")
>
> Signed-off-by: Bruce Richardson <bruce.richardson@intel.com>
> ---
> V2: no changes to this patch, dropped patch 2 from series.

Tested-by: David Marchand <david.marchand@redhat.com>

I initially encountered the issue on a fc30 when checking doc generation.
fc30 is now EOL and the problem does not happen anymore on fc31.

Still worth fixing from my pov.

Thanks Bruce.


-- 
David Marchand

Re: [dpdk-dev] [PATCH v2] doc: fix doc build when sphinx reports version to stderr

[Thomas Monjalon]

22/06/2020 16:20, David Marchand:
> On Mon, Jun 22, 2020 at 4:00 PM Bruce Richardson
> <bruce.richardson@intel.com> wrote:
> >
> > When sphinx-build reports its version information to stderr rather
> > than stdout, the wrapper script misses it, and then fails to run.
> > We can fix this by redirecting stderr to stdout for the version
> > query call.
> >
> > Fixes: f5ab2074cfba ("doc: rebuild with meson whenever a file changes")
> >
> > Signed-off-by: Bruce Richardson <bruce.richardson@intel.com>
> > ---
> > V2: no changes to this patch, dropped patch 2 from series.
> 
> Tested-by: David Marchand <david.marchand@redhat.com>
> 
> I initially encountered the issue on a fc30 when checking doc generation.
> fc30 is now EOL and the problem does not happen anymore on fc31.
> 
> Still worth fixing from my pov.
> 
> Thanks Bruce.

Applied, thanks