[PATCH] doc: build manpages as well as html output

[PATCH] doc: build manpages as well as html output

[Bruce Richardson]

Doxygen can produce manpage output as well as html output for the DPDK
APIs. However, we need to do this as a separate task as the manpage
output needs to be placed in a different location post-install to the
html output (/usr/local/share/man vs /usr/local/share/doc/).

Changes required are:
* Add configurable options for manpage output and html output to the
  doxygen config template. (Remove option for html output path as it's
  always "html")
* Modify API meson.build file to configure two separate doxygen config
  files, for HTML and manpages respectively.
* Change doxygen wrapper script to have separate output log files for
  the manpage and HTML jobs, to avoid conflicts
* Add "custom_targets" to meson.build file to build the HTML pages and
  the manpages, with individual install locations for each.
* Where supported by meson version, call "mandb" post-install to update
  the man database to ensure the new manpages can be found.

Signed-off-by: Bruce Richardson <bruce.richardson@intel.com>
---
 doc/api/doxy-api.conf.in    |  7 ++---
 doc/api/generate_doxygen.py |  2 +-
 doc/api/meson.build         | 52 +++++++++++++++++++++++++++++++------
 3 files changed, 49 insertions(+), 12 deletions(-)

diff --git a/doc/api/doxy-api.conf.in b/doc/api/doxy-api.conf.in
index 967aa11701..6e5be9b469 100644
--- a/doc/api/doxy-api.conf.in
+++ b/doc/api/doxy-api.conf.in
@@ -125,10 +125,11 @@ EXAMPLE_RECURSIVE       = YES
 
 OUTPUT_DIRECTORY        = @OUTPUT@
 STRIP_FROM_PATH         = @STRIP_FROM_PATH@
-GENERATE_HTML           = YES
-HTML_OUTPUT             = @HTML_OUTPUT@
+GENERATE_HTML           = @GENERATE_HTML@
+HTML_OUTPUT             = html
 GENERATE_LATEX          = NO
-GENERATE_MAN            = NO
+GENERATE_MAN            = @GENERATE_MAN@
+MAN_LINKS               = YES
 
 HAVE_DOT                = NO
 
diff --git a/doc/api/generate_doxygen.py b/doc/api/generate_doxygen.py
index d3a22869f6..c704f13018 100755
--- a/doc/api/generate_doxygen.py
+++ b/doc/api/generate_doxygen.py
@@ -7,7 +7,7 @@
 
 pattern = re.compile('^Preprocessing (.*)...$')
 out_dir, *doxygen_command = sys.argv[1:]
-out_file = os.path.join(os.path.dirname(out_dir), 'doxygen.out')
+out_file = os.path.join(out_dir + '.out')
 dep_file = f'{out_dir}.d'
 with open(out_file, 'w') as out:
     subprocess.run(doxygen_command, check=True, stdout=out)
diff --git a/doc/api/meson.build b/doc/api/meson.build
index 2876a78a7e..104b91517b 100644
--- a/doc/api/meson.build
+++ b/doc/api/meson.build
@@ -29,11 +29,11 @@ example = custom_target('examples.dox',
         install_dir: htmldir,
         build_by_default: get_option('enable_docs'))
 
+#set up common doxygen configuration
 cdata = configuration_data()
 cdata.set('VERSION', meson.project_version())
 cdata.set('API_EXAMPLES', join_paths(dpdk_build_root, 'doc', 'api', 'examples.dox'))
 cdata.set('OUTPUT', join_paths(dpdk_build_root, 'doc', 'api'))
-cdata.set('HTML_OUTPUT', 'html')
 cdata.set('TOPDIR', dpdk_source_root)
 cdata.set('STRIP_FROM_PATH', ' '.join([dpdk_source_root, join_paths(dpdk_build_root, 'doc', 'api')]))
 cdata.set('WARN_AS_ERROR', 'NO')
@@ -41,14 +41,31 @@ if get_option('werror')
     cdata.set('WARN_AS_ERROR', 'YES')
 endif
 
-doxy_conf = configure_file(input: 'doxy-api.conf.in',
-        output: 'doxy-api.conf',
-        configuration: cdata)
+# configure HTML doxygen run
+html_cdata = configuration_data()
+html_cdata.merge_from(cdata)
+html_cdata.set('GENERATE_HTML', 'YES')
+html_cdata.set('GENERATE_MAN', 'NO')
 
-doxy_build = custom_target('doxygen',
+doxy_html_conf = configure_file(input: 'doxy-api.conf.in',
+        output: 'doxy-api-html.conf',
+        configuration: html_cdata)
+
+# configure manpage doxygen run
+man_cdata = configuration_data()
+man_cdata.merge_from(cdata)
+man_cdata.set('GENERATE_HTML', 'NO')
+man_cdata.set('GENERATE_MAN', 'YES')
+
+doxy_man_conf = configure_file(input: 'doxy-api.conf.in',
+        output: 'doxy-api-man.conf',
+        configuration: man_cdata)
+
+# do doxygen runs
+doxy_html_build = custom_target('doxygen-html',
         depends: example,
         depend_files: 'doxy-api-index.md',
-        input: doxy_conf,
+        input: doxy_html_conf,
         output: 'html',
         depfile: 'html.d',
         command: [generate_doxygen, '@OUTPUT@', doxygen, '@INPUT@'],
@@ -56,5 +73,24 @@ doxy_build = custom_target('doxygen',
         install_dir: htmldir,
         build_by_default: get_option('enable_docs'))
 
-doc_targets += doxy_build
-doc_target_names += 'Doxygen_API'
+doc_targets += doxy_html_build
+doc_target_names += 'Doxygen_API(HTML)'
+
+doxy_man_build = custom_target('doxygen-man',
+        depends: example,
+        depend_files: 'doxy-api-index.md',
+        input: doxy_man_conf,
+        output: 'man',
+        depfile: 'man.d',
+        command: [generate_doxygen, '@OUTPUT@', doxygen, '@INPUT@'],
+        install: get_option('enable_docs'),
+        install_dir: get_option('datadir'),
+        build_by_default: get_option('enable_docs'))
+
+doc_targets += doxy_man_build
+doc_target_names += 'Doxygen_API(Manpage)'
+
+mandb = find_program('mandb', required: false)
+if mandb.found() and get_option('enable_docs') and meson.version().version_compare('>=0.55.0')
+    meson.add_install_script(mandb)
+endif
-- 
2.39.2

Re: [PATCH] doc: build manpages as well as html output

[Jerin Jacob]

On Thu, Jun 1, 2023 at 9:08 PM Bruce Richardson
<bruce.richardson@intel.com> wrote:
>
> Doxygen can produce manpage output as well as html output for the DPDK
> APIs. However, we need to do this as a separate task as the manpage
> output needs to be placed in a different location post-install to the
> html output (/usr/local/share/man vs /usr/local/share/doc/).
>
> Changes required are:
> * Add configurable options for manpage output and html output to the
>   doxygen config template. (Remove option for html output path as it's
>   always "html")
> * Modify API meson.build file to configure two separate doxygen config
>   files, for HTML and manpages respectively.
> * Change doxygen wrapper script to have separate output log files for
>   the manpage and HTML jobs, to avoid conflicts
> * Add "custom_targets" to meson.build file to build the HTML pages and
>   the manpages, with individual install locations for each.
> * Where supported by meson version, call "mandb" post-install to update
>   the man database to ensure the new manpages can be found.
>
> Signed-off-by: Bruce Richardson <bruce.richardson@intel.com>

> +
> +mandb = find_program('mandb', required: false)
> +if mandb.found() and get_option('enable_docs') and meson.version().version_compare('>=0.55.0')
> +    meson.add_install_script(mandb)

It does not look like just executing mandb it is adding these man
pages to database

log:
Running custom install script '/usr/bin/mandb'
Purging old database entries in /home/jerin/.local/man...
Processing manual pages under /home/jerin/.local/man...
Checking for stray cats under /home/jerin/.local/man...
Processing manual pages under /home/jerin/.local/man/cat1...
Purging old database entries in /home/jerin/.local/share/man...
Processing manual pages under /home/jerin/.local/share/man...
Checking for stray cats under /home/jerin/.local/share/man...
Processing manual pages under /home/jerin/.local/share/man/cat1...
0 man subdirectories contained newer manual pages.
0 manual pages were added.
0 stray cats were added.
0 old database entries were purged.

[main][dpdk.org] $ man  rte_flow_create
No manual entry for rte_flow_create

# Following works by providing the path i.e man pages created properly
only db update is missing
man --manpath=/tmp/i/usr/local/share/man/ rte_flow_create


> +endif
> --
> 2.39.2
>

Re: [PATCH] doc: build manpages as well as html output

[Bruce Richardson]

On Mon, Jun 05, 2023 at 10:50:48AM +0530, Jerin Jacob wrote:
> On Thu, Jun 1, 2023 at 9:08 PM Bruce Richardson
> <bruce.richardson@intel.com> wrote:
> >
> > Doxygen can produce manpage output as well as html output for the DPDK
> > APIs. However, we need to do this as a separate task as the manpage
> > output needs to be placed in a different location post-install to the
> > html output (/usr/local/share/man vs /usr/local/share/doc/).
> >
> > Changes required are:
> > * Add configurable options for manpage output and html output to the
> >   doxygen config template. (Remove option for html output path as it's
> >   always "html")
> > * Modify API meson.build file to configure two separate doxygen config
> >   files, for HTML and manpages respectively.
> > * Change doxygen wrapper script to have separate output log files for
> >   the manpage and HTML jobs, to avoid conflicts
> > * Add "custom_targets" to meson.build file to build the HTML pages and
> >   the manpages, with individual install locations for each.
> > * Where supported by meson version, call "mandb" post-install to update
> >   the man database to ensure the new manpages can be found.
> >
> > Signed-off-by: Bruce Richardson <bruce.richardson@intel.com>
> 
> > +
> > +mandb = find_program('mandb', required: false)
> > +if mandb.found() and get_option('enable_docs') and meson.version().version_compare('>=0.55.0')
> > +    meson.add_install_script(mandb)
> 
> It does not look like just executing mandb it is adding these man
> pages to database
> 
> log:
> Running custom install script '/usr/bin/mandb'
> Purging old database entries in /home/jerin/.local/man...
> Processing manual pages under /home/jerin/.local/man...
> Checking for stray cats under /home/jerin/.local/man...
> Processing manual pages under /home/jerin/.local/man/cat1...
> Purging old database entries in /home/jerin/.local/share/man...
> Processing manual pages under /home/jerin/.local/share/man...
> Checking for stray cats under /home/jerin/.local/share/man...
> Processing manual pages under /home/jerin/.local/share/man/cat1...
> 0 man subdirectories contained newer manual pages.
> 0 manual pages were added.
> 0 stray cats were added.
> 0 old database entries were purged.
> 
> [main][dpdk.org] $ man  rte_flow_create
> No manual entry for rte_flow_create
> 
> # Following works by providing the path i.e man pages created properly
> only db update is missing
> man --manpath=/tmp/i/usr/local/share/man/ rte_flow_create
> 
Yes, that is correct.

If you install to a non-standard location, then yes you need to update
manpath yourself. However, in case you install to a "standard" location,
then running mandb will update the database for you. I believe this is the
behaviour we should have. I view it as the same as installing binaries in a
standard vs non-standard path - if the binaries are placed in a standard
location then they are found automatically, but if installed in a custom
location, then the user is responsible for ensuring all paths are correct.

/Bruce

Re: [PATCH] doc: build manpages as well as html output

[Jerin Jacob]

On Tue, Jun 6, 2023 at 2:49 PM Bruce Richardson
<bruce.richardson@intel.com> wrote:
>
> On Mon, Jun 05, 2023 at 10:50:48AM +0530, Jerin Jacob wrote:
> > On Thu, Jun 1, 2023 at 9:08 PM Bruce Richardson
> > <bruce.richardson@intel.com> wrote:
> > >
> > > Doxygen can produce manpage output as well as html output for the DPDK
> > > APIs. However, we need to do this as a separate task as the manpage
> > > output needs to be placed in a different location post-install to the
> > > html output (/usr/local/share/man vs /usr/local/share/doc/).
> > >
> > > Changes required are:
> > > * Add configurable options for manpage output and html output to the
> > >   doxygen config template. (Remove option for html output path as it's
> > >   always "html")
> > > * Modify API meson.build file to configure two separate doxygen config
> > >   files, for HTML and manpages respectively.
> > > * Change doxygen wrapper script to have separate output log files for
> > >   the manpage and HTML jobs, to avoid conflicts
> > > * Add "custom_targets" to meson.build file to build the HTML pages and
> > >   the manpages, with individual install locations for each.
> > > * Where supported by meson version, call "mandb" post-install to update
> > >   the man database to ensure the new manpages can be found.
> > >
> > > Signed-off-by: Bruce Richardson <bruce.richardson@intel.com>
> >
> > > +
> > > +mandb = find_program('mandb', required: false)
> > > +if mandb.found() and get_option('enable_docs') and meson.version().version_compare('>=0.55.0')
> > > +    meson.add_install_script(mandb)
> >
> > It does not look like just executing mandb it is adding these man
> > pages to database
> >
> > log:
> > Running custom install script '/usr/bin/mandb'
> > Purging old database entries in /home/jerin/.local/man...
> > Processing manual pages under /home/jerin/.local/man...
> > Checking for stray cats under /home/jerin/.local/man...
> > Processing manual pages under /home/jerin/.local/man/cat1...
> > Purging old database entries in /home/jerin/.local/share/man...
> > Processing manual pages under /home/jerin/.local/share/man...
> > Checking for stray cats under /home/jerin/.local/share/man...
> > Processing manual pages under /home/jerin/.local/share/man/cat1...
> > 0 man subdirectories contained newer manual pages.
> > 0 manual pages were added.
> > 0 stray cats were added.
> > 0 old database entries were purged.
> >
> > [main][dpdk.org] $ man  rte_flow_create
> > No manual entry for rte_flow_create
> >
> > # Following works by providing the path i.e man pages created properly
> > only db update is missing
> > man --manpath=/tmp/i/usr/local/share/man/ rte_flow_create
> >
> Yes, that is correct.
>
> If you install to a non-standard location, then yes you need to update
> manpath yourself. However, in case you install to a "standard" location,
> then running mandb will update the database for you. I believe this is the
> behaviour we should have. I view it as the same as installing binaries in a
> standard vs non-standard path - if the binaries are placed in a standard
> location then they are found automatically, but if installed in a custom
> location, then the user is responsible for ensuring all paths are correct.

OK. Then I think, we can move "meson.add_install_script(mandb)"
invocation under !DESTDIR not provided.
As if DESTDIR is provided then there is no use for mandb.

>
> /Bruce

Re: [PATCH] doc: build manpages as well as html output

[Bruce Richardson]

On Tue, Jun 06, 2023 at 03:16:27PM +0530, Jerin Jacob wrote:
> On Tue, Jun 6, 2023 at 2:49 PM Bruce Richardson
> <bruce.richardson@intel.com> wrote:
> >
> > On Mon, Jun 05, 2023 at 10:50:48AM +0530, Jerin Jacob wrote:
> > > On Thu, Jun 1, 2023 at 9:08 PM Bruce Richardson
> > > <bruce.richardson@intel.com> wrote:
> > > >
> > > > Doxygen can produce manpage output as well as html output for the DPDK
> > > > APIs. However, we need to do this as a separate task as the manpage
> > > > output needs to be placed in a different location post-install to the
> > > > html output (/usr/local/share/man vs /usr/local/share/doc/).
> > > >
> > > > Changes required are:
> > > > * Add configurable options for manpage output and html output to the
> > > >   doxygen config template. (Remove option for html output path as it's
> > > >   always "html")
> > > > * Modify API meson.build file to configure two separate doxygen config
> > > >   files, for HTML and manpages respectively.
> > > > * Change doxygen wrapper script to have separate output log files for
> > > >   the manpage and HTML jobs, to avoid conflicts
> > > > * Add "custom_targets" to meson.build file to build the HTML pages and
> > > >   the manpages, with individual install locations for each.
> > > > * Where supported by meson version, call "mandb" post-install to update
> > > >   the man database to ensure the new manpages can be found.
> > > >
> > > > Signed-off-by: Bruce Richardson <bruce.richardson@intel.com>
> > >
> > > > +
> > > > +mandb = find_program('mandb', required: false)
> > > > +if mandb.found() and get_option('enable_docs') and meson.version().version_compare('>=0.55.0')
> > > > +    meson.add_install_script(mandb)
> > >
> > > It does not look like just executing mandb it is adding these man
> > > pages to database
> > >
> > > log:
> > > Running custom install script '/usr/bin/mandb'
> > > Purging old database entries in /home/jerin/.local/man...
> > > Processing manual pages under /home/jerin/.local/man...
> > > Checking for stray cats under /home/jerin/.local/man...
> > > Processing manual pages under /home/jerin/.local/man/cat1...
> > > Purging old database entries in /home/jerin/.local/share/man...
> > > Processing manual pages under /home/jerin/.local/share/man...
> > > Checking for stray cats under /home/jerin/.local/share/man...
> > > Processing manual pages under /home/jerin/.local/share/man/cat1...
> > > 0 man subdirectories contained newer manual pages.
> > > 0 manual pages were added.
> > > 0 stray cats were added.
> > > 0 old database entries were purged.
> > >
> > > [main][dpdk.org] $ man  rte_flow_create
> > > No manual entry for rte_flow_create
> > >
> > > # Following works by providing the path i.e man pages created properly
> > > only db update is missing
> > > man --manpath=/tmp/i/usr/local/share/man/ rte_flow_create
> > >
> > Yes, that is correct.
> >
> > If you install to a non-standard location, then yes you need to update
> > manpath yourself. However, in case you install to a "standard" location,
> > then running mandb will update the database for you. I believe this is the
> > behaviour we should have. I view it as the same as installing binaries in a
> > standard vs non-standard path - if the binaries are placed in a standard
> > location then they are found automatically, but if installed in a custom
> > location, then the user is responsible for ensuring all paths are correct.
> 
> OK. Then I think, we can move "meson.add_install_script(mandb)"
> invocation under !DESTDIR not provided.
> As if DESTDIR is provided then there is no use for mandb.
> 
That would mean adding a new build script to run mandb, since:
* meson doesn't directly support getting environment variables
* DESTDIR may only be set at install time rather than at configuration
  time, so we'll always need to set up the build script

Furthermore, I'd point out that DESTDIR is not going to avoid unnecessary
runs of mandb, since using a special "prefix" setting would also cause the
manpages to not be found.

Overall, I'd view the complexity of avoiding the mandb call not to be
worth it, since calling mandb is largely harmless - bar the time taken to
run it. However, if you feel strongly it should be skipped when installing
with DESTDIR, I'll spin a v2 with the wrapper script added. Let me know
what you think?

/Bruce

Re: [PATCH] doc: build manpages as well as html output

[Jerin Jacob]

On Tue, Jun 6, 2023 at 3:49 PM Bruce Richardson
<bruce.richardson@intel.com> wrote:
>
> On Tue, Jun 06, 2023 at 03:16:27PM +0530, Jerin Jacob wrote:
> > On Tue, Jun 6, 2023 at 2:49 PM Bruce Richardson
> > <bruce.richardson@intel.com> wrote:
> > >
> > > On Mon, Jun 05, 2023 at 10:50:48AM +0530, Jerin Jacob wrote:
> > > > On Thu, Jun 1, 2023 at 9:08 PM Bruce Richardson
> > > > <bruce.richardson@intel.com> wrote:
> > > > >
> > > > > Doxygen can produce manpage output as well as html output for the DPDK
> > > > > APIs. However, we need to do this as a separate task as the manpage
> > > > > output needs to be placed in a different location post-install to the
> > > > > html output (/usr/local/share/man vs /usr/local/share/doc/).
> > > > >
> > > > > Changes required are:
> > > > > * Add configurable options for manpage output and html output to the
> > > > >   doxygen config template. (Remove option for html output path as it's
> > > > >   always "html")
> > > > > * Modify API meson.build file to configure two separate doxygen config
> > > > >   files, for HTML and manpages respectively.
> > > > > * Change doxygen wrapper script to have separate output log files for
> > > > >   the manpage and HTML jobs, to avoid conflicts
> > > > > * Add "custom_targets" to meson.build file to build the HTML pages and
> > > > >   the manpages, with individual install locations for each.
> > > > > * Where supported by meson version, call "mandb" post-install to update
> > > > >   the man database to ensure the new manpages can be found.
> > > > >
> > > > > Signed-off-by: Bruce Richardson <bruce.richardson@intel.com>
> > > >
> > > > > +
> > > > > +mandb = find_program('mandb', required: false)
> > > > > +if mandb.found() and get_option('enable_docs') and meson.version().version_compare('>=0.55.0')
> > > > > +    meson.add_install_script(mandb)
> > > >
> > > > It does not look like just executing mandb it is adding these man
> > > > pages to database
> > > >
> > > > log:
> > > > Running custom install script '/usr/bin/mandb'
> > > > Purging old database entries in /home/jerin/.local/man...
> > > > Processing manual pages under /home/jerin/.local/man...
> > > > Checking for stray cats under /home/jerin/.local/man...
> > > > Processing manual pages under /home/jerin/.local/man/cat1...
> > > > Purging old database entries in /home/jerin/.local/share/man...
> > > > Processing manual pages under /home/jerin/.local/share/man...
> > > > Checking for stray cats under /home/jerin/.local/share/man...
> > > > Processing manual pages under /home/jerin/.local/share/man/cat1...
> > > > 0 man subdirectories contained newer manual pages.
> > > > 0 manual pages were added.
> > > > 0 stray cats were added.
> > > > 0 old database entries were purged.
> > > >
> > > > [main][dpdk.org] $ man  rte_flow_create
> > > > No manual entry for rte_flow_create
> > > >
> > > > # Following works by providing the path i.e man pages created properly
> > > > only db update is missing
> > > > man --manpath=/tmp/i/usr/local/share/man/ rte_flow_create
> > > >
> > > Yes, that is correct.
> > >
> > > If you install to a non-standard location, then yes you need to update
> > > manpath yourself. However, in case you install to a "standard" location,
> > > then running mandb will update the database for you. I believe this is the
> > > behaviour we should have. I view it as the same as installing binaries in a
> > > standard vs non-standard path - if the binaries are placed in a standard
> > > location then they are found automatically, but if installed in a custom
> > > location, then the user is responsible for ensuring all paths are correct.
> >
> > OK. Then I think, we can move "meson.add_install_script(mandb)"
> > invocation under !DESTDIR not provided.
> > As if DESTDIR is provided then there is no use for mandb.
> >
> That would mean adding a new build script to run mandb, since:
> * meson doesn't directly support getting environment variables
> * DESTDIR may only be set at install time rather than at configuration
>   time, so we'll always need to set up the build script
>
> Furthermore, I'd point out that DESTDIR is not going to avoid unnecessary
> runs of mandb, since using a special "prefix" setting would also cause the
> manpages to not be found.
>
> Overall, I'd view the complexity of avoiding the mandb call not to be
> worth it, since calling mandb is largely harmless - bar the time taken to
> run it. However, if you feel strongly it should be skipped when installing
> with DESTDIR

No strong option. OK to skip DESTDIR check.

>, I'll spin a v2 with the wrapper script added. Let me know
> what you think?

I think, it is good to update doc/guides/contributing/documentation.rst
on how to use the man page(i.e use  with --manpage)


>
> /Bruce

Re: [PATCH] doc: build manpages as well as html output

[Bruce Richardson]

On Tue, Jun 06, 2023 at 04:19:27PM +0530, Jerin Jacob wrote:
> On Tue, Jun 6, 2023 at 3:49 PM Bruce Richardson
> <bruce.richardson@intel.com> wrote:
> >
> > On Tue, Jun 06, 2023 at 03:16:27PM +0530, Jerin Jacob wrote:
> > > On Tue, Jun 6, 2023 at 2:49 PM Bruce Richardson
> > > <bruce.richardson@intel.com> wrote:
> > > >
> > > > On Mon, Jun 05, 2023 at 10:50:48AM +0530, Jerin Jacob wrote:
> > > > > On Thu, Jun 1, 2023 at 9:08 PM Bruce Richardson
> > > > > <bruce.richardson@intel.com> wrote:
> > > > > >
> > > > > > Doxygen can produce manpage output as well as html output for the DPDK
> > > > > > APIs. However, we need to do this as a separate task as the manpage
> > > > > > output needs to be placed in a different location post-install to the
> > > > > > html output (/usr/local/share/man vs /usr/local/share/doc/).
> > > > > >
> > > > > > Changes required are:
> > > > > > * Add configurable options for manpage output and html output to the
> > > > > >   doxygen config template. (Remove option for html output path as it's
> > > > > >   always "html")
> > > > > > * Modify API meson.build file to configure two separate doxygen config
> > > > > >   files, for HTML and manpages respectively.
> > > > > > * Change doxygen wrapper script to have separate output log files for
> > > > > >   the manpage and HTML jobs, to avoid conflicts
> > > > > > * Add "custom_targets" to meson.build file to build the HTML pages and
> > > > > >   the manpages, with individual install locations for each.
> > > > > > * Where supported by meson version, call "mandb" post-install to update
> > > > > >   the man database to ensure the new manpages can be found.
> > > > > >
> > > > > > Signed-off-by: Bruce Richardson <bruce.richardson@intel.com>
> > > > >
> > > > > > +
> > > > > > +mandb = find_program('mandb', required: false)
> > > > > > +if mandb.found() and get_option('enable_docs') and meson.version().version_compare('>=0.55.0')
> > > > > > +    meson.add_install_script(mandb)
> > > > >
> > > > > It does not look like just executing mandb it is adding these man
> > > > > pages to database
> > > > >
> > > > > log:
> > > > > Running custom install script '/usr/bin/mandb'
> > > > > Purging old database entries in /home/jerin/.local/man...
> > > > > Processing manual pages under /home/jerin/.local/man...
> > > > > Checking for stray cats under /home/jerin/.local/man...
> > > > > Processing manual pages under /home/jerin/.local/man/cat1...
> > > > > Purging old database entries in /home/jerin/.local/share/man...
> > > > > Processing manual pages under /home/jerin/.local/share/man...
> > > > > Checking for stray cats under /home/jerin/.local/share/man...
> > > > > Processing manual pages under /home/jerin/.local/share/man/cat1...
> > > > > 0 man subdirectories contained newer manual pages.
> > > > > 0 manual pages were added.
> > > > > 0 stray cats were added.
> > > > > 0 old database entries were purged.
> > > > >
> > > > > [main][dpdk.org] $ man  rte_flow_create
> > > > > No manual entry for rte_flow_create
> > > > >
> > > > > # Following works by providing the path i.e man pages created properly
> > > > > only db update is missing
> > > > > man --manpath=/tmp/i/usr/local/share/man/ rte_flow_create
> > > > >
> > > > Yes, that is correct.
> > > >
> > > > If you install to a non-standard location, then yes you need to update
> > > > manpath yourself. However, in case you install to a "standard" location,
> > > > then running mandb will update the database for you. I believe this is the
> > > > behaviour we should have. I view it as the same as installing binaries in a
> > > > standard vs non-standard path - if the binaries are placed in a standard
> > > > location then they are found automatically, but if installed in a custom
> > > > location, then the user is responsible for ensuring all paths are correct.
> > >
> > > OK. Then I think, we can move "meson.add_install_script(mandb)"
> > > invocation under !DESTDIR not provided.
> > > As if DESTDIR is provided then there is no use for mandb.
> > >
> > That would mean adding a new build script to run mandb, since:
> > * meson doesn't directly support getting environment variables
> > * DESTDIR may only be set at install time rather than at configuration
> >   time, so we'll always need to set up the build script
> >
> > Furthermore, I'd point out that DESTDIR is not going to avoid unnecessary
> > runs of mandb, since using a special "prefix" setting would also cause the
> > manpages to not be found.
> >
> > Overall, I'd view the complexity of avoiding the mandb call not to be
> > worth it, since calling mandb is largely harmless - bar the time taken to
> > run it. However, if you feel strongly it should be skipped when installing
> > with DESTDIR
> 
> No strong option. OK to skip DESTDIR check.
> 
> >, I'll spin a v2 with the wrapper script added. Let me know
> > what you think?
> 
> I think, it is good to update doc/guides/contributing/documentation.rst
> on how to use the man page(i.e use  with --manpage)
> 

Ok, will spin a v2.

[PATCH v2] doc: build manpages as well as html output

[Bruce Richardson]

Doxygen can produce manpage output as well as html output for the DPDK
APIs. However, we need to do this as a separate task as the manpage
output needs to be placed in a different location post-install to the
html output (/usr/local/share/man vs /usr/local/share/doc/).

Changes required are:
* Add configurable options for manpage output and html output to the
  doxygen config template. (Remove option for html output path as it's
  always "html")
* Modify API meson.build file to configure two separate doxygen config
  files, for HTML and manpages respectively.
* Change doxygen wrapper script to have separate output log files for
  the manpage and HTML jobs, to avoid conflicts
* Add "custom_targets" to meson.build file to build the HTML pages and
  the manpages, with individual install locations for each.
* Where supported by meson version, call "mandb" post-install to update
  the man database to ensure the new manpages can be found.

Signed-off-by: Bruce Richardson <bruce.richardson@intel.com>

---
V2: add doc update about building or using the manpages
---
 doc/api/doxy-api.conf.in                  |  7 +--
 doc/api/generate_doxygen.py               |  2 +-
 doc/api/meson.build                       | 52 +++++++++++++++++++----
 doc/guides/contributing/documentation.rst | 12 +++++-
 4 files changed, 60 insertions(+), 13 deletions(-)

diff --git a/doc/api/doxy-api.conf.in b/doc/api/doxy-api.conf.in
index 967aa11701..6e5be9b469 100644
--- a/doc/api/doxy-api.conf.in
+++ b/doc/api/doxy-api.conf.in
@@ -125,10 +125,11 @@ EXAMPLE_RECURSIVE       = YES
 
 OUTPUT_DIRECTORY        = @OUTPUT@
 STRIP_FROM_PATH         = @STRIP_FROM_PATH@
-GENERATE_HTML           = YES
-HTML_OUTPUT             = @HTML_OUTPUT@
+GENERATE_HTML           = @GENERATE_HTML@
+HTML_OUTPUT             = html
 GENERATE_LATEX          = NO
-GENERATE_MAN            = NO
+GENERATE_MAN            = @GENERATE_MAN@
+MAN_LINKS               = YES
 
 HAVE_DOT                = NO
 
diff --git a/doc/api/generate_doxygen.py b/doc/api/generate_doxygen.py
index d3a22869f6..c704f13018 100755
--- a/doc/api/generate_doxygen.py
+++ b/doc/api/generate_doxygen.py
@@ -7,7 +7,7 @@
 
 pattern = re.compile('^Preprocessing (.*)...$')
 out_dir, *doxygen_command = sys.argv[1:]
-out_file = os.path.join(os.path.dirname(out_dir), 'doxygen.out')
+out_file = os.path.join(out_dir + '.out')
 dep_file = f'{out_dir}.d'
 with open(out_file, 'w') as out:
     subprocess.run(doxygen_command, check=True, stdout=out)
diff --git a/doc/api/meson.build b/doc/api/meson.build
index 2876a78a7e..104b91517b 100644
--- a/doc/api/meson.build
+++ b/doc/api/meson.build
@@ -29,11 +29,11 @@ example = custom_target('examples.dox',
         install_dir: htmldir,
         build_by_default: get_option('enable_docs'))
 
+#set up common doxygen configuration
 cdata = configuration_data()
 cdata.set('VERSION', meson.project_version())
 cdata.set('API_EXAMPLES', join_paths(dpdk_build_root, 'doc', 'api', 'examples.dox'))
 cdata.set('OUTPUT', join_paths(dpdk_build_root, 'doc', 'api'))
-cdata.set('HTML_OUTPUT', 'html')
 cdata.set('TOPDIR', dpdk_source_root)
 cdata.set('STRIP_FROM_PATH', ' '.join([dpdk_source_root, join_paths(dpdk_build_root, 'doc', 'api')]))
 cdata.set('WARN_AS_ERROR', 'NO')
@@ -41,14 +41,31 @@ if get_option('werror')
     cdata.set('WARN_AS_ERROR', 'YES')
 endif
 
-doxy_conf = configure_file(input: 'doxy-api.conf.in',
-        output: 'doxy-api.conf',
-        configuration: cdata)
+# configure HTML doxygen run
+html_cdata = configuration_data()
+html_cdata.merge_from(cdata)
+html_cdata.set('GENERATE_HTML', 'YES')
+html_cdata.set('GENERATE_MAN', 'NO')
 
-doxy_build = custom_target('doxygen',
+doxy_html_conf = configure_file(input: 'doxy-api.conf.in',
+        output: 'doxy-api-html.conf',
+        configuration: html_cdata)
+
+# configure manpage doxygen run
+man_cdata = configuration_data()
+man_cdata.merge_from(cdata)
+man_cdata.set('GENERATE_HTML', 'NO')
+man_cdata.set('GENERATE_MAN', 'YES')
+
+doxy_man_conf = configure_file(input: 'doxy-api.conf.in',
+        output: 'doxy-api-man.conf',
+        configuration: man_cdata)
+
+# do doxygen runs
+doxy_html_build = custom_target('doxygen-html',
         depends: example,
         depend_files: 'doxy-api-index.md',
-        input: doxy_conf,
+        input: doxy_html_conf,
         output: 'html',
         depfile: 'html.d',
         command: [generate_doxygen, '@OUTPUT@', doxygen, '@INPUT@'],
@@ -56,5 +73,24 @@ doxy_build = custom_target('doxygen',
         install_dir: htmldir,
         build_by_default: get_option('enable_docs'))
 
-doc_targets += doxy_build
-doc_target_names += 'Doxygen_API'
+doc_targets += doxy_html_build
+doc_target_names += 'Doxygen_API(HTML)'
+
+doxy_man_build = custom_target('doxygen-man',
+        depends: example,
+        depend_files: 'doxy-api-index.md',
+        input: doxy_man_conf,
+        output: 'man',
+        depfile: 'man.d',
+        command: [generate_doxygen, '@OUTPUT@', doxygen, '@INPUT@'],
+        install: get_option('enable_docs'),
+        install_dir: get_option('datadir'),
+        build_by_default: get_option('enable_docs'))
+
+doc_targets += doxy_man_build
+doc_target_names += 'Doxygen_API(Manpage)'
+
+mandb = find_program('mandb', required: false)
+if mandb.found() and get_option('enable_docs') and meson.version().version_compare('>=0.55.0')
+    meson.add_install_script(mandb)
+endif
diff --git a/doc/guides/contributing/documentation.rst b/doc/guides/contributing/documentation.rst
index 7fcbb7fc43..e0b672ffbb 100644
--- a/doc/guides/contributing/documentation.rst
+++ b/doc/guides/contributing/documentation.rst
@@ -182,7 +182,17 @@ To build the documentation::
 
 See :doc:`../linux_gsg/build_dpdk` for more detail on compiling DPDK with meson.
 
-The output is generated in the directories ``build/doc/html/{api,guides}``.
+The output is generated in the directory ``build/doc/``, with:
+
+* HTML versions of the guide docs, e.g. Getting Started Guides, Programmers Guide, in ``build/doc/guides/html``
+* HTML version of the API documentation in ``build/doc/api/html``
+* Man-page version of the API documentation in ``build/doc/api/man``.
+  If not installing DPDK system-wise, these pages can be accessed by adding this directory to the ``MANPATH`` environment variable.
+  For example:
+
+.. code-block:: console
+
+    export MANPATH=:/path/to/build/doc/api/man
 
 .. Note::
 
-- 
2.39.2

Re: [PATCH v2] doc: build manpages as well as html output

[David Marchand]

On Tue, Jun 6, 2023 at 3:13 PM Bruce Richardson
<bruce.richardson@intel.com> wrote:
>
> Doxygen can produce manpage output as well as html output for the DPDK
> APIs. However, we need to do this as a separate task as the manpage
> output needs to be placed in a different location post-install to the
> html output (/usr/local/share/man vs /usr/local/share/doc/).
>
> Changes required are:
> * Add configurable options for manpage output and html output to the
>   doxygen config template. (Remove option for html output path as it's
>   always "html")
> * Modify API meson.build file to configure two separate doxygen config
>   files, for HTML and manpages respectively.
> * Change doxygen wrapper script to have separate output log files for
>   the manpage and HTML jobs, to avoid conflicts
> * Add "custom_targets" to meson.build file to build the HTML pages and
>   the manpages, with individual install locations for each.
> * Where supported by meson version, call "mandb" post-install to update
>   the man database to ensure the new manpages can be found.
>
> Signed-off-by: Bruce Richardson <bruce.richardson@intel.com>

Nice.. we should have done this sooner :-).

Reviewed-by: David Marchand <david.marchand@redhat.com>


There may be some polishing to do later.
Looking at the result for rte_eal_init, I see that the generated
manual starts with a reference to the rte_eal.h header with a path
relative to its location in the DPDK tree.
$ MANPATH=build-gcc/install/share/man man rte_eal_init | head -5
lib/eal/include/rte_eal.h(3)
                                           DPDK

lib/eal/include/rte_eal.h(3)

NAME
       lib/eal/include/rte_eal.h

At least, it is possible to ask for this header man with "man
rte_eal.h", but it is a bit confusing.
Is there something we can do on this side?


-- 
David Marchand

Re: [PATCH v2] doc: build manpages as well as html output

[Bruce Richardson]

On Tue, Jul 04, 2023 at 10:21:22AM +0200, David Marchand wrote:
> On Tue, Jun 6, 2023 at 3:13 PM Bruce Richardson
> <bruce.richardson@intel.com> wrote:
> >
> > Doxygen can produce manpage output as well as html output for the DPDK
> > APIs. However, we need to do this as a separate task as the manpage
> > output needs to be placed in a different location post-install to the
> > html output (/usr/local/share/man vs /usr/local/share/doc/).
> >
> > Changes required are:
> > * Add configurable options for manpage output and html output to the
> >   doxygen config template. (Remove option for html output path as it's
> >   always "html")
> > * Modify API meson.build file to configure two separate doxygen config
> >   files, for HTML and manpages respectively.
> > * Change doxygen wrapper script to have separate output log files for
> >   the manpage and HTML jobs, to avoid conflicts
> > * Add "custom_targets" to meson.build file to build the HTML pages and
> >   the manpages, with individual install locations for each.
> > * Where supported by meson version, call "mandb" post-install to update
> >   the man database to ensure the new manpages can be found.
> >
> > Signed-off-by: Bruce Richardson <bruce.richardson@intel.com>
> 
> Nice.. we should have done this sooner :-).
> 
> Reviewed-by: David Marchand <david.marchand@redhat.com>
> 
> 
> There may be some polishing to do later.
> Looking at the result for rte_eal_init, I see that the generated
> manual starts with a reference to the rte_eal.h header with a path
> relative to its location in the DPDK tree.
> $ MANPATH=build-gcc/install/share/man man rte_eal_init | head -5
> lib/eal/include/rte_eal.h(3)
>                                            DPDK
> 
> lib/eal/include/rte_eal.h(3)
> 
> NAME
>        lib/eal/include/rte_eal.h
> 
> At least, it is possible to ask for this header man with "man
> rte_eal.h", but it is a bit confusing.
> Is there something we can do on this side?
> 

Not sure, not really familiar with how doxygen works generating manpages
and the options supported, etc. etc. Mainly I just looked at the
build-system side to support this, since I really missed having manpages
for DPDK functions to quickly check parameter order.

/Bruce

Re: [PATCH v2] doc: build manpages as well as html output

[David Marchand]

On Mon, Jul 17, 2023 at 1:09 PM Bruce Richardson
<bruce.richardson@intel.com> wrote:
> > There may be some polishing to do later.
> > Looking at the result for rte_eal_init, I see that the generated
> > manual starts with a reference to the rte_eal.h header with a path
> > relative to its location in the DPDK tree.
> > $ MANPATH=build-gcc/install/share/man man rte_eal_init | head -5
> > lib/eal/include/rte_eal.h(3)
> >                                            DPDK
> >
> > lib/eal/include/rte_eal.h(3)
> >
> > NAME
> >        lib/eal/include/rte_eal.h
> >
> > At least, it is possible to ask for this header man with "man
> > rte_eal.h", but it is a bit confusing.
> > Is there something we can do on this side?
> >
>
> Not sure, not really familiar with how doxygen works generating manpages
> and the options supported, etc. etc. Mainly I just looked at the
> build-system side to support this, since I really missed having manpages
> for DPDK functions to quickly check parameter order.

doxygen is invoked on the sources tree, so this is probably the reason
why such path is in the generated manual.
Maybe Thomas has an idea how we could handle this.

In any case, I am fine with this patch as it is now.


-- 
David Marchand

Re: [PATCH v2] doc: build manpages as well as html output

[Bruce Richardson]

On Thu, Aug 03, 2023 at 11:18:09AM +0200, David Marchand wrote:
> On Mon, Jul 17, 2023 at 1:09 PM Bruce Richardson
> <bruce.richardson@intel.com> wrote:
> > > There may be some polishing to do later.
> > > Looking at the result for rte_eal_init, I see that the generated
> > > manual starts with a reference to the rte_eal.h header with a path
> > > relative to its location in the DPDK tree.
> > > $ MANPATH=build-gcc/install/share/man man rte_eal_init | head -5
> > > lib/eal/include/rte_eal.h(3)
> > >                                            DPDK
> > >
> > > lib/eal/include/rte_eal.h(3)
> > >
> > > NAME
> > >        lib/eal/include/rte_eal.h
> > >
> > > At least, it is possible to ask for this header man with "man
> > > rte_eal.h", but it is a bit confusing.
> > > Is there something we can do on this side?
> > >
> >
> > Not sure, not really familiar with how doxygen works generating manpages
> > and the options supported, etc. etc. Mainly I just looked at the
> > build-system side to support this, since I really missed having manpages
> > for DPDK functions to quickly check parameter order.
> 
> doxygen is invoked on the sources tree, so this is probably the reason
> why such path is in the generated manual.
> Maybe Thomas has an idea how we could handle this.
> 
> In any case, I am fine with this patch as it is now.
>
I think I've found a fix for this.
doxygen setting "FULL_PATH_NAMES" defaults to "YES", which is what we want
for the HTML pages. However, setting it to "NO", I believe is correct for
generating the manpages.

Patch V3 on its way.

/Bruce 

[PATCH v3] doc: build manpages as well as html output

[Bruce Richardson]

Doxygen can produce manpage output as well as html output for the DPDK
APIs. However, we need to do this as a separate task as the manpage
output needs to be placed in a different location post-install to the
html output (/usr/local/share/man vs /usr/local/share/doc/).

Changes required are:
* Add configurable options for manpage output and html output to the
  doxygen config template. (Remove option for html output path as it's
  always "html")
* Modify API meson.build file to configure two separate doxygen config
  files, for HTML and manpages respectively.
* Change doxygen wrapper script to have separate output log files for
  the manpage and HTML jobs, to avoid conflicts
* Add "custom_targets" to meson.build file to build the HTML pages and
  the manpages, with individual install locations for each.
* Where supported by meson version, call "mandb" post-install to update
  the man database to ensure the new manpages can be found.

Signed-off-by: Bruce Richardson <bruce.richardson@intel.com>
Reviewed-by: David Marchand <david.marchand@redhat.com>

---

V3: don't use full file paths when generating manpages

V2: add doc update about building or using the manpages
---
 doc/api/doxy-api.conf.in                  |  8 ++--
 doc/api/generate_doxygen.py               |  2 +-
 doc/api/meson.build                       | 54 +++++++++++++++++++----
 doc/guides/contributing/documentation.rst | 12 ++++-
 4 files changed, 63 insertions(+), 13 deletions(-)

diff --git a/doc/api/doxy-api.conf.in b/doc/api/doxy-api.conf.in
index 1a4210b948..c77bdb328b 100644
--- a/doc/api/doxy-api.conf.in
+++ b/doc/api/doxy-api.conf.in
@@ -124,11 +124,13 @@ EXAMPLE_PATTERNS        = *.c
 EXAMPLE_RECURSIVE       = YES
 
 OUTPUT_DIRECTORY        = @OUTPUT@
+FULL_PATH_NAMES         = @FULL_PATH_NAMES@
 STRIP_FROM_PATH         = @STRIP_FROM_PATH@
-GENERATE_HTML           = YES
-HTML_OUTPUT             = @HTML_OUTPUT@
+GENERATE_HTML           = @GENERATE_HTML@
+HTML_OUTPUT             = html
 GENERATE_LATEX          = NO
-GENERATE_MAN            = NO
+GENERATE_MAN            = @GENERATE_MAN@
+MAN_LINKS               = YES
 
 HAVE_DOT                = NO
 
diff --git a/doc/api/generate_doxygen.py b/doc/api/generate_doxygen.py
index d3a22869f6..c704f13018 100755
--- a/doc/api/generate_doxygen.py
+++ b/doc/api/generate_doxygen.py
@@ -7,7 +7,7 @@
 
 pattern = re.compile('^Preprocessing (.*)...$')
 out_dir, *doxygen_command = sys.argv[1:]
-out_file = os.path.join(os.path.dirname(out_dir), 'doxygen.out')
+out_file = os.path.join(out_dir + '.out')
 dep_file = f'{out_dir}.d'
 with open(out_file, 'w') as out:
     subprocess.run(doxygen_command, check=True, stdout=out)
diff --git a/doc/api/meson.build b/doc/api/meson.build
index 2876a78a7e..9f83b34bb3 100644
--- a/doc/api/meson.build
+++ b/doc/api/meson.build
@@ -29,11 +29,11 @@ example = custom_target('examples.dox',
         install_dir: htmldir,
         build_by_default: get_option('enable_docs'))
 
+#set up common doxygen configuration
 cdata = configuration_data()
 cdata.set('VERSION', meson.project_version())
 cdata.set('API_EXAMPLES', join_paths(dpdk_build_root, 'doc', 'api', 'examples.dox'))
 cdata.set('OUTPUT', join_paths(dpdk_build_root, 'doc', 'api'))
-cdata.set('HTML_OUTPUT', 'html')
 cdata.set('TOPDIR', dpdk_source_root)
 cdata.set('STRIP_FROM_PATH', ' '.join([dpdk_source_root, join_paths(dpdk_build_root, 'doc', 'api')]))
 cdata.set('WARN_AS_ERROR', 'NO')
@@ -41,14 +41,33 @@ if get_option('werror')
     cdata.set('WARN_AS_ERROR', 'YES')
 endif
 
-doxy_conf = configure_file(input: 'doxy-api.conf.in',
-        output: 'doxy-api.conf',
-        configuration: cdata)
+# configure HTML doxygen run
+html_cdata = configuration_data()
+html_cdata.merge_from(cdata)
+html_cdata.set('GENERATE_HTML', 'YES')
+html_cdata.set('GENERATE_MAN', 'NO')
+html_cdata.set('FULL_PATH_NAMES', 'YES')
 
-doxy_build = custom_target('doxygen',
+doxy_html_conf = configure_file(input: 'doxy-api.conf.in',
+        output: 'doxy-api-html.conf',
+        configuration: html_cdata)
+
+# configure manpage doxygen run
+man_cdata = configuration_data()
+man_cdata.merge_from(cdata)
+man_cdata.set('GENERATE_HTML', 'NO')
+man_cdata.set('GENERATE_MAN', 'YES')
+man_cdata.set('FULL_PATH_NAMES', 'NO')
+
+doxy_man_conf = configure_file(input: 'doxy-api.conf.in',
+        output: 'doxy-api-man.conf',
+        configuration: man_cdata)
+
+# do doxygen runs
+doxy_html_build = custom_target('doxygen-html',
         depends: example,
         depend_files: 'doxy-api-index.md',
-        input: doxy_conf,
+        input: doxy_html_conf,
         output: 'html',
         depfile: 'html.d',
         command: [generate_doxygen, '@OUTPUT@', doxygen, '@INPUT@'],
@@ -56,5 +75,24 @@ doxy_build = custom_target('doxygen',
         install_dir: htmldir,
         build_by_default: get_option('enable_docs'))
 
-doc_targets += doxy_build
-doc_target_names += 'Doxygen_API'
+doc_targets += doxy_html_build
+doc_target_names += 'Doxygen_API(HTML)'
+
+doxy_man_build = custom_target('doxygen-man',
+        depends: example,
+        depend_files: 'doxy-api-index.md',
+        input: doxy_man_conf,
+        output: 'man',
+        depfile: 'man.d',
+        command: [generate_doxygen, '@OUTPUT@', doxygen, '@INPUT@'],
+        install: get_option('enable_docs'),
+        install_dir: get_option('datadir'),
+        build_by_default: get_option('enable_docs'))
+
+doc_targets += doxy_man_build
+doc_target_names += 'Doxygen_API(Manpage)'
+
+mandb = find_program('mandb', required: false)
+if mandb.found() and get_option('enable_docs') and meson.version().version_compare('>=0.55.0')
+    meson.add_install_script(mandb)
+endif
diff --git a/doc/guides/contributing/documentation.rst b/doc/guides/contributing/documentation.rst
index 7fcbb7fc43..e0b672ffbb 100644
--- a/doc/guides/contributing/documentation.rst
+++ b/doc/guides/contributing/documentation.rst
@@ -182,7 +182,17 @@ To build the documentation::
 
 See :doc:`../linux_gsg/build_dpdk` for more detail on compiling DPDK with meson.
 
-The output is generated in the directories ``build/doc/html/{api,guides}``.
+The output is generated in the directory ``build/doc/``, with:
+
+* HTML versions of the guide docs, e.g. Getting Started Guides, Programmers Guide, in ``build/doc/guides/html``
+* HTML version of the API documentation in ``build/doc/api/html``
+* Man-page version of the API documentation in ``build/doc/api/man``.
+  If not installing DPDK system-wise, these pages can be accessed by adding this directory to the ``MANPATH`` environment variable.
+  For example:
+
+.. code-block:: console
+
+    export MANPATH=:/path/to/build/doc/api/man
 
 .. Note::
 
-- 
2.39.2

Re: [PATCH v3] doc: build manpages as well as html output

[David Marchand]

On Thu, Aug 3, 2023 at 6:44 PM Bruce Richardson
<bruce.richardson@intel.com> wrote:
>
> Doxygen can produce manpage output as well as html output for the DPDK
> APIs. However, we need to do this as a separate task as the manpage
> output needs to be placed in a different location post-install to the
> html output (/usr/local/share/man vs /usr/local/share/doc/).
>
> Changes required are:
> * Add configurable options for manpage output and html output to the
>   doxygen config template. (Remove option for html output path as it's
>   always "html")
> * Modify API meson.build file to configure two separate doxygen config
>   files, for HTML and manpages respectively.
> * Change doxygen wrapper script to have separate output log files for
>   the manpage and HTML jobs, to avoid conflicts
> * Add "custom_targets" to meson.build file to build the HTML pages and
>   the manpages, with individual install locations for each.
> * Where supported by meson version, call "mandb" post-install to update
>   the man database to ensure the new manpages can be found.
>
> Signed-off-by: Bruce Richardson <bruce.richardson@intel.com>
> Reviewed-by: David Marchand <david.marchand@redhat.com>
>
> ---
>
> V3: don't use full file paths when generating manpages

Nice!
Thank you Bruce.


-- 
David Marchand

Re: [PATCH v3] doc: build manpages as well as html output

[Thomas Monjalon]

3/08/2023, Bruce Richardson:
> +#set up common doxygen configuration

A space is missing here

> +man_cdata.set('FULL_PATH_NAMES', 'NO')

Why it has to be disabled? Maybe add a comment?

> +    meson.add_install_script(mandb)

When is it executed exactly?
Will it update the database in case we install in a staging directory,
when preparing a package for later deploying on another machine?

> +.. code-block:: console
> +
> +    export MANPATH=:/path/to/build/doc/api/man

Styling consideration: I think we should indent with 3 spaces
so it is aligned with "code-block".

Re: [PATCH v3] doc: build manpages as well as html output

[Bruce Richardson]

On Tue, Aug 29, 2023 at 11:28:01AM +0200, Thomas Monjalon wrote:
> 3/08/2023, Bruce Richardson:
> > +#set up common doxygen configuration
> 
> A space is missing here
> 
Ack

> > +man_cdata.set('FULL_PATH_NAMES', 'NO')
> 
> Why it has to be disabled? Maybe add a comment?
> 

Sure, will add comment. It's so that the man pages don't include
full paths to the headers, but are just e.g. "rte_eal.h".

> > +    meson.add_install_script(mandb)
> 
> When is it executed exactly?
> Will it update the database in case we install in a staging directory,
> when preparing a package for later deploying on another machine?

Yes, it will. Unfortunately, I can't find any way to just call mandb if we
are installing in a system manpage location on the local machine. Therefore,
I had two options:
1. don't update the manpage database. In this case, the user won't be able to
actually get the newly install manpages
2. always update the local manpage database. In this case, the user
installing the docs will find them, but anyone installing to staging will
experience a slight delay while their local mandb is updated.

I went for #2 on the basis that the delay in the staging case is pretty
harmless, while not actually finding the manpages is more serious.

However, I'm open to other suggestions on how to work this?

> 
> > +.. code-block:: console
> > +
> > +    export MANPATH=:/path/to/build/doc/api/man
> 
> Styling consideration: I think we should indent with 3 spaces
> so it is aligned with "code-block".
> 
Ack.

Re: [PATCH v3] doc: build manpages as well as html output

[Thomas Monjalon]

29/08/2023 12:10, Bruce Richardson:
> On Tue, Aug 29, 2023 at 11:28:01AM +0200, Thomas Monjalon wrote:
> > 3/08/2023, Bruce Richardson:
> > > +    meson.add_install_script(mandb)
> > 
> > When is it executed exactly?
> > Will it update the database in case we install in a staging directory,
> > when preparing a package for later deploying on another machine?
> 
> Yes, it will. Unfortunately, I can't find any way to just call mandb if we
> are installing in a system manpage location on the local machine. Therefore,
> I had two options:
> 1. don't update the manpage database. In this case, the user won't be able to
> actually get the newly install manpages

The user can update the manpage database himself.
And if installing from a package, it should have been done automatically.

> 2. always update the local manpage database. In this case, the user
> installing the docs will find them, but anyone installing to staging will
> experience a slight delay while their local mandb is updated.
> 
> I went for #2 on the basis that the delay in the staging case is pretty
> harmless, while not actually finding the manpages is more serious.
> 
> However, I'm open to other suggestions on how to work this?

My concern is polluting the machine of the packager.
What happens when the staging directory is removed?
Is it an error when opening a manpage later?

Re: [PATCH v3] doc: build manpages as well as html output

[Bruce Richardson]

On Wed, Aug 30, 2023 at 11:47:00AM +0200, Thomas Monjalon wrote:
> 29/08/2023 12:10, Bruce Richardson:
> > On Tue, Aug 29, 2023 at 11:28:01AM +0200, Thomas Monjalon wrote:
> > > 3/08/2023, Bruce Richardson:
> > > > +    meson.add_install_script(mandb)
> > > 
> > > When is it executed exactly?  Will it update the database in case we
> > > install in a staging directory, when preparing a package for later
> > > deploying on another machine?
> > 
> > Yes, it will. Unfortunately, I can't find any way to just call mandb if
> > we are installing in a system manpage location on the local machine.
> > Therefore, I had two options: 1. don't update the manpage database. In
> > this case, the user won't be able to actually get the newly install
> > manpages
> 
> The user can update the manpage database himself.  And if installing from
> a package, it should have been done automatically.
> 
> > 2. always update the local manpage database. In this case, the user
> > installing the docs will find them, but anyone installing to staging
> > will experience a slight delay while their local mandb is updated.
> > 
> > I went for #2 on the basis that the delay in the staging case is pretty
> > harmless, while not actually finding the manpages is more serious.
> > 
> > However, I'm open to other suggestions on how to work this?
> 
> My concern is polluting the machine of the packager.  What happens when
> the staging directory is removed?  Is it an error when opening a manpage
> later?
> 
Running mandb doesn't add the manpages from the staging directory to the
database, it simply triggers a scan of the registered manpage directories
on the system, adding new pages found there. It's the man equivalent of
doing an "ldconfig".

/Bruce

Re: [PATCH v3] doc: build manpages as well as html output

[Thomas Monjalon]

30/08/2023 12:20, Bruce Richardson:
> On Wed, Aug 30, 2023 at 11:47:00AM +0200, Thomas Monjalon wrote:
> > 29/08/2023 12:10, Bruce Richardson:
> > > On Tue, Aug 29, 2023 at 11:28:01AM +0200, Thomas Monjalon wrote:
> > > > 3/08/2023, Bruce Richardson:
> > > > > +    meson.add_install_script(mandb)
> > > > 
> > > > When is it executed exactly?  Will it update the database in case we
> > > > install in a staging directory, when preparing a package for later
> > > > deploying on another machine?
> > > 
> > > Yes, it will. Unfortunately, I can't find any way to just call mandb if
> > > we are installing in a system manpage location on the local machine.
> > > Therefore, I had two options: 1. don't update the manpage database. In
> > > this case, the user won't be able to actually get the newly install
> > > manpages
> > 
> > The user can update the manpage database himself.  And if installing from
> > a package, it should have been done automatically.
> > 
> > > 2. always update the local manpage database. In this case, the user
> > > installing the docs will find them, but anyone installing to staging
> > > will experience a slight delay while their local mandb is updated.
> > > 
> > > I went for #2 on the basis that the delay in the staging case is pretty
> > > harmless, while not actually finding the manpages is more serious.
> > > 
> > > However, I'm open to other suggestions on how to work this?
> > 
> > My concern is polluting the machine of the packager.  What happens when
> > the staging directory is removed?  Is it an error when opening a manpage
> > later?
> > 
> Running mandb doesn't add the manpages from the staging directory to the
> database, it simply triggers a scan of the registered manpage directories
> on the system, adding new pages found there. It's the man equivalent of
> doing an "ldconfig".

Ah OK
So there is no more concern, thanks.

[PATCH v4] doc: build manpages as well as html output

[Bruce Richardson]

Doxygen can produce manpage output as well as html output for the DPDK
APIs. However, we need to do this as a separate task as the manpage
output needs to be placed in a different location post-install to the
html output (/usr/local/share/man vs /usr/local/share/doc/).

Changes required are:
* Add configurable options for manpage output and html output to the
  doxygen config template. (Remove option for html output path as it's
  always "html")
* Modify API meson.build file to configure two separate doxygen config
  files, for HTML and manpages respectively.
* Change doxygen wrapper script to have separate output log files for
  the manpage and HTML jobs, to avoid conflicts
* Add "custom_targets" to meson.build file to build the HTML pages and
  the manpages, with individual install locations for each.
* Where supported by meson version, call "mandb" post-install to update
  the man database to ensure the new manpages can be found. If the
  manpages are not installed in system location i.e. one not in MANPATH,
  then this update will have no effect, as only system locations are
  scanned.

Signed-off-by: Bruce Richardson <bruce.richardson@intel.com>
Reviewed-by: David Marchand <david.marchand@redhat.com>

---

V4: fix whitespace issues.
    add clarifying notes about mandb updating the database by
    scanning system locations only.

V3: don't use full file paths when generating manpages

V2: add doc update about building or using the manpages
---
 doc/api/doxy-api.conf.in                  |  8 ++--
 doc/api/generate_doxygen.py               |  2 +-
 doc/api/meson.build                       | 58 +++++++++++++++++++----
 doc/guides/contributing/documentation.rst | 12 ++++-
 4 files changed, 67 insertions(+), 13 deletions(-)

diff --git a/doc/api/doxy-api.conf.in b/doc/api/doxy-api.conf.in
index a88accd907..1ba72e6221 100644
--- a/doc/api/doxy-api.conf.in
+++ b/doc/api/doxy-api.conf.in
@@ -123,11 +123,13 @@ EXAMPLE_PATTERNS        = *.c
 EXAMPLE_RECURSIVE       = YES
 
 OUTPUT_DIRECTORY        = @OUTPUT@
+FULL_PATH_NAMES         = @FULL_PATH_NAMES@
 STRIP_FROM_PATH         = @STRIP_FROM_PATH@
-GENERATE_HTML           = YES
-HTML_OUTPUT             = @HTML_OUTPUT@
+GENERATE_HTML           = @GENERATE_HTML@
+HTML_OUTPUT             = html
 GENERATE_LATEX          = NO
-GENERATE_MAN            = NO
+GENERATE_MAN            = @GENERATE_MAN@
+MAN_LINKS               = YES
 
 HAVE_DOT                = NO
 
diff --git a/doc/api/generate_doxygen.py b/doc/api/generate_doxygen.py
index d3a22869f6..c704f13018 100755
--- a/doc/api/generate_doxygen.py
+++ b/doc/api/generate_doxygen.py
@@ -7,7 +7,7 @@
 
 pattern = re.compile('^Preprocessing (.*)...$')
 out_dir, *doxygen_command = sys.argv[1:]
-out_file = os.path.join(os.path.dirname(out_dir), 'doxygen.out')
+out_file = os.path.join(out_dir + '.out')
 dep_file = f'{out_dir}.d'
 with open(out_file, 'w') as out:
     subprocess.run(doxygen_command, check=True, stdout=out)
diff --git a/doc/api/meson.build b/doc/api/meson.build
index 2876a78a7e..8b0ced423b 100644
--- a/doc/api/meson.build
+++ b/doc/api/meson.build
@@ -29,11 +29,11 @@ example = custom_target('examples.dox',
         install_dir: htmldir,
         build_by_default: get_option('enable_docs'))
 
+# set up common doxygen configuration
 cdata = configuration_data()
 cdata.set('VERSION', meson.project_version())
 cdata.set('API_EXAMPLES', join_paths(dpdk_build_root, 'doc', 'api', 'examples.dox'))
 cdata.set('OUTPUT', join_paths(dpdk_build_root, 'doc', 'api'))
-cdata.set('HTML_OUTPUT', 'html')
 cdata.set('TOPDIR', dpdk_source_root)
 cdata.set('STRIP_FROM_PATH', ' '.join([dpdk_source_root, join_paths(dpdk_build_root, 'doc', 'api')]))
 cdata.set('WARN_AS_ERROR', 'NO')
@@ -41,14 +41,35 @@ if get_option('werror')
     cdata.set('WARN_AS_ERROR', 'YES')
 endif
 
-doxy_conf = configure_file(input: 'doxy-api.conf.in',
-        output: 'doxy-api.conf',
-        configuration: cdata)
+# configure HTML doxygen run
+html_cdata = configuration_data()
+html_cdata.merge_from(cdata)
+html_cdata.set('GENERATE_HTML', 'YES')
+html_cdata.set('GENERATE_MAN', 'NO')
+html_cdata.set('FULL_PATH_NAMES', 'YES')
 
-doxy_build = custom_target('doxygen',
+doxy_html_conf = configure_file(input: 'doxy-api.conf.in',
+        output: 'doxy-api-html.conf',
+        configuration: html_cdata)
+
+# configure manpage doxygen run
+man_cdata = configuration_data()
+man_cdata.merge_from(cdata)
+man_cdata.set('GENERATE_HTML', 'NO')
+man_cdata.set('GENERATE_MAN', 'YES')
+# for manpages, have the pages only titled with the header name,
+# rather than the full path to the header
+man_cdata.set('FULL_PATH_NAMES', 'NO')
+
+doxy_man_conf = configure_file(input: 'doxy-api.conf.in',
+        output: 'doxy-api-man.conf',
+        configuration: man_cdata)
+
+# do doxygen runs
+doxy_html_build = custom_target('doxygen-html',
         depends: example,
         depend_files: 'doxy-api-index.md',
-        input: doxy_conf,
+        input: doxy_html_conf,
         output: 'html',
         depfile: 'html.d',
         command: [generate_doxygen, '@OUTPUT@', doxygen, '@INPUT@'],
@@ -56,5 +77,26 @@ doxy_build = custom_target('doxygen',
         install_dir: htmldir,
         build_by_default: get_option('enable_docs'))
 
-doc_targets += doxy_build
-doc_target_names += 'Doxygen_API'
+doc_targets += doxy_html_build
+doc_target_names += 'Doxygen_API(HTML)'
+
+doxy_man_build = custom_target('doxygen-man',
+        depends: example,
+        depend_files: 'doxy-api-index.md',
+        input: doxy_man_conf,
+        output: 'man',
+        depfile: 'man.d',
+        command: [generate_doxygen, '@OUTPUT@', doxygen, '@INPUT@'],
+        install: get_option('enable_docs'),
+        install_dir: get_option('datadir'),
+        build_by_default: get_option('enable_docs'))
+
+doc_targets += doxy_man_build
+doc_target_names += 'Doxygen_API(Manpage)'
+
+# refresh the manpage database on install
+# if DPDK manpages are installed to a staging directory, not in MANPATH, this has no effect
+mandb = find_program('mandb', required: false)
+if mandb.found() and get_option('enable_docs') and meson.version().version_compare('>=0.55.0')
+    meson.add_install_script(mandb)
+endif
diff --git a/doc/guides/contributing/documentation.rst b/doc/guides/contributing/documentation.rst
index 79616e5610..4eb62fc36a 100644
--- a/doc/guides/contributing/documentation.rst
+++ b/doc/guides/contributing/documentation.rst
@@ -182,7 +182,17 @@ To build the documentation::
 
 See :doc:`../linux_gsg/build_dpdk` for more detail on compiling DPDK with meson.
 
-The output is generated in the directories ``build/doc/html/{api,guides}``.
+The output is generated in the directory ``build/doc/``, with:
+
+* HTML versions of the guide docs, e.g. Getting Started Guides, Programmers Guide, in ``build/doc/guides/html``
+* HTML version of the API documentation in ``build/doc/api/html``
+* Man-page version of the API documentation in ``build/doc/api/man``.
+  If not installing DPDK system-wise, these pages can be accessed by adding this directory to the ``MANPATH`` environment variable.
+  For example:
+
+.. code-block:: console
+
+   export MANPATH=:/path/to/build/doc/api/man
 
 .. Note::
 
-- 
2.39.2

Re: [PATCH v4] doc: build manpages as well as html output

[Thomas Monjalon]

31/08/2023 11:49, Bruce Richardson:
> Doxygen can produce manpage output as well as html output for the DPDK
> APIs. However, we need to do this as a separate task as the manpage
> output needs to be placed in a different location post-install to the
> html output (/usr/local/share/man vs /usr/local/share/doc/).
> 
> Changes required are:
> * Add configurable options for manpage output and html output to the
>   doxygen config template. (Remove option for html output path as it's
>   always "html")
> * Modify API meson.build file to configure two separate doxygen config
>   files, for HTML and manpages respectively.
> * Change doxygen wrapper script to have separate output log files for
>   the manpage and HTML jobs, to avoid conflicts
> * Add "custom_targets" to meson.build file to build the HTML pages and
>   the manpages, with individual install locations for each.
> * Where supported by meson version, call "mandb" post-install to update
>   the man database to ensure the new manpages can be found. If the
>   manpages are not installed in system location i.e. one not in MANPATH,
>   then this update will have no effect, as only system locations are
>   scanned.
> 
> Signed-off-by: Bruce Richardson <bruce.richardson@intel.com>
> Reviewed-by: David Marchand <david.marchand@redhat.com>

Acked-by: Thomas Monjalon <thomas@monjalon.net>

RE: [PATCH v4] doc: build manpages as well as html output

[Morten Brørup]

> From: Thomas Monjalon [mailto:thomas@monjalon.net]
> Sent: Thursday, 31 August 2023 12.12
> 
> 31/08/2023 11:49, Bruce Richardson:
> > Doxygen can produce manpage output as well as html output for the DPDK
> > APIs. However, we need to do this as a separate task as the manpage
> > output needs to be placed in a different location post-install to the
> > html output (/usr/local/share/man vs /usr/local/share/doc/).
> >
> > Changes required are:
> > * Add configurable options for manpage output and html output to the
> >   doxygen config template. (Remove option for html output path as it's
> >   always "html")
> > * Modify API meson.build file to configure two separate doxygen config
> >   files, for HTML and manpages respectively.
> > * Change doxygen wrapper script to have separate output log files for
> >   the manpage and HTML jobs, to avoid conflicts
> > * Add "custom_targets" to meson.build file to build the HTML pages and
> >   the manpages, with individual install locations for each.
> > * Where supported by meson version, call "mandb" post-install to update
> >   the man database to ensure the new manpages can be found. If the
> >   manpages are not installed in system location i.e. one not in MANPATH,
> >   then this update will have no effect, as only system locations are
> >   scanned.
> >
> > Signed-off-by: Bruce Richardson <bruce.richardson@intel.com>
> > Reviewed-by: David Marchand <david.marchand@redhat.com>
> 
> Acked-by: Thomas Monjalon <thomas@monjalon.net>
> 

Having followed the discussion where the "cross build" details were clarified...

Acked-by: Morten Brørup <mb@smartsharesystems.com>

Re: [PATCH v4] doc: build manpages as well as html output

[Thomas Monjalon]

31/08/2023 17:48, Morten Brørup:
> > From: Thomas Monjalon [mailto:thomas@monjalon.net]
> > Sent: Thursday, 31 August 2023 12.12
> > 
> > 31/08/2023 11:49, Bruce Richardson:
> > > Doxygen can produce manpage output as well as html output for the DPDK
> > > APIs. However, we need to do this as a separate task as the manpage
> > > output needs to be placed in a different location post-install to the
> > > html output (/usr/local/share/man vs /usr/local/share/doc/).
> > >
> > > Changes required are:
> > > * Add configurable options for manpage output and html output to the
> > >   doxygen config template. (Remove option for html output path as it's
> > >   always "html")
> > > * Modify API meson.build file to configure two separate doxygen config
> > >   files, for HTML and manpages respectively.
> > > * Change doxygen wrapper script to have separate output log files for
> > >   the manpage and HTML jobs, to avoid conflicts
> > > * Add "custom_targets" to meson.build file to build the HTML pages and
> > >   the manpages, with individual install locations for each.
> > > * Where supported by meson version, call "mandb" post-install to update
> > >   the man database to ensure the new manpages can be found. If the
> > >   manpages are not installed in system location i.e. one not in MANPATH,
> > >   then this update will have no effect, as only system locations are
> > >   scanned.
> > >
> > > Signed-off-by: Bruce Richardson <bruce.richardson@intel.com>
> > > Reviewed-by: David Marchand <david.marchand@redhat.com>
> > 
> > Acked-by: Thomas Monjalon <thomas@monjalon.net>
> > 
> 
> Having followed the discussion where the "cross build" details were clarified...
> 
> Acked-by: Morten Brørup <mb@smartsharesystems.com>

Applied, thanks.