From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id A5EA243E51; Fri, 12 Apr 2024 12:14:33 +0200 (CEST) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 6DABA40A76; Fri, 12 Apr 2024 12:14:15 +0200 (CEST) Received: from mail-lf1-f54.google.com (mail-lf1-f54.google.com [209.85.167.54]) by mails.dpdk.org (Postfix) with ESMTP id 9668D406BC for ; Fri, 12 Apr 2024 12:14:12 +0200 (CEST) Received: by mail-lf1-f54.google.com with SMTP id 2adb3069b0e04-516d47ce662so1146456e87.1 for ; Fri, 12 Apr 2024 03:14:12 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=pantheon.tech; s=google; t=1712916852; x=1713521652; darn=dpdk.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=0fT4rFTx5S2snWFzvDXZDIgiH/T0bh2PnzsPQGA9zEY=; b=gFKvkiSaDDceIP4B3bN+bWE2rgGHWOhSwjMiGpBp/NKQzvJ0aq1wAozUgGD8GADzLG wGiY12/gC6WpS7By6TYZFeQavrgmV+uboMZWj9/BujxIYPAf4TEKYgoN5tXOV2nisAmz A4B/MSeUaY+j+pAJcfzcvnHH260waPpzw7jbhZ8S3TxLoeiw7KHUSM1Ccv/tl2Q+qrnT ZYpfoNQyR+/fzxDmCSXDbcVN1g3ezY+xavdOS9G4JkrMbxH8d/6qJGKrGXfQnlr3383x ajyGKXUbMpUTsqNiy0TDjwP/5id4CEUnb22lYrBL0MQZnaeP5HY4ikE3TSv1mANobUbl RVKQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1712916852; x=1713521652; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=0fT4rFTx5S2snWFzvDXZDIgiH/T0bh2PnzsPQGA9zEY=; b=GyknQfc60P513lzEXjK6hE/AMZHrp5gpTPPMA998/XeW4i9vcgW5uoc20TlE/TprIO YmwLv+6Lbd3x89dBoyZwsBCoSQQhQTIlUnpm89s5qbVEC3JSfYaWJ4biFqf9zal+sqn3 pyInzZixTUR+1Eo/M8Ufn/5wMLnQkqmLINs6+K5sHn2mOW01J6SqDo38MU0aMyYKbq2C B1pQs97RSqOAepob/NQcdE95PTtYecFmvem56UW8HSniwls2ouKjRuX5IGOIMrx6KIDE JaG8Onh6NauC/MOm5cvzS8ioinxwWNNqVlSCoJC2UtAKLr+M3aELoPFSeDwMHEKPQObX gbOQ== X-Gm-Message-State: AOJu0Yz30Urp5IHSB8JkEwcJv3AmzvZ2LJMaaomBAS0Rc70rNQbI6SYo Rt9/Mxb7fENpufjKFEdiVlGZ9h0RPLGvQPalH4xJVlAxxb7LmT/LywUwatkEHbY= X-Google-Smtp-Source: AGHT+IGX6s7lxpvnC/KQhmzcDdaLnmXryl79y9p2YrqAmAXUUUTblRZoYBpjesOGNeBFnQ/XtDC9Rw== X-Received: by 2002:a05:6512:3d11:b0:518:9844:648d with SMTP id d17-20020a0565123d1100b005189844648dmr51850lfv.60.1712916851956; Fri, 12 Apr 2024 03:14:11 -0700 (PDT) Received: from jlinkes-PT-Latitude-5530.pantheon.local ([84.245.121.75]) by smtp.gmail.com with ESMTPSA id k11-20020a170906578b00b00a51be2b75f3sm1640802ejq.35.2024.04.12.03.14.10 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 12 Apr 2024 03:14:11 -0700 (PDT) From: =?UTF-8?q?Juraj=20Linke=C5=A1?= To: thomas@monjalon.net, Honnappa.Nagarahalli@arm.com, bruce.richardson@intel.com, jspewock@iol.unh.edu, probb@iol.unh.edu, paul.szczepanek@arm.com, Luca.Vizzarro@arm.com, npratte@iol.unh.edu Cc: dev@dpdk.org, =?UTF-8?q?Juraj=20Linke=C5=A1?= Subject: [PATCH v4 3/3] dts: add API doc generation Date: Fri, 12 Apr 2024 12:14:05 +0200 Message-Id: <20240412101405.94377-4-juraj.linkes@pantheon.tech> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20240412101405.94377-1-juraj.linkes@pantheon.tech> References: <20231115133606.42081-1-juraj.linkes@pantheon.tech> <20240412101405.94377-1-juraj.linkes@pantheon.tech> MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org The tool used to generate DTS API docs is Sphinx, which is already in use in DPDK. The same configuration is used to preserve style with one DTS-specific configuration (so that the DPDK docs are unchanged) that modifies how the sidebar displays the content. Sphinx generates the documentation from Python docstrings. The docstring format is the Google format [0] which requires the sphinx.ext.napoleon extension. The other extension, sphinx.ext.intersphinx, enables linking to object in external documentations, such as the Python documentation. There are two requirements for building DTS docs: * The same Python version as DTS or higher, because Sphinx imports the code. * Also the same Python packages as DTS, for the same reason. [0] https://google.github.io/styleguide/pyguide.html#38-comments-and-docstrings Signed-off-by: Juraj Linkeš Reviewed-by: Jeremy Spewock Tested-by: Nicholas Pratte --- buildtools/call-sphinx-build.py | 33 +++++++++++++++++++--------- doc/api/doxy-api-index.md | 3 +++ doc/api/doxy-api.conf.in | 2 ++ doc/api/meson.build | 11 +++++++--- doc/guides/conf.py | 39 ++++++++++++++++++++++++++++----- doc/guides/meson.build | 1 + doc/guides/tools/dts.rst | 34 +++++++++++++++++++++++++++- dts/doc/meson.build | 27 +++++++++++++++++++++++ dts/meson.build | 16 ++++++++++++++ meson.build | 1 + 10 files changed, 148 insertions(+), 19 deletions(-) create mode 100644 dts/doc/meson.build create mode 100644 dts/meson.build diff --git a/buildtools/call-sphinx-build.py b/buildtools/call-sphinx-build.py index 39a60d09fa..aea771a64e 100755 --- a/buildtools/call-sphinx-build.py +++ b/buildtools/call-sphinx-build.py @@ -3,37 +3,50 @@ # Copyright(c) 2019 Intel Corporation # +import argparse import sys import os from os.path import join from subprocess import run, PIPE, STDOUT from packaging.version import Version -# assign parameters to variables -(sphinx, version, src, dst, *extra_args) = sys.argv[1:] +parser = argparse.ArgumentParser() +parser.add_argument('sphinx') +parser.add_argument('version') +parser.add_argument('src') +parser.add_argument('dst') +parser.add_argument('--dts-root', default=None) +args, extra_args = parser.parse_known_args() # set the version in environment for sphinx to pick up -os.environ['DPDK_VERSION'] = version +os.environ['DPDK_VERSION'] = args.version +if args.dts_root: + os.environ['DTS_ROOT'] = args.dts_root # for sphinx version >= 1.7 add parallelism using "-j auto" -ver = run([sphinx, '--version'], stdout=PIPE, +ver = run([args.sphinx, '--version'], stdout=PIPE, stderr=STDOUT).stdout.decode().split()[-1] -sphinx_cmd = [sphinx] + extra_args +sphinx_cmd = [args.sphinx] + extra_args if Version(ver) >= Version('1.7'): sphinx_cmd += ['-j', 'auto'] # find all the files sphinx will process so we can write them as dependencies srcfiles = [] -for root, dirs, files in os.walk(src): +for root, dirs, files in os.walk(args.src): srcfiles.extend([join(root, f) for f in files]) +if not os.path.exists(args.dst): + os.makedirs(args.dst) + # run sphinx, putting the html output in a "html" directory -with open(join(dst, 'sphinx_html.out'), 'w') as out: - process = run(sphinx_cmd + ['-b', 'html', src, join(dst, 'html')], - stdout=out) +with open(join(args.dst, 'sphinx_html.out'), 'w') as out: + process = run( + sphinx_cmd + ['-b', 'html', args.src, join(args.dst, 'html')], + stdout=out + ) # create a gcc format .d file giving all the dependencies of this doc build -with open(join(dst, '.html.d'), 'w') as d: +with open(join(args.dst, '.html.d'), 'w') as d: d.write('html: ' + ' '.join(srcfiles) + '\n') sys.exit(process.returncode) diff --git a/doc/api/doxy-api-index.md b/doc/api/doxy-api-index.md index 8c1eb8fafa..d5f823b7f0 100644 --- a/doc/api/doxy-api-index.md +++ b/doc/api/doxy-api-index.md @@ -243,3 +243,6 @@ The public API headers are grouped by topics: [experimental APIs](@ref rte_compat.h), [ABI versioning](@ref rte_function_versioning.h), [version](@ref rte_version.h) + +- **tests**: + [**DTS**](@dts_api_main_page) diff --git a/doc/api/doxy-api.conf.in b/doc/api/doxy-api.conf.in index 27afec8b3b..2e08c6a452 100644 --- a/doc/api/doxy-api.conf.in +++ b/doc/api/doxy-api.conf.in @@ -123,6 +123,8 @@ SEARCHENGINE = YES SORT_MEMBER_DOCS = NO SOURCE_BROWSER = YES +ALIASES = "dts_api_main_page=@DTS_API_MAIN_PAGE@" + EXAMPLE_PATH = @TOPDIR@/examples EXAMPLE_PATTERNS = *.c EXAMPLE_RECURSIVE = YES diff --git a/doc/api/meson.build b/doc/api/meson.build index 5b50692df9..ffc75d7b5a 100644 --- a/doc/api/meson.build +++ b/doc/api/meson.build @@ -1,6 +1,7 @@ # SPDX-License-Identifier: BSD-3-Clause # Copyright(c) 2018 Luca Boccassi +doc_api_build_dir = meson.current_build_dir() doxygen = find_program('doxygen', required: get_option('enable_docs')) if not doxygen.found() @@ -32,14 +33,18 @@ example = custom_target('examples.dox', # set up common Doxygen configuration cdata = configuration_data() cdata.set('VERSION', meson.project_version()) -cdata.set('API_EXAMPLES', join_paths(dpdk_build_root, 'doc', 'api', 'examples.dox')) -cdata.set('OUTPUT', join_paths(dpdk_build_root, 'doc', 'api')) +cdata.set('API_EXAMPLES', join_paths(doc_api_build_dir, 'examples.dox')) +cdata.set('OUTPUT', doc_api_build_dir) cdata.set('TOPDIR', dpdk_source_root) -cdata.set('STRIP_FROM_PATH', ' '.join([dpdk_source_root, join_paths(dpdk_build_root, 'doc', 'api')])) +cdata.set('STRIP_FROM_PATH', ' '.join([dpdk_source_root, doc_api_build_dir])) cdata.set('WARN_AS_ERROR', 'NO') if get_option('werror') cdata.set('WARN_AS_ERROR', 'YES') endif +# A local reference must be relative to the main index.html page +# The path below can't be taken from the DTS meson file as that would +# require recursive subdir traversal (doc, dts, then doc again) +cdata.set('DTS_API_MAIN_PAGE', join_paths('..', 'dts', 'html', 'index.html')) # configure HTML Doxygen run html_cdata = configuration_data() diff --git a/doc/guides/conf.py b/doc/guides/conf.py index 0f7ff5282d..b442a1f76c 100644 --- a/doc/guides/conf.py +++ b/doc/guides/conf.py @@ -7,10 +7,9 @@ from sphinx import __version__ as sphinx_version from os import listdir from os import environ -from os.path import basename -from os.path import dirname +from os.path import basename, dirname from os.path import join as path_join -from sys import argv, stderr +from sys import argv, stderr, path import configparser @@ -24,6 +23,37 @@ file=stderr) pass +# Napoleon enables the Google format of Python doscstrings, used in DTS +# Intersphinx allows linking to external projects, such as Python docs, also used in DTS +extensions = ['sphinx.ext.napoleon', 'sphinx.ext.intersphinx'] + +# DTS Python docstring options +autodoc_default_options = { + 'members': True, + 'member-order': 'bysource', + 'show-inheritance': True, +} +autodoc_class_signature = 'separated' +autodoc_typehints = 'both' +autodoc_typehints_format = 'short' +autodoc_typehints_description_target = 'documented' +napoleon_numpy_docstring = False +napoleon_attr_annotations = True +napoleon_preprocess_types = True +add_module_names = False +toc_object_entries = True +toc_object_entries_show_parents = 'hide' +intersphinx_mapping = {'python': ('https://docs.python.org/3', None)} + +dts_root = environ.get('DTS_ROOT') +if dts_root: + path.append(dts_root) + # DTS Sidebar config + html_theme_options = { + 'collapse_navigation': False, + 'navigation_depth': -1, + } + stop_on_error = ('-W' in argv) project = 'Data Plane Development Kit' @@ -35,8 +65,7 @@ html_show_copyright = False highlight_language = 'none' -release = environ.setdefault('DPDK_VERSION', "None") -version = release +version = environ.setdefault('DPDK_VERSION', "None") master_doc = 'index' diff --git a/doc/guides/meson.build b/doc/guides/meson.build index 51f81da2e3..8933d75f6b 100644 --- a/doc/guides/meson.build +++ b/doc/guides/meson.build @@ -1,6 +1,7 @@ # SPDX-License-Identifier: BSD-3-Clause # Copyright(c) 2018 Intel Corporation +doc_guides_source_dir = meson.current_source_dir() sphinx = find_program('sphinx-build', required: get_option('enable_docs')) if not sphinx.found() diff --git a/doc/guides/tools/dts.rst b/doc/guides/tools/dts.rst index 47b218b2c6..d1c3c2af7a 100644 --- a/doc/guides/tools/dts.rst +++ b/doc/guides/tools/dts.rst @@ -280,7 +280,12 @@ and try not to divert much from it. The :ref:`DTS developer tools ` will issue warnings when some of the basics are not met. -The code must be properly documented with docstrings. +The API documentation, which is a helpful reference when developing, may be accessed +in the code directly or generated with the :ref:`API docs build steps `. +When adding new files or modifying the directory structure, the corresponding changes must +be made to DTS api doc sources in ``dts/doc``. + +Speaking of which, the code must be properly documented with docstrings. The style must conform to the `Google style `_. See an example of the style `here @@ -415,6 +420,33 @@ the DTS code check and format script. Refer to the script for usage: ``devtools/dts-check-format.sh -h``. +.. _building_api_docs: + +Building DTS API docs +--------------------- + +To build DTS API docs, install the dependencies with Poetry, then enter its shell: + +.. code-block:: console + + poetry install --no-root --with docs + poetry shell + +The documentation is built using the standard DPDK build system. After executing the meson command +and entering Poetry's shell, build the documentation with: + +.. code-block:: console + + ninja -C build dts-doc + +The output is generated in ``build/doc/api/dts/html``. + +.. Note:: + + Make sure to fix any Sphinx warnings when adding or updating docstrings. Also make sure to run + the ``devtools/dts-check-format.sh`` script and address any issues it finds. + + Configuration Schema -------------------- diff --git a/dts/doc/meson.build b/dts/doc/meson.build new file mode 100644 index 0000000000..01b7b51034 --- /dev/null +++ b/dts/doc/meson.build @@ -0,0 +1,27 @@ +# SPDX-License-Identifier: BSD-3-Clause +# Copyright(c) 2023 PANTHEON.tech s.r.o. + +sphinx = find_program('sphinx-build', required: false) +sphinx_apidoc = find_program('sphinx-apidoc', required: false) + +if not sphinx.found() or not sphinx_apidoc.found() + subdir_done() +endif + +dts_doc_api_build_dir = join_paths(doc_api_build_dir, 'dts') + +extra_sphinx_args = ['-E', '-c', doc_guides_source_dir, '--dts-root', dts_dir] +if get_option('werror') + extra_sphinx_args += '-W' +endif + +htmldir = join_paths(get_option('datadir'), 'doc', 'dpdk', 'dts') +dts_api_html = custom_target('dts_api_html', + output: 'html', + command: [sphinx_wrapper, sphinx, meson.project_version(), + meson.current_source_dir(), dts_doc_api_build_dir, extra_sphinx_args], + build_by_default: false, + install: get_option('enable_docs'), + install_dir: htmldir) +doc_targets += dts_api_html +doc_target_names += 'DTS_API_HTML' diff --git a/dts/meson.build b/dts/meson.build new file mode 100644 index 0000000000..e8ce0f06ac --- /dev/null +++ b/dts/meson.build @@ -0,0 +1,16 @@ +# SPDX-License-Identifier: BSD-3-Clause +# Copyright(c) 2023 PANTHEON.tech s.r.o. + +doc_targets = [] +doc_target_names = [] +dts_dir = meson.current_source_dir() + +subdir('doc') + +if doc_targets.length() == 0 + message = 'No docs targets found' +else + message = 'Built docs:' +endif +run_target('dts-doc', command: [echo, message, doc_target_names], + depends: doc_targets) diff --git a/meson.build b/meson.build index 8b248d4505..835973a0ce 100644 --- a/meson.build +++ b/meson.build @@ -87,6 +87,7 @@ subdir('app') # build docs subdir('doc') +subdir('dts') # build any examples explicitly requested - useful for developers - and # install any example code into the appropriate install path -- 2.34.1