From mboxrd@z Thu Jan  1 00:00:00 1970
Return-Path: <dev-bounces@dpdk.org>
Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124])
	by inbox.dpdk.org (Postfix) with ESMTP id 31C7145769;
	Thu,  8 Aug 2024 10:54:56 +0200 (CEST)
Received: from mails.dpdk.org (localhost [127.0.0.1])
	by mails.dpdk.org (Postfix) with ESMTP id 0062040A80;
	Thu,  8 Aug 2024 10:54:56 +0200 (CEST)
Received: from mail-ej1-f47.google.com (mail-ej1-f47.google.com
 [209.85.218.47]) by mails.dpdk.org (Postfix) with ESMTP id DF23740696
 for <dev@dpdk.org>; Thu,  8 Aug 2024 10:54:54 +0200 (CEST)
Received: by mail-ej1-f47.google.com with SMTP id
 a640c23a62f3a-a7aa212c1c9so96242666b.2
 for <dev@dpdk.org>; Thu, 08 Aug 2024 01:54:54 -0700 (PDT)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
 d=pantheon.tech; s=google; t=1723107294; x=1723712094; darn=dpdk.org;
 h=content-transfer-encoding:mime-version:references:in-reply-to
 :message-id:date:subject:cc:to:from:from:to:cc:subject:date
 :message-id:reply-to;
 bh=B4aES13Q9yGtW4vmNEWrWE9Ykcph8ofbjrhQh3jTVEE=;
 b=eXEvDruZ4yYatGp57u4AXEVefZCNq/Q24sRfsNzTHpzbA+lbS+NvU6EH4CytMRid9f
 WQ4tymncX276tmBJqDNyZoI7w+rV4/EcLJNPaIMTnAlyycWo5R+B1rz3uFOyAvUGmiGm
 xwT7DovMTsaKtou1ruI3DSjB7D7G2+wQtdhai6xirphKFmRvqCAPA9WHAKLwHhzz7PQy
 S+FzDMKdj3uUXDMOUvuYH6EFrhWtWVzKg/YF23Ii4UvxXQRgSQqo3uSDkgRWqsoo8rpJ
 4475zdP8oOqq7eFuquwcB14CZ4kT9Q0+Z25UsOciTKU0dCF2e1bitoHCiD0IQlnuqubh
 TT6A==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
 d=1e100.net; s=20230601; t=1723107294; x=1723712094;
 h=content-transfer-encoding:mime-version:references:in-reply-to
 :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc
 :subject:date:message-id:reply-to;
 bh=B4aES13Q9yGtW4vmNEWrWE9Ykcph8ofbjrhQh3jTVEE=;
 b=kpt2H2gz007aqLv6o9qxzUlg2RzbDQKOD8fTBPOFWriFDBvDev8h7OKhrVuw7Va2Qm
 d/Eu7XglmNRrzEQJYWdV6DkeCajm0a3CLqU8PxN90sN4AgN/c/BeGECSJ2xg5IWZXrSn
 tfsKNoQLdDAOLjNGe2zg4xxp7JYGpuKgNPgDUS0L6ksAApkhpLQE7mhKtgXGcd5DMUUz
 45lXagnZju93r51bI2XZQSV99wxLHl90ao0HkGgKXgWCwbCQXPtM4dV7Vujw5XSJuoQ9
 wrCEJck6e2L+goazFuIwMugV2MgJRT0bYAFKbMZixm6msZE8GKDZ0BucxXQ3JVOjP444
 G44g==
X-Gm-Message-State: AOJu0Yx27e2gDNgx4vVnCVNt+P/UBbcLYpUMhoSwFQvzrZuPLdnAVIWG
 g/+8LlX8jDeupqSo+QIZM71EIIhAXv4luw5aeaJqva48IegTwatbvLG0lpqjj0DRmeg1jmZ/jKF
 bUJs=
X-Google-Smtp-Source: AGHT+IG2fwZS3PLInxGLw5vtUfma/gzx+gvtzz0u+8wWMmIBga8DNMEiHqAIkylx+QhUZNNOGhagcA==
X-Received: by 2002:a17:907:7283:b0:a7a:bae8:f29d with SMTP id
 a640c23a62f3a-a8090c2561cmr72378966b.6.1723107294309; 
 Thu, 08 Aug 2024 01:54:54 -0700 (PDT)
Received: from jlinkes-PT-Latitude-5530.pantheon.local ([84.245.121.236])
 by smtp.gmail.com with ESMTPSA id
 a640c23a62f3a-a7dc9ec8879sm719996266b.211.2024.08.08.01.54.53
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Thu, 08 Aug 2024 01:54:53 -0700 (PDT)
From: =?UTF-8?q?Juraj=20Linke=C5=A1?= <juraj.linkes@pantheon.tech>
To: thomas@monjalon.net, Honnappa.Nagarahalli@arm.com,
 bruce.richardson@intel.com, jspewock@iol.unh.edu, probb@iol.unh.edu,
 paul.szczepanek@arm.com, Luca.Vizzarro@arm.com, npratte@iol.unh.edu
Cc: dev@dpdk.org, =?UTF-8?q?Juraj=20Linke=C5=A1?= <juraj.linkes@pantheon.tech>
Subject: [PATCH v16 0/5] API docs generation
Date: Thu,  8 Aug 2024 10:54:47 +0200
Message-Id: <20240808085452.426702-1-juraj.linkes@pantheon.tech>
X-Mailer: git-send-email 2.34.1
In-Reply-To: <20231115133606.42081-1-juraj.linkes@pantheon.tech>
References: <20231115133606.42081-1-juraj.linkes@pantheon.tech>
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit
X-BeenThere: dev@dpdk.org
X-Mailman-Version: 2.1.29
Precedence: list
List-Id: DPDK patches and discussions <dev.dpdk.org>
List-Unsubscribe: <https://mails.dpdk.org/options/dev>,
 <mailto:dev-request@dpdk.org?subject=unsubscribe>
List-Archive: <http://mails.dpdk.org/archives/dev/>
List-Post: <mailto:dev@dpdk.org>
List-Help: <mailto:dev-request@dpdk.org?subject=help>
List-Subscribe: <https://mails.dpdk.org/listinfo/dev>,
 <mailto:dev-request@dpdk.org?subject=subscribe>
Errors-To: dev-bounces@dpdk.org

The generation is done with Sphinx, which DPDK already uses, with
slightly modified configuration of the sidebar present in an if block.

DTS dependencies do not need to be installed, but there is the option to
install doc build dependencies with Poetry:
poetry install --with docs

The build itself may be run with:
meson setup <meson_build_dir> -Denable_docs=true
ninja -C <meson_build_dir>

The above will do a full DPDK build with docs. To build just docs:
meson setup <meson_build_dir>
ninja -C <meson_build_dir> dts-doc

Python3.10 is required to build the DTS API docs.

The patchset contains the .rst sources which Sphinx uses to generate the
html pages. These were first generated with the sphinx-apidoc utility
and modified to provide a better look. The documentation just doesn't
look that good without the modifications and there isn't enough
configuration options to achieve that without manual changes to the .rst
files. This introduces extra maintenance which involves adding new .rst
files when a new Python module is added or changing the .rst structure
if the Python directory/file structure is changed (moved, renamed
files). This small maintenance burden is outweighed by the flexibility
afforded by the ability to make manual changes to the .rst files.

v10:
Fix dts doc generation issue: Only copy the custom rss file if it exists.

v11:
Added the config option autodoc_mock_imports, which eliminates the need
for DTS dependencies. Added a script that find out which imports need to
be added to autodoc_mock_imports. The script also check the required
Python version for building DTS docs.
Removed tags from the two affected patches which will need to be
reviewed again.

v12:
Added paramiko to the required dependencies of get-dts-deps.py.

v13:
Fixed build error:
TypeError: unsupported operand type(s) for |: 'NoneType' and 'Transport'

v14:
Fixed install error:
ERROR: File 'dts/doc/html' could not be found
This required me to put the built docs into dts/doc which is outside the
DPDK API doc dir, resulting in linking between DPDK and DTS api docs not
working properly. I addressed this by adding a symlink to the build dir.
This way the link works after installing the docs and the symlink is
just one extra file in the build dir.

v15:
Moved DTS API sources to doc/api/dts. This simplifies a lot of things in
the build, but mainly makes a lot of sense. Now the source, build and
install paths are the same so there isn't any need for any symlinks or
other workarounds.
Also added a symlink to the custom.css file so that it works with
call-sphinx-build.py without any modifications.

v16:
Renamed the dependency python file to get-dts-runtime-deps.py a modified
it to only get runtime dependencies. We don't need to check docs
dependencies (Sphinx) as we don't need to mock those.
Also moved all new Sphinx configuration into the DTS if branch to make
sure it won't ever affect the DPDK doc build.

Juraj Linkeš (5):
  dts: update params and parser docstrings
  dts: replace the or operator in third party types
  dts: add doc generation dependencies
  dts: add API doc sources
  dts: add API doc generation

 buildtools/call-sphinx-build.py               |   2 +
 buildtools/get-dts-runtime-deps.py            |  72 +++
 buildtools/meson.build                        |   1 +
 doc/api/doxy-api-index.md                     |   3 +
 doc/api/doxy-api.conf.in                      |   2 +
 doc/api/dts/conf_yaml_schema.json             |   1 +
 doc/api/dts/custom.css                        |   1 +
 doc/api/dts/framework.config.rst              |  12 +
 doc/api/dts/framework.config.types.rst        |   6 +
 doc/api/dts/framework.exception.rst           |   6 +
 doc/api/dts/framework.logger.rst              |   6 +
 doc/api/dts/framework.params.eal.rst          |   6 +
 doc/api/dts/framework.params.rst              |  14 +
 doc/api/dts/framework.params.testpmd.rst      |   6 +
 doc/api/dts/framework.params.types.rst        |   6 +
 doc/api/dts/framework.parser.rst              |   6 +
 .../framework.remote_session.dpdk_shell.rst   |   6 +
 ...ote_session.interactive_remote_session.rst |   6 +
 ...ework.remote_session.interactive_shell.rst |   6 +
 .../framework.remote_session.python_shell.rst |   6 +
 ...ramework.remote_session.remote_session.rst |   6 +
 doc/api/dts/framework.remote_session.rst      |  18 +
 .../framework.remote_session.ssh_session.rst  |   6 +
 ...framework.remote_session.testpmd_shell.rst |   6 +
 doc/api/dts/framework.runner.rst              |   6 +
 doc/api/dts/framework.settings.rst            |   6 +
 doc/api/dts/framework.test_result.rst         |   6 +
 doc/api/dts/framework.test_suite.rst          |   6 +
 doc/api/dts/framework.testbed_model.cpu.rst   |   6 +
 .../framework.testbed_model.linux_session.rst |   6 +
 doc/api/dts/framework.testbed_model.node.rst  |   6 +
 .../framework.testbed_model.os_session.rst    |   6 +
 doc/api/dts/framework.testbed_model.port.rst  |   6 +
 .../framework.testbed_model.posix_session.rst |   6 +
 doc/api/dts/framework.testbed_model.rst       |  26 +
 .../dts/framework.testbed_model.sut_node.rst  |   6 +
 .../dts/framework.testbed_model.tg_node.rst   |   6 +
 ..._generator.capturing_traffic_generator.rst |   6 +
 ...mework.testbed_model.traffic_generator.rst |  14 +
 ....testbed_model.traffic_generator.scapy.rst |   6 +
 ...el.traffic_generator.traffic_generator.rst |   6 +
 ...framework.testbed_model.virtual_device.rst |   6 +
 doc/api/dts/framework.utils.rst               |   6 +
 doc/api/dts/index.rst                         |  43 ++
 doc/api/dts/meson.build                       |  29 +
 doc/api/meson.build                           |  13 +
 doc/guides/conf.py                            |  44 +-
 doc/guides/contributing/documentation.rst     |   2 +
 doc/guides/contributing/patches.rst           |   4 +
 doc/guides/tools/dts.rst                      |  39 +-
 doc/meson.build                               |   1 +
 dts/framework/params/__init__.py              |   4 +-
 dts/framework/params/eal.py                   |   7 +-
 dts/framework/params/types.py                 |   3 +-
 dts/framework/parser.py                       |   4 +-
 .../interactive_remote_session.py             |   3 +-
 dts/poetry.lock                               | 521 +++++++++++++++++-
 dts/pyproject.toml                            |   8 +
 58 files changed, 1055 insertions(+), 22 deletions(-)
 create mode 100755 buildtools/get-dts-runtime-deps.py
 create mode 120000 doc/api/dts/conf_yaml_schema.json
 create mode 120000 doc/api/dts/custom.css
 create mode 100644 doc/api/dts/framework.config.rst
 create mode 100644 doc/api/dts/framework.config.types.rst
 create mode 100644 doc/api/dts/framework.exception.rst
 create mode 100644 doc/api/dts/framework.logger.rst
 create mode 100644 doc/api/dts/framework.params.eal.rst
 create mode 100644 doc/api/dts/framework.params.rst
 create mode 100644 doc/api/dts/framework.params.testpmd.rst
 create mode 100644 doc/api/dts/framework.params.types.rst
 create mode 100644 doc/api/dts/framework.parser.rst
 create mode 100644 doc/api/dts/framework.remote_session.dpdk_shell.rst
 create mode 100644 doc/api/dts/framework.remote_session.interactive_remote_session.rst
 create mode 100644 doc/api/dts/framework.remote_session.interactive_shell.rst
 create mode 100644 doc/api/dts/framework.remote_session.python_shell.rst
 create mode 100644 doc/api/dts/framework.remote_session.remote_session.rst
 create mode 100644 doc/api/dts/framework.remote_session.rst
 create mode 100644 doc/api/dts/framework.remote_session.ssh_session.rst
 create mode 100644 doc/api/dts/framework.remote_session.testpmd_shell.rst
 create mode 100644 doc/api/dts/framework.runner.rst
 create mode 100644 doc/api/dts/framework.settings.rst
 create mode 100644 doc/api/dts/framework.test_result.rst
 create mode 100644 doc/api/dts/framework.test_suite.rst
 create mode 100644 doc/api/dts/framework.testbed_model.cpu.rst
 create mode 100644 doc/api/dts/framework.testbed_model.linux_session.rst
 create mode 100644 doc/api/dts/framework.testbed_model.node.rst
 create mode 100644 doc/api/dts/framework.testbed_model.os_session.rst
 create mode 100644 doc/api/dts/framework.testbed_model.port.rst
 create mode 100644 doc/api/dts/framework.testbed_model.posix_session.rst
 create mode 100644 doc/api/dts/framework.testbed_model.rst
 create mode 100644 doc/api/dts/framework.testbed_model.sut_node.rst
 create mode 100644 doc/api/dts/framework.testbed_model.tg_node.rst
 create mode 100644 doc/api/dts/framework.testbed_model.traffic_generator.capturing_traffic_generator.rst
 create mode 100644 doc/api/dts/framework.testbed_model.traffic_generator.rst
 create mode 100644 doc/api/dts/framework.testbed_model.traffic_generator.scapy.rst
 create mode 100644 doc/api/dts/framework.testbed_model.traffic_generator.traffic_generator.rst
 create mode 100644 doc/api/dts/framework.testbed_model.virtual_device.rst
 create mode 100644 doc/api/dts/framework.utils.rst
 create mode 100644 doc/api/dts/index.rst
 create mode 100644 doc/api/dts/meson.build

-- 
2.34.1