From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id D12C445833; Wed, 21 Aug 2024 17:02:58 +0200 (CEST) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 5B36B41141; Wed, 21 Aug 2024 17:02:58 +0200 (CEST) Received: from mail-ej1-f46.google.com (mail-ej1-f46.google.com [209.85.218.46]) by mails.dpdk.org (Postfix) with ESMTP id AD362410FA for ; Wed, 21 Aug 2024 17:02:56 +0200 (CEST) Received: by mail-ej1-f46.google.com with SMTP id a640c23a62f3a-a7a8a4f21aeso831056966b.2 for ; Wed, 21 Aug 2024 08:02:56 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=pantheon.tech; s=google; t=1724252576; x=1724857376; darn=dpdk.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=e3yjstboNtwYIv1QxWZo1ly7Be4JJFA6fSsoPYpYEGE=; b=PI/oQ+cart9Oq+mZ0NeQe5owSVrUuRFNiW8I8OMX2lzICRJ2N11zucDxGlAPT3tLVe ObGPNI91FIByZEgV5xOAxrz11mnhtmgqrdObuqKaT/bZHv+XKYRmuUz/7BD5P/9WYiD3 ut/E+9EsienQu46B7hCP9d9ypQcTFoWZ48gzgki94k9BCX5hKULenXVsup2uVkazI18c ZdMbenhAKq57k6clKCwpYSa+5A6BebRI5j3AjPvjzV/xpTtzsQQmkEVTwFq6m1+XBXnq tEiDHZV0GPHUWrC9eZaaLMhPkoBPIIJM+NYruRDm4QFLcel+QdxWrpVFqrQ0fVsaEE/v EK6g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1724252576; x=1724857376; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=e3yjstboNtwYIv1QxWZo1ly7Be4JJFA6fSsoPYpYEGE=; b=YWQD8obIq9mpJrLBt/ywOP24knrEF6d5PK4075s3YlD6pGKgCZhom5Lkysx+GrPeqJ XOGgyaf+hXEU4tEY5IOLQxXFiHdtT4h4bCLMN6erJnnHV2ozw3ks6qDFyOuSM+UbnFs8 b09aleG0Y4hZQAmt9paN/RaRblYZGlpv3QDpQJorAcgzeUv4jrXzRd0qH0H7FNqjSADI 6Ptefcz9TEi1j+ZetIYJi/QWu6zyEKVMQYHVgQ2ReruyG8odqMqsC/Bc7+p40BHHEE/b rzs+NMk24c7XE58rYaTvgOLS6okp6yeGg+ELKsuSOPkDKNxN1DXHmeUI9Ci4xjhW+XP9 NeBw== X-Gm-Message-State: AOJu0Yw7ud69VoGODqXLobUsPXo28KQXp/PW1Nu/YwtxbJhPWy1jGh3f 0WVgZsCcJtFRW01b3iJbr9+NJ/dlEZLH2kcuqIG4ya34SXdxfDbGIeTnqDthsrc= X-Google-Smtp-Source: AGHT+IGHpz04bEPp0knynH+yN6Wf2BFDs/ik/2g7mnBwY0oxfY/n/4c+mEDXRv/lDT/LdskXi8txVQ== X-Received: by 2002:a17:907:f168:b0:a86:859a:ed65 with SMTP id a640c23a62f3a-a86859aeea9mr54372766b.56.1724252575897; Wed, 21 Aug 2024 08:02:55 -0700 (PDT) Received: from localhost.localdomain ([84.245.121.236]) by smtp.gmail.com with ESMTPSA id a640c23a62f3a-a83838cef1esm914030766b.48.2024.08.21.08.02.54 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 21 Aug 2024 08:02:55 -0700 (PDT) From: =?UTF-8?q?Juraj=20Linke=C5=A1?= To: thomas@monjalon.net, Honnappa.Nagarahalli@arm.com, jspewock@iol.unh.edu, probb@iol.unh.edu, paul.szczepanek@arm.com, Luca.Vizzarro@arm.com, npratte@iol.unh.edu, dmarx@iol.unh.edu, alex.chapman@arm.com Cc: dev@dpdk.org, =?UTF-8?q?Juraj=20Linke=C5=A1?= Subject: [PATCH v19 0/5] DTS API docs generation Date: Wed, 21 Aug 2024 17:02:49 +0200 Message-Id: <20240821150254.158912-1-juraj.linkes@pantheon.tech> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20231115133606.42081-1-juraj.linkes@pantheon.tech> References: <20231115133606.42081-1-juraj.linkes@pantheon.tech> MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org The generation is done with Sphinx, which DPDK already uses, with slightly modified configuration of the sidebar present in an if block. DTS dependencies do not need to be installed, but there is the option to install doc build dependencies with Poetry: poetry install --with docs The build itself may be run with: meson setup -Denable_docs=true ninja -C The above will do a full DPDK build with docs. To build just docs: meson setup ninja -C doc Python3.10 is required to build the DTS API docs. The patchset contains the .rst sources which Sphinx uses to generate the html pages. These were first generated with the sphinx-apidoc utility and modified to provide a better look. The documentation just doesn't look that good without the modifications and there isn't enough configuration options to achieve that without manual changes to the .rst files. This introduces extra maintenance which involves adding new .rst files when a new Python module is added or changing the .rst structure if the Python directory/file structure is changed (moved, renamed files). This small maintenance burden is outweighed by the flexibility afforded by the ability to make manual changes to the .rst files. v10: Fix dts doc generation issue: Only copy the custom rss file if it exists. v11: Added the config option autodoc_mock_imports, which eliminates the need for DTS dependencies. Added a script that find out which imports need to be added to autodoc_mock_imports. The script also check the required Python version for building DTS docs. Removed tags from the two affected patches which will need to be reviewed again. v12: Added paramiko to the required dependencies of get-dts-deps.py. v13: Fixed build error: TypeError: unsupported operand type(s) for |: 'NoneType' and 'Transport' v14: Fixed install error: ERROR: File 'dts/doc/html' could not be found This required me to put the built docs into dts/doc which is outside the DPDK API doc dir, resulting in linking between DPDK and DTS api docs not working properly. I addressed this by adding a symlink to the build dir. This way the link works after installing the docs and the symlink is just one extra file in the build dir. v15: Moved DTS API sources to doc/api/dts. This simplifies a lot of things in the build, but mainly makes a lot of sense. Now the source, build and install paths are the same so there isn't any need for any symlinks or other workarounds. Also added a symlink to the custom.css file so that it works with call-sphinx-build.py without any modifications. v16: Renamed the dependency python file to get-dts-runtime-deps.py a modified it to only get runtime dependencies. We don't need to check docs dependencies (Sphinx) as we don't need to mock those. Also moved all new Sphinx configuration into the DTS if branch to make sure it won't ever affect the DPDK doc build. v17: Removed the dts-doc build target to mirror the functionality of using -Denable_docs=true. Moved DTS-specific meson build code to doc/api/dts/meson.build. Added comments to get_missing_imports() and the top level docstring of get-dts-runtime-deps.py to explain the function is there to be imported. v18: Added PyYAML to get-dts-runtime-deps.py. v19: Fixed a regression in get-dts-runtime-deps.py introduced in v18: AttributeError: 'dict' object has no attribute 'strip' Juraj Linkeš (5): dts: update params and parser docstrings dts: replace the or operator in third party types dts: add doc generation dependencies dts: add API doc sources dts: add API doc generation buildtools/call-sphinx-build.py | 2 + buildtools/get-dts-runtime-deps.py | 95 ++++ buildtools/meson.build | 1 + doc/api/doxy-api-index.md | 3 + doc/api/doxy-api.conf.in | 2 + doc/api/dts/conf_yaml_schema.json | 1 + doc/api/dts/custom.css | 1 + doc/api/dts/framework.config.rst | 12 + doc/api/dts/framework.config.types.rst | 6 + doc/api/dts/framework.exception.rst | 6 + doc/api/dts/framework.logger.rst | 6 + doc/api/dts/framework.params.eal.rst | 6 + doc/api/dts/framework.params.rst | 14 + doc/api/dts/framework.params.testpmd.rst | 6 + doc/api/dts/framework.params.types.rst | 6 + doc/api/dts/framework.parser.rst | 6 + .../framework.remote_session.dpdk_shell.rst | 6 + ...ote_session.interactive_remote_session.rst | 6 + ...ework.remote_session.interactive_shell.rst | 6 + .../framework.remote_session.python_shell.rst | 6 + ...ramework.remote_session.remote_session.rst | 6 + doc/api/dts/framework.remote_session.rst | 18 + .../framework.remote_session.ssh_session.rst | 6 + ...framework.remote_session.testpmd_shell.rst | 6 + doc/api/dts/framework.runner.rst | 6 + doc/api/dts/framework.settings.rst | 6 + doc/api/dts/framework.test_result.rst | 6 + doc/api/dts/framework.test_suite.rst | 6 + doc/api/dts/framework.testbed_model.cpu.rst | 6 + .../framework.testbed_model.linux_session.rst | 6 + doc/api/dts/framework.testbed_model.node.rst | 6 + .../framework.testbed_model.os_session.rst | 6 + doc/api/dts/framework.testbed_model.port.rst | 6 + .../framework.testbed_model.posix_session.rst | 6 + doc/api/dts/framework.testbed_model.rst | 26 + .../dts/framework.testbed_model.sut_node.rst | 6 + .../dts/framework.testbed_model.tg_node.rst | 6 + ..._generator.capturing_traffic_generator.rst | 6 + ...mework.testbed_model.traffic_generator.rst | 14 + ....testbed_model.traffic_generator.scapy.rst | 6 + ...el.traffic_generator.traffic_generator.rst | 6 + ...framework.testbed_model.virtual_device.rst | 6 + doc/api/dts/framework.utils.rst | 6 + doc/api/dts/index.rst | 43 ++ doc/api/dts/meson.build | 31 ++ doc/api/meson.build | 6 +- doc/guides/conf.py | 44 +- doc/guides/contributing/documentation.rst | 2 + doc/guides/contributing/patches.rst | 4 + doc/guides/tools/dts.rst | 39 +- doc/meson.build | 1 + dts/framework/params/__init__.py | 4 +- dts/framework/params/eal.py | 7 +- dts/framework/params/types.py | 3 +- dts/framework/parser.py | 4 +- .../interactive_remote_session.py | 3 +- dts/poetry.lock | 521 +++++++++++++++++- dts/pyproject.toml | 8 + 58 files changed, 1072 insertions(+), 23 deletions(-) create mode 100755 buildtools/get-dts-runtime-deps.py create mode 120000 doc/api/dts/conf_yaml_schema.json create mode 120000 doc/api/dts/custom.css create mode 100644 doc/api/dts/framework.config.rst create mode 100644 doc/api/dts/framework.config.types.rst create mode 100644 doc/api/dts/framework.exception.rst create mode 100644 doc/api/dts/framework.logger.rst create mode 100644 doc/api/dts/framework.params.eal.rst create mode 100644 doc/api/dts/framework.params.rst create mode 100644 doc/api/dts/framework.params.testpmd.rst create mode 100644 doc/api/dts/framework.params.types.rst create mode 100644 doc/api/dts/framework.parser.rst create mode 100644 doc/api/dts/framework.remote_session.dpdk_shell.rst create mode 100644 doc/api/dts/framework.remote_session.interactive_remote_session.rst create mode 100644 doc/api/dts/framework.remote_session.interactive_shell.rst create mode 100644 doc/api/dts/framework.remote_session.python_shell.rst create mode 100644 doc/api/dts/framework.remote_session.remote_session.rst create mode 100644 doc/api/dts/framework.remote_session.rst create mode 100644 doc/api/dts/framework.remote_session.ssh_session.rst create mode 100644 doc/api/dts/framework.remote_session.testpmd_shell.rst create mode 100644 doc/api/dts/framework.runner.rst create mode 100644 doc/api/dts/framework.settings.rst create mode 100644 doc/api/dts/framework.test_result.rst create mode 100644 doc/api/dts/framework.test_suite.rst create mode 100644 doc/api/dts/framework.testbed_model.cpu.rst create mode 100644 doc/api/dts/framework.testbed_model.linux_session.rst create mode 100644 doc/api/dts/framework.testbed_model.node.rst create mode 100644 doc/api/dts/framework.testbed_model.os_session.rst create mode 100644 doc/api/dts/framework.testbed_model.port.rst create mode 100644 doc/api/dts/framework.testbed_model.posix_session.rst create mode 100644 doc/api/dts/framework.testbed_model.rst create mode 100644 doc/api/dts/framework.testbed_model.sut_node.rst create mode 100644 doc/api/dts/framework.testbed_model.tg_node.rst create mode 100644 doc/api/dts/framework.testbed_model.traffic_generator.capturing_traffic_generator.rst create mode 100644 doc/api/dts/framework.testbed_model.traffic_generator.rst create mode 100644 doc/api/dts/framework.testbed_model.traffic_generator.scapy.rst create mode 100644 doc/api/dts/framework.testbed_model.traffic_generator.traffic_generator.rst create mode 100644 doc/api/dts/framework.testbed_model.virtual_device.rst create mode 100644 doc/api/dts/framework.utils.rst create mode 100644 doc/api/dts/index.rst create mode 100644 doc/api/dts/meson.build -- 2.34.1