From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: <dev-bounces@dpdk.org> Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 7D66A459C6; Wed, 18 Sep 2024 09:38:42 +0200 (CEST) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 69D1F42E72; Wed, 18 Sep 2024 09:38:42 +0200 (CEST) Received: from mail-ej1-f45.google.com (mail-ej1-f45.google.com [209.85.218.45]) by mails.dpdk.org (Postfix) with ESMTP id 34DF542E67 for <dev@dpdk.org>; Wed, 18 Sep 2024 09:38:41 +0200 (CEST) Received: by mail-ej1-f45.google.com with SMTP id a640c23a62f3a-a8b155b5e9eso904443566b.1 for <dev@dpdk.org>; Wed, 18 Sep 2024 00:38:41 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=pantheon.tech; s=google; t=1726645121; x=1727249921; darn=dpdk.org; h=content-transfer-encoding:in-reply-to:from:content-language :references:cc:to:subject:user-agent:mime-version:date:message-id :from:to:cc:subject:date:message-id:reply-to; bh=oWYyUv4L6vvGk+r7pU7jRASiOs1mEcOeq61/LPzo17A=; b=nwloa3z3tZHnPu6S69rdtV8FjW0/NhSKXftQKmgSm7Oy20QKr9lt1yPhuY93C7ihSp XrKMpC3HiBAGqKiyl/OBUbOQ1SJ0svwX0l77iTPhqqUaaDGNdLPWLhS8xnVB4nV/BJap JbydkMDdvi9bYnP1cv5UhzrBkIGwwsc3vCGXIaI6Nws030Uu1lMLzN2COYnSj+4XTOsJ MLSZwma+Y3G0QoIfG/rtFYImQ9YkkvzjYWOBS5HM0Rh+/7PlDjkVFnDnKpj7tz7JlR+7 ml/O6LmQFTeswq64KqLh/eZWwp1JJz24QBlCY37DvR4g/4PiI/18nz9k17lYAnTyZxH+ FYUA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1726645121; x=1727249921; h=content-transfer-encoding:in-reply-to:from:content-language :references:cc:to:subject:user-agent:mime-version:date:message-id :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=oWYyUv4L6vvGk+r7pU7jRASiOs1mEcOeq61/LPzo17A=; b=dxjC+YrDJTR08bbvr3vTqReriMvpdJNHtmdeAlU89QG7qv3dAzbt3RpTPFDFMZHuQ6 vhE/8PtfgPJQFCfObONggDL+0VGgmOhw7BDtQBt5K/lXL/zj+UNrkaJY9TBBXTYMYWrM GHAbsrRbNOoTax9YtCUefgh44bdFlYZNSDiBOKWQRzvnbdeW9S0ialji0cl1qqGG7a3y 6BCZxhjOD6hj5/EL/fKHGUGyuc1Xi6y0MhcXonySEA64jNXzVSLKLBIrv79+5n1qXWEG sLZUkwWsTzqtd/QzH/E3YNuez5EG6BgWsNZv6wwbp2dv5Z4Yw7j6TmQbYpRTZ5ZIRfue gtPQ== X-Gm-Message-State: AOJu0YwwdN2Z9jJc7Jo+233vP0d1BvK156XzlkP3g4FEQ0EE6+PcnHHs gYvxP/UchTfQy1HqnMcK4VL4e8BS+341WH+YJ9XgqO/Sw3JLdDVNu+yv4f9kF8M= X-Google-Smtp-Source: AGHT+IFeZUzpoF4tV/J7uLXaatsPVQylIJRLYmpMZd/Hz8U52wvloPyWr/Wm6kJBEE6+svsru+q+dg== X-Received: by 2002:a17:907:e8d:b0:a86:80ef:4fe5 with SMTP id a640c23a62f3a-a9029617930mr2194037466b.47.1726645120447; Wed, 18 Sep 2024 00:38:40 -0700 (PDT) Received: from [192.168.200.22] ([84.245.121.62]) by smtp.gmail.com with ESMTPSA id a640c23a62f3a-a90610968c4sm548693866b.4.2024.09.18.00.38.39 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Wed, 18 Sep 2024 00:38:40 -0700 (PDT) Message-ID: <0666a68e-3a4d-43f2-a5b3-7ec7a53cd3f5@pantheon.tech> Date: Wed, 18 Sep 2024 09:38:38 +0200 MIME-Version: 1.0 User-Agent: Mozilla Thunderbird Subject: Re: [PATCH v19 0/5] DTS API docs generation To: thomas@monjalon.net, Honnappa.Nagarahalli@arm.com, jspewock@iol.unh.edu, probb@iol.unh.edu, paul.szczepanek@arm.com, Luca.Vizzarro@arm.com, npratte@iol.unh.edu, dmarx@iol.unh.edu, alex.chapman@arm.com Cc: dev@dpdk.org References: <20231115133606.42081-1-juraj.linkes@pantheon.tech> <20240821150254.158912-1-juraj.linkes@pantheon.tech> Content-Language: en-US From: =?UTF-8?Q?Juraj_Linke=C5=A1?= <juraj.linkes@pantheon.tech> In-Reply-To: <20240821150254.158912-1-juraj.linkes@pantheon.tech> Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 8bit X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions <dev.dpdk.org> List-Unsubscribe: <https://mails.dpdk.org/options/dev>, <mailto:dev-request@dpdk.org?subject=unsubscribe> List-Archive: <http://mails.dpdk.org/archives/dev/> List-Post: <mailto:dev@dpdk.org> List-Help: <mailto:dev-request@dpdk.org?subject=help> List-Subscribe: <https://mails.dpdk.org/listinfo/dev>, <mailto:dev-request@dpdk.org?subject=subscribe> Errors-To: dev-bounces@dpdk.org On 21. 8. 2024 17:02, Juraj Linkeš wrote: > The generation is done with Sphinx, which DPDK already uses, with > slightly modified configuration of the sidebar present in an if block. > > DTS dependencies do not need to be installed, but there is the option to > install doc build dependencies with Poetry: > poetry install --with docs > > The build itself may be run with: > meson setup <meson_build_dir> -Denable_docs=true > ninja -C <meson_build_dir> > > The above will do a full DPDK build with docs. To build just docs: > meson setup <meson_build_dir> > ninja -C <meson_build_dir> doc > > Python3.10 is required to build the DTS API docs. > > The patchset contains the .rst sources which Sphinx uses to generate the > html pages. These were first generated with the sphinx-apidoc utility > and modified to provide a better look. The documentation just doesn't > look that good without the modifications and there isn't enough > configuration options to achieve that without manual changes to the .rst > files. This introduces extra maintenance which involves adding new .rst > files when a new Python module is added or changing the .rst structure > if the Python directory/file structure is changed (moved, renamed > files). This small maintenance burden is outweighed by the flexibility > afforded by the ability to make manual changes to the .rst files. > > v10: > Fix dts doc generation issue: Only copy the custom rss file if it exists. > > v11: > Added the config option autodoc_mock_imports, which eliminates the need > for DTS dependencies. Added a script that find out which imports need to > be added to autodoc_mock_imports. The script also check the required > Python version for building DTS docs. > Removed tags from the two affected patches which will need to be > reviewed again. > > v12: > Added paramiko to the required dependencies of get-dts-deps.py. > > v13: > Fixed build error: > TypeError: unsupported operand type(s) for |: 'NoneType' and 'Transport' > > v14: > Fixed install error: > ERROR: File 'dts/doc/html' could not be found > This required me to put the built docs into dts/doc which is outside the > DPDK API doc dir, resulting in linking between DPDK and DTS api docs not > working properly. I addressed this by adding a symlink to the build dir. > This way the link works after installing the docs and the symlink is > just one extra file in the build dir. > > v15: > Moved DTS API sources to doc/api/dts. This simplifies a lot of things in > the build, but mainly makes a lot of sense. Now the source, build and > install paths are the same so there isn't any need for any symlinks or > other workarounds. > Also added a symlink to the custom.css file so that it works with > call-sphinx-build.py without any modifications. > > v16: > Renamed the dependency python file to get-dts-runtime-deps.py a modified > it to only get runtime dependencies. We don't need to check docs > dependencies (Sphinx) as we don't need to mock those. > Also moved all new Sphinx configuration into the DTS if branch to make > sure it won't ever affect the DPDK doc build. > > v17: > Removed the dts-doc build target to mirror the functionality of using > -Denable_docs=true. > Moved DTS-specific meson build code to doc/api/dts/meson.build. > Added comments to get_missing_imports() and the top level docstring of > get-dts-runtime-deps.py to explain the function is there to be imported. > > v18: > Added PyYAML to get-dts-runtime-deps.py. > > v19: > Fixed a regression in get-dts-runtime-deps.py introduced in v18: > AttributeError: 'dict' object has no attribute 'strip' > > Juraj Linkeš (5): > dts: update params and parser docstrings > dts: replace the or operator in third party types > dts: add doc generation dependencies > dts: add API doc sources > dts: add API doc generation > > buildtools/call-sphinx-build.py | 2 + > buildtools/get-dts-runtime-deps.py | 95 ++++ > buildtools/meson.build | 1 + > doc/api/doxy-api-index.md | 3 + > doc/api/doxy-api.conf.in | 2 + > doc/api/dts/conf_yaml_schema.json | 1 + > doc/api/dts/custom.css | 1 + > doc/api/dts/framework.config.rst | 12 + > doc/api/dts/framework.config.types.rst | 6 + > doc/api/dts/framework.exception.rst | 6 + > doc/api/dts/framework.logger.rst | 6 + > doc/api/dts/framework.params.eal.rst | 6 + > doc/api/dts/framework.params.rst | 14 + > doc/api/dts/framework.params.testpmd.rst | 6 + > doc/api/dts/framework.params.types.rst | 6 + > doc/api/dts/framework.parser.rst | 6 + > .../framework.remote_session.dpdk_shell.rst | 6 + > ...ote_session.interactive_remote_session.rst | 6 + > ...ework.remote_session.interactive_shell.rst | 6 + > .../framework.remote_session.python_shell.rst | 6 + > ...ramework.remote_session.remote_session.rst | 6 + > doc/api/dts/framework.remote_session.rst | 18 + > .../framework.remote_session.ssh_session.rst | 6 + > ...framework.remote_session.testpmd_shell.rst | 6 + > doc/api/dts/framework.runner.rst | 6 + > doc/api/dts/framework.settings.rst | 6 + > doc/api/dts/framework.test_result.rst | 6 + > doc/api/dts/framework.test_suite.rst | 6 + > doc/api/dts/framework.testbed_model.cpu.rst | 6 + > .../framework.testbed_model.linux_session.rst | 6 + > doc/api/dts/framework.testbed_model.node.rst | 6 + > .../framework.testbed_model.os_session.rst | 6 + > doc/api/dts/framework.testbed_model.port.rst | 6 + > .../framework.testbed_model.posix_session.rst | 6 + > doc/api/dts/framework.testbed_model.rst | 26 + > .../dts/framework.testbed_model.sut_node.rst | 6 + > .../dts/framework.testbed_model.tg_node.rst | 6 + > ..._generator.capturing_traffic_generator.rst | 6 + > ...mework.testbed_model.traffic_generator.rst | 14 + > ....testbed_model.traffic_generator.scapy.rst | 6 + > ...el.traffic_generator.traffic_generator.rst | 6 + > ...framework.testbed_model.virtual_device.rst | 6 + > doc/api/dts/framework.utils.rst | 6 + > doc/api/dts/index.rst | 43 ++ > doc/api/dts/meson.build | 31 ++ > doc/api/meson.build | 6 +- > doc/guides/conf.py | 44 +- > doc/guides/contributing/documentation.rst | 2 + > doc/guides/contributing/patches.rst | 4 + > doc/guides/tools/dts.rst | 39 +- > doc/meson.build | 1 + > dts/framework/params/__init__.py | 4 +- > dts/framework/params/eal.py | 7 +- > dts/framework/params/types.py | 3 +- > dts/framework/parser.py | 4 +- > .../interactive_remote_session.py | 3 +- > dts/poetry.lock | 521 +++++++++++++++++- > dts/pyproject.toml | 8 + > 58 files changed, 1072 insertions(+), 23 deletions(-) > create mode 100755 buildtools/get-dts-runtime-deps.py > create mode 120000 doc/api/dts/conf_yaml_schema.json > create mode 120000 doc/api/dts/custom.css > create mode 100644 doc/api/dts/framework.config.rst > create mode 100644 doc/api/dts/framework.config.types.rst > create mode 100644 doc/api/dts/framework.exception.rst > create mode 100644 doc/api/dts/framework.logger.rst > create mode 100644 doc/api/dts/framework.params.eal.rst > create mode 100644 doc/api/dts/framework.params.rst > create mode 100644 doc/api/dts/framework.params.testpmd.rst > create mode 100644 doc/api/dts/framework.params.types.rst > create mode 100644 doc/api/dts/framework.parser.rst > create mode 100644 doc/api/dts/framework.remote_session.dpdk_shell.rst > create mode 100644 doc/api/dts/framework.remote_session.interactive_remote_session.rst > create mode 100644 doc/api/dts/framework.remote_session.interactive_shell.rst > create mode 100644 doc/api/dts/framework.remote_session.python_shell.rst > create mode 100644 doc/api/dts/framework.remote_session.remote_session.rst > create mode 100644 doc/api/dts/framework.remote_session.rst > create mode 100644 doc/api/dts/framework.remote_session.ssh_session.rst > create mode 100644 doc/api/dts/framework.remote_session.testpmd_shell.rst > create mode 100644 doc/api/dts/framework.runner.rst > create mode 100644 doc/api/dts/framework.settings.rst > create mode 100644 doc/api/dts/framework.test_result.rst > create mode 100644 doc/api/dts/framework.test_suite.rst > create mode 100644 doc/api/dts/framework.testbed_model.cpu.rst > create mode 100644 doc/api/dts/framework.testbed_model.linux_session.rst > create mode 100644 doc/api/dts/framework.testbed_model.node.rst > create mode 100644 doc/api/dts/framework.testbed_model.os_session.rst > create mode 100644 doc/api/dts/framework.testbed_model.port.rst > create mode 100644 doc/api/dts/framework.testbed_model.posix_session.rst > create mode 100644 doc/api/dts/framework.testbed_model.rst > create mode 100644 doc/api/dts/framework.testbed_model.sut_node.rst > create mode 100644 doc/api/dts/framework.testbed_model.tg_node.rst > create mode 100644 doc/api/dts/framework.testbed_model.traffic_generator.capturing_traffic_generator.rst > create mode 100644 doc/api/dts/framework.testbed_model.traffic_generator.rst > create mode 100644 doc/api/dts/framework.testbed_model.traffic_generator.scapy.rst > create mode 100644 doc/api/dts/framework.testbed_model.traffic_generator.traffic_generator.rst > create mode 100644 doc/api/dts/framework.testbed_model.virtual_device.rst > create mode 100644 doc/api/dts/framework.utils.rst > create mode 100644 doc/api/dts/index.rst > create mode 100644 doc/api/dts/meson.build > Applied to dpdk-next-dts with slight modifications suggested by Thomas in v19. Thanks everyone for reviews.