Re: [PATCH v7 3/3] doc: revise coding guidelines section

[Patrick Robb <probb@iol.unh.edu>]

[-- Attachment #1: Type: text/plain, Size: 7816 bytes --]

On Wed, Jul 16, 2025 at 4:54 PM Dean Marx <dmarx@iol.unh.edu> wrote:

> code.
>
>
>  How To Write a Test Suite
>  -------------------------
>
> -All test suites inherit from ``TestSuite`` defined in
> ``dts/framework/test_suite.py``.
> -There are four types of methods that comprise a test suite:
> +All test suites are classes that inherit from TestSuite, defined in
> dts/framework/test_suite.py. A typical suite contains:
> +
> +Test Cases
> +
> +   Test cases are defined as methods and must be decorated appropriately.
> +   Use the @func_test and/or @perf_test decorators from TestSuite above
> each test case method.
> +   For example:
> +
> +Example::
> +
> +   @func_test
> +   def test_basic_link(self):
> +      """your testcase docstring here"""
> +      # your testcase code here
>

This looks good except for the fact that the whole example block should be
indented once. Otherwise it's not actually under "Test Cases," and also it
consumes the text blurb immediately following the example.

I just did a git fixup which indents it and that looks good after reviewing
the guides docs build output file. I'm pushing to my GHA robot fork to make
sure it still passes the checks but unless there are any fails I will just
push to next-dts with my fixup.


>
> -#. **Test cases**
> +   Functional test cases should use the @func_test decorator, and
> performance test cases should use @perf_test.
> +   A test suite may include any number of functional and/or performance
> test cases.
> +   Each suite should focus on testing a single feature (one feature = one
> test suite).
>
> -   | Test cases are methods that start with a particular prefix.
> -   | Functional test cases start with ``test_``, e.g.
> ``test_hello_world_single_core``.
> -   | Performance test cases start with ``test_perf_``, e.g.
> ``test_perf_nic_single_core``.
> -   | A test suite may have any number of functional and/or performance
> test cases.
> -     However, these test cases must test the same feature,
> -     following the rule of one feature = one test suite.
> -     Test cases for one feature don't need to be grouped in just one test
> suite, though.
> -     If the feature requires many testing scenarios to cover,
> -     the test cases would be better off spread over multiple test suites
> -     so that each test suite doesn't take too long to execute.
> +Setup and Teardown Hooks
>
> -#. **Setup and Teardown methods**
> +   Setup and teardown methods can be defined at both the suite and test
> case levels.
>
> -   | There are setup and teardown methods for the whole test suite and
> each individual test case.
> -   | Methods ``set_up_suite`` and ``tear_down_suite`` will be executed
> -     before any and after all test cases have been executed, respectively.
> -   | Methods ``set_up_test_case`` and ``tear_down_test_case`` will be
> executed
> -     before and after each test case, respectively.
> -   | These methods don't need to be implemented if there's no need for
> them in a test suite.
> -     In that case, nothing will happen when they are executed.
> +   Suite-level:
>
> -#. **Configuration, traffic and other logic**
> +   * set_up_suite() — runs once before any test cases in the suite
>
> -   The ``TestSuite`` class contains a variety of methods for anything that
> -   a test suite setup, a teardown, or a test case may need to do.
> +   * tear_down_suite() — runs once after all test cases have completed
>
> -   The test suites also frequently use a DPDK app, such as testpmd, in
> interactive mode
> -   and use the interactive shell instances directly.
> +   Case-level:
>
> -   These are the two main ways to call the framework logic in test suites.
> -   If there's any functionality or logic missing from the framework,
> -   it should be implemented so that the test suites can use one of these
> two ways.
> +   * set_up_test_case() — runs before each individual test case
>
> -   Test suites may also be configured individually using a file provided
> at the command line.
> -   The file is a simple mapping of test suite names to their
> corresponding configurations.
> +   * tear_down_test_case() — runs after each individual test case
>
> -   Any test suite can be designed to require custom configuration
> attributes or optional ones.
> -   Any optional attributes should supply a default value for the test
> suite to use.
> +   These methods are optional. If not implemented, the framework will
> simply skip them.
>
> -#. **Test case verification**
> +   The TestSuite class provides a variety of methods for setup, teardown,
> and test logic.
> +   Test suites often use DPDK applications (e.g., testpmd) in interactive
> mode and interact with them via shell instances.
>
> -   Test case verification should be done with the ``verify`` method,
> which records the result.
> -   The method should be called at the end of each test case.
> +Leveraging the DTS framework in writing testsuites:
>
> -#. **Other methods**
> +   One should avoid directly importing DTS framework code to their
> +   testsuites where possible. Instead, for performing common processes
> +   required in testsuites, one should use (or add to) the list of methods
> +   provided in the Testsuite class (the base class of all testsuites). For
> +   instance, for sending a list of packets, one should work through the
> packet
> +   transmitting function already made available in the TestSuite class,
> +   instead of directly importing the DTS traffic generator class and using
> +   that class in one's testsuite implementation. It is also acceptable to
> +   import and instantiate classes for various DPDK applications. For
> instance,
> +   writing a testsuite for a simple packet forwarding operation would
> involve
> +   importing the DTS TestPmdShell class, instantiating TestPmdShell,
> calling
> +   TestPmdShell's start() method, and then sending traffic via one of the
> +   traffic transmitting functions exposed in the Testsuite class.
>
> -   Of course, all test suite code should adhere to coding standards.
> -   Only the above methods will be treated specially and any other methods
> may be defined
> -   (which should be mostly private methods needed by each particular test
> suite).
> -   Any specific features (such as NIC configuration) required by a test
> suite
> -   should be implemented in the ``SutNode`` class (and the underlying
> classes that ``SutNode`` uses)
> -   and used by the test suite via the ``sut_node`` field.
> +Test Case Verification
> +
> +   Use the verify method to assert conditions and record test results.
> +   This should typically be called at the end of each test case.
> +   Example: self.verify(link_up, "Link should be up after configuration.")
> +
> +Other Methods
> +
> +   All test suite code should follow the project's coding standards.
> +   Only test cases, setup/teardown hooks, and verification methods are
> treated specially by the framework.
> +   Additional private methods may be added as needed in your testsuite
> implementation.
>
>
>  .. _dts_dev_tools:
> @@ -493,13 +491,10 @@ Building DTS API docs
>  The documentation is built using the standard DPDK build system.
>  See :doc:`../linux_gsg/build_dpdk` for more details on compiling DPDK
> with meson.
>
> -The :ref:`doc build dependencies <doc_dependencies>` may be installed
> with Poetry:
> -
>  .. code-block:: console
>
>     poetry install --only docs
>     poetry install --with docs  # an alternative that will also install
> DTS dependencies
> -   poetry shell
>
>  After executing the meson command, build the documentation with:
>
> --
> 2.50.1
>
>

[-- Attachment #2: Type: text/html, Size: 8773 bytes --]