From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id A5BCC46130; Fri, 24 Jan 2025 19:19:01 +0100 (CET) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 7B7C0402CE; Fri, 24 Jan 2025 19:19:01 +0100 (CET) Received: from mail-lf1-f48.google.com (mail-lf1-f48.google.com [209.85.167.48]) by mails.dpdk.org (Postfix) with ESMTP id C402C4028A for ; Fri, 24 Jan 2025 19:18:59 +0100 (CET) Received: by mail-lf1-f48.google.com with SMTP id 2adb3069b0e04-53e36a1cf4fso360844e87.0 for ; Fri, 24 Jan 2025 10:18:59 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=iol.unh.edu; s=unh-iol; t=1737742739; x=1738347539; darn=dpdk.org; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:from:to:cc:subject:date :message-id:reply-to; bh=EkSpIKLxD90UGGFNCPPK3cqz/QF2HDCNVMLE3wLWRfk=; b=AtEZYxIc7SJxuhlE85JDXkj8ob5O5ykDexhkaCVmFgy3+rIWTLUWJyRNPJCwr68UPh dumT6K7Lpbs0luPuyGWin5alZNzTXiyH5nWeAjabu3+YHtZXKjBTJXdbP5mFwhG52Uuu HX6OQG6l5qJQcWO0wipqXY/c2bCGUuuGyySEo= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1737742739; x=1738347539; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=EkSpIKLxD90UGGFNCPPK3cqz/QF2HDCNVMLE3wLWRfk=; b=MmYRdDqrhVuyEh0ogB/5Yg+15OCsyhOgTx5vIosagZY0cCyOGE5r4ZX7QsHFuLBMyw IkvSOEqEg6JodU3hKvO4u6bd55D1Z+5AZTBQEpiiFsBo46AFOMd8Yq/U842IBwijbhcX uhul1xBy6qrHZG6egjtpJet+ors/TOUKLKv8/3p7ojJHREQb7FzTx/JUf++/d60ZFIrm P+QIZemW8M+VUxzbhl0haz6oxdDSqhOualwy5PIAVcATEtwXAP7JlEMV4iInF0fzi5I8 FRvC8ns32FrHAkQllomCy2+gWihB67ksoU+aPWYtvc7JrH1zHCVykSXgqkIcOasYkcvf C+bw== X-Gm-Message-State: AOJu0YyA82sDWGOq4+0VDxXapBlswZSV7qUWeRmypRs1vwl0s0oEDjI3 xJ12woXOfkX+/0z1wFM5CrL82NOeEpqO7m9KIJNNpyck1BbdWiDMVCIudldD+O1+yWJykhtVpT0 uxi6ZnuKWeM++r2zsknZ7hOcZRFtWPJ6VLYyLw8Pdmx9CLiUI X-Gm-Gg: ASbGncuvUjrZlh0GTwLiK7SDxK4XfwZjZP3xNiQr4fyNY1Ik0FtAME8fApKD3I3UVeF 7b9bzyNFgOv8AyfdagrWSJZ3PZIOqZvaQYGwUTCQ6q+GAo1B/9FhMfz/Ocbc7kdz3yZZSWVsse1 01yX7YEIKSPb/gHg2gG7rg X-Google-Smtp-Source: AGHT+IHXO0S5n8roXfZfMkfwpOCKhYhqm+cVMs6STaNEdmpOL6dCrjpJcWMQOHXdgZb3Ts5AlcP9PX5te+Xi20VpZ4k= X-Received: by 2002:a05:651c:1505:b0:300:1f36:8fea with SMTP id 38308e7fff4ca-3072cb1f622mr41959111fa.7.1737742738573; Fri, 24 Jan 2025 10:18:58 -0800 (PST) MIME-Version: 1.0 References: <20240613201831.9748-3-npratte@iol.unh.edu> <20250124113909.137128-1-luca.vizzarro@arm.com> <20250124113909.137128-7-luca.vizzarro@arm.com> In-Reply-To: <20250124113909.137128-7-luca.vizzarro@arm.com> From: Nicholas Pratte Date: Fri, 24 Jan 2025 13:18:47 -0500 X-Gm-Features: AWEUYZktoRex6SWdINKcCyCAXQvmr06eJ6qbT4j_iprrRfyghP6WyRnA7xXiD-g Message-ID: Subject: Re: [PATCH v4 6/7] dts: split configuration file To: Luca Vizzarro Cc: dev@dpdk.org, Paul Szczepanek , Dean Marx , Patrick Robb Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org This is great! Before Jeremy left, he suggested going a step further and putting the config in a directory of its own, potentially offering more flexibility. Something we could consider looking into in the future, if there is time. Reviewed-by: Nicholas Pratte On Fri, Jan 24, 2025 at 6:39=E2=80=AFAM Luca Vizzarro wrote: > > To avoid the creation of a big monolithic configuration file, nodes and > test runs are now split into distinct files. This also allows > flexibility to run different test runs on the same nodes. > > Since there are now 2 distinct configuration files, there are also 2 > command line arguments to specify them. > > Bugzilla ID: 1344 > > Signed-off-by: Nicholas Pratte > Signed-off-by: Luca Vizzarro > Reviewed-by: Paul Szczepanek > Reviewed-by: Dean Marx > --- > doc/guides/tools/dts.rst | 78 ++- > dts/.gitignore | 4 + > dts/conf.yaml | 84 --- > dts/framework/config/__init__.py | 526 ++---------------- > dts/framework/config/common.py | 59 ++ > dts/framework/config/node.py | 144 +++++ > dts/framework/config/test_run.py | 290 ++++++++++ > dts/framework/runner.py | 11 +- > dts/framework/settings.py | 37 +- > dts/framework/test_result.py | 2 +- > dts/framework/testbed_model/node.py | 6 +- > dts/framework/testbed_model/os_session.py | 2 +- > dts/framework/testbed_model/port.py | 2 +- > dts/framework/testbed_model/sut_node.py | 6 +- > dts/framework/testbed_model/tg_node.py | 2 +- > dts/framework/testbed_model/topology.py | 2 +- > .../traffic_generator/__init__.py | 2 +- > .../testbed_model/traffic_generator/scapy.py | 2 +- > .../traffic_generator/traffic_generator.py | 2 +- > dts/nodes.example.yaml | 53 ++ > dts/test_runs.example.yaml | 33 ++ > dts/tests/TestSuite_smoke_tests.py | 2 +- > 22 files changed, 729 insertions(+), 620 deletions(-) > create mode 100644 dts/.gitignore > delete mode 100644 dts/conf.yaml > create mode 100644 dts/framework/config/common.py > create mode 100644 dts/framework/config/node.py > create mode 100644 dts/framework/config/test_run.py > create mode 100644 dts/nodes.example.yaml > create mode 100644 dts/test_runs.example.yaml > > diff --git a/doc/guides/tools/dts.rst b/doc/guides/tools/dts.rst > index abc389b42a..6fc4eb8dac 100644 > --- a/doc/guides/tools/dts.rst > +++ b/doc/guides/tools/dts.rst > @@ -210,8 +210,10 @@ DTS configuration is split into nodes and test runs, > and must respect the model definitions > as documented in the DTS API docs under the ``config`` page. > The root of the configuration is represented by the ``Configuration`` mo= del. > -By default, DTS will try to use the ``dts/conf.yaml`` :ref:`config file = `, > -which is a template that illustrates what can be configured in DTS. > +By default, DTS will try to use the ``dts/test_runs.example.yaml`` > +:ref:`config file `, and ``dts/nodes.ex= ample.yaml`` > +:ref:`config file ` which are templates tha= t > +illustrate what can be configured in DTS. > > The user must have :ref:`administrator privileges ` > which don't require password authentication. > @@ -225,16 +227,19 @@ DTS is run with ``main.py`` located in the ``dts`` = directory after entering Poet > .. code-block:: console > > (dts-py3.10) $ ./main.py --help > - usage: main.py [-h] [--config-file FILE_PATH] [--output-dir DIR_PATH]= [-t SECONDS] [-v] [--dpdk-tree DIR_PATH | --tarball FILE_PATH] [--remote-s= ource] > - [--precompiled-build-dir DIR_NAME] [--compile-timeout = SECONDS] [--test-suite TEST_SUITE [TEST_CASES ...]] [--re-run N_TIMES] > - [--random-seed NUMBER] > + usage: main.py [-h] [--test-runs-config-file FILE_PATH] [--nodes-conf= ig-file FILE_PATH] [--output-dir DIR_PATH] [-t SECONDS] [-v] > + [--dpdk-tree DIR_PATH | --tarball FILE_PATH] [--remote= -source] [--precompiled-build-dir DIR_NAME] > + [--compile-timeout SECONDS] [--test-suite TEST_SUITE [= TEST_CASES ...]] [--re-run N_TIMES] [--random-seed NUMBER] > > - Run DPDK test suites. All options may be specified with the environme= nt variables provided in brackets. Command line arguments have higher prior= ity. > + Run DPDK test suites. All options may be specified with the environme= nt variables provided in brackets. Command line arguments have higher > + priority. > > options: > -h, --help show this help message and exit > - --config-file FILE_PATH > - [DTS_CFG_FILE] The configuration file that de= scribes the test cases, SUTs and DPDK build configs. (default: conf.yaml) > + --test-runs-config-file FILE_PATH > + [DTS_TEST_RUNS_CFG_FILE] The configuration fi= le that describes the test cases and DPDK build options. (default: test-run= s.conf.yaml) > + --nodes-config-file FILE_PATH > + [DTS_NODES_CFG_FILE] The configuration file t= hat describes the SUT and TG nodes. (default: nodes.conf.yaml) > --output-dir DIR_PATH, --output DIR_PATH > [DTS_OUTPUT_DIR] Output directory where DTS l= ogs and results are saved. (default: output) > -t SECONDS, --timeout SECONDS > @@ -243,31 +248,31 @@ DTS is run with ``main.py`` located in the ``dts`` = directory after entering Poet > --compile-timeout SECONDS > [DTS_COMPILE_TIMEOUT] The timeout for compili= ng DPDK. (default: 1200) > --test-suite TEST_SUITE [TEST_CASES ...] > - [DTS_TEST_SUITES] A list containing a test su= ite with test cases. The first parameter is the test suite name, and the re= st are > - test case names, which are optional. May be s= pecified multiple times. To specify multiple test suites in the environment > - variable, join the lists with a comma. Exampl= es: --test-suite suite case case --test-suite suite case ... | > - DTS_TEST_SUITES=3D'suite case case, suite cas= e, ...' | --test-suite suite --test-suite suite case ... | DTS_TEST_SUITES= =3D'suite, > - suite case, ...' (default: []) > + [DTS_TEST_SUITES] A list containing a test su= ite with test cases. The first parameter is the test suite name, and > + the rest are test case names, which are optio= nal. May be specified multiple times. To specify multiple test suites > + in the environment variable, join the lists w= ith a comma. Examples: --test-suite suite case case --test-suite > + suite case ... | DTS_TEST_SUITES=3D'suite cas= e case, suite case, ...' | --test-suite suite --test-suite suite case > + ... | DTS_TEST_SUITES=3D'suite, suite case, .= ..' (default: []) > --re-run N_TIMES, --re_run N_TIMES > [DTS_RERUN] Re-run each test case the specifi= ed number of times if a test failure occurs. (default: 0) > - --random-seed NUMBER [DTS_RANDOM_SEED] The seed to use with the ps= eudo-random generator. If not specified, the configuration value is used in= stead. > - If that's also not specified, a random seed i= s generated. (default: None) > + --random-seed NUMBER [DTS_RANDOM_SEED] The seed to use with the ps= eudo-random generator. If not specified, the configuration value is > + used instead. If that's also not specified, a= random seed is generated. (default: None) > > DPDK Build Options: > - Arguments in this group (and subgroup) will be applied to a DPDKLoc= ation when the DPDK tree, tarball or revision will be provided, other argum= ents > - like remote source and build dir are optional. A DPDKLocation from = settings are used instead of from config if construct successful. > + Arguments in this group (and subgroup) will be applied to a DPDKLoc= ation when the DPDK tree, tarball or revision will be provided, > + other arguments like remote source and build dir are optional. A DP= DKLocation from settings are used instead of from config if > + construct successful. > > - --dpdk-tree DIR_PATH [DTS_DPDK_TREE] The path to the DPDK source t= ree directory to test. Cannot be used in conjunction with --tarball. (defau= lt: > - None) > + --dpdk-tree DIR_PATH [DTS_DPDK_TREE] The path to the DPDK source t= ree directory to test. Cannot be used in conjunction with --tarball. > + (default: None) > --tarball FILE_PATH, --snapshot FILE_PATH > - [DTS_DPDK_TARBALL] The path to the DPDK sourc= e tarball to test. DPDK must be contained in a folder with the same name as= the > - tarball file. Cannot be used in conjunction w= ith --dpdk-tree. (default: None) > - --remote-source [DTS_REMOTE_SOURCE] Set this option if either= the DPDK source tree or tarball to be used are located on the SUT node. Ca= n only > - be used with --dpdk-tree or --tarball. (defau= lt: False) > + [DTS_DPDK_TARBALL] The path to the DPDK sourc= e tarball to test. DPDK must be contained in a folder with the same > + name as the tarball file. Cannot be used in c= onjunction with --dpdk-tree. (default: None) > + --remote-source [DTS_REMOTE_SOURCE] Set this option if either= the DPDK source tree or tarball to be used are located on the SUT > + node. Can only be used with --dpdk-tree or --= tarball. (default: False) > --precompiled-build-dir DIR_NAME > - [DTS_PRECOMPILED_BUILD_DIR] Define the subdir= ectory under the DPDK tree root directory where the pre-compiled binaries a= re > - located. If set, DTS will build DPDK under th= e `build` directory instead. Can only be used with --dpdk-tree or --tarball= . > - (default: None) > + [DTS_PRECOMPILED_BUILD_DIR] Define the subdir= ectory under the DPDK tree root directory or tarball where the pre- > + compiled binaries are located. (default: None= ) > > > The brackets contain the names of environment variables that set the sam= e thing. > @@ -467,7 +472,7 @@ The output is generated in ``build/doc/api/dts/html``= . > Configuration Example > --------------------- > > -The following example (which can be found in ``dts/conf.yaml``) sets up = two nodes: > +The following example configuration files sets up two nodes: > > * ``SUT1`` which is already setup with the DPDK build requirements and a= ny other > required for execution; > @@ -479,6 +484,21 @@ And they both have two network ports which are physi= cally connected to each othe > This example assumes that you have setup SSH keys in both the system = under test > and traffic generator nodes. > > -.. literalinclude:: ../../../dts/conf.yaml > +.. _test_runs_configuration_example: > + > +``dts/test_runs.example.yaml`` > +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ > + > +.. literalinclude:: ../../../dts/test_runs.example.yaml > + :language: yaml > + :start-at: # Define > + > +.. _nodes_configuration_example: > + > + > +``dts/nodes.example.yaml`` > +~~~~~~~~~~~~~~~~~~~~~~~~~~ > + > +.. literalinclude:: ../../../dts/nodes.example.yaml > :language: yaml > - :start-at: test_runs: > + :start-at: # Define > diff --git a/dts/.gitignore b/dts/.gitignore > new file mode 100644 > index 0000000000..d53a2f3b7e > --- /dev/null > +++ b/dts/.gitignore > @@ -0,0 +1,4 @@ > +# default configuration files for DTS > +nodes.yaml > +test_runs.yaml > + > diff --git a/dts/conf.yaml b/dts/conf.yaml > deleted file mode 100644 > index bc78882d0d..0000000000 > --- a/dts/conf.yaml > +++ /dev/null > @@ -1,84 +0,0 @@ > -# SPDX-License-Identifier: BSD-3-Clause > -# Copyright 2022-2023 The DPDK contributors > -# Copyright 2023 Arm Limited > - > -test_runs: > - # define one test run environment > - - dpdk_build: > - dpdk_location: > - # dpdk_tree: Commented out because `tarball` is defined. > - tarball: dpdk-tarball.tar.xz > - # Either `dpdk_tree` or `tarball` can be defined, but not both. > - remote: false # Optional, defaults to false. If it's true, the `= dpdk_tree` or `tarball` > - # is located on the SUT node, instead of the execu= tion host. > - > - # precompiled_build_dir: Commented out because `build_options` is = defined. > - build_options: > - # the combination of the following two makes CC=3D"ccache gcc" > - compiler: gcc > - compiler_wrapper: ccache # Optional. > - # If `precompiled_build_dir` is defined, DPDK has been pre-built a= nd the build directory is > - # in a subdirectory of DPDK tree root directory. Otherwise, will b= e using the `build_options` > - # to build the DPDK from source. Either `precompiled_build_dir` or= `build_options` can be > - # defined, but not both. > - perf: false # disable performance testing > - func: true # enable functional testing > - skip_smoke_tests: false # optional > - test_suites: # the following test suites will be run in their entire= ty > - - hello_world > - vdevs: # optional; if removed, vdevs won't be used in the execution > - - "crypto_openssl" > - # The machine running the DPDK test executable > - system_under_test_node: "SUT 1" > - # Traffic generator node to use for this execution environment > - traffic_generator_node: "TG 1" > -nodes: > - # Define a system under test node, having two network ports physically > - # connected to the corresponding ports in TG 1 (the peer node) > - - name: "SUT 1" > - hostname: sut1.change.me.localhost > - user: dtsuser > - os: linux > - ports: > - # sets up the physical link between "SUT 1"@0000:00:08.0 and "TG 1= "@0000:00:08.0 > - - pci: "0000:00:08.0" > - os_driver_for_dpdk: vfio-pci # OS driver that DPDK will use > - os_driver: i40e # OS driver to bind when the tests = are not running > - peer_node: "TG 1" > - peer_pci: "0000:00:08.0" > - # sets up the physical link between "SUT 1"@0000:00:08.1 and "TG 1= "@0000:00:08.1 > - - pci: "0000:00:08.1" > - os_driver_for_dpdk: vfio-pci > - os_driver: i40e > - peer_node: "TG 1" > - peer_pci: "0000:00:08.1" > - hugepages_2mb: # optional; if removed, will use system hugepage conf= iguration > - number_of: 256 > - force_first_numa: false > - dpdk_config: > - lcores: "" # use all available logical cores (Skips first core) > - memory_channels: 4 # tells DPDK to use 4 memory channels > - # Define a Scapy traffic generator node, having two network ports > - # physically connected to the corresponding ports in SUT 1 (the peer n= ode). > - - name: "TG 1" > - hostname: tg1.change.me.localhost > - user: dtsuser > - os: linux > - ports: > - # sets up the physical link between "TG 1"@0000:00:08.0 and "SUT 1= "@0000:00:08.0 > - - pci: "0000:00:08.0" > - os_driver_for_dpdk: rdma > - os_driver: rdma > - peer_node: "SUT 1" > - peer_pci: "0000:00:08.0" > - # sets up the physical link between "SUT 1"@0000:00:08.0 and "TG 1= "@0000:00:08.0 > - - pci: "0000:00:08.1" > - os_driver_for_dpdk: rdma > - os_driver: rdma > - peer_node: "SUT 1" > - peer_pci: "0000:00:08.1" > - hugepages_2mb: # optional; if removed, will use system hugepage conf= iguration > - number_of: 256 > - force_first_numa: false > - traffic_generator: > - type: SCAPY > diff --git a/dts/framework/config/__init__.py b/dts/framework/config/__in= it__.py > index 6ae98d0387..adbd4e952d 100644 > --- a/dts/framework/config/__init__.py > +++ b/dts/framework/config/__init__.py > @@ -8,20 +8,15 @@ > > This package offers classes that hold real-time information about the te= stbed, hold test run > configuration describing the tested testbed and a loader function, :func= :`load_config`, which loads > -the YAML test run configuration file and validates it against the :class= :`Configuration` Pydantic > -model. > +the YAML configuration files and validates them against the :class:`Conf= iguration` Pydantic > +model, which fields are directly mapped. > > -The YAML test run configuration file is parsed into a dictionary, parts = of which are used throughout > -this package. The allowed keys and types inside this dictionary map dire= ctly to the > -:class:`Configuration` model, its fields and sub-models. > +The configuration files are split in: > > -The test run configuration has two main sections: > - > - * The :class:`TestRunConfiguration` which defines what tests are goi= ng to be run > - and how DPDK will be built. It also references the testbed where t= hese tests and DPDK > - are going to be run, > - * The nodes of the testbed are defined in the other section, > - a :class:`list` of :class:`NodeConfiguration` objects. > + * A list of test run which are represented by :class:`~.test_run.Tes= tRunConfiguration` > + defining what tests are going to be run and how DPDK will be built= . It also references > + the testbed where these tests and DPDK are going to be run, > + * A list of the nodes of the testbed which ar represented by :class:= `~.node.NodeConfiguration`. > > The real-time information about testbed is supposed to be gathered at ru= ntime. > > @@ -32,467 +27,24 @@ > and makes it thread safe should we ever want to move in that direc= tion. > """ > > -import tarfile > -from collections.abc import Callable, MutableMapping > -from enum import Enum, auto, unique > from functools import cached_property > -from pathlib import Path, PurePath > -from typing import TYPE_CHECKING, Annotated, Any, Literal, NamedTuple, T= ypedDict, cast > +from pathlib import Path > +from typing import Annotated, Any, Literal, NamedTuple, TypeVar, cast > > import yaml > -from pydantic import ( > - BaseModel, > - ConfigDict, > - Field, > - ValidationError, > - ValidationInfo, > - field_validator, > - model_validator, > -) > +from pydantic import Field, TypeAdapter, ValidationError, field_validato= r, model_validator > from typing_extensions import Self > > from framework.exception import ConfigurationError > -from framework.settings import Settings > -from framework.utils import REGEX_FOR_PCI_ADDRESS, StrEnum > - > -if TYPE_CHECKING: > - from framework.test_suite import TestSuiteSpec > - > - > -class ValidationContext(TypedDict): > - """A context dictionary to use for validation.""" > - > - #: The command line settings. > - settings: Settings > - > - > -def load_fields_from_settings( > - *fields: str | tuple[str, str], > -) -> Callable[[Any, ValidationInfo], Any]: > - """Before model validator that injects values from :attr:`Validation= Context.settings`. > - > - Args: > - *fields: The name of the fields to apply the argument value to. = If the settings field name > - is not the same as the configuration field, supply a tuple w= ith the respective names. > - > - Returns: > - Pydantic before model validator. > - """ > - > - def _loader(data: Any, info: ValidationInfo) -> Any: > - if not isinstance(data, MutableMapping): > - return data > - > - settings =3D cast(ValidationContext, info.context)["settings"] > - for field in fields: > - if isinstance(field, tuple): > - settings_field =3D field[0] > - config_field =3D field[1] > - else: > - settings_field =3D config_field =3D field > - > - if settings_data :=3D getattr(settings, settings_field): > - data[config_field] =3D settings_data > - > - return data > - > - return _loader > - > - > -class FrozenModel(BaseModel): > - """A pre-configured :class:`~pydantic.BaseModel`.""" > - > - #: Fields are set as read-only and any extra fields are forbidden. > - model_config =3D ConfigDict(frozen=3DTrue, extra=3D"forbid") > - > - > -@unique > -class OS(StrEnum): > - r"""The supported operating systems of :class:`~framework.testbed_mo= del.node.Node`\s.""" > - > - #: > - linux =3D auto() > - #: > - freebsd =3D auto() > - #: > - windows =3D auto() > - > - > -@unique > -class Compiler(StrEnum): > - r"""The supported compilers of :class:`~framework.testbed_model.node= .Node`\s.""" > - > - #: > - gcc =3D auto() > - #: > - clang =3D auto() > - #: > - icc =3D auto() > - #: > - msvc =3D auto() > - > - > -@unique > -class TrafficGeneratorType(str, Enum): > - """The supported traffic generators.""" > - > - #: > - SCAPY =3D "SCAPY" > - > - > -class HugepageConfiguration(FrozenModel): > - r"""The hugepage configuration of :class:`~framework.testbed_model.n= ode.Node`\s.""" > - > - #: The number of hugepages to allocate. > - number_of: int > - #: If :data:`True`, the hugepages will be configured on the first NU= MA node. > - force_first_numa: bool > - > - > -class PortConfig(FrozenModel): > - r"""The port configuration of :class:`~framework.testbed_model.node.= Node`\s.""" > - > - #: The PCI address of the port. > - pci: str =3D Field(pattern=3DREGEX_FOR_PCI_ADDRESS) > - #: The driver that the kernel should bind this device to for DPDK to= use it. > - os_driver_for_dpdk: str =3D Field(examples=3D["vfio-pci", "mlx5_core= "]) > - #: The operating system driver name when the operating system contro= ls the port. > - os_driver: str =3D Field(examples=3D["i40e", "ice", "mlx5_core"]) > - #: The name of the peer node this port is connected to. > - peer_node: str > - #: The PCI address of the peer port connected to this port. > - peer_pci: str =3D Field(pattern=3DREGEX_FOR_PCI_ADDRESS) > - > - > -class TrafficGeneratorConfig(FrozenModel): > - """A protocol required to define traffic generator types.""" > - > - #: The traffic generator type the child class is required to define = to be distinguished among > - #: others. > - type: TrafficGeneratorType > - > - > -class ScapyTrafficGeneratorConfig(TrafficGeneratorConfig): > - """Scapy traffic generator specific configuration.""" > - > - type: Literal[TrafficGeneratorType.SCAPY] > - > - > -#: A union type discriminating traffic generators by the `type` field. > -TrafficGeneratorConfigTypes =3D Annotated[ScapyTrafficGeneratorConfig, F= ield(discriminator=3D"type")] > - > -#: Comma-separated list of logical cores to use. An empty string or ```a= ny``` means use all lcores. > -LogicalCores =3D Annotated[ > - str, > - Field( > - examples=3D["1,2,3,4,5,18-22", "10-15", "any"], > - pattern=3Dr"^(([0-9]+|([0-9]+-[0-9]+))(,([0-9]+|([0-9]+-[0-9]+))= )*)?$|any", > - ), > -] > - > - > -class NodeConfiguration(FrozenModel): > - r"""The configuration of :class:`~framework.testbed_model.node.Node`= \s.""" > - > - #: The name of the :class:`~framework.testbed_model.node.Node`. > - name: str > - #: The hostname of the :class:`~framework.testbed_model.node.Node`. = Can also be an IP address. > - hostname: str > - #: The name of the user used to connect to the :class:`~framework.te= stbed_model.node.Node`. > - user: str > - #: The password of the user. The use of passwords is heavily discour= aged, please use SSH keys. > - password: str | None =3D None > - #: The operating system of the :class:`~framework.testbed_model.node= .Node`. > - os: OS > - #: An optional hugepage configuration. > - hugepages: HugepageConfiguration | None =3D Field(None, alias=3D"hug= epages_2mb") > - #: The ports that can be used in testing. > - ports: list[PortConfig] =3D Field(min_length=3D1) > - > - > -class DPDKConfiguration(FrozenModel): > - """Configuration of the DPDK EAL parameters.""" > - > - #: A comma delimited list of logical cores to use when running DPDK.= ```any```, an empty > - #: string or omitting this field means use any core except for the f= irst one. The first core > - #: will only be used if explicitly set. > - lcores: LogicalCores =3D "" > - > - #: The number of memory channels to use when running DPDK. > - memory_channels: int =3D 1 > - > - @property > - def use_first_core(self) -> bool: > - """Returns :data:`True` if `lcores` explicitly selects the first= core.""" > - return "0" in self.lcores > - > - > -class SutNodeConfiguration(NodeConfiguration): > - """:class:`~framework.testbed_model.sut_node.SutNode` specific confi= guration.""" > - > - #: The runtime configuration for DPDK. > - dpdk_config: DPDKConfiguration > - > - > -class TGNodeConfiguration(NodeConfiguration): > - """:class:`~framework.testbed_model.tg_node.TGNode` specific configu= ration.""" > - > - #: The configuration of the traffic generator present on the TG node= . > - traffic_generator: TrafficGeneratorConfigTypes > - > - > -#: Union type for all the node configuration types. > -NodeConfigurationTypes =3D TGNodeConfiguration | SutNodeConfiguration > - > - > -def resolve_path(path: Path) -> Path: > - """Resolve a path into a real path.""" > - return path.resolve() > - > - > -class BaseDPDKLocation(FrozenModel): > - """DPDK location base class. > - > - The path to the DPDK sources and type of location. > - """ > - > - #: Specifies whether to find DPDK on the SUT node or on the local ho= st. Which are respectively > - #: represented by :class:`RemoteDPDKLocation` and :class:`LocalDPDKT= reeLocation`. > - remote: bool =3D False > - > - > -class LocalDPDKLocation(BaseDPDKLocation): > - """Local DPDK location base class. > - > - This class is meant to represent any location that is present only l= ocally. > - """ > - > - remote: Literal[False] =3D False > - > - > -class LocalDPDKTreeLocation(LocalDPDKLocation): > - """Local DPDK tree location. > > - This class makes a distinction from :class:`RemoteDPDKTreeLocation` = by enforcing on the fly > - validation. > - """ > - > - #: The path to the DPDK source tree directory on the local host pass= ed as string. > - dpdk_tree: Path > - > - #: Resolve the local DPDK tree path. > - resolve_dpdk_tree_path =3D field_validator("dpdk_tree")(resolve_path= ) > - > - @model_validator(mode=3D"after") > - def validate_dpdk_tree_path(self) -> Self: > - """Validate the provided DPDK tree path.""" > - assert self.dpdk_tree.exists(), "DPDK tree not found in local fi= lesystem." > - assert self.dpdk_tree.is_dir(), "The DPDK tree path must be a di= rectory." > - return self > - > - > -class LocalDPDKTarballLocation(LocalDPDKLocation): > - """Local DPDK tarball location. > - > - This class makes a distinction from :class:`RemoteDPDKTarballLocatio= n` by enforcing on the fly > - validation. > - """ > - > - #: The path to the DPDK tarball on the local host passed as string. > - tarball: Path > - > - #: Resolve the local tarball path. > - resolve_tarball_path =3D field_validator("tarball")(resolve_path) > - > - @model_validator(mode=3D"after") > - def validate_tarball_path(self) -> Self: > - """Validate the provided tarball.""" > - assert self.tarball.exists(), "DPDK tarball not found in local f= ilesystem." > - assert tarfile.is_tarfile(self.tarball), "The DPDK tarball must = be a valid tar archive." > - return self > - > - > -class RemoteDPDKLocation(BaseDPDKLocation): > - """Remote DPDK location base class. > - > - This class is meant to represent any location that is present only r= emotely. > - """ > - > - remote: Literal[True] =3D True > - > - > -class RemoteDPDKTreeLocation(RemoteDPDKLocation): > - """Remote DPDK tree location. > - > - This class is distinct from :class:`LocalDPDKTreeLocation` which enf= orces on the fly validation. > - """ > - > - #: The path to the DPDK source tree directory on the remote node pas= sed as string. > - dpdk_tree: PurePath > - > - > -class RemoteDPDKTarballLocation(RemoteDPDKLocation): > - """Remote DPDK tarball location. > - > - This class is distinct from :class:`LocalDPDKTarballLocation` which = enforces on the fly > - validation. > - """ > - > - #: The path to the DPDK tarball on the remote node passed as string. > - tarball: PurePath > - > - > -#: Union type for different DPDK locations. > -DPDKLocation =3D ( > - LocalDPDKTreeLocation > - | LocalDPDKTarballLocation > - | RemoteDPDKTreeLocation > - | RemoteDPDKTarballLocation > +from .common import FrozenModel, ValidationContext > +from .node import ( > + NodeConfiguration, > + NodeConfigurationTypes, > + SutNodeConfiguration, > + TGNodeConfiguration, > ) > - > - > -class BaseDPDKBuildConfiguration(FrozenModel): > - """The base configuration for different types of build. > - > - The configuration contain the location of the DPDK and configuration= used for building it. > - """ > - > - #: The location of the DPDK tree. > - dpdk_location: DPDKLocation > - > - dpdk_location_from_settings =3D model_validator(mode=3D"before")( > - load_fields_from_settings("dpdk_location") > - ) > - > - > -class DPDKPrecompiledBuildConfiguration(BaseDPDKBuildConfiguration): > - """DPDK precompiled build configuration.""" > - > - #: If it's defined, DPDK has been pre-compiled and the build directo= ry is located in a > - #: subdirectory of `~dpdk_location.dpdk_tree` or `~dpdk_location.tar= ball` root directory. > - precompiled_build_dir: str =3D Field(min_length=3D1) > - > - build_dir_from_settings =3D model_validator(mode=3D"before")( > - load_fields_from_settings("precompiled_build_dir") > - ) > - > - > -class DPDKBuildOptionsConfiguration(FrozenModel): > - """DPDK build options configuration. > - > - The build options used for building DPDK. > - """ > - > - #: The compiler executable to use. > - compiler: Compiler > - #: This string will be put in front of the compiler when executing t= he build. Useful for adding > - #: wrapper commands, such as ``ccache``. > - compiler_wrapper: str =3D "" > - > - > -class DPDKUncompiledBuildConfiguration(BaseDPDKBuildConfiguration): > - """DPDK uncompiled build configuration.""" > - > - #: The build options to compiled DPDK with. > - build_options: DPDKBuildOptionsConfiguration > - > - > -#: Union type for different build configurations. > -DPDKBuildConfiguration =3D DPDKPrecompiledBuildConfiguration | DPDKUncom= piledBuildConfiguration > - > - > -class TestSuiteConfig(FrozenModel): > - """Test suite configuration. > - > - Information about a single test suite to be executed. This can also = be represented as a string > - instead of a mapping, example: > - > - .. code:: yaml > - > - test_runs: > - - test_suites: > - # As string representation: > - - hello_world # test all of `hello_world`, or > - - hello_world hello_world_single_core # test only `hello_wor= ld_single_core` > - # or as model fields: > - - test_suite: hello_world > - test_cases: [hello_world_single_core] # without this field= all test cases are run > - """ > - > - #: The name of the test suite module without the starting ``TestSuit= e_``. > - test_suite_name: str =3D Field(alias=3D"test_suite") > - #: The names of test cases from this test suite to execute. If empty= , all test cases will be > - #: executed. > - test_cases_names: list[str] =3D Field(default_factory=3Dlist, alias= =3D"test_cases") > - > - @cached_property > - def test_suite_spec(self) -> "TestSuiteSpec": > - """The specification of the requested test suite.""" > - from framework.test_suite import find_by_name > - > - test_suite_spec =3D find_by_name(self.test_suite_name) > - assert ( > - test_suite_spec is not None > - ), f"{self.test_suite_name} is not a valid test suite module nam= e." > - return test_suite_spec > - > - @model_validator(mode=3D"before") > - @classmethod > - def convert_from_string(cls, data: Any) -> Any: > - """Convert the string representation of the model into a valid m= apping.""" > - if isinstance(data, str): > - [test_suite, *test_cases] =3D data.split() > - return dict(test_suite=3Dtest_suite, test_cases=3Dtest_cases= ) > - return data > - > - @model_validator(mode=3D"after") > - def validate_names(self) -> Self: > - """Validate the supplied test suite and test cases names. > - > - This validator relies on the cached property `test_suite_spec` t= o run for the first > - time in this call, therefore triggering the assertions if needed= . > - """ > - available_test_cases =3D map( > - lambda t: t.name, self.test_suite_spec.class_obj.get_test_ca= ses() > - ) > - for requested_test_case in self.test_cases_names: > - assert requested_test_case in available_test_cases, ( > - f"{requested_test_case} is not a valid test case " > - f"of test suite {self.test_suite_name}." > - ) > - > - return self > - > - > -class TestRunConfiguration(FrozenModel): > - """The configuration of a test run. > - > - The configuration contains testbed information, what tests to execut= e > - and with what DPDK build. > - """ > - > - #: The DPDK configuration used to test. > - dpdk_config: DPDKBuildConfiguration =3D Field(alias=3D"dpdk_build") > - #: Whether to run performance tests. > - perf: bool > - #: Whether to run functional tests. > - func: bool > - #: Whether to skip smoke tests. > - skip_smoke_tests: bool =3D False > - #: The names of test suites and/or test cases to execute. > - test_suites: list[TestSuiteConfig] =3D Field(min_length=3D1) > - #: The SUT node name to use in this test run. > - system_under_test_node: str > - #: The TG node name to use in this test run. > - traffic_generator_node: str > - #: The names of virtual devices to test. > - vdevs: list[str] =3D Field(default_factory=3Dlist) > - #: The seed to use for pseudo-random generation. > - random_seed: int | None =3D None > - > - fields_from_settings =3D model_validator(mode=3D"before")( > - load_fields_from_settings("test_suites", "random_seed") > - ) > +from .test_run import TestRunConfiguration > > > class TestRunWithNodesConfiguration(NamedTuple): > @@ -506,13 +58,18 @@ class TestRunWithNodesConfiguration(NamedTuple): > tg_node_config: TGNodeConfiguration > > > +TestRunsConfig =3D Annotated[list[TestRunConfiguration], Field(min_lengt= h=3D1)] > + > +NodesConfig =3D Annotated[list[NodeConfigurationTypes], Field(min_length= =3D1)] > + > + > class Configuration(FrozenModel): > """DTS testbed and test configuration.""" > > #: Test run configurations. > - test_runs: list[TestRunConfiguration] =3D Field(min_length=3D1) > + test_runs: TestRunsConfig > #: Node configurations. > - nodes: list[NodeConfigurationTypes] =3D Field(min_length=3D1) > + nodes: NodesConfig > > @cached_property > def test_runs_with_nodes(self) -> list[TestRunWithNodesConfiguration= ]: > @@ -596,30 +153,37 @@ def validate_test_runs_with_nodes(self) -> Self: > return self > > > -def load_config(settings: Settings) -> Configuration: > - """Load DTS test run configuration from a file. > +T =3D TypeVar("T") > + > + > +def _load_and_parse_model(file_path: Path, model_type: T, ctx: Validatio= nContext) -> T: > + with open(file_path) as f: > + try: > + data =3D yaml.safe_load(f) > + return TypeAdapter(model_type).validate_python(data, context= =3Dcast(dict[str, Any], ctx)) > + except ValidationError as e: > + msg =3D f"failed to load the configuration file {file_path}" > + raise ConfigurationError(msg) from e > + > > - Load the YAML test run configuration file, validate it, and create a= test run configuration > - object. > +def load_config(ctx: ValidationContext) -> Configuration: > + """Load the DTS configuration from files. > > - The YAML test run configuration file is specified in the :option:`--= config-file` command line > - argument or the :envvar:`DTS_CFG_FILE` environment variable. > + Load the YAML configuration files, validate them, and create a confi= guration object. > > Args: > - config_file_path: The path to the YAML test run configuration fi= le. > - settings: The settings provided by the user on the command line. > + ctx: The context required for validation. > > Returns: > The parsed test run configuration. > > Raises: > - ConfigurationError: If the supplied configuration file is invali= d. > + ConfigurationError: If the supplied configuration files are inva= lid. > """ > - with open(settings.config_file_path, "r") as f: > - config_data =3D yaml.safe_load(f) > + test_runs =3D _load_and_parse_model(ctx["settings"].test_runs_config= _path, TestRunsConfig, ctx) > + nodes =3D _load_and_parse_model(ctx["settings"].nodes_config_path, N= odesConfig, ctx) > > try: > - context =3D ValidationContext(settings=3Dsettings) > - return Configuration.model_validate(config_data, context=3Dconte= xt) > + return Configuration.model_validate({"test_runs": test_runs, "no= des": nodes}, context=3Dctx) > except ValidationError as e: > - raise ConfigurationError("failed to load the supplied configurat= ion") from e > + raise ConfigurationError("the configurations supplied are invali= d") from e > diff --git a/dts/framework/config/common.py b/dts/framework/config/common= .py > new file mode 100644 > index 0000000000..25265cb9da > --- /dev/null > +++ b/dts/framework/config/common.py > @@ -0,0 +1,59 @@ > +# SPDX-License-Identifier: BSD-3-Clause > +# Copyright (c) 2025 Arm Limited > + > +"""Common definitions and objects for the configuration.""" > + > +from collections.abc import Callable, MutableMapping > +from typing import TYPE_CHECKING, Any, TypedDict, cast > + > +from pydantic import BaseModel, ConfigDict, ValidationInfo > + > +if TYPE_CHECKING: > + from framework.settings import Settings > + > + > +class ValidationContext(TypedDict): > + """A context dictionary to use for validation.""" > + > + #: The command line settings. > + settings: "Settings" > + > + > +def load_fields_from_settings( > + *fields: str | tuple[str, str], > +) -> Callable[[Any, ValidationInfo], Any]: > + """Before model validator that injects values from :attr:`Validation= Context.settings`. > + > + Args: > + *fields: The name of the fields to apply the argument value to. = If the settings field name > + is not the same as the configuration field, supply a tuple w= ith the respective names. > + > + Returns: > + Pydantic before model validator. > + """ > + > + def _loader(data: Any, info: ValidationInfo) -> Any: > + if not isinstance(data, MutableMapping): > + return data > + > + settings =3D cast(ValidationContext, info.context)["settings"] > + for field in fields: > + if isinstance(field, tuple): > + settings_field =3D field[0] > + config_field =3D field[1] > + else: > + settings_field =3D config_field =3D field > + > + if settings_data :=3D getattr(settings, settings_field): > + data[config_field] =3D settings_data > + > + return data > + > + return _loader > + > + > +class FrozenModel(BaseModel): > + """A pre-configured :class:`~pydantic.BaseModel`.""" > + > + #: Fields are set as read-only and any extra fields are forbidden. > + model_config =3D ConfigDict(frozen=3DTrue, extra=3D"forbid") > diff --git a/dts/framework/config/node.py b/dts/framework/config/node.py > new file mode 100644 > index 0000000000..a7ace514d9 > --- /dev/null > +++ b/dts/framework/config/node.py > @@ -0,0 +1,144 @@ > +# SPDX-License-Identifier: BSD-3-Clause > +# Copyright(c) 2010-2021 Intel Corporation > +# Copyright(c) 2022-2023 University of New Hampshire > +# Copyright(c) 2023 PANTHEON.tech s.r.o. > +# Copyright(c) 2024 Arm Limited > + > +"""Configuration models representing a node. > + > +The root model of a node configuration is :class:`NodeConfiguration`. > +""" > + > +from enum import Enum, auto, unique > +from typing import Annotated, Literal > + > +from pydantic import Field > + > +from framework.utils import REGEX_FOR_PCI_ADDRESS, StrEnum > + > +from .common import FrozenModel > + > + > +@unique > +class OS(StrEnum): > + r"""The supported operating systems of :class:`~framework.testbed_mo= del.node.Node`\s.""" > + > + #: > + linux =3D auto() > + #: > + freebsd =3D auto() > + #: > + windows =3D auto() > + > + > +@unique > +class TrafficGeneratorType(str, Enum): > + """The supported traffic generators.""" > + > + #: > + SCAPY =3D "SCAPY" > + > + > +class HugepageConfiguration(FrozenModel): > + r"""The hugepage configuration of :class:`~framework.testbed_model.n= ode.Node`\s.""" > + > + #: The number of hugepages to allocate. > + number_of: int > + #: If :data:`True`, the hugepages will be configured on the first NU= MA node. > + force_first_numa: bool > + > + > +class PortConfig(FrozenModel): > + r"""The port configuration of :class:`~framework.testbed_model.node.= Node`\s.""" > + > + #: The PCI address of the port. > + pci: str =3D Field(pattern=3DREGEX_FOR_PCI_ADDRESS) > + #: The driver that the kernel should bind this device to for DPDK to= use it. > + os_driver_for_dpdk: str =3D Field(examples=3D["vfio-pci", "mlx5_core= "]) > + #: The operating system driver name when the operating system contro= ls the port. > + os_driver: str =3D Field(examples=3D["i40e", "ice", "mlx5_core"]) > + #: The name of the peer node this port is connected to. > + peer_node: str > + #: The PCI address of the peer port connected to this port. > + peer_pci: str =3D Field(pattern=3DREGEX_FOR_PCI_ADDRESS) > + > + > +class TrafficGeneratorConfig(FrozenModel): > + """A protocol required to define traffic generator types.""" > + > + #: The traffic generator type the child class is required to define = to be distinguished among > + #: others. > + type: TrafficGeneratorType > + > + > +class ScapyTrafficGeneratorConfig(TrafficGeneratorConfig): > + """Scapy traffic generator specific configuration.""" > + > + type: Literal[TrafficGeneratorType.SCAPY] > + > + > +#: A union type discriminating traffic generators by the `type` field. > +TrafficGeneratorConfigTypes =3D Annotated[ScapyTrafficGeneratorConfig, F= ield(discriminator=3D"type")] > + > +#: Comma-separated list of logical cores to use. An empty string or ```a= ny``` means use all lcores. > +LogicalCores =3D Annotated[ > + str, > + Field( > + examples=3D["1,2,3,4,5,18-22", "10-15", "any"], > + pattern=3Dr"^(([0-9]+|([0-9]+-[0-9]+))(,([0-9]+|([0-9]+-[0-9]+))= )*)?$|any", > + ), > +] > + > + > +class NodeConfiguration(FrozenModel): > + r"""The configuration of :class:`~framework.testbed_model.node.Node`= \s.""" > + > + #: The name of the :class:`~framework.testbed_model.node.Node`. > + name: str > + #: The hostname of the :class:`~framework.testbed_model.node.Node`. = Can also be an IP address. > + hostname: str > + #: The name of the user used to connect to the :class:`~framework.te= stbed_model.node.Node`. > + user: str > + #: The password of the user. The use of passwords is heavily discour= aged, please use SSH keys. > + password: str | None =3D None > + #: The operating system of the :class:`~framework.testbed_model.node= .Node`. > + os: OS > + #: An optional hugepage configuration. > + hugepages: HugepageConfiguration | None =3D Field(None, alias=3D"hug= epages_2mb") > + #: The ports that can be used in testing. > + ports: list[PortConfig] =3D Field(min_length=3D1) > + > + > +class DPDKConfiguration(FrozenModel): > + """Configuration of the DPDK EAL parameters.""" > + > + #: A comma delimited list of logical cores to use when running DPDK.= ```any```, an empty > + #: string or omitting this field means use any core except for the f= irst one. The first core > + #: will only be used if explicitly set. > + lcores: LogicalCores =3D "" > + > + #: The number of memory channels to use when running DPDK. > + memory_channels: int =3D 1 > + > + @property > + def use_first_core(self) -> bool: > + """Returns :data:`True` if `lcores` explicitly selects the first= core.""" > + return "0" in self.lcores > + > + > +class SutNodeConfiguration(NodeConfiguration): > + """:class:`~framework.testbed_model.sut_node.SutNode` specific confi= guration.""" > + > + #: The runtime configuration for DPDK. > + dpdk_config: DPDKConfiguration > + > + > +class TGNodeConfiguration(NodeConfiguration): > + """:class:`~framework.testbed_model.tg_node.TGNode` specific configu= ration.""" > + > + #: The configuration of the traffic generator present on the TG node= . > + traffic_generator: TrafficGeneratorConfigTypes > + > + > +#: Union type for all the node configuration types. > +NodeConfigurationTypes =3D TGNodeConfiguration | SutNodeConfiguration > diff --git a/dts/framework/config/test_run.py b/dts/framework/config/test= _run.py > new file mode 100644 > index 0000000000..dc0e46047d > --- /dev/null > +++ b/dts/framework/config/test_run.py > @@ -0,0 +1,290 @@ > +# SPDX-License-Identifier: BSD-3-Clause > +# Copyright(c) 2010-2021 Intel Corporation > +# Copyright(c) 2022-2023 University of New Hampshire > +# Copyright(c) 2023 PANTHEON.tech s.r.o. > +# Copyright(c) 2024 Arm Limited > + > +"""Configuration models representing a test run. > + > +The root model of a test run configuration is :class:`TestRunConfigurati= on`. > +""" > + > +import tarfile > +from enum import auto, unique > +from functools import cached_property > +from pathlib import Path, PurePath > +from typing import Any, Literal > + > +from pydantic import Field, field_validator, model_validator > +from typing_extensions import TYPE_CHECKING, Self > + > +from framework.utils import StrEnum > + > +from .common import FrozenModel, load_fields_from_settings > + > +if TYPE_CHECKING: > + from framework.test_suite import TestSuiteSpec > + > + > +@unique > +class Compiler(StrEnum): > + r"""The supported compilers of :class:`~framework.testbed_model.node= .Node`\s.""" > + > + #: > + gcc =3D auto() > + #: > + clang =3D auto() > + #: > + icc =3D auto() > + #: > + msvc =3D auto() > + > + > +def resolve_path(path: Path) -> Path: > + """Resolve a path into a real path.""" > + return path.resolve() > + > + > +class BaseDPDKLocation(FrozenModel): > + """DPDK location base class. > + > + The path to the DPDK sources and type of location. > + """ > + > + #: Specifies whether to find DPDK on the SUT node or on the local ho= st. Which are respectively > + #: represented by :class:`RemoteDPDKLocation` and :class:`LocalDPDKT= reeLocation`. > + remote: bool =3D False > + > + > +class LocalDPDKLocation(BaseDPDKLocation): > + """Local DPDK location base class. > + > + This class is meant to represent any location that is present only l= ocally. > + """ > + > + remote: Literal[False] =3D False > + > + > +class LocalDPDKTreeLocation(LocalDPDKLocation): > + """Local DPDK tree location. > + > + This class makes a distinction from :class:`RemoteDPDKTreeLocation` = by enforcing on the fly > + validation. > + """ > + > + #: The path to the DPDK source tree directory on the local host pass= ed as string. > + dpdk_tree: Path > + > + #: Resolve the local DPDK tree path. > + resolve_dpdk_tree_path =3D field_validator("dpdk_tree")(resolve_path= ) > + > + @model_validator(mode=3D"after") > + def validate_dpdk_tree_path(self) -> Self: > + """Validate the provided DPDK tree path.""" > + assert self.dpdk_tree.exists(), "DPDK tree not found in local fi= lesystem." > + assert self.dpdk_tree.is_dir(), "The DPDK tree path must be a di= rectory." > + return self > + > + > +class LocalDPDKTarballLocation(LocalDPDKLocation): > + """Local DPDK tarball location. > + > + This class makes a distinction from :class:`RemoteDPDKTarballLocatio= n` by enforcing on the fly > + validation. > + """ > + > + #: The path to the DPDK tarball on the local host passed as string. > + tarball: Path > + > + #: Resolve the local tarball path. > + resolve_tarball_path =3D field_validator("tarball")(resolve_path) > + > + @model_validator(mode=3D"after") > + def validate_tarball_path(self) -> Self: > + """Validate the provided tarball.""" > + assert self.tarball.exists(), "DPDK tarball not found in local f= ilesystem." > + assert tarfile.is_tarfile(self.tarball), "The DPDK tarball must = be a valid tar archive." > + return self > + > + > +class RemoteDPDKLocation(BaseDPDKLocation): > + """Remote DPDK location base class. > + > + This class is meant to represent any location that is present only r= emotely. > + """ > + > + remote: Literal[True] =3D True > + > + > +class RemoteDPDKTreeLocation(RemoteDPDKLocation): > + """Remote DPDK tree location. > + > + This class is distinct from :class:`LocalDPDKTreeLocation` which enf= orces on the fly validation. > + """ > + > + #: The path to the DPDK source tree directory on the remote node pas= sed as string. > + dpdk_tree: PurePath > + > + > +class RemoteDPDKTarballLocation(RemoteDPDKLocation): > + """Remote DPDK tarball location. > + > + This class is distinct from :class:`LocalDPDKTarballLocation` which = enforces on the fly > + validation. > + """ > + > + #: The path to the DPDK tarball on the remote node passed as string. > + tarball: PurePath > + > + > +#: Union type for different DPDK locations. > +DPDKLocation =3D ( > + LocalDPDKTreeLocation > + | LocalDPDKTarballLocation > + | RemoteDPDKTreeLocation > + | RemoteDPDKTarballLocation > +) > + > + > +class BaseDPDKBuildConfiguration(FrozenModel): > + """The base configuration for different types of build. > + > + The configuration contain the location of the DPDK and configuration= used for building it. > + """ > + > + #: The location of the DPDK tree. > + dpdk_location: DPDKLocation > + > + dpdk_location_from_settings =3D model_validator(mode=3D"before")( > + load_fields_from_settings("dpdk_location") > + ) > + > + > +class DPDKPrecompiledBuildConfiguration(BaseDPDKBuildConfiguration): > + """DPDK precompiled build configuration.""" > + > + #: If it's defined, DPDK has been pre-compiled and the build directo= ry is located in a > + #: subdirectory of `~dpdk_location.dpdk_tree` or `~dpdk_location.tar= ball` root directory. > + precompiled_build_dir: str =3D Field(min_length=3D1) > + > + build_dir_from_settings =3D model_validator(mode=3D"before")( > + load_fields_from_settings("precompiled_build_dir") > + ) > + > + > +class DPDKBuildOptionsConfiguration(FrozenModel): > + """DPDK build options configuration. > + > + The build options used for building DPDK. > + """ > + > + #: The compiler executable to use. > + compiler: Compiler > + #: This string will be put in front of the compiler when executing t= he build. Useful for adding > + #: wrapper commands, such as ``ccache``. > + compiler_wrapper: str =3D "" > + > + > +class DPDKUncompiledBuildConfiguration(BaseDPDKBuildConfiguration): > + """DPDK uncompiled build configuration.""" > + > + #: The build options to compiled DPDK with. > + build_options: DPDKBuildOptionsConfiguration > + > + > +#: Union type for different build configurations. > +DPDKBuildConfiguration =3D DPDKPrecompiledBuildConfiguration | DPDKUncom= piledBuildConfiguration > + > + > +class TestSuiteConfig(FrozenModel): > + """Test suite configuration. > + > + Information about a single test suite to be executed. This can also = be represented as a string > + instead of a mapping, example: > + > + .. code:: yaml > + > + test_runs: > + - test_suites: > + # As string representation: > + - hello_world # test all of `hello_world`, or > + - hello_world hello_world_single_core # test only `hello_wor= ld_single_core` > + # or as model fields: > + - test_suite: hello_world > + test_cases: [hello_world_single_core] # without this field= all test cases are run > + """ > + > + #: The name of the test suite module without the starting ``TestSuit= e_``. > + test_suite_name: str =3D Field(alias=3D"test_suite") > + #: The names of test cases from this test suite to execute. If empty= , all test cases will be > + #: executed. > + test_cases_names: list[str] =3D Field(default_factory=3Dlist, alias= =3D"test_cases") > + > + @cached_property > + def test_suite_spec(self) -> "TestSuiteSpec": > + """The specification of the requested test suite.""" > + from framework.test_suite import find_by_name > + > + test_suite_spec =3D find_by_name(self.test_suite_name) > + assert ( > + test_suite_spec is not None > + ), f"{self.test_suite_name} is not a valid test suite module nam= e." > + return test_suite_spec > + > + @model_validator(mode=3D"before") > + @classmethod > + def convert_from_string(cls, data: Any) -> Any: > + """Convert the string representation of the model into a valid m= apping.""" > + if isinstance(data, str): > + [test_suite, *test_cases] =3D data.split() > + return dict(test_suite=3Dtest_suite, test_cases=3Dtest_cases= ) > + return data > + > + @model_validator(mode=3D"after") > + def validate_names(self) -> Self: > + """Validate the supplied test suite and test cases names. > + > + This validator relies on the cached property `test_suite_spec` t= o run for the first > + time in this call, therefore triggering the assertions if needed= . > + """ > + available_test_cases =3D map( > + lambda t: t.name, self.test_suite_spec.class_obj.get_test_ca= ses() > + ) > + for requested_test_case in self.test_cases_names: > + assert requested_test_case in available_test_cases, ( > + f"{requested_test_case} is not a valid test case " > + f"of test suite {self.test_suite_name}." > + ) > + > + return self > + > + > +class TestRunConfiguration(FrozenModel): > + """The configuration of a test run. > + > + The configuration contains testbed information, what tests to execut= e > + and with what DPDK build. > + """ > + > + #: The DPDK configuration used to test. > + dpdk_config: DPDKBuildConfiguration =3D Field(alias=3D"dpdk_build") > + #: Whether to run performance tests. > + perf: bool > + #: Whether to run functional tests. > + func: bool > + #: Whether to skip smoke tests. > + skip_smoke_tests: bool =3D False > + #: The names of test suites and/or test cases to execute. > + test_suites: list[TestSuiteConfig] =3D Field(min_length=3D1) > + #: The SUT node name to use in this test run. > + system_under_test_node: str > + #: The TG node name to use in this test run. > + traffic_generator_node: str > + #: The names of virtual devices to test. > + vdevs: list[str] =3D Field(default_factory=3Dlist) > + #: The seed to use for pseudo-random generation. > + random_seed: int | None =3D None > + > + fields_from_settings =3D model_validator(mode=3D"before")( > + load_fields_from_settings("test_suites", "random_seed") > + ) > diff --git a/dts/framework/runner.py b/dts/framework/runner.py > index e46a8c1a4f..9f9789cf49 100644 > --- a/dts/framework/runner.py > +++ b/dts/framework/runner.py > @@ -25,17 +25,22 @@ > from types import MethodType > from typing import Iterable > > +from framework.config.common import ValidationContext > from framework.testbed_model.capability import Capability, get_supported= _capabilities > from framework.testbed_model.sut_node import SutNode > from framework.testbed_model.tg_node import TGNode > > from .config import ( > Configuration, > + load_config, > +) > +from .config.node import ( > SutNodeConfiguration, > + TGNodeConfiguration, > +) > +from .config.test_run import ( > TestRunConfiguration, > TestSuiteConfig, > - TGNodeConfiguration, > - load_config, > ) > from .exception import BlockingTestSuiteError, SSHTimeoutError, TestCase= VerifyError > from .logger import DTSLogger, DtsStage, get_dts_logger > @@ -81,7 +86,7 @@ class DTSRunner: > > def __init__(self): > """Initialize the instance with configuration, logger, result an= d string constants.""" > - self._configuration =3D load_config(SETTINGS) > + self._configuration =3D load_config(ValidationContext(settings= =3DSETTINGS)) > self._logger =3D get_dts_logger() > if not os.path.exists(SETTINGS.output_dir): > os.makedirs(SETTINGS.output_dir) > diff --git a/dts/framework/settings.py b/dts/framework/settings.py > index 873d400bec..cf82a7c18f 100644 > --- a/dts/framework/settings.py > +++ b/dts/framework/settings.py > @@ -14,10 +14,15 @@ > > The command line arguments along with the supported environment variable= s are: > > -.. option:: --config-file > -.. envvar:: DTS_CFG_FILE > +.. option:: --test-runs-config-file > +.. envvar:: DTS_TEST_RUNS_CFG_FILE > > - The path to the YAML test run configuration file. > + The path to the YAML configuration file of the test runs. > + > +.. option:: --nodes-config-file > +.. envvar:: DTS_NODES_CFG_FILE > + > + The path to the YAML configuration file of the nodes. > > .. option:: --output-dir, --output > .. envvar:: DTS_OUTPUT_DIR > @@ -102,7 +107,7 @@ > > from pydantic import ValidationError > > -from .config import ( > +from .config.test_run import ( > DPDKLocation, > LocalDPDKTarballLocation, > LocalDPDKTreeLocation, > @@ -120,7 +125,9 @@ class Settings: > """ > > #: > - config_file_path: Path =3D Path(__file__).parent.parent.joinpath("co= nf.yaml") > + test_runs_config_path: Path =3D Path(__file__).parent.parent.joinpat= h("test_runs.yaml") > + #: > + nodes_config_path: Path =3D Path(__file__).parent.parent.joinpath("n= odes.yaml") > #: > output_dir: str =3D "output" > #: > @@ -316,14 +323,24 @@ def _get_parser() -> _DTSArgumentParser: > ) > > action =3D parser.add_argument( > - "--config-file", > - default=3DSETTINGS.config_file_path, > + "--test-runs-config-file", > + default=3DSETTINGS.test_runs_config_path, > + type=3DPath, > + help=3D"The configuration file that describes the test cases and= DPDK build options.", > + metavar=3D"FILE_PATH", > + dest=3D"test_runs_config_path", > + ) > + _add_env_var_to_action(action, "TEST_RUNS_CFG_FILE") > + > + action =3D parser.add_argument( > + "--nodes-config-file", > + default=3DSETTINGS.nodes_config_path, > type=3DPath, > - help=3D"The configuration file that describes the test cases, SU= Ts and DPDK build configs.", > + help=3D"The configuration file that describes the SUT and TG nod= es.", > metavar=3D"FILE_PATH", > - dest=3D"config_file_path", > + dest=3D"nodes_config_path", > ) > - _add_env_var_to_action(action, "CFG_FILE") > + _add_env_var_to_action(action, "NODES_CFG_FILE") > > action =3D parser.add_argument( > "--output-dir", > diff --git a/dts/framework/test_result.py b/dts/framework/test_result.py > index 0060155ef9..bffbc52505 100644 > --- a/dts/framework/test_result.py > +++ b/dts/framework/test_result.py > @@ -32,7 +32,7 @@ > > from framework.testbed_model.capability import Capability > > -from .config import TestRunConfiguration, TestSuiteConfig > +from .config.test_run import TestRunConfiguration, TestSuiteConfig > from .exception import DTSError, ErrorSeverity > from .logger import DTSLogger > from .test_suite import TestCase, TestSuite > diff --git a/dts/framework/testbed_model/node.py b/dts/framework/testbed_= model/node.py > index 6c2dfd6185..e53a321499 100644 > --- a/dts/framework/testbed_model/node.py > +++ b/dts/framework/testbed_model/node.py > @@ -15,10 +15,12 @@ > > from abc import ABC > > -from framework.config import ( > +from framework.config.node import ( > OS, > - DPDKBuildConfiguration, > NodeConfiguration, > +) > +from framework.config.test_run import ( > + DPDKBuildConfiguration, > TestRunConfiguration, > ) > from framework.exception import ConfigurationError > diff --git a/dts/framework/testbed_model/os_session.py b/dts/framework/te= stbed_model/os_session.py > index e436886692..6d5fce40ff 100644 > --- a/dts/framework/testbed_model/os_session.py > +++ b/dts/framework/testbed_model/os_session.py > @@ -28,7 +28,7 @@ > from dataclasses import dataclass > from pathlib import Path, PurePath, PurePosixPath > > -from framework.config import NodeConfiguration > +from framework.config.node import NodeConfiguration > from framework.logger import DTSLogger > from framework.remote_session import ( > InteractiveRemoteSession, > diff --git a/dts/framework/testbed_model/port.py b/dts/framework/testbed_= model/port.py > index 566f4c5b46..7177da3371 100644 > --- a/dts/framework/testbed_model/port.py > +++ b/dts/framework/testbed_model/port.py > @@ -10,7 +10,7 @@ > > from dataclasses import dataclass > > -from framework.config import PortConfig > +from framework.config.node import PortConfig > > > @dataclass(slots=3DTrue, frozen=3DTrue) > diff --git a/dts/framework/testbed_model/sut_node.py b/dts/framework/test= bed_model/sut_node.py > index d8f1f9d452..483733cede 100644 > --- a/dts/framework/testbed_model/sut_node.py > +++ b/dts/framework/testbed_model/sut_node.py > @@ -16,7 +16,10 @@ > from dataclasses import dataclass > from pathlib import Path, PurePath > > -from framework.config import ( > +from framework.config.node import ( > + SutNodeConfiguration, > +) > +from framework.config.test_run import ( > DPDKBuildConfiguration, > DPDKBuildOptionsConfiguration, > DPDKPrecompiledBuildConfiguration, > @@ -25,7 +28,6 @@ > LocalDPDKTreeLocation, > RemoteDPDKTarballLocation, > RemoteDPDKTreeLocation, > - SutNodeConfiguration, > TestRunConfiguration, > ) > from framework.exception import ConfigurationError, RemoteFileNotFoundEr= ror > diff --git a/dts/framework/testbed_model/tg_node.py b/dts/framework/testb= ed_model/tg_node.py > index 3071bbd645..86cd278efb 100644 > --- a/dts/framework/testbed_model/tg_node.py > +++ b/dts/framework/testbed_model/tg_node.py > @@ -11,7 +11,7 @@ > > from scapy.packet import Packet > > -from framework.config import TGNodeConfiguration > +from framework.config.node import TGNodeConfiguration > from framework.testbed_model.traffic_generator.capturing_traffic_generat= or import ( > PacketFilteringConfig, > ) > diff --git a/dts/framework/testbed_model/topology.py b/dts/framework/test= bed_model/topology.py > index 0bad59d2a4..caee9b22ea 100644 > --- a/dts/framework/testbed_model/topology.py > +++ b/dts/framework/testbed_model/topology.py > @@ -16,7 +16,7 @@ > else: > from aenum import NoAliasEnum > > -from framework.config import PortConfig > +from framework.config.node import PortConfig > from framework.exception import ConfigurationError > > from .port import Port > diff --git a/dts/framework/testbed_model/traffic_generator/__init__.py b/= dts/framework/testbed_model/traffic_generator/__init__.py > index e501f6d5ee..922875f401 100644 > --- a/dts/framework/testbed_model/traffic_generator/__init__.py > +++ b/dts/framework/testbed_model/traffic_generator/__init__.py > @@ -14,7 +14,7 @@ > and a capturing traffic generator is required. > """ > > -from framework.config import ScapyTrafficGeneratorConfig, TrafficGenerat= orConfig > +from framework.config.node import ScapyTrafficGeneratorConfig, TrafficGe= neratorConfig > from framework.exception import ConfigurationError > from framework.testbed_model.node import Node > > diff --git a/dts/framework/testbed_model/traffic_generator/scapy.py b/dts= /framework/testbed_model/traffic_generator/scapy.py > index a16cdf6758..c9c7dac54a 100644 > --- a/dts/framework/testbed_model/traffic_generator/scapy.py > +++ b/dts/framework/testbed_model/traffic_generator/scapy.py > @@ -20,7 +20,7 @@ > from scapy.layers.l2 import Ether > from scapy.packet import Packet > > -from framework.config import OS, ScapyTrafficGeneratorConfig > +from framework.config.node import OS, ScapyTrafficGeneratorConfig > from framework.remote_session.python_shell import PythonShell > from framework.testbed_model.node import Node > from framework.testbed_model.port import Port > diff --git a/dts/framework/testbed_model/traffic_generator/traffic_genera= tor.py b/dts/framework/testbed_model/traffic_generator/traffic_generator.py > index a07538cc98..9b4d5dc80a 100644 > --- a/dts/framework/testbed_model/traffic_generator/traffic_generator.py > +++ b/dts/framework/testbed_model/traffic_generator/traffic_generator.py > @@ -12,7 +12,7 @@ > > from scapy.packet import Packet > > -from framework.config import TrafficGeneratorConfig > +from framework.config.node import TrafficGeneratorConfig > from framework.logger import DTSLogger, get_dts_logger > from framework.testbed_model.node import Node > from framework.testbed_model.port import Port > diff --git a/dts/nodes.example.yaml b/dts/nodes.example.yaml > new file mode 100644 > index 0000000000..454d97ab5d > --- /dev/null > +++ b/dts/nodes.example.yaml > @@ -0,0 +1,53 @@ > +# SPDX-License-Identifier: BSD-3-Clause > +# Copyright 2022-2023 The DPDK contributors > +# Copyright 2023 Arm Limited > + > +# Define a system under test node, having two network ports physically > +# connected to the corresponding ports in TG 1 (the peer node) > +- name: "SUT 1" > + hostname: sut1.change.me.localhost > + user: dtsuser > + os: linux > + ports: > + # sets up the physical link between "SUT 1"@0000:00:08.0 and "TG 1"@= 0000:00:08.0 > + - pci: "0000:00:08.0" > + os_driver_for_dpdk: vfio-pci # OS driver that DPDK will use > + os_driver: i40e # OS driver to bind when the tests ar= e not running > + peer_node: "TG 1" > + peer_pci: "0000:00:08.0" > + # sets up the physical link between "SUT 1"@0000:00:08.1 and "TG 1"@= 0000:00:08.1 > + - pci: "0000:00:08.1" > + os_driver_for_dpdk: vfio-pci > + os_driver: i40e > + peer_node: "TG 1" > + peer_pci: "0000:00:08.1" > + hugepages_2mb: # optional; if removed, will use system hugepage config= uration > + number_of: 256 > + force_first_numa: false > + dpdk_config: > + lcores: "" # use all available logical cores (Skips first core) > + memory_channels: 4 # tells DPDK to use 4 memory channels > +# Define a Scapy traffic generator node, having two network ports > +# physically connected to the corresponding ports in SUT 1 (the peer nod= e). > +- name: "TG 1" > + hostname: tg1.change.me.localhost > + user: dtsuser > + os: linux > + ports: > + # sets up the physical link between "TG 1"@0000:00:08.0 and "SUT 1"@= 0000:00:08.0 > + - pci: "0000:00:08.0" > + os_driver_for_dpdk: rdma > + os_driver: rdma > + peer_node: "SUT 1" > + peer_pci: "0000:00:08.0" > + # sets up the physical link between "SUT 1"@0000:00:08.0 and "TG 1"@= 0000:00:08.0 > + - pci: "0000:00:08.1" > + os_driver_for_dpdk: rdma > + os_driver: rdma > + peer_node: "SUT 1" > + peer_pci: "0000:00:08.1" > + hugepages_2mb: # optional; if removed, will use system hugepage config= uration > + number_of: 256 > + force_first_numa: false > + traffic_generator: > + type: SCAPY > diff --git a/dts/test_runs.example.yaml b/dts/test_runs.example.yaml > new file mode 100644 > index 0000000000..5b6afb153e > --- /dev/null > +++ b/dts/test_runs.example.yaml > @@ -0,0 +1,33 @@ > +# SPDX-License-Identifier: BSD-3-Clause > +# Copyright 2022-2023 The DPDK contributors > +# Copyright 2023 Arm Limited > + > +# Define one test run environment > +- dpdk_build: > + dpdk_location: > + # dpdk_tree: Commented out because `tarball` is defined. > + tarball: dpdk-tarball.tar.xz > + # Either `dpdk_tree` or `tarball` can be defined, but not both. > + remote: false # Optional, defaults to false. If it's true, the `dp= dk_tree` or `tarball` > + # is located on the SUT node, instead of the executi= on host. > + > + # precompiled_build_dir: Commented out because `build_options` is de= fined. > + build_options: > + # the combination of the following two makes CC=3D"ccache gcc" > + compiler: gcc > + compiler_wrapper: ccache # Optional. > + # If `precompiled_build_dir` is defined, DPDK has been pre-built and= the build directory is > + # in a subdirectory of DPDK tree root directory. Otherwise, will be = using the `build_options` > + # to build the DPDK from source. Either `precompiled_build_dir` or `= build_options` can be > + # defined, but not both. > + perf: false # disable performance testing > + func: true # enable functional testing > + skip_smoke_tests: false # optional > + test_suites: # the following test suites will be run in their entirety > + - hello_world > + vdevs: # optional; if removed, vdevs won't be used in the execution > + - "crypto_openssl" > + # The machine running the DPDK test executable > + system_under_test_node: "SUT 1" > + # Traffic generator node to use for this execution environment > + traffic_generator_node: "TG 1" > \ No newline at end of file > diff --git a/dts/tests/TestSuite_smoke_tests.py b/dts/tests/TestSuite_smo= ke_tests.py > index ab5ad44850..7ed266dac0 100644 > --- a/dts/tests/TestSuite_smoke_tests.py > +++ b/dts/tests/TestSuite_smoke_tests.py > @@ -14,7 +14,7 @@ > > import re > > -from framework.config import PortConfig > +from framework.config.node import PortConfig > from framework.remote_session.testpmd_shell import TestPmdShell > from framework.settings import SETTINGS > from framework.test_suite import TestSuite, func_test > -- > 2.43.0 >