From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 5A53545562; Wed, 3 Jul 2024 16:58:44 +0200 (CEST) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 4A7CD427B8; Wed, 3 Jul 2024 16:58:44 +0200 (CEST) Received: from mail-qt1-f180.google.com (mail-qt1-f180.google.com [209.85.160.180]) by mails.dpdk.org (Postfix) with ESMTP id 3F53242DCE for ; Wed, 3 Jul 2024 16:58:42 +0200 (CEST) Received: by mail-qt1-f180.google.com with SMTP id d75a77b69052e-4467b4bcf80so905761cf.0 for ; Wed, 03 Jul 2024 07:58:42 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=iol.unh.edu; s=unh-iol; t=1720018721; x=1720623521; darn=dpdk.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=jajAfJ2Z912nDimDfynYzIGMmm9JV2rOdheTjBOYT/8=; b=brNB+DPSxcw+DjdF/xBtdq4HjkjKUWByjbAIznn3zFylU/BFkweLqfbpZXVRo42+88 glATB5yy17gc9gjUk63GL13oAWjqIIIVHhk5QrPO7EjaX5jvkymayYGviSJ9hkqG5JEW jcroHXy5UnkHva61rsX77jWFuf8t/ktCnt/hs= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1720018721; x=1720623521; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=jajAfJ2Z912nDimDfynYzIGMmm9JV2rOdheTjBOYT/8=; b=KGNegxMtf/xKStwhHnNGO2K7C78v6W2or1fGq9jmeKUo9XdT6NBwLu2N9sYopZTFGb GGAQSZhs6ZvXQK+vDxDZLsmIjmh7nuzZkBjjNbZUnOUurFF4VTPNVfY/OdN2HGByeiwW 6qzfhgwj0+GCcBXcwR4Zpi+qhMLy9CjPfxBHYprSRDqkF0Z829c9FozH57IFWVSTCAnn keEihdX8NNvEB5s5k7rb+M2WQ7jSvYhZAJYc4MVXRo4AcGG5NafnmcRbJPhaI1hQo3TH QKIOzWf5xYBSpmh4bVpbCCP/m0U6uPsBhG7SxZp3G/UgBhjWKBHIDc/7UpOxnOybISrF DELw== X-Gm-Message-State: AOJu0YyoJTm0Tom3WNN/O0kMAUe3drh53LCAIbgx/ALPUyuXbYaq0b3R WOEonQ7T27tWrZDNpxYteptrOmOKc4Ia6BbnFHa8p5IUcbspdA6SU5U5HNeoFA0= X-Google-Smtp-Source: AGHT+IFUxNN2M8ubEtbQfTHHxPqsZq8ojPsp8GdsrSeUzhi/WXLex4mWBDhw/AQ58j7wlYwOJfKNtg== X-Received: by 2002:a05:620a:5687:b0:79b:eca2:b807 with SMTP id af79cd13be357-79d7baba8f6mr1232715885a.3.1720018721310; Wed, 03 Jul 2024 07:58:41 -0700 (PDT) Received: from localhost.unh.edu ([2606:4100:3880:1271:e2f8:4ec3:8bf3:864c]) by smtp.gmail.com with ESMTPSA id af79cd13be357-79d6932ff9csm568365085a.133.2024.07.03.07.58.40 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 03 Jul 2024 07:58:40 -0700 (PDT) From: Nicholas Pratte To: juraj.linkes@pantheon.tech, dmarx@iol.unh.edu, luca.vizzarro@arm.com, Honnappa.Nagarahalli@arm.com, paul.szczepanek@arm.com, probb@iol.unh.edu, jspewock@iol.unh.edu, yoan.picchi@foss.arm.com Cc: dev@dpdk.org, Nicholas Pratte Subject: [PATCH v2] dts: Testbed And Node Configuration Split Date: Wed, 3 Jul 2024 10:57:50 -0400 Message-ID: <20240703145749.23416-2-npratte@iol.unh.edu> X-Mailer: git-send-email 2.44.0 In-Reply-To: <20240610193410.17968-2-npratte@iol.unh.edu> References: <20240610193410.17968-2-npratte@iol.unh.edu> MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org This implementation splits the execution and node configuration components of the conf.yaml into two separate config files. A new command line argument is added, allowing the user to specify both a node configuration file and an execution configuration file. Be default, these config files are now named node_conf.yaml and execution_conf.yaml, respectively. To assert these changes, the schema calls for one of these objects, nodes or executions, in each file, but neither config file can have both. To avoid excess refactoring, both config files are merged together early on in the load_config process. Bugzilla ID: 1344 Signed-off-by: Nicholas Pratte --- dts/execution_conf.yaml | 26 ++ dts/framework/config/__init__.py | 26 +- dts/framework/config/conf_yaml_schema.json | 340 ++++++++++----------- dts/framework/runner.py | 4 +- dts/framework/settings.py | 36 ++- dts/node_conf.yaml | 56 ++++ dts/testbed_conf.yaml | 26 ++ 7 files changed, 325 insertions(+), 189 deletions(-) create mode 100644 dts/execution_conf.yaml create mode 100644 dts/node_conf.yaml create mode 100644 dts/testbed_conf.yaml diff --git a/dts/execution_conf.yaml b/dts/execution_conf.yaml new file mode 100644 index 0000000000..af2180eac2 --- /dev/null +++ b/dts/execution_conf.yaml @@ -0,0 +1,26 @@ +# SPDX-License-Identifier: BSD-3-Clause +# Copyright 2022-2023 The DPDK contributors +# Copyright 2023 Arm Limited + +executions: + # define one execution environment + - build_targets: + - arch: x86_64 + os: linux + cpu: native + # the combination of the following two makes CC="ccache gcc" + compiler: gcc + compiler_wrapper: ccache + perf: false # disable performance testing + func: true # enable functional testing + skip_smoke_tests: false # optional + test_suites: # the following test suites will be run in their entirety + - hello_world + - os_udp + # The machine running the DPDK test executable + system_under_test_node: + node_name: "SUT 1" + vdevs: # optional; if removed, vdevs won't be used in the execution + - "crypto_openssl" + # Traffic generator node to use for this execution environment + traffic_generator_node: "TG 1" \ No newline at end of file diff --git a/dts/framework/config/__init__.py b/dts/framework/config/__init__.py index df60a5030e..2322f3d996 100644 --- a/dts/framework/config/__init__.py +++ b/dts/framework/config/__init__.py @@ -573,29 +573,39 @@ def from_dict(cls, d: ConfigurationDict) -> Self: return cls(test_runs=test_runs) -def load_config(config_file_path: Path) -> Configuration: +def load_config(node_config_file_path: Path, exec_config_file_path: Path) -> Configuration: """Load DTS test run configuration from a file. - Load the YAML test run configuration file + Load both the YAML testbed and execution configuration files and :download:`the configuration file schema `, - validate the test run configuration file, and create a test run configuration object. + validate both configuration files to create a test run configuration object. The YAML test run configuration file is specified in the :option:`--config-file` command line argument or the :envvar:`DTS_CFG_FILE` environment variable. Args: - config_file_path: The path to the YAML test run configuration file. + node_config_file_path: The path to the testbed configuration YAML file. + exec_config_file_path: The path to the execution configuration YAML file. Returns: The parsed test run configuration. """ - with open(config_file_path, "r") as f: - config_data = yaml.safe_load(f) + with open(node_config_file_path, "r") as f: + node_config_data = yaml.safe_load(f) + with open(exec_config_file_path, "r") as f: + execution_config_data = yaml.safe_load(f) schema_path = os.path.join(Path(__file__).parent.resolve(), "conf_yaml_schema.json") with open(schema_path, "r") as f: schema = json.load(f) - config = warlock.model_factory(schema, name="_Config")(config_data) - config_obj: Configuration = Configuration.from_dict(dict(config)) # type: ignore[arg-type] + config = { + **dict(warlock.model_factory(schema, name="_Config")(node_config_data)), + **dict(warlock.model_factory(schema, name="_Config")(execution_config_data)), + } + if "nodes" not in config or "executions" not in config: + raise ConfigurationError( + f"{'node' if 'nodes' not in config else 'execution'} data not configured." + ) + config_obj: Configuration = Configuration.from_dict(config) # type: ignore[arg-type] return config_obj diff --git a/dts/framework/config/conf_yaml_schema.json b/dts/framework/config/conf_yaml_schema.json index f02a310bb5..1962ed05dd 100644 --- a/dts/framework/config/conf_yaml_schema.json +++ b/dts/framework/config/conf_yaml_schema.json @@ -211,192 +211,192 @@ "additionalProperties": false } }, - "type": "object", - "properties": { - "nodes": { - "type": "array", - "items": { - "type": "object", - "properties": { - "name": { - "type": "string", - "description": "A unique identifier for this node" - }, - "hostname": { - "type": "string", - "description": "A hostname from which the node running DTS can access this node. This can also be an IP address." - }, - "user": { - "type": "string", - "description": "The user to access this node with." - }, - "password": { - "type": "string", - "description": "The password to use on this node. Use only as a last resort. SSH keys are STRONGLY preferred." - }, - "arch": { - "$ref": "#/definitions/ARCH" - }, - "os": { - "$ref": "#/definitions/OS" - }, - "lcores": { - "type": "string", - "pattern": "^(([0-9]+|([0-9]+-[0-9]+))(,([0-9]+|([0-9]+-[0-9]+)))*)?$", - "description": "Optional comma-separated list of logical cores to use, e.g.: 1,2,3,4,5,18-22. Defaults to 1. An empty string means use all lcores." - }, - "use_first_core": { - "type": "boolean", - "description": "Indicate whether DPDK should use the first physical core. It won't be used by default." - }, - "memory_channels": { - "type": "integer", - "description": "How many memory channels to use. Optional, defaults to 1." - }, - "hugepages_2mb": { - "$ref": "#/definitions/hugepages_2mb" - }, - "ports": { - "type": "array", - "items": { - "type": "object", - "description": "Each port should be described on both sides of the connection. This makes configuration slightly more verbose but greatly simplifies implementation. If there are inconsistencies, then DTS will not run until that issue is fixed. An example inconsistency would be port 1, node 1 says it is connected to port 1, node 2, but port 1, node 2 says it is connected to port 2, node 1.", - "properties": { - "pci": { - "$ref": "#/definitions/pci_address", - "description": "The local PCI address of the port" - }, - "os_driver_for_dpdk": { - "type": "string", - "description": "The driver that the kernel should bind this device to for DPDK to use it. (ex: vfio-pci)" - }, - "os_driver": { - "type": "string", - "description": "The driver normally used by this port (ex: i40e)" - }, - "peer_node": { - "type": "string", - "description": "The name of the node the peer port is on" + "oneOf": [ + { + "type": "object", + "properties": { + "nodes": { + "type": "array", + "items": { + "type": "object", + "properties": { + "name": { + "type": "string", + "description": "A unique identifier for this node" + }, + "hostname": { + "type": "string", + "description": "A hostname from which the node running DTS can access this node. This can also be an IP address." + }, + "user": { + "type": "string", + "description": "The user to access this node with." + }, + "password": { + "type": "string", + "description": "The password to use on this node. Use only as a last resort. SSH keys are STRONGLY preferred." + }, + "arch": { + "$ref": "#/definitions/ARCH" + }, + "os": { + "$ref": "#/definitions/OS" + }, + "lcores": { + "type": "string", + "pattern": "^(([0-9]+|([0-9]+-[0-9]+))(,([0-9]+|([0-9]+-[0-9]+)))*)?$", + "description": "Optional comma-separated list of logical cores to use, e.g.: 1,2,3,4,5,18-22. Defaults to 1. An empty string means use all lcores." + }, + "use_first_core": { + "type": "boolean", + "description": "Indicate whether DPDK should use the first physical core. It won't be used by default." + }, + "memory_channels": { + "type": "integer", + "description": "How many memory channels to use. Optional, defaults to 1." + }, + "hugepages_2mb": { + "$ref": "#/definitions/hugepages_2mb" + }, + "ports": { + "type": "array", + "items": { + "type": "object", + "description": "Each port should be described on both sides of the connection. This makes configuration slightly more verbose but greatly simplifies implementation. If there are inconsistencies, then DTS will not run until that issue is fixed. An example inconsistency would be port 1, node 1 says it is connected to port 1, node 2, but port 1, node 2 says it is connected to port 2, node 1.", + "properties": { + "pci": { + "$ref": "#/definitions/pci_address", + "description": "The local PCI address of the port" + }, + "os_driver_for_dpdk": { + "type": "string", + "description": "The driver that the kernel should bind this device to for DPDK to use it. (ex: vfio-pci)" + }, + "os_driver": { + "type": "string", + "description": "The driver normally used by this port (ex: i40e)" + }, + "peer_node": { + "type": "string", + "description": "The name of the node the peer port is on" + }, + "peer_pci": { + "$ref": "#/definitions/pci_address", + "description": "The PCI address of the peer port" + } + }, + "additionalProperties": false, + "required": [ + "pci", + "os_driver_for_dpdk", + "os_driver", + "peer_node", + "peer_pci" + ] }, - "peer_pci": { - "$ref": "#/definitions/pci_address", - "description": "The PCI address of the peer port" - } + "minimum": 1 }, - "additionalProperties": false, - "required": [ - "pci", - "os_driver_for_dpdk", - "os_driver", - "peer_node", - "peer_pci" - ] - }, - "minimum": 1 - }, - "traffic_generator": { - "oneOf": [ - { - "type": "object", - "description": "Scapy traffic generator. Used for functional testing.", - "properties": { - "type": { - "type": "string", - "enum": [ - "SCAPY" - ] + "traffic_generator": { + "oneOf": [ + { + "type": "object", + "description": "Scapy traffic generator. Used for functional testing.", + "properties": { + "type": { + "type": "string", + "enum": [ + "SCAPY" + ] + } + } } - } + ] } - ] - } - }, - "additionalProperties": false, - "required": [ - "name", - "hostname", - "user", - "arch", - "os" - ] - }, - "minimum": 1 - }, - "test_runs": { - "type": "array", - "items": { - "type": "object", - "properties": { - "build_targets": { - "type": "array", - "items": { - "$ref": "#/definitions/build_target" }, - "minimum": 1 - }, - "perf": { - "type": "boolean", - "description": "Enable performance testing." - }, - "func": { - "type": "boolean", - "description": "Enable functional testing." - }, - "test_suites": { - "type": "array", - "items": { - "oneOf": [ - { - "$ref": "#/definitions/test_suite" - }, - { - "$ref": "#/definitions/test_target" - } - ] - } - }, - "skip_smoke_tests": { - "description": "Optional field that allows you to skip smoke testing", - "type": "boolean" + "additionalProperties": false, + "required": [ + "name", + "hostname", + "user", + "arch", + "os" + ] }, - "system_under_test_node": { - "type":"object", + "minimum": 1 + }, + "executions": { + "type": "array", + "items": { + "type": "object", "properties": { - "node_name": { - "$ref": "#/definitions/node_name" + "build_targets": { + "type": "array", + "items": { + "$ref": "#/definitions/build_target" + }, + "minimum": 1 + }, + "perf": { + "type": "boolean", + "description": "Enable performance testing." + }, + "func": { + "type": "boolean", + "description": "Enable functional testing." }, - "vdevs": { - "description": "Optional list of names of vdevs to be used in the test run", + "test_suites": { "type": "array", "items": { - "type": "string" + "oneOf": [ + { + "$ref": "#/definitions/test_suite" + }, + { + "$ref": "#/definitions/test_target" + } + ] } + }, + "skip_smoke_tests": { + "description": "Optional field that allows you to skip smoke testing", + "type": "boolean" + }, + "system_under_test_node": { + "type":"object", + "properties": { + "node_name": { + "$ref": "#/definitions/node_name" + }, + "vdevs": { + "description": "Optional list of names of vdevs to be used in execution", + "type": "array", + "items": { + "type": "string" + } + } + }, + "required": [ + "node_name" + ] + }, + "traffic_generator_node": { + "$ref": "#/definitions/node_name" } }, + "additionalProperties": false, "required": [ - "node_name" + "build_targets", + "perf", + "func", + "test_suites", + "system_under_test_node", + "traffic_generator_node" ] }, - "traffic_generator_node": { - "$ref": "#/definitions/node_name" - } - }, - "additionalProperties": false, - "required": [ - "build_targets", - "perf", - "func", - "test_suites", - "system_under_test_node", - "traffic_generator_node" - ] + "minimum": 1 + } }, - "minimum": 1 + "additionalProperties": false } - }, - "required": [ - "test_runs", - "nodes" - ], - "additionalProperties": false + ] } diff --git a/dts/framework/runner.py b/dts/framework/runner.py index 6b6f6a05f5..ac54189025 100644 --- a/dts/framework/runner.py +++ b/dts/framework/runner.py @@ -85,7 +85,9 @@ class DTSRunner: def __init__(self): """Initialize the instance with configuration, logger, result and string constants.""" - self._configuration = load_config(SETTINGS.config_file_path) + self._configuration = load_config( + SETTINGS.node_config_file_path, SETTINGS.exec_config_file_path + ) self._logger = get_dts_logger() if not os.path.exists(SETTINGS.output_dir): os.makedirs(SETTINGS.output_dir) diff --git a/dts/framework/settings.py b/dts/framework/settings.py index f95876113f..48f46d5a46 100644 --- a/dts/framework/settings.py +++ b/dts/framework/settings.py @@ -14,10 +14,15 @@ The command line arguments along with the supported environment variables are: -.. option:: --config-file -.. envvar:: DTS_CFG_FILE +.. option:: --node-config-file +.. envvar:: DTS_NODE_CFG_FILE - The path to the YAML test run configuration file. + The path to the YAML testbed configuration file. + +.. option:: --exec-config-file +.. envvar:: DTS_EXEC_CFG_FILE + + The path to the YAML execution configuration file. .. option:: --output-dir, --output .. envvar:: DTS_OUTPUT_DIR @@ -98,7 +103,9 @@ class Settings: """ #: - config_file_path: Path = Path(__file__).parent.parent.joinpath("conf.yaml") + node_config_file_path: Path = Path(__file__).parent.parent.joinpath("node_conf.yaml") + #: + exec_config_file_path: Path = Path(__file__).parent.parent.joinpath("execution_conf.yaml") #: output_dir: str = "output" #: @@ -267,14 +274,23 @@ def _get_parser() -> _DTSArgumentParser: ) action = parser.add_argument( - "--config-file", - default=SETTINGS.config_file_path, + "--node-config-file", + default=SETTINGS.node_config_file_path, type=Path, - help="The configuration file that describes the test cases, SUTs and targets.", - metavar="FILE_PATH", - dest="config_file_path", + help="[DTS_NODE_CFG_FILE] The configuration file that describes the testbed devices.", + metavar="NODE_FILE_PATH", + dest="node_config_file_path", + ) + _add_env_var_to_action(action, "NODE_CFG_FILE") + action = parser.add_argument( + "--exec-config-file", + default=SETTINGS.exec_config_file_path, + type=Path, + help="[DTS_EXEC_CFG_FILE] The configuration file that describes the test cases.", + metavar="EXEC_FILE_PATH", + dest="exec_config_file_path", ) - _add_env_var_to_action(action, "CFG_FILE") + _add_env_var_to_action(action, "EXEC_CFG_FILE") action = parser.add_argument( "--output-dir", diff --git a/dts/node_conf.yaml b/dts/node_conf.yaml new file mode 100644 index 0000000000..c7e64a00e0 --- /dev/null +++ b/dts/node_conf.yaml @@ -0,0 +1,56 @@ +# SPDX-License-Identifier: BSD-3-Clause +# Copyright 2022-2023 The DPDK contributors +# Copyright 2023 Arm Limited + +nodes: + # Define a system under test node, having two network ports physically + # connected to the corresponding ports in TG 1 (the peer node) + - name: "SUT 1" + hostname: sut1.change.me.localhost + user: dtsuser + arch: x86_64 + os: linux + lcores: "" # use all the available logical cores + use_first_core: false # tells DPDK to use any physical core + memory_channels: 4 # tells DPDK to use 4 memory channels + hugepages: # optional; if removed, will use system hugepage configuration + amount: 256 + force_first_numa: false + ports: + # sets up the physical link between "SUT 1"@000:00:08.0 and "TG 1"@0000:00:08.0 + - pci: "0000:00:08.0" + os_driver_for_dpdk: vfio-pci # OS driver that DPDK will use + os_driver: i40e # OS driver to bind when the tests are not running + peer_node: "TG 1" + peer_pci: "0000:00:08.0" + # sets up the physical link between "SUT 1"@000:00:08.1 and "TG 1"@0000:00:08.1 + - pci: "0000:00:08.1" + os_driver_for_dpdk: vfio-pci + os_driver: i40e + peer_node: "TG 1" + peer_pci: "0000:00:08.1" + # Define a Scapy traffic generator node, having two network ports + # physically connected to the corresponding ports in SUT 1 (the peer node). + - name: "TG 1" + hostname: tg1.change.me.localhost + user: dtsuser + arch: x86_64 + os: linux + ports: + # sets up the physical link between "TG 1"@000:00:08.0 and "SUT 1"@0000:00:08.0 + - pci: "0000:00:08.0" + os_driver_for_dpdk: rdma + os_driver: rdma + peer_node: "SUT 1" + peer_pci: "0000:00:08.0" + # sets up the physical link between "SUT 1"@000:00:08.0 and "TG 1"@0000:00:08.0 + - pci: "0000:00:08.1" + os_driver_for_dpdk: rdma + os_driver: rdma + peer_node: "SUT 1" + peer_pci: "0000:00:08.1" + hugepages: # optional; if removed, will use system hugepage configuration + amount: 256 + force_first_numa: false + traffic_generator: + type: SCAPY diff --git a/dts/testbed_conf.yaml b/dts/testbed_conf.yaml new file mode 100644 index 0000000000..af2180eac2 --- /dev/null +++ b/dts/testbed_conf.yaml @@ -0,0 +1,26 @@ +# SPDX-License-Identifier: BSD-3-Clause +# Copyright 2022-2023 The DPDK contributors +# Copyright 2023 Arm Limited + +executions: + # define one execution environment + - build_targets: + - arch: x86_64 + os: linux + cpu: native + # the combination of the following two makes CC="ccache gcc" + compiler: gcc + compiler_wrapper: ccache + perf: false # disable performance testing + func: true # enable functional testing + skip_smoke_tests: false # optional + test_suites: # the following test suites will be run in their entirety + - hello_world + - os_udp + # The machine running the DPDK test executable + system_under_test_node: + node_name: "SUT 1" + vdevs: # optional; if removed, vdevs won't be used in the execution + - "crypto_openssl" + # Traffic generator node to use for this execution environment + traffic_generator_node: "TG 1" \ No newline at end of file -- 2.44.0