Re: [PATCH v1 1/3] dts: rework port attributes in config module

["Juraj Linkeš" <juraj.linkes@pantheon.tech>]

As a general note, it's possible we should wait with these changes for 
the Pydantic changes which could simplify a lot of what we want to do 
with the config (not just this, but also the split and removing excess 
attributes).

On 21. 8. 2024 20:43, Nicholas Pratte wrote:
> The current design requires that a peer pci port is identified so that
> test suites can create the correct port links. While this can work, it
> also creates a lot of room for user error. Instead, devices should be
> given a unique identifier which is referenced in defined test runs.
> 
> Both defined testbeds for the SUT and TG must have an equal number of
> specified ports. In each given array or ports, SUT port 0 is connected
> to TG port 0, SUT port 1 is connected to TG port 1, etc.
> 
> Bugzilla ID: 1478
> 
> Signed-off-by: Nicholas Pratte <npratte@iol.unh.edu>
> ---
>   dts/conf.yaml                              | 32 ++++++-------
>   dts/framework/config/__init__.py           | 12 +++--
>   dts/framework/config/conf_yaml_schema.json | 52 +++++++++++++---------
>   dts/framework/config/types.py              | 19 +++++---
>   4 files changed, 69 insertions(+), 46 deletions(-)
> 
> diff --git a/dts/conf.yaml b/dts/conf.yaml
> index 7d95016e68..16214ee267 100644
> --- a/dts/conf.yaml
> +++ b/dts/conf.yaml
> @@ -20,10 +20,17 @@ test_runs:
>       # The machine running the DPDK test executable
>       system_under_test_node:
>         node_name: "SUT 1"
> +      test_bed:

test_bed is the whole thing we're testing. Let's find a better name. 
We're already using unique identifiers for SUTs and TGs and referencing 
those with node_name, so we could reference port identifiers with 
port_names.

> diff --git a/dts/framework/config/__init__.py b/dts/framework/config/__init__.py
> index df60a5030e..534821ed22 100644
> --- a/dts/framework/config/__init__.py
> +++ b/dts/framework/config/__init__.py
> @@ -151,11 +151,10 @@ class PortConfig:
>       """
>   
>       node: str
> +    name: str
>       pci: str
>       os_driver_for_dpdk: str
>       os_driver: str
> -    peer_node: str
> -    peer_pci: str
>   
>       @classmethod
>       def from_dict(cls, node: str, d: PortConfigDict) -> Self:
> @@ -487,12 +486,19 @@ def from_dict(
>               system_under_test_node, SutNodeConfiguration
>           ), f"Invalid SUT configuration {system_under_test_node}"
>   
> -        tg_name = d["traffic_generator_node"]
> +        tg_name = d["traffic_generator_node"]["node_name"]
>           assert tg_name in node_map, f"Unknown TG {tg_name} in test run {d}"
>           traffic_generator_node = node_map[tg_name]
>           assert isinstance(
>               traffic_generator_node, TGNodeConfiguration
>           ), f"Invalid TG configuration {traffic_generator_node}"
> +        assert len(traffic_generator_node.ports) == len(
> +            system_under_test_node.ports
> +        ), "Insufficient ports defined on nodes."

This checks the number of ports specified in the nodes section, but 
these don't necessarily have to correspond. Consider this scenario:

TG has four ports, two connected to SUT1, two connected to SUT2

With the above, we could have two different test run configurations, one 
with TG and SUT1 and the other with TG and SUT2.

We should check that the number of SUT/TG ports specified in test_run is 
the same. The message should also be more precise - something like "The 
number of ports on SUT/TG nodes specified in test_run is different.".

> +        for port_name in d["system_under_test_node"]["test_bed"]:
> +            assert port_name in {port.name: port for port in system_under_test_node.ports}

This is missing an error message. And we're also creating the dictionary 
for each port, the dictionary could be created before the cycle. Or we 
could just look at sets of ports of nodes and those specified in test_run:
sut_test_run_ports = {port_name for port_name in 
d["system_under_test_node"]["test_bed"]}
tg_test_run_ports = {port_name for port_name in 
d["traffic_generator_node"]["test_bed"]}

assert not sut_test_run_ports - {port.name for port in 
system_under_test_node.ports}, "err msg"
assert not tg_test_run_ports - {port.name for port in 
traffic_generator_node.ports}, "err msg"

We should also enforce that the port name are unique across the node config.

> +        for port_name in d["traffic_generator_node"]["test_bed"]:
> +            assert port_name in {port.name: port for port in traffic_generator_node.ports}
>   
>           vdevs = (
>               d["system_under_test_node"]["vdevs"] if "vdevs" in d["system_under_test_node"] else []
> diff --git a/dts/framework/config/conf_yaml_schema.json b/dts/framework/config/conf_yaml_schema.json
> index f02a310bb5..91667b01cc 100644
> --- a/dts/framework/config/conf_yaml_schema.json
> +++ b/dts/framework/config/conf_yaml_schema.json
> @@ -6,6 +6,10 @@
>         "type": "string",
>         "description": "A unique identifier for a node"
>       },
> +    "port_name": {
> +      "type": "string",
> +      "description": "A unique identifier for a node's NIC port."
> +    },
>       "NIC": {
>         "type": "string",
>         "enum": [
> @@ -190,6 +194,24 @@
>           "pmd_buffer_scatter"
>         ]
>       },
> +    "test_run_node": {
> +      "type": "object",
> +      "properties": {
> +        "node_name": {
> +          "$ref": "#/definitions/node_name"
> +        },
> +        "test_bed": {
> +          "type": "array",
> +          "items": {
> +            "$ref": "#/definitions/port_name"
> +          }
> +        }
> +      },
> +      "required": [
> +        "node_name",
> +        "test_bed"
> +      ]

Ports are not actually strictly required, we do have some simple tests 
that don't require any ports (hello world and some of the smoke tests).