From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id C595845954; Tue, 10 Sep 2024 10:57:54 +0200 (CEST) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id B2F60427B6; Tue, 10 Sep 2024 10:57:52 +0200 (CEST) Received: from mail-lf1-f44.google.com (mail-lf1-f44.google.com [209.85.167.44]) by mails.dpdk.org (Postfix) with ESMTP id 5045B427A7 for ; Tue, 10 Sep 2024 10:57:51 +0200 (CEST) Received: by mail-lf1-f44.google.com with SMTP id 2adb3069b0e04-5365cf5de24so3746928e87.1 for ; Tue, 10 Sep 2024 01:57:51 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=pantheon.tech; s=google; t=1725958671; x=1726563471; darn=dpdk.org; h=content-transfer-encoding:in-reply-to:from:content-language :references:cc:to:subject:user-agent:mime-version:date:message-id :from:to:cc:subject:date:message-id:reply-to; bh=2O8NLRuuDCLeShy2zW0+tDNrdn59qiXSyemR62wMVrM=; b=d6bdWoF8HoPvxVEzC/8tT3XRKEyjbQpYex4QkbMoVsiOzsjX8DAwogD7Hkz1Qm9NI5 wtMomArsMmHFo8MNqee/FRUMMRs+gUnM9acgk1tCXib/oArNH88kQLYEIPwirztYYpii i4EzCJci/2Lh7oJw8slx1upaxvxP9VL60JNmK6uT6qU2RFOd6Q4409OGXWIIllp65tYn +y0Ml262JrSzCyeYwEnaLPolXLo8PSEu8YxGxasiMqB/aOMhY6ANNEL364GdWiT4Lr1m vWgk0QbIurNRPh1obwCgLpL2BEAk8hBa9mqPgUpsnV7y1i/ChEFweHh1CDynhNtCu4+M jnbg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1725958671; x=1726563471; h=content-transfer-encoding:in-reply-to:from:content-language :references:cc:to:subject:user-agent:mime-version:date:message-id :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=2O8NLRuuDCLeShy2zW0+tDNrdn59qiXSyemR62wMVrM=; b=jfhCjvi+oooM/B1P6zTLg/eOesyCsux7XhCIkPMAb11CoxJhklG4ZP50KEsSH7khQz 4pKgPn6RUr1cgTprW0AXa7F5i6/JgSyuod5D5rDM1o07XjcvE3HSvURwSL9aFZTbaFl1 HM219ZiAVrGFMCOYEZRRv9wxcTgpH4VovWzT3UpAUDt7E5G6DE0ItLMOUVXcfRbo+/ce EVTAtZnHZll5/5iiE6Shoyh1MFOOL25E+624qD/wgcBr0f+ZW098ogoXQsHeDjZnkAxb 6+LhwOY2xWxbcp8iHd3YW+ftXWgES3WlmZuUe6UIwB0q1suMvER+7kFSDir80yXa7KWJ d70g== X-Forwarded-Encrypted: i=1; AJvYcCVctMGe4PEDT6ZRw8h1bdBxU6IPuMo8fhJGOqV++axdWKA+3XQhViw7Nea0VhC/DSRlSsg=@dpdk.org X-Gm-Message-State: AOJu0YzS87Fx9djgw5cQEWH/cnVtVHgoZEQWUuejD7f96cqFr94JS4C1 jUhpZQ1bUmSMXzZMpzwPu8tilUq6lNMxyxV44DC1/swURjW36rx9VK8bn2krAdE= X-Google-Smtp-Source: AGHT+IGFlnPtBIkPO2+06zqtf2A0ZIbfolRCKT9SHkoZxjBuuFaM5JOHqguZLd97JMuailDGOU8uWw== X-Received: by 2002:a05:6512:2389:b0:533:901:e441 with SMTP id 2adb3069b0e04-536587a54f5mr6911892e87.10.1725958670393; Tue, 10 Sep 2024 01:57:50 -0700 (PDT) Received: from [10.12.0.236] (81.89.53.154.host.vnet.sk. [81.89.53.154]) by smtp.gmail.com with ESMTPSA id a640c23a62f3a-a8d25d5ddc0sm447085766b.188.2024.09.10.01.57.49 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Tue, 10 Sep 2024 01:57:50 -0700 (PDT) Message-ID: Date: Tue, 10 Sep 2024 10:57:49 +0200 MIME-Version: 1.0 User-Agent: Mozilla Thunderbird Subject: Re: [PATCH v1] dts: Testbed And Node Configuration Split To: Jeremy Spewock , Nicholas Pratte Cc: luca.vizzarro@arm.com, probb@iol.unh.edu, bruce.richardson@intel.com, yoan.picchi@foss.arm.com, paul.szczepanek@arm.com, Honnappa.Nagarahalli@arm.com, dev@dpdk.org References: <20240610193410.17968-2-npratte@iol.unh.edu> Content-Language: en-US From: =?UTF-8?Q?Juraj_Linke=C5=A1?= In-Reply-To: Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 8bit X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org On 14. 6. 2024 20:32, Jeremy Spewock wrote: > On Mon, Jun 10, 2024 at 3:37 PM Nicholas Pratte wrote: >> >> >> >>> -def load_config(config_file_path: Path) -> Configuration: >>> +def load_config(node_config_file_path: Path, exec_config_file_path: Path) -> Configuration: >>> """Load DTS test run configuration from a file. >>> >>> - Load the YAML test run configuration file >>> + Load both the YAML testbed and execution configuration files >>> and :download:`the configuration file schema `, >>> - validate the test run configuration file, and create a test run configuration object. >>> + validate both configuration files to create a test run configuration object. >>> >>> The YAML test run configuration file is specified in the :option:`--config-file` command line >>> argument or the :envvar:`DTS_CFG_FILE` environment variable. >>> >>> Args: >>> - config_file_path: The path to the YAML test run configuration file. >>> + node_config_file_path: The path to the testbed configuration YAML file. >>> + exec_config_file_path: The path to the execution configuration YAML file. >>> >>> Returns: >>> The parsed test run configuration. >>> """ >>> - with open(config_file_path, "r") as f: >>> - config_data = yaml.safe_load(f) >>> + with open(node_config_file_path, "r") as f: >>> + node_config_data = yaml.safe_load(f) >>> + with open(exec_config_file_path, "r") as f: >>> + execution_config_data = yaml.safe_load(f) >>> >>> schema_path = os.path.join(Path(__file__).parent.resolve(), "conf_yaml_schema.json") >>> >>> with open(schema_path, "r") as f: >>> schema = json.load(f) >>> - config = warlock.model_factory(schema, name="_Config")(config_data) >>> - config_obj: Configuration = Configuration.from_dict(dict(config)) # type: ignore[arg-type] >>> + config = { >>> + **dict(warlock.model_factory(schema, name="_Config")(node_config_data)), >>> + **dict(warlock.model_factory(schema, name="_Config")(execution_config_data)), >>> + } >>> + if "nodes" not in config or "executions" not in config: >>> + raise ConfigurationError( >>> + f"{'node' if 'nodes' not in config else 'execution'} data not configured." >>> + ) >>> + config_obj: Configuration = Configuration.from_dict(config) # type: ignore[arg-type] >>> return config_obj >> >> There is an alternative approach to this that I currently have >> implemented on a separate local branch. Essentially, both configs are >> merged together before being validated by the schema, effectively >> removing the need to change the schema at all as well as the above >> assertion that both nodes and executions are in the two config files. >> However, using this alternative method might mean that there is no added >> control that prevents users from making funky errors when creating >> both config files. I could also just be overthinking or missing >> something. > > That's an interesting idea and on one hand it makes the schema a > little less confusing because it wouldn't need to say "oneOf" when > both of the attributes are actually required for DTS to run, but on > the other it removes the ability for developers to know if they made a > mistake without actually running DTS. Making it a runtime error > wouldn't be the worst thing since it would at least be very early in > the run that the error is shown, but it loses some of its purpose I > feel if your IDE doesn't also show you some of the errors before > actually using the config. I could go either way on it personally. > Since we're splitting the file, we can also split the schema. I like splitting the schema since it'll be much clearer which files the schemas describe (if we have everything in one schema, we have to carefully look for which parts describe which file). It's also going to be easier to maintain.