From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id E0E6A45D04; Thu, 14 Nov 2024 13:39:32 +0100 (CET) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id A554C4028A; Thu, 14 Nov 2024 13:39:32 +0100 (CET) Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by mails.dpdk.org (Postfix) with ESMTP id 573E84025D for ; Thu, 14 Nov 2024 13:39:31 +0100 (CET) Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 375DA2BC4; Thu, 14 Nov 2024 04:40:00 -0800 (PST) Received: from [10.1.39.61] (JR4XG4HTQC.cambridge.arm.com [10.1.39.61]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id 86D763F6A8; Thu, 14 Nov 2024 04:39:29 -0800 (PST) Message-ID: <48bf44a8-d483-4022-bebf-b90d50c7c5a3@arm.com> Date: Thu, 14 Nov 2024 12:39:27 +0000 MIME-Version: 1.0 User-Agent: Mozilla Thunderbird Subject: Re: [PATCH v3 3/4] dts: add per-test-suite configuration Content-Language: en-GB To: Dean Marx Cc: dev@dpdk.org, Paul Szczepanek , Patrick Robb References: <20240906161355.701688-1-luca.vizzarro@arm.com> <20241108134532.130681-1-luca.vizzarro@arm.com> <20241108134532.130681-4-luca.vizzarro@arm.com> From: Luca Vizzarro In-Reply-To: Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Hi Dean! Thank you for your review! On 13/11/2024 21:44, Dean Marx wrote: > I like the idea of mapping the suite to specific test cases, and for > the most part the custom configuration option as well. The only > thing that I feel should be different is the way the code generation > is documented, I think it might be worth providing an example within > conf.yaml through a comment near the suites section, rather than > just in the dts.rst file. It might be a little more clear where to > create the custom config class as well. So... this is a weird one. I've integrated the code generation under dts-check-format.sh, so nobody should really worry about it, as that should be run before submitting. If it's not, then the CI will eventually pick it up when dts-check-format will be in place. >> +class HelloWorldConfig(TestSuiteConfig): >> + """Example custom configuration for the `TestHelloWorld` >> test suite.""" >> + >> + #: Timeout for the DPDK apps. >> + timeout: int = 50 >> -- >> 2.43.0 > > Additionally, I was a bit confused by the custom config examples, do > these fields (timeout, my_custom_field) actually affect the suite in > any way as of this patch? Or is this just so that we can potentially > add configuration options through this method in the future? The hello_world suite doesn't really need this, I've just added it as an example. But yes test suites are affected. If you see the hello_world suite, it calls `self.config.timeout` which is a reference to this field. This class specifically is now wholly integrated in the configuration: test_runs: - test_suites: hello_world: # this is a mapping to HelloWorldConfig # like before the behavior of this field is: # - if unset or empty list, all test cases are run # - if individual test cases are listed, those are run test_cases: [single_core] # field inherited from TestSuiteConfig timeout: 20 # optional field from HelloWorldConfig What I've added for the simplicity is to be able to represent any TestSuiteConfig as a string instead of a mapping: test_runs: - test_suites: hello_world: all # special keyword which enables all test cases # or hello_world: single_core all_cores # a whitespace-delimited list of test suites Mind that if the underlying test suite configuration has a mandatory field that has no default, then the string representation will fail and Pydantic will complain that the (required) field is missing. Please let me know if I answered every query, and don't hesitate to ask for further clarifications. Best, Luca