From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id D6A8C4590C; Thu, 5 Sep 2024 10:07:37 +0200 (CEST) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 74EF040264; Thu, 5 Sep 2024 10:07:37 +0200 (CEST) Received: from mail-ej1-f43.google.com (mail-ej1-f43.google.com [209.85.218.43]) by mails.dpdk.org (Postfix) with ESMTP id 41EAD4025C for ; Thu, 5 Sep 2024 10:07:36 +0200 (CEST) Received: by mail-ej1-f43.google.com with SMTP id a640c23a62f3a-a89c8db505bso79964066b.0 for ; Thu, 05 Sep 2024 01:07:36 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=pantheon.tech; s=google; t=1725523656; x=1726128456; darn=dpdk.org; h=content-transfer-encoding:in-reply-to:from:content-language :references:cc:to:subject:user-agent:mime-version:date:message-id :from:to:cc:subject:date:message-id:reply-to; bh=WiqXkY6sMr//y9hU5RjB5O5pYYCQ7JgzwhZptu++/gM=; b=MFn8acaEXvQeufxURPKi2BendfNi8PGY4D9v7zV9dvSlfinXcH9P6D7OknEZo2eak3 0sINqPPGN3TeJA3E6u1liwh735JD/GHfi0Z1DMGPVoM25TV+JKJcupU9MUFRsbIDsgEb kDjvReORVhqWrJ5723fN6dK44ZVbXmGzTDu/H6aY/nqhiBcO5hXTVrEihvPbFlTFgdxr 5kK1Vh58+zJTHZ+eQ/HDRARhwfrM2xspKoDg0e2NFVx68HthovQCGKylBrM1wfAimhgj mebO394k84MiMSlfBppDQG97FdNsPmJlJ1YXDWTF3HCYcfhQLLOp5PdqX7o8OUxx2qRH HK3g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1725523656; x=1726128456; h=content-transfer-encoding:in-reply-to:from:content-language :references:cc:to:subject:user-agent:mime-version:date:message-id :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=WiqXkY6sMr//y9hU5RjB5O5pYYCQ7JgzwhZptu++/gM=; b=Hqd0aVQMh5AdztPgzRad9shUNk/iF5EtToQYNu4sqrbTzCfQje92OOH4Y/i371p8Ur 1/h5l+VqkDDyeS4hPzOZvOLKzac5n4SLr3vMsykqN3fbjj1Qbqnoelfvs6CaD2y0EZg+ H6hb4AM274Au/gU8QmPGGjBvCB3iO2FQehQR9Ry9SaVp5P4kW7PHwnhJ7bZIjzSHz5/n K8bimtk9F4yQtLzaxTu4rFLehMk/VDkPJ4wlsx+QWPfLw2dr8zc3wemiPu83nLC7VO3L pjpyJgRY01snAPrMTq8wMYahr9df4L2rtJvpNggEjAqsISLUWthopkDT9TbeLaWiFCyj XW6Q== X-Forwarded-Encrypted: i=1; AJvYcCWYbZgNvh+L0+5eTZXJHWcbv/cMC0JjWlHlySPshtpLdymTRM6+FqEPLE1dY8jfUpBOls8=@dpdk.org X-Gm-Message-State: AOJu0YyxPz4rLv1ebWJMc4dcez1D5CwMi8G3OeJNmCYS3nMZTqLhmiFm zwR37nOXWypy+WX1pjCmgxwBtMIfgBf9Q3IWmAJ7lmeqBSsuWQ6NSO2goJ4CZC0= X-Google-Smtp-Source: AGHT+IGp0+3buwNMgODTt9zzqPwuQDcGskmJbkKtUDk+2s/KUdw+IfHaArCyqq2l5GCv8NhJ4I4mlA== X-Received: by 2002:a17:906:7310:b0:a86:b762:52ec with SMTP id a640c23a62f3a-a8a32f9f1e2mr529592466b.51.1725523655574; Thu, 05 Sep 2024 01:07:35 -0700 (PDT) Received: from [192.168.200.22] ([84.245.121.62]) by smtp.gmail.com with ESMTPSA id a640c23a62f3a-a8a623c2b8csm97897966b.181.2024.09.05.01.07.34 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Thu, 05 Sep 2024 01:07:35 -0700 (PDT) Message-ID: Date: Thu, 5 Sep 2024 10:07:33 +0200 MIME-Version: 1.0 User-Agent: Mozilla Thunderbird Subject: Re: [PATCH v3 03/12] dts: add test case decorators To: Jeremy Spewock Cc: thomas@monjalon.net, Honnappa.Nagarahalli@arm.com, probb@iol.unh.edu, paul.szczepanek@arm.com, Luca.Vizzarro@arm.com, npratte@iol.unh.edu, dmarx@iol.unh.edu, alex.chapman@arm.com, dev@dpdk.org References: <20240301155416.96960-1-juraj.linkes@pantheon.tech> <20240821145315.97974-1-juraj.linkes@pantheon.tech> <20240821145315.97974-4-juraj.linkes@pantheon.tech> Content-Language: en-US From: =?UTF-8?Q?Juraj_Linke=C5=A1?= In-Reply-To: Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 8bit X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org On 26. 8. 2024 18:50, Jeremy Spewock wrote: > On Wed, Aug 21, 2024 at 10:53 AM Juraj Linkeš > wrote: > >> class DTSRunner: >> @@ -232,9 +231,9 @@ def _get_test_suites_with_cases( >> >> for test_suite_config in test_suite_configs: >> test_suite_class = self._get_test_suite_class(test_suite_config.test_suite) >> - test_cases = [] >> - func_test_cases, perf_test_cases = self._filter_test_cases( >> - test_suite_class, test_suite_config.test_cases >> + test_cases: list[type[TestCase]] = [] > > If TestCase is just a class, why is the `type[]` in the annotation > required? Are these not specific instances of the TestCase class? I > figured they would need to be in order for you to run the specific > test case methods. Maybe this has something to do with the class being > a Protocol? > The *_test decorators return type[TestCase]. The functions (test methods) are cast to type[TestCase] (which kinda makes them subclasses of TestCase). This was a suggestion from Luca and I took it as as. Maybe the functions could be cast as instances of TestCase, but I didn't try that. >> + func_test_cases, perf_test_cases = test_suite_class.get_test_cases( >> + test_suite_config.test_cases >> ) >> if func: >> test_cases.extend(func_test_cases) >> @@ -309,57 +308,6 @@ def is_test_suite(object) -> bool: >> f"Couldn't find any valid test suites in {test_suite_module.__name__}." >> ) >> > >> @@ -120,6 +123,68 @@ def _process_links(self) -> None: >> ): >> self._port_links.append(PortLink(sut_port=sut_port, tg_port=tg_port)) >> >> + @classmethod >> + def get_test_cases( >> + cls, test_case_sublist: Sequence[str] | None = None >> + ) -> tuple[set[type["TestCase"]], set[type["TestCase"]]]: >> + """Filter `test_case_subset` from this class. >> + >> + Test cases are regular (or bound) methods decorated with :func:`func_test` >> + or :func:`perf_test`. >> + >> + Args: >> + test_case_sublist: Test case names to filter from this class. >> + If empty or :data:`None`, return all test cases. >> + >> + Returns: >> + The filtered test case functions. This method returns functions as opposed to methods, >> + as methods are bound to instances and this method only has access to the class. >> + >> + Raises: >> + ConfigurationError: If a test case from `test_case_subset` is not found. >> + """ >> + > >> + for test_case_name, test_case_function in inspect.getmembers(cls, is_test_case): >> + if test_case_name in test_case_sublist_copy: >> + # if test_case_sublist_copy is non-empty, remove the found test case >> + # so that we can look at the remainder at the end >> + test_case_sublist_copy.remove(test_case_name) >> + elif test_case_sublist: >> + # if the original list is not empty (meaning we're filtering test cases), >> + # we're dealing with a test case we would've > > I think this part of the comment about "we're dealing with a test case > we would've removed in the other branch" confused me a little bit. It > could just be a me thing, but I think this would have been more clear > for me if it was something more like "The original list is not empty > (meaning we're filtering test cases). Since we didn't remove this test > case in the other branch, it doesn't match the filter and we don't > want to run it." > We should remove any confusion. I'll change it - your wording sound good. >> + # removed in the other branch; since we didn't, we don't want to run it >> + continue >> + >> + match test_case_function.test_type: >> + case TestCaseType.PERFORMANCE: >> + perf_test_cases.add(test_case_function) >> + case TestCaseType.FUNCTIONAL: >> + func_test_cases.add(test_case_function) >> + >> + if test_case_sublist_copy: >> + raise ConfigurationError( >> + f"Test cases {test_case_sublist_copy} not found among functions of {cls.__name__}." >> + ) >> + >> + return func_test_cases, perf_test_cases >> + > >> 2.34.1 >>