From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 393EE46130; Fri, 24 Jan 2025 19:20:18 +0100 (CET) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 133E940B98; Fri, 24 Jan 2025 19:20:18 +0100 (CET) Received: from mail-lj1-f169.google.com (mail-lj1-f169.google.com [209.85.208.169]) by mails.dpdk.org (Postfix) with ESMTP id CAB6D4028A for ; Fri, 24 Jan 2025 19:20:16 +0100 (CET) Received: by mail-lj1-f169.google.com with SMTP id 38308e7fff4ca-30224cee64dso2582241fa.0 for ; Fri, 24 Jan 2025 10:20:16 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=iol.unh.edu; s=unh-iol; t=1737742816; x=1738347616; darn=dpdk.org; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:from:to:cc:subject:date :message-id:reply-to; bh=8PtJ8kIhO5KO4qYs5C9RFvE5oi7d3TTxFyX5kRwQab8=; b=Ard6Xx49/yZu6rQ/xdFRkQAHr6XdSPJF/Zf08xsk5GnjCzj80psqL7KUwAZPFV3HLu 9x/fU56PsRQSvZ9c3SNxwi+jLBF5zjurenY92RYtzvdg2JH9FrBY2CQqok/xQhenwu8v VCKfiKYlsupJ6YUg0ekN1aD+9S+BbkDYTJqV8= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1737742816; x=1738347616; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=8PtJ8kIhO5KO4qYs5C9RFvE5oi7d3TTxFyX5kRwQab8=; b=IMHbErB4Qe5uws49hihpfWs/OGw+1YuZ/NoeYYWfTjHwZOR+HPWm+Vvtp27gA9AF3b XOUGCwjh1mElKz4w6QHgtPhJ+ORZGBQBBsNuMxp1irYBPr+Do7AeWicU635bFtbndXFk Fe0ZRPm1+9DUGaX2Tz2Q7r73R9suB0vBh9OftOeJcRWI094ArEbMiQyccfnRe2bE8oXp 8H3924bKY/9LjU9GVsF5d0JnJc6E6IcFN6POx7dr70/HUu2TPCsU5vN222uVl9C5q5+J QwtDg41uq8sPhjs5MWbBDgaTsvFckJcrKazfxjy/R4IxSovKPpkcSE0Ao5zI/h5gD8sZ d9dg== X-Gm-Message-State: AOJu0YzyTeXmchnJcrOkYj7eb25KI651+4BDReqSFhYYT51c5zs3sd1M d7uqeOSnWRqWHVG966HFQcm97UhozRbidhALUnvt8cim4RbMpdW1P3IbOWT0BftsW+m4fC3XmnO /UtnZcGxXh+KKu7QHRrLkY3O9jwj9vMMgv0+oSA== X-Gm-Gg: ASbGncubyEZJWoVNvvE463TAC3wdppnwrUyfpAw2MjrtrYphYXHafnxNHj7B4jWLm9q s6ldCwp1p+JUhGhtgE8+UlD+5RNeoeztcv47i+GMP/IKdEJYQ+KH9FlW+w2FTaDdDR2ZtdG6Bnn wCTLYVbzR64WimM8eebXfF X-Google-Smtp-Source: AGHT+IH+78WLFv+VBAie3AhBw03Y1iJUetm6osVpFX6u/UG1agdgvcwW0YXJWw+1ssRKLoCkMkBpB94Gkiy5veLHCds= X-Received: by 2002:a05:651c:198c:b0:302:289f:3c5f with SMTP id 38308e7fff4ca-3072cb1133amr35538921fa.8.1737742816212; Fri, 24 Jan 2025 10:20:16 -0800 (PST) MIME-Version: 1.0 References: <20240613201831.9748-3-npratte@iol.unh.edu> <20250124113909.137128-1-luca.vizzarro@arm.com> <20250124113909.137128-8-luca.vizzarro@arm.com> In-Reply-To: <20250124113909.137128-8-luca.vizzarro@arm.com> From: Nicholas Pratte Date: Fri, 24 Jan 2025 13:20:04 -0500 X-Gm-Features: AWEUYZmKugIXGfd_hJ9rfbGgWEftTtRdkzAbYeLGQWFAObSjYTe9e0puSmRNTTE Message-ID: Subject: Re: [PATCH v4 7/7] dts: run all test suites by default To: Luca Vizzarro Cc: dev@dpdk.org, Paul Szczepanek , Dean Marx , Patrick Robb Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Nice touch! Reviewed-by: Nicholas Pratte On Fri, Jan 24, 2025 at 6:39=E2=80=AFAM Luca Vizzarro wrote: > > The configuration requires the user to explicitly set the requested test > suites in the files. Sometimes we want to run all the test suites and > don't want to manually specify all of them. It is therefore reasonable > to change the default behaviour to automatically run all the available > test suites if none are specified. > > Bugzilla ID: 1360 > > Signed-off-by: Luca Vizzarro > Reviewed-by: Paul Szczepanek > Reviewed-by: Dean Marx > --- > dts/framework/config/test_run.py | 16 +++++++++++++++- > dts/test_runs.example.yaml | 3 ++- > 2 files changed, 17 insertions(+), 2 deletions(-) > > diff --git a/dts/framework/config/test_run.py b/dts/framework/config/test= _run.py > index dc0e46047d..006410b467 100644 > --- a/dts/framework/config/test_run.py > +++ b/dts/framework/config/test_run.py > @@ -259,6 +259,20 @@ def validate_names(self) -> Self: > return self > > > +def fetch_all_test_suites() -> list[TestSuiteConfig]: > + """Returns all the available test suites as configuration objects. > + > + This function does not include the smoke tests. > + """ > + from framework.test_suite import AVAILABLE_TEST_SUITES > + > + return [ > + TestSuiteConfig(test_suite=3Dtest_suite.name) > + for test_suite in AVAILABLE_TEST_SUITES > + if test_suite.name !=3D "smoke_tests" > + ] > + > + > class TestRunConfiguration(FrozenModel): > """The configuration of a test run. > > @@ -275,7 +289,7 @@ class TestRunConfiguration(FrozenModel): > #: Whether to skip smoke tests. > skip_smoke_tests: bool =3D False > #: The names of test suites and/or test cases to execute. > - test_suites: list[TestSuiteConfig] =3D Field(min_length=3D1) > + test_suites: list[TestSuiteConfig] =3D Field(default_factory=3Dfetch= _all_test_suites) > #: The SUT node name to use in this test run. > system_under_test_node: str > #: The TG node name to use in this test run. > diff --git a/dts/test_runs.example.yaml b/dts/test_runs.example.yaml > index 5b6afb153e..5cc167ebe1 100644 > --- a/dts/test_runs.example.yaml > +++ b/dts/test_runs.example.yaml > @@ -23,6 +23,7 @@ > perf: false # disable performance testing > func: true # enable functional testing > skip_smoke_tests: false # optional > + # by removing the `test_suites` field, this test run will run every te= st suite available > test_suites: # the following test suites will be run in their entirety > - hello_world > vdevs: # optional; if removed, vdevs won't be used in the execution > @@ -30,4 +31,4 @@ > # The machine running the DPDK test executable > system_under_test_node: "SUT 1" > # Traffic generator node to use for this execution environment > - traffic_generator_node: "TG 1" > \ No newline at end of file > + traffic_generator_node: "TG 1" > -- > 2.43.0 >