[PATCH v1 0/1] dts: add EAL sanity check suite

[PATCH v1 0/1] dts: add EAL sanity check suite

[Dean Marx]

Adds a test suite to DTS which the user can have as
a sanity check for their EAL configuration. This can
be used as a replacement for the hello_world suite.

Dean Marx (1):
  dts: add EAL sanity check suite

 dts/tests/TestSuite_eal.py | 28 ++++++++++++++++++++++++++++
 1 file changed, 28 insertions(+)
 create mode 100644 dts/tests/TestSuite_eal.py

-- 
2.44.0

[PATCH v1 1/1] dts: add EAL sanity check suite

[Dean Marx]

Add a test suite to replace hello_world which simply
starts and stops a testpmd session. The user can use
this as a sanity check to verify their configuration.

Signed-off-by: Dean Marx <dmarx@iol.unh.edu>
---
 dts/tests/TestSuite_eal.py | 28 ++++++++++++++++++++++++++++
 1 file changed, 28 insertions(+)
 create mode 100644 dts/tests/TestSuite_eal.py

diff --git a/dts/tests/TestSuite_eal.py b/dts/tests/TestSuite_eal.py
new file mode 100644
index 0000000000..35660178f0
--- /dev/null
+++ b/dts/tests/TestSuite_eal.py
@@ -0,0 +1,28 @@
+# SPDX-License-Identifier: BSD-3-Clause
+# Copyright(c) 2024 University of New Hampshire
+
+"""DPDK EAL sanity check suite.
+
+Starts and stops a testpmd session to verify EAL parameters
+are properly configured.
+"""
+
+from framework.remote_session.testpmd_shell import TestPmdShell
+from framework.test_suite import TestSuite, func_test
+
+
+class TestEal(TestSuite):
+    """EAL test suite. One test case, which starts and stops a testpmd session."""
+
+    @func_test
+    def test_verify_eal(self) -> None:
+        """EAL sanity test.
+
+        Steps:
+            Start testpmd session and check status.
+        Verify:
+            The testpmd session is alive after starting.
+        """
+        with TestPmdShell(node=self.sut_node) as testpmd:
+            testpmd.start()
+            self.verify(True, "True")
-- 
2.44.0

Re: [PATCH v1 1/1] dts: add EAL sanity check suite

[Paul Szczepanek]

On 09/12/2024 21:54, Dean Marx wrote:
> Add a test suite to replace hello_world which simply
> starts and stops a testpmd session. The user can use
> this as a sanity check to verify their configuration.
> 
> Signed-off-by: Dean Marx <dmarx@iol.unh.edu>
> ---
>  dts/tests/TestSuite_eal.py | 28 ++++++++++++++++++++++++++++
>  1 file changed, 28 insertions(+)
>  create mode 100644 dts/tests/TestSuite_eal.py
> 
> diff --git a/dts/tests/TestSuite_eal.py b/dts/tests/TestSuite_eal.py
> new file mode 100644
> index 0000000000..35660178f0
> --- /dev/null
> +++ b/dts/tests/TestSuite_eal.py
> @@ -0,0 +1,28 @@
> +# SPDX-License-Identifier: BSD-3-Clause
> +# Copyright(c) 2024 University of New Hampshire
> +
> +"""DPDK EAL sanity check suite.
> +
> +Starts and stops a testpmd session to verify EAL parameters
> +are properly configured.
> +"""
> +
> +from framework.remote_session.testpmd_shell import TestPmdShell
> +from framework.test_suite import TestSuite, func_test
> +
> +
> +class TestEal(TestSuite):
> +    """EAL test suite. One test case, which starts and stops a testpmd session."""
> +
> +    @func_test
> +    def test_verify_eal(self) -> None:
> +        """EAL sanity test.
> +
> +        Steps:
> +            Start testpmd session and check status.
> +        Verify:
> +            The testpmd session is alive after starting.
> +        """
> +        with TestPmdShell(node=self.sut_node) as testpmd:
> +            testpmd.start()
> +            self.verify(True, "True")

Not sure the self.verify(True, "True") is needed.

Reviewed-by: Paul Szczepanek <paul.szczepanek@arm.com>

Re: [PATCH v1 1/1] dts: add EAL sanity check suite

[Stephen Hemminger]

On Fri, 20 Dec 2024 15:49:10 +0000
Paul Szczepanek <paul.szczepanek@arm.com> wrote:

> On 09/12/2024 21:54, Dean Marx wrote:
> > Add a test suite to replace hello_world which simply
> > starts and stops a testpmd session. The user can use
> > this as a sanity check to verify their configuration.
> > 
> > Signed-off-by: Dean Marx <dmarx@iol.unh.edu>
> > ---
> >  dts/tests/TestSuite_eal.py | 28 ++++++++++++++++++++++++++++
> >  1 file changed, 28 insertions(+)
> >  create mode 100644 dts/tests/TestSuite_eal.py
> > 
> > diff --git a/dts/tests/TestSuite_eal.py b/dts/tests/TestSuite_eal.py
> > new file mode 100644
> > index 0000000000..35660178f0
> > --- /dev/null
> > +++ b/dts/tests/TestSuite_eal.py
> > @@ -0,0 +1,28 @@
> > +# SPDX-License-Identifier: BSD-3-Clause
> > +# Copyright(c) 2024 University of New Hampshire
> > +
> > +"""DPDK EAL sanity check suite.

Do not use the term "sanity check".

https://inclusivenaming.org/word-lists/tier-2/sanity-check/

[PATCH v2 1/1] dts: add EAL confidence check suite

[Dean Marx]

Add a test suite to replace hello_world which simply
starts and stops a testpmd session. The user can use
this as a confidence check to verify their configuration.

Signed-off-by: Dean Marx <dmarx@iol.unh.edu>
Reviewed-by: Paul Szczepanek <paul.szczepanek@arm.com>
---
 dts/tests/TestSuite_eal.py | 27 +++++++++++++++++++++++++++
 1 file changed, 27 insertions(+)
 create mode 100644 dts/tests/TestSuite_eal.py

diff --git a/dts/tests/TestSuite_eal.py b/dts/tests/TestSuite_eal.py
new file mode 100644
index 0000000000..7cf96e61cd
--- /dev/null
+++ b/dts/tests/TestSuite_eal.py
@@ -0,0 +1,27 @@
+# SPDX-License-Identifier: BSD-3-Clause
+# Copyright(c) 2024 University of New Hampshire
+
+"""DPDK EAL confidence check suite.
+
+Starts and stops a testpmd session to verify EAL parameters
+are properly configured.
+"""
+
+from framework.remote_session.testpmd_shell import TestPmdShell
+from framework.test_suite import TestSuite, func_test
+
+
+class TestEal(TestSuite):
+    """EAL test suite. One test case, which starts and stops a testpmd session."""
+
+    @func_test
+    def test_verify_eal(self) -> None:
+        """EAL confidence test.
+
+        Steps:
+            Start testpmd session and check status.
+        Verify:
+            The testpmd session is alive after starting.
+        """
+        with TestPmdShell(node=self.sut_node) as testpmd:
+            testpmd.start()
-- 
2.44.0

Re: [PATCH v2 1/1] dts: add EAL confidence check suite

[Patrick Robb]

[-- Attachment #1: Type: text/plain, Size: 1863 bytes --]

Hey Dean,

I would retain the hello world name if I were you, in place of "eal" as you
use here. I don't want to give off the impression that we are testing EAL
in some intentional way, as opposed to providing a reference testsuite.

On Mon, Dec 23, 2024 at 12:56 PM Dean Marx <dmarx@iol.unh.edu> wrote:

> Add a test suite to replace hello_world which simply
> starts and stops a testpmd session. The user can use
> this as a confidence check to verify their configuration.
>
> Signed-off-by: Dean Marx <dmarx@iol.unh.edu>
> Reviewed-by: Paul Szczepanek <paul.szczepanek@arm.com>
> ---
>  dts/tests/TestSuite_eal.py | 27 +++++++++++++++++++++++++++
>  1 file changed, 27 insertions(+)
>  create mode 100644 dts/tests/TestSuite_eal.py
>
> diff --git a/dts/tests/TestSuite_eal.py b/dts/tests/TestSuite_eal.py
> new file mode 100644
> index 0000000000..7cf96e61cd
> --- /dev/null
> +++ b/dts/tests/TestSuite_eal.py
> @@ -0,0 +1,27 @@
> +# SPDX-License-Identifier: BSD-3-Clause
> +# Copyright(c) 2024 University of New Hampshire
> +
> +"""DPDK EAL confidence check suite.
> +
> +Starts and stops a testpmd session to verify EAL parameters
> +are properly configured.
> +"""
> +
> +from framework.remote_session.testpmd_shell import TestPmdShell
> +from framework.test_suite import TestSuite, func_test
> +
> +
> +class TestEal(TestSuite):
> +    """EAL test suite. One test case, which starts and stops a testpmd
> session."""
> +
> +    @func_test
> +    def test_verify_eal(self) -> None:
> +        """EAL confidence test.
> +
> +        Steps:
> +            Start testpmd session and check status.
> +        Verify:
> +            The testpmd session is alive after starting.
> +        """
> +        with TestPmdShell(node=self.sut_node) as testpmd:
> +            testpmd.start()
> --
> 2.44.0
>
>

[-- Attachment #2: Type: text/html, Size: 2554 bytes --]

[PATCH v3] dts: reform hello world test suite

[Dean Marx]

Add a test suite to replace hello_world which simply
starts and stops a testpmd session. The user can use
this as a confidence check to verify their configuration.

Signed-off-by: Dean Marx <dmarx@iol.unh.edu>
Reviewed-by: Paul Szczepanek <paul.szczepanek@arm.com>
---
 dts/tests/TestSuite_hello_world.py | 68 ++++++------------------------
 1 file changed, 12 insertions(+), 56 deletions(-)

diff --git a/dts/tests/TestSuite_hello_world.py b/dts/tests/TestSuite_hello_world.py
index 734f006026..75f168bc9f 100644
--- a/dts/tests/TestSuite_hello_world.py
+++ b/dts/tests/TestSuite_hello_world.py
@@ -1,71 +1,27 @@
 # SPDX-License-Identifier: BSD-3-Clause
-# Copyright(c) 2010-2014 Intel Corporation
+# Copyright(c) 2024 University of New Hampshire
 
-"""The DPDK hello world app test suite.
+"""DPDK Hello World test suite.
 
-Run the helloworld example app and verify it prints a message for each used core.
-No other EAL parameters apart from cores are used.
+Starts and stops a testpmd session to verify EAL parameters
+are properly configured.
 """
 
-from framework.remote_session.dpdk_shell import compute_eal_params
+from framework.remote_session.testpmd_shell import TestPmdShell
 from framework.test_suite import TestSuite, func_test
-from framework.testbed_model.capability import TopologyType, requires
-from framework.testbed_model.cpu import (
-    LogicalCoreCount,
-    LogicalCoreCountFilter,
-    LogicalCoreList,
-)
 
 
-@requires(topology_type=TopologyType.no_link)
 class TestHelloWorld(TestSuite):
-    """DPDK hello world app test suite."""
-
-    def set_up_suite(self) -> None:
-        """Set up the test suite.
-
-        Setup:
-            Build the app we're about to test - helloworld.
-        """
-        self.app_helloworld_path = self.sut_node.build_dpdk_app("helloworld")
-
-    @func_test
-    def hello_world_single_core(self) -> None:
-        """Single core test case.
-
-        Steps:
-            Run the helloworld app on the first usable logical core.
-        Verify:
-            The app prints a message from the used core:
-            "hello from core <core_id>"
-        """
-        # get the first usable core
-        lcore_amount = LogicalCoreCount(1, 1, 1)
-        lcores = LogicalCoreCountFilter(self.sut_node.lcores, lcore_amount).filter()
-        eal_para = compute_eal_params(self.sut_node, lcore_filter_specifier=lcore_amount)
-        result = self.sut_node.run_dpdk_app(self.app_helloworld_path, eal_para)
-        self.verify(
-            f"hello from core {int(lcores[0])}" in result.stdout,
-            f"helloworld didn't start on lcore{lcores[0]}",
-        )
+    """Hello World test suite. One test case, which starts and stops a testpmd session."""
 
     @func_test
-    def hello_world_all_cores(self) -> None:
-        """All cores test case.
+    def test_hello_world(self) -> None:
+        """EAL confidence test.
 
         Steps:
-            Run the helloworld app on all usable logical cores.
+            Start testpmd session and check status.
         Verify:
-            The app prints a message from all used cores:
-            "hello from core <core_id>"
+            The testpmd session throws no errors.
         """
-        # get the maximum logical core number
-        eal_para = compute_eal_params(
-            self.sut_node, lcore_filter_specifier=LogicalCoreList(self.sut_node.lcores)
-        )
-        result = self.sut_node.run_dpdk_app(self.app_helloworld_path, eal_para, 50)
-        for lcore in self.sut_node.lcores:
-            self.verify(
-                f"hello from core {int(lcore)}" in result.stdout,
-                f"helloworld didn't start on lcore{lcore}",
-            )
+        with TestPmdShell(node=self.sut_node) as testpmd:
+            testpmd.start()
-- 
2.44.0

Re: [PATCH v3] dts: reform hello world test suite

[Luca Vizzarro]

Reviewed-by: Luca Vizzarro <luca.vizzarro@arm.com>

Re: [PATCH v3] dts: reform hello world test suite

[Patrick Robb]

[-- Attachment #1: Type: text/plain, Size: 4317 bytes --]

Actually, would you want to use logger to print a "hello world" message. It
might make the testsuite name make more sense hah.

On Mon, Jan 6, 2025 at 11:57 AM Dean Marx <dmarx@iol.unh.edu> wrote:

> Add a test suite to replace hello_world which simply
> starts and stops a testpmd session. The user can use
> this as a confidence check to verify their configuration.
>
> Signed-off-by: Dean Marx <dmarx@iol.unh.edu>
> Reviewed-by: Paul Szczepanek <paul.szczepanek@arm.com>
> ---
>  dts/tests/TestSuite_hello_world.py | 68 ++++++------------------------
>  1 file changed, 12 insertions(+), 56 deletions(-)
>
> diff --git a/dts/tests/TestSuite_hello_world.py
> b/dts/tests/TestSuite_hello_world.py
> index 734f006026..75f168bc9f 100644
> --- a/dts/tests/TestSuite_hello_world.py
> +++ b/dts/tests/TestSuite_hello_world.py
> @@ -1,71 +1,27 @@
>  # SPDX-License-Identifier: BSD-3-Clause
> -# Copyright(c) 2010-2014 Intel Corporation
> +# Copyright(c) 2024 University of New Hampshire
>
> -"""The DPDK hello world app test suite.
> +"""DPDK Hello World test suite.
>
> -Run the helloworld example app and verify it prints a message for each
> used core.
> -No other EAL parameters apart from cores are used.
> +Starts and stops a testpmd session to verify EAL parameters
> +are properly configured.
>  """
>
> -from framework.remote_session.dpdk_shell import compute_eal_params
> +from framework.remote_session.testpmd_shell import TestPmdShell
>  from framework.test_suite import TestSuite, func_test
> -from framework.testbed_model.capability import TopologyType, requires
> -from framework.testbed_model.cpu import (
> -    LogicalCoreCount,
> -    LogicalCoreCountFilter,
> -    LogicalCoreList,
> -)
>
>
> -@requires(topology_type=TopologyType.no_link)
>  class TestHelloWorld(TestSuite):
> -    """DPDK hello world app test suite."""
> -
> -    def set_up_suite(self) -> None:
> -        """Set up the test suite.
> -
> -        Setup:
> -            Build the app we're about to test - helloworld.
> -        """
> -        self.app_helloworld_path =
> self.sut_node.build_dpdk_app("helloworld")
> -
> -    @func_test
> -    def hello_world_single_core(self) -> None:
> -        """Single core test case.
> -
> -        Steps:
> -            Run the helloworld app on the first usable logical core.
> -        Verify:
> -            The app prints a message from the used core:
> -            "hello from core <core_id>"
> -        """
> -        # get the first usable core
> -        lcore_amount = LogicalCoreCount(1, 1, 1)
> -        lcores = LogicalCoreCountFilter(self.sut_node.lcores,
> lcore_amount).filter()
> -        eal_para = compute_eal_params(self.sut_node,
> lcore_filter_specifier=lcore_amount)
> -        result = self.sut_node.run_dpdk_app(self.app_helloworld_path,
> eal_para)
> -        self.verify(
> -            f"hello from core {int(lcores[0])}" in result.stdout,
> -            f"helloworld didn't start on lcore{lcores[0]}",
> -        )
> +    """Hello World test suite. One test case, which starts and stops a
> testpmd session."""
>
>      @func_test
> -    def hello_world_all_cores(self) -> None:
> -        """All cores test case.
> +    def test_hello_world(self) -> None:
> +        """EAL confidence test.
>
>          Steps:
> -            Run the helloworld app on all usable logical cores.
> +            Start testpmd session and check status.
>          Verify:
> -            The app prints a message from all used cores:
> -            "hello from core <core_id>"
> +            The testpmd session throws no errors.
>          """
> -        # get the maximum logical core number
> -        eal_para = compute_eal_params(
> -            self.sut_node,
> lcore_filter_specifier=LogicalCoreList(self.sut_node.lcores)
> -        )
> -        result = self.sut_node.run_dpdk_app(self.app_helloworld_path,
> eal_para, 50)
> -        for lcore in self.sut_node.lcores:
> -            self.verify(
> -                f"hello from core {int(lcore)}" in result.stdout,
> -                f"helloworld didn't start on lcore{lcore}",
> -            )
> +        with TestPmdShell(node=self.sut_node) as testpmd:
> +            testpmd.start()
> --
> 2.44.0
>
>

[-- Attachment #2: Type: text/html, Size: 5491 bytes --]

[PATCH v4] dts: reform hello world test suite

[Dean Marx]

Add a test suite to replace hello_world which simply
starts and stops a testpmd session. The user can use
this as a confidence check to verify their configuration.

Signed-off-by: Dean Marx <dmarx@iol.unh.edu>
Reviewed-by: Paul Szczepanek <paul.szczepanek@arm.com>
Reviewed-by: Luca Vizzarro <luca.vizzarro@arm.com>
---
 dts/tests/TestSuite_hello_world.py | 70 ++++++------------------------
 1 file changed, 14 insertions(+), 56 deletions(-)

diff --git a/dts/tests/TestSuite_hello_world.py b/dts/tests/TestSuite_hello_world.py
index 734f006026..bad35f3267 100644
--- a/dts/tests/TestSuite_hello_world.py
+++ b/dts/tests/TestSuite_hello_world.py
@@ -1,71 +1,29 @@
 # SPDX-License-Identifier: BSD-3-Clause
-# Copyright(c) 2010-2014 Intel Corporation
+# Copyright(c) 2024 University of New Hampshire
 
-"""The DPDK hello world app test suite.
+"""DPDK Hello World test suite.
 
-Run the helloworld example app and verify it prints a message for each used core.
-No other EAL parameters apart from cores are used.
+Starts and stops a testpmd session to verify EAL parameters
+are properly configured.
 """
 
-from framework.remote_session.dpdk_shell import compute_eal_params
+from framework.remote_session.testpmd_shell import TestPmdShell
+from framework import logger
 from framework.test_suite import TestSuite, func_test
-from framework.testbed_model.capability import TopologyType, requires
-from framework.testbed_model.cpu import (
-    LogicalCoreCount,
-    LogicalCoreCountFilter,
-    LogicalCoreList,
-)
 
 
-@requires(topology_type=TopologyType.no_link)
 class TestHelloWorld(TestSuite):
-    """DPDK hello world app test suite."""
-
-    def set_up_suite(self) -> None:
-        """Set up the test suite.
-
-        Setup:
-            Build the app we're about to test - helloworld.
-        """
-        self.app_helloworld_path = self.sut_node.build_dpdk_app("helloworld")
-
-    @func_test
-    def hello_world_single_core(self) -> None:
-        """Single core test case.
-
-        Steps:
-            Run the helloworld app on the first usable logical core.
-        Verify:
-            The app prints a message from the used core:
-            "hello from core <core_id>"
-        """
-        # get the first usable core
-        lcore_amount = LogicalCoreCount(1, 1, 1)
-        lcores = LogicalCoreCountFilter(self.sut_node.lcores, lcore_amount).filter()
-        eal_para = compute_eal_params(self.sut_node, lcore_filter_specifier=lcore_amount)
-        result = self.sut_node.run_dpdk_app(self.app_helloworld_path, eal_para)
-        self.verify(
-            f"hello from core {int(lcores[0])}" in result.stdout,
-            f"helloworld didn't start on lcore{lcores[0]}",
-        )
+    """Hello World test suite. One test case, which starts and stops a testpmd session."""
 
     @func_test
-    def hello_world_all_cores(self) -> None:
-        """All cores test case.
+    def test_hello_world(self) -> None:
+        """EAL confidence test.
 
         Steps:
-            Run the helloworld app on all usable logical cores.
+            Start testpmd session and check status.
         Verify:
-            The app prints a message from all used cores:
-            "hello from core <core_id>"
+            The testpmd session throws no errors.
         """
-        # get the maximum logical core number
-        eal_para = compute_eal_params(
-            self.sut_node, lcore_filter_specifier=LogicalCoreList(self.sut_node.lcores)
-        )
-        result = self.sut_node.run_dpdk_app(self.app_helloworld_path, eal_para, 50)
-        for lcore in self.sut_node.lcores:
-            self.verify(
-                f"hello from core {int(lcore)}" in result.stdout,
-                f"helloworld didn't start on lcore{lcore}",
-            )
+        with TestPmdShell(node=self.sut_node) as testpmd:
+            testpmd.start()
+        self._logger.log(1, "Hello World!")
-- 
2.44.0

Re: [PATCH v4] dts: reform hello world test suite

[Paul Szczepanek]

On 07/01/2025 21:34, Dean Marx wrote:> +        with
TestPmdShell(node=self.sut_node) as testpmd:
> +            testpmd.start()
> +        self._logger.log(1, "Hello World!")

You should not be using private members like _logger
you can use get_dts_logger to get the logger but honestly I think we
should not encourage tests to use the logger directly.
Maybe we should add a TestSuite class method to log.