[PATCH v4 0/7] dts: Port scatter suite over

[PATCH v4 0/7] dts: Port scatter suite over

[jspewock]

From: Jeremy Spewock <jspewock@iol.unh.edu>

v4:

Addresses comments on the previous version. Main things which were added
include specific methods within the testpmd shell to avoid the need of
calling send_command directly, as well as docstring comment updates.

This series had to get resubmitted so that it would show up as one
series on patchwork. Previously each patch was given its own series by
mistake. Seeing as there was no change otherwise, submitting this as v4
once again.

I think this might be due to me trying to submit with --in-reply-to
after having re-ordered/renamed commits from the previous version.
Because of this, I am resubmitting v4 as a new thread. Apologies for the
multiple emails and the slight inconvience of the lack of thread.

Jeremy Spewock (7):
  dts: add required methods to testpmd_shell
  dts: allow passing parameters into interactive apps
  dts: add optional packet filtering to scapy sniffer
  dts: add pci addresses to EAL parameters
  dts: allow configuring MTU of ports
  dts: add scatter to the yaml schema
  dts: add scatter test suite

 dts/framework/config/conf_yaml_schema.json    |   3 +-
 dts/framework/exception.py                    |   4 +
 dts/framework/remote_session/linux_session.py |   8 ++
 dts/framework/remote_session/os_session.py    |   9 ++
 .../remote_session/remote/testpmd_shell.py    |  94 +++++++++++++++-
 dts/framework/test_suite.py                   |  14 ++-
 .../capturing_traffic_generator.py            |  22 +++-
 dts/framework/testbed_model/scapy.py          |  28 ++++-
 dts/framework/testbed_model/sut_node.py       |  27 ++++-
 dts/framework/testbed_model/tg_node.py        |  12 +-
 dts/tests/TestSuite_pmd_buffer_scatter.py     | 105 ++++++++++++++++++
 11 files changed, 312 insertions(+), 14 deletions(-)
 create mode 100644 dts/tests/TestSuite_pmd_buffer_scatter.py

-- 
2.43.0

[PATCH v4 1/7] dts: add required methods to testpmd_shell

[jspewock]

From: Jeremy Spewock <jspewock@iol.unh.edu>

Added a method within the testpmd interactive shell that polls the
status of ports and verifies that the link status on a given port is
"up." Polling will continue until either the link comes up, or the
timeout is reached. Also added methods for starting and stopping packet
forwarding in testpmd and a method for setting the forwarding mode on
testpmd. The method for starting packet forwarding will also attempt to
verify that forwarding did indeed start by default.

Signed-off-by: Jeremy Spewock <jspewock@iol.unh.edu>
---
 dts/framework/exception.py                    |  4 +
 .../remote_session/remote/testpmd_shell.py    | 92 +++++++++++++++++++
 2 files changed, 96 insertions(+)

diff --git a/dts/framework/exception.py b/dts/framework/exception.py
index b362e42924..e36db20e32 100644
--- a/dts/framework/exception.py
+++ b/dts/framework/exception.py
@@ -119,6 +119,10 @@ def __str__(self) -> str:
         return f"Command {self.command} returned a non-zero exit code: {self.command_return_code}"
 
 
+class InteractiveCommandExecutionError(DTSError):
+    severity: ClassVar[ErrorSeverity] = ErrorSeverity.REMOTE_CMD_EXEC_ERR
+
+
 class RemoteDirectoryExistsError(DTSError):
     """
     Raised when a remote directory to be created already exists.
diff --git a/dts/framework/remote_session/remote/testpmd_shell.py b/dts/framework/remote_session/remote/testpmd_shell.py
index 08ac311016..b5e4cba9b3 100644
--- a/dts/framework/remote_session/remote/testpmd_shell.py
+++ b/dts/framework/remote_session/remote/testpmd_shell.py
@@ -1,9 +1,15 @@
 # SPDX-License-Identifier: BSD-3-Clause
 # Copyright(c) 2023 University of New Hampshire
 
+import time
+from enum import auto
 from pathlib import PurePath
 from typing import Callable
 
+from framework.exception import InteractiveCommandExecutionError
+from framework.settings import SETTINGS
+from framework.utils import StrEnum
+
 from .interactive_shell import InteractiveShell
 
 
@@ -17,6 +23,37 @@ def __str__(self) -> str:
         return self.pci_address
 
 
+class TestPmdForwardingModes(StrEnum):
+    r"""The supported packet forwarding modes for :class:`~TestPmdShell`\s"""
+
+    #:
+    io = auto()
+    #:
+    mac = auto()
+    #:
+    macswap = auto()
+    #:
+    flowgen = auto()
+    #:
+    rxonly = auto()
+    #:
+    txonly = auto()
+    #:
+    csum = auto()
+    #:
+    icmpecho = auto()
+    #:
+    ieee1588 = auto()
+    #:
+    noisy = auto()
+    #:
+    fivetswap = "5tswap"
+    #:
+    shared_rxq = "shared-rxq"
+    #:
+    recycle_mbufs = auto()
+
+
 class TestPmdShell(InteractiveShell):
     path: PurePath = PurePath("app", "dpdk-testpmd")
     dpdk_app: bool = True
@@ -28,6 +65,27 @@ def _start_application(self, get_privileged_command: Callable[[str], str] | None
         self._app_args += " -- -i"
         super()._start_application(get_privileged_command)
 
+    def start(self, verify: bool = True) -> None:
+        """Start packet forwarding with the current configuration.
+
+        Args:
+            verify: If :data:`True` , a second start command will be sent in an attempt to verify
+                packet forwarding started as expected.
+
+        Raises:
+            InteractiveCommandExecutionError: If `verify` is :data:`True` and forwarding fails to
+                start.
+        """
+        self.send_command("start")
+        if verify:
+            # If forwarding was already started, sending "start" again should tell us
+            if "Packet forwarding already started" not in self.send_command("start"):
+                raise InteractiveCommandExecutionError("Testpmd failed to start packet forwarding.")
+
+    def stop(self) -> None:
+        """Stop packet forwarding."""
+        self.send_command("stop")
+
     def get_devices(self) -> list[TestPmdDevice]:
         """Get a list of device names that are known to testpmd
 
@@ -43,3 +101,37 @@ def get_devices(self) -> list[TestPmdDevice]:
             if "device name:" in line.lower():
                 dev_list.append(TestPmdDevice(line))
         return dev_list
+
+    def wait_link_status_up(self, port_id: int, timeout=SETTINGS.timeout) -> bool:
+        """Wait until the link status on the given port is "up".
+
+        Arguments:
+            port_id: Port to check the link status on.
+            timeout: Time to wait for the link to come up. The default value for this
+                argument is set using the :option:`-t, --timeout` command-line argument
+                or the :envvar:`DTS_TIMEOUT` environment variable.
+
+        Returns:
+            If the link came up in time or not.
+        """
+        time_to_stop = time.time() + timeout
+        while time.time() < time_to_stop:
+            port_info = self.send_command(f"show port info {port_id}")
+            if "Link status: up" in port_info:
+                break
+            time.sleep(0.5)
+        else:
+            self._logger.error(f"The link for port {port_id} did not come up in the given timeout.")
+        return "Link status: up" in port_info
+
+    def set_forward_mode(self, mode: TestPmdForwardingModes):
+        """Set packet forwarding mode.
+
+        Args:
+            mode: The forwarding mode to use.
+        """
+        self.send_command(f"set fwd {mode.value}")
+
+    def close(self) -> None:
+        self.send_command("exit", "")
+        return super().close()
-- 
2.43.0

Re: [PATCH v4 1/7] dts: add required methods to testpmd_shell

[Juraj Linkeš]

The subject could be improved. That these methods are required is
kinda obvious. We should try to actually include some useful
information in the subject, such as "add basic methods to testpmd
shell", but even that is not saying much. Maybe "add startup
verification and forwarding to testpmd shell" - I actually like
something like this.

On Mon, Dec 18, 2023 at 7:13 PM <jspewock@iol.unh.edu> wrote:
>
> From: Jeremy Spewock <jspewock@iol.unh.edu>
>
> Added a method within the testpmd interactive shell that polls the
> status of ports and verifies that the link status on a given port is
> "up." Polling will continue until either the link comes up, or the
> timeout is reached. Also added methods for starting and stopping packet
> forwarding in testpmd and a method for setting the forwarding mode on
> testpmd. The method for starting packet forwarding will also attempt to
> verify that forwarding did indeed start by default.
>

The body should not explain what we're adding, but why we're adding it.


> Signed-off-by: Jeremy Spewock <jspewock@iol.unh.edu>
> ---
>  dts/framework/exception.py                    |  4 +
>  .../remote_session/remote/testpmd_shell.py    | 92 +++++++++++++++++++
>  2 files changed, 96 insertions(+)
>
> diff --git a/dts/framework/exception.py b/dts/framework/exception.py
> index b362e42924..e36db20e32 100644
> --- a/dts/framework/exception.py
> +++ b/dts/framework/exception.py
> @@ -119,6 +119,10 @@ def __str__(self) -> str:
>          return f"Command {self.command} returned a non-zero exit code: {self.command_return_code}"
>
>
> +class InteractiveCommandExecutionError(DTSError):
> +    severity: ClassVar[ErrorSeverity] = ErrorSeverity.REMOTE_CMD_EXEC_ERR
> +
> +
>  class RemoteDirectoryExistsError(DTSError):
>      """
>      Raised when a remote directory to be created already exists.
> diff --git a/dts/framework/remote_session/remote/testpmd_shell.py b/dts/framework/remote_session/remote/testpmd_shell.py
> index 08ac311016..b5e4cba9b3 100644
> --- a/dts/framework/remote_session/remote/testpmd_shell.py
> +++ b/dts/framework/remote_session/remote/testpmd_shell.py
> @@ -1,9 +1,15 @@
>  # SPDX-License-Identifier: BSD-3-Clause
>  # Copyright(c) 2023 University of New Hampshire
>
> +import time
> +from enum import auto
>  from pathlib import PurePath
>  from typing import Callable
>
> +from framework.exception import InteractiveCommandExecutionError
> +from framework.settings import SETTINGS
> +from framework.utils import StrEnum
> +
>  from .interactive_shell import InteractiveShell
>
>
> @@ -17,6 +23,37 @@ def __str__(self) -> str:
>          return self.pci_address
>
>
> +class TestPmdForwardingModes(StrEnum):
> +    r"""The supported packet forwarding modes for :class:`~TestPmdShell`\s"""
> +
> +    #:
> +    io = auto()
> +    #:
> +    mac = auto()
> +    #:
> +    macswap = auto()
> +    #:
> +    flowgen = auto()
> +    #:
> +    rxonly = auto()
> +    #:
> +    txonly = auto()
> +    #:
> +    csum = auto()
> +    #:
> +    icmpecho = auto()
> +    #:
> +    ieee1588 = auto()
> +    #:
> +    noisy = auto()
> +    #:
> +    fivetswap = "5tswap"
> +    #:
> +    shared_rxq = "shared-rxq"
> +    #:
> +    recycle_mbufs = auto()
> +
> +
>  class TestPmdShell(InteractiveShell):
>      path: PurePath = PurePath("app", "dpdk-testpmd")
>      dpdk_app: bool = True
> @@ -28,6 +65,27 @@ def _start_application(self, get_privileged_command: Callable[[str], str] | None
>          self._app_args += " -- -i"
>          super()._start_application(get_privileged_command)
>
> +    def start(self, verify: bool = True) -> None:
> +        """Start packet forwarding with the current configuration.
> +
> +        Args:
> +            verify: If :data:`True` , a second start command will be sent in an attempt to verify
> +                packet forwarding started as expected.
> +

Isn't there a better way to verify this? Like with some show command?
Or is this how it's supposed to be used?

> +        Raises:
> +            InteractiveCommandExecutionError: If `verify` is :data:`True` and forwarding fails to
> +                start.
> +        """
> +        self.send_command("start")
> +        if verify:
> +            # If forwarding was already started, sending "start" again should tell us
> +            if "Packet forwarding already started" not in self.send_command("start"):
> +                raise InteractiveCommandExecutionError("Testpmd failed to start packet forwarding.")
> +
> +    def stop(self) -> None:
> +        """Stop packet forwarding."""
> +        self.send_command("stop")
> +

Do we want to do verification here as well? Is there a reason to do
such verification?

>      def get_devices(self) -> list[TestPmdDevice]:
>          """Get a list of device names that are known to testpmd
>
> @@ -43,3 +101,37 @@ def get_devices(self) -> list[TestPmdDevice]:
>              if "device name:" in line.lower():
>                  dev_list.append(TestPmdDevice(line))
>          return dev_list
> +
> +    def wait_link_status_up(self, port_id: int, timeout=SETTINGS.timeout) -> bool:
> +        """Wait until the link status on the given port is "up".
> +
> +        Arguments:
> +            port_id: Port to check the link status on.
> +            timeout: Time to wait for the link to come up. The default value for this
> +                argument is set using the :option:`-t, --timeout` command-line argument
> +                or the :envvar:`DTS_TIMEOUT` environment variable.
> +
> +        Returns:
> +            If the link came up in time or not.

This is a bit of a pet peeve of mine - Whether the link came up.

> +        """
> +        time_to_stop = time.time() + timeout
> +        while time.time() < time_to_stop:
> +            port_info = self.send_command(f"show port info {port_id}")
> +            if "Link status: up" in port_info:
> +                break
> +            time.sleep(0.5)

How long does it usually take? If it's in the order of seconds, then
0.5 seems fine, if it's faster, the sleep should probably be shorter.

> +        else:
> +            self._logger.error(f"The link for port {port_id} did not come up in the given timeout.")
> +        return "Link status: up" in port_info
> +
> +    def set_forward_mode(self, mode: TestPmdForwardingModes):
> +        """Set packet forwarding mode.
> +
> +        Args:
> +            mode: The forwarding mode to use.
> +        """
> +        self.send_command(f"set fwd {mode.value}")
> +

Again the verification - does it make sense to verify this as well?

> +    def close(self) -> None:
> +        self.send_command("exit", "")
> +        return super().close()
> --
> 2.43.0
>

Re: [PATCH v4 1/7] dts: add required methods to testpmd_shell

[Jeremy Spewock]

[-- Attachment #1: Type: text/plain, Size: 7048 bytes --]

On Tue, Dec 19, 2023 at 11:45 AM Juraj Linkeš <juraj.linkes@pantheon.tech>
wrote:

> The subject could be improved. That these methods are required is
> kinda obvious. We should try to actually include some useful
> information in the subject, such as "add basic methods to testpmd
> shell", but even that is not saying much. Maybe "add startup
> verification and forwarding to testpmd shell" - I actually like
> something like this.
>

The subject on this commit was something that I was having trouble with for
quite some time, I was trying to think of something that would be
descriptive enough and not too long so I opted to go for the more vague
subject and explain it in the body, but I like this subject much better and
will change to using that in the next version.


> On Mon, Dec 18, 2023 at 7:13 PM <jspewock@iol.unh.edu> wrote:
> >
> > From: Jeremy Spewock <jspewock@iol.unh.edu>
> >
> > Added a method within the testpmd interactive shell that polls the
> > status of ports and verifies that the link status on a given port is
> > "up." Polling will continue until either the link comes up, or the
> > timeout is reached. Also added methods for starting and stopping packet
> > forwarding in testpmd and a method for setting the forwarding mode on
> > testpmd. The method for starting packet forwarding will also attempt to
> > verify that forwarding did indeed start by default.
> >
>
> The body should not explain what we're adding, but why we're adding it.
>

Very good point and I'll keep this in mind in the future.


> >
> > +    def start(self, verify: bool = True) -> None:
> > +        """Start packet forwarding with the current configuration.
> > +
> > +        Args:
> > +            verify: If :data:`True` , a second start command will be
> sent in an attempt to verify
> > +                packet forwarding started as expected.
> > +
>
> Isn't there a better way to verify this? Like with some show command?
> Or is this how it's supposed to be used?
>

I looked through documentation and the outputs of many of the show commands
and I wasn't able to find one that indicated that forwarding had actually
started.  Clearly testpmd holds this state somewhere but I couldn't find
where. I agree that this method of verification doesn't seem perfect but I
wasn't able to find a more automated method of doing so since the start
command has no output otherwise.

One place that does seem to display if forwarding has started is using the
command: `show port (port_id) rxq|txq (queue_id) desc (desc_id) status`
(specifically the rxq variant) but this seems to take a few seconds to
update its state between available and done when you stop forwarding so it
seems like it could lead to inaccurate verification. This would also make
assumptions on the number of rx and tx queues which I'm unsure if we would
want to make.


>
> > +        Raises:
> > +            InteractiveCommandExecutionError: If `verify` is
> :data:`True` and forwarding fails to
> > +                start.
> > +        """
> > +        self.send_command("start")
> > +        if verify:
> > +            # If forwarding was already started, sending "start" again
> should tell us
> > +            if "Packet forwarding already started" not in
> self.send_command("start"):
> > +                raise InteractiveCommandExecutionError("Testpmd failed
> to start packet forwarding.")
> > +
> > +    def stop(self) -> None:
> > +        """Stop packet forwarding."""
> > +        self.send_command("stop")
> > +
>
> Do we want to do verification here as well? Is there a reason to do
> such verification?
>

I figured there wasn't much of a reason as your two options when you run
this command are you aren't already forwarding in which case this command
still gets you into the state you want to be in, or you are forwarding and
it stops you which also puts you into the correct state.

I guess that assumes that there can't be an error when trying to stop
forwarding, so I could add a verification step for that purpose, but I
don't think it would commonly be the case that stopping fails. There isn't
really harm in verifying this for safety though, so I'll add it.


>
> >      def get_devices(self) -> list[TestPmdDevice]:
> >          """Get a list of device names that are known to testpmd
> >
> > @@ -43,3 +101,37 @@ def get_devices(self) -> list[TestPmdDevice]:
> >              if "device name:" in line.lower():
> >                  dev_list.append(TestPmdDevice(line))
> >          return dev_list
> > +
> > +    def wait_link_status_up(self, port_id: int,
> timeout=SETTINGS.timeout) -> bool:
> > +        """Wait until the link status on the given port is "up".
> > +
> > +        Arguments:
> > +            port_id: Port to check the link status on.
> > +            timeout: Time to wait for the link to come up. The default
> value for this
> > +                argument is set using the :option:`-t, --timeout`
> command-line argument
> > +                or the :envvar:`DTS_TIMEOUT` environment variable.
> > +
> > +        Returns:
> > +            If the link came up in time or not.
>
> This is a bit of a pet peeve of mine - Whether the link came up.
>

Definitely would sound better that way, I'll make the change.


>
> > +        """
> > +        time_to_stop = time.time() + timeout
> > +        while time.time() < time_to_stop:
> > +            port_info = self.send_command(f"show port info {port_id}")
> > +            if "Link status: up" in port_info:
> > +                break
> > +            time.sleep(0.5)
>
> How long does it usually take? If it's in the order of seconds, then
> 0.5 seems fine, if it's faster, the sleep should probably be shorter.
>

Generally this is something that I've actually seen be up instantly when
you open an interactive terminal in testpmd, so another option could be to
just make this a single verification check. The reason it was left this way
right now was just under the assumption that if a link was down for some
reason it might be able to recover within the timeout, but whether that
takes seconds or not would depend on why the link was down.


>
> > +        else:
> > +            self._logger.error(f"The link for port {port_id} did not
> come up in the given timeout.")
> > +        return "Link status: up" in port_info
> > +
> > +    def set_forward_mode(self, mode: TestPmdForwardingModes):
> > +        """Set packet forwarding mode.
> > +
> > +        Args:
> > +            mode: The forwarding mode to use.
> > +        """
> > +        self.send_command(f"set fwd {mode.value}")
> > +
>
> Again the verification - does it make sense to verify this as well?
>

That's a good point and is something easy to verify, I'll add this as well.


>
> > +    def close(self) -> None:
> > +        self.send_command("exit", "")
> > +        return super().close()
> > --
> > 2.43.0
> >
>

[-- Attachment #2: Type: text/html, Size: 10500 bytes --]

Re: [PATCH v4 1/7] dts: add required methods to testpmd_shell

[Juraj Linkeš]

On Thu, Dec 21, 2023 at 8:38 PM Jeremy Spewock <jspewock@iol.unh.edu> wrote:
>
>
>
> On Tue, Dec 19, 2023 at 11:45 AM Juraj Linkeš <juraj.linkes@pantheon.tech> wrote:
>>
>> The subject could be improved. That these methods are required is
>> kinda obvious. We should try to actually include some useful
>> information in the subject, such as "add basic methods to testpmd
>> shell", but even that is not saying much. Maybe "add startup
>> verification and forwarding to testpmd shell" - I actually like
>> something like this.
>
>
> The subject on this commit was something that I was having trouble with for quite some time, I was trying to think of something that would be descriptive enough and not too long so I opted to go for the more vague subject and explain it in the body, but I like this subject much better and will change to using that in the next version.
>
>>
>> On Mon, Dec 18, 2023 at 7:13 PM <jspewock@iol.unh.edu> wrote:
>> >
>> > From: Jeremy Spewock <jspewock@iol.unh.edu>
>> >
>> > Added a method within the testpmd interactive shell that polls the
>> > status of ports and verifies that the link status on a given port is
>> > "up." Polling will continue until either the link comes up, or the
>> > timeout is reached. Also added methods for starting and stopping packet
>> > forwarding in testpmd and a method for setting the forwarding mode on
>> > testpmd. The method for starting packet forwarding will also attempt to
>> > verify that forwarding did indeed start by default.
>> >
>>
>> The body should not explain what we're adding, but why we're adding it.
>
>
> Very good point and I'll keep this in mind in the future.
>
>>
>> >
>> > +    def start(self, verify: bool = True) -> None:
>> > +        """Start packet forwarding with the current configuration.
>> > +
>> > +        Args:
>> > +            verify: If :data:`True` , a second start command will be sent in an attempt to verify
>> > +                packet forwarding started as expected.
>> > +
>>
>> Isn't there a better way to verify this? Like with some show command?
>> Or is this how it's supposed to be used?
>
>
> I looked through documentation and the outputs of many of the show commands and I wasn't able to find one that indicated that forwarding had actually started.  Clearly testpmd holds this state somewhere but I couldn't find where. I agree that this method of verification doesn't seem perfect but I wasn't able to find a more automated method of doing so since the start command has no output otherwise.
>
> One place that does seem to display if forwarding has started is using the command: `show port (port_id) rxq|txq (queue_id) desc (desc_id) status` (specifically the rxq variant) but this seems to take a few seconds to update its state between available and done when you stop forwarding so it seems like it could lead to inaccurate verification. This would also make assumptions on the number of rx and tx queues which I'm unsure if we would want to make.
>

Yea, let's not assume anything if we can avoid it. I guess there
doesn't really have to be a dedicated command for verification since
doing it this way is kinda doing the same thing, except maybe this
could put the testpmd application into a faulty state if we try to
start forwarding a second time if it hasn't already been started by
the first execution. I imagine this is not a risk since it's the way
it was done in the original DTS.

>>
>>
>> > +        Raises:
>> > +            InteractiveCommandExecutionError: If `verify` is :data:`True` and forwarding fails to
>> > +                start.
>> > +        """
>> > +        self.send_command("start")
>> > +        if verify:
>> > +            # If forwarding was already started, sending "start" again should tell us
>> > +            if "Packet forwarding already started" not in self.send_command("start"):
>> > +                raise InteractiveCommandExecutionError("Testpmd failed to start packet forwarding.")
>> > +
>> > +    def stop(self) -> None:
>> > +        """Stop packet forwarding."""
>> > +        self.send_command("stop")
>> > +
>>
>> Do we want to do verification here as well? Is there a reason to do
>> such verification?
>
>
> I figured there wasn't much of a reason as your two options when you run this command are you aren't already forwarding in which case this command still gets you into the state you want to be in, or you are forwarding and it stops you which also puts you into the correct state.
>
> I guess that assumes that there can't be an error when trying to stop forwarding,

Yes, that would be the reason to do the verification.

> so I could add a verification step for that purpose, but I don't think it would commonly be the case that stopping fails. There isn't really harm in verifying this for safety though, so I'll add it.
>

Yes, let's add it if we're not absolutely sure we don't need it. The
worst case scenario is it would help with debugging if the
verification fails.

>>
>>
>> >      def get_devices(self) -> list[TestPmdDevice]:
>> >          """Get a list of device names that are known to testpmd
>> >
>> > @@ -43,3 +101,37 @@ def get_devices(self) -> list[TestPmdDevice]:
>> >              if "device name:" in line.lower():
>> >                  dev_list.append(TestPmdDevice(line))
>> >          return dev_list
>> > +
>> > +    def wait_link_status_up(self, port_id: int, timeout=SETTINGS.timeout) -> bool:
>> > +        """Wait until the link status on the given port is "up".
>> > +
>> > +        Arguments:
>> > +            port_id: Port to check the link status on.
>> > +            timeout: Time to wait for the link to come up. The default value for this
>> > +                argument is set using the :option:`-t, --timeout` command-line argument
>> > +                or the :envvar:`DTS_TIMEOUT` environment variable.
>> > +
>> > +        Returns:
>> > +            If the link came up in time or not.
>>
>> This is a bit of a pet peeve of mine - Whether the link came up.
>
>
> Definitely would sound better that way, I'll make the change.
>
>>
>>
>> > +        """
>> > +        time_to_stop = time.time() + timeout
>> > +        while time.time() < time_to_stop:
>> > +            port_info = self.send_command(f"show port info {port_id}")
>> > +            if "Link status: up" in port_info:
>> > +                break
>> > +            time.sleep(0.5)
>>
>> How long does it usually take? If it's in the order of seconds, then
>> 0.5 seems fine, if it's faster, the sleep should probably be shorter.
>
>
> Generally this is something that I've actually seen be up instantly when you open an interactive terminal in testpmd, so another option could be to just make this a single verification check. The reason it was left this way right now was just under the assumption that if a link was down for some reason it might be able to recover within the timeout, but whether that takes seconds or not would depend on why the link was down.
>

Ok, let's leave this as is then.

>>
>>
>> > +        else:
>> > +            self._logger.error(f"The link for port {port_id} did not come up in the given timeout.")
>> > +        return "Link status: up" in port_info
>> > +
>> > +    def set_forward_mode(self, mode: TestPmdForwardingModes):
>> > +        """Set packet forwarding mode.
>> > +
>> > +        Args:
>> > +            mode: The forwarding mode to use.
>> > +        """
>> > +        self.send_command(f"set fwd {mode.value}")
>> > +
>>
>> Again the verification - does it make sense to verify this as well?
>
>
> That's a good point and is something easy to verify, I'll add this as well.
>
>>
>>
>> > +    def close(self) -> None:
>> > +        self.send_command("exit", "")
>> > +        return super().close()
>> > --
>> > 2.43.0
>> >

[PATCH v4 2/7] dts: allow passing parameters into interactive apps

[jspewock]

From: Jeremy Spewock <jspewock@iol.unh.edu>

Modified interactive applications to allow for the ability to pass
parameters into the app on start up. Also modified the way EAL
parameters are handled so that the trailing "--" separator is added be
default after all EAL parameters.

Signed-off-by: Jeremy Spewock <jspewock@iol.unh.edu>
---
 .../remote_session/remote/testpmd_shell.py       |  2 +-
 dts/framework/testbed_model/sut_node.py          | 16 ++++++++++------
 2 files changed, 11 insertions(+), 7 deletions(-)

diff --git a/dts/framework/remote_session/remote/testpmd_shell.py b/dts/framework/remote_session/remote/testpmd_shell.py
index b5e4cba9b3..369807a33e 100644
--- a/dts/framework/remote_session/remote/testpmd_shell.py
+++ b/dts/framework/remote_session/remote/testpmd_shell.py
@@ -62,7 +62,7 @@ class TestPmdShell(InteractiveShell):
 
     def _start_application(self, get_privileged_command: Callable[[str], str] | None) -> None:
         """See "_start_application" in InteractiveShell."""
-        self._app_args += " -- -i"
+        self._app_args += " -i"
         super()._start_application(get_privileged_command)
 
     def start(self, verify: bool = True) -> None:
diff --git a/dts/framework/testbed_model/sut_node.py b/dts/framework/testbed_model/sut_node.py
index 7f75043bd3..9c92232d9e 100644
--- a/dts/framework/testbed_model/sut_node.py
+++ b/dts/framework/testbed_model/sut_node.py
@@ -361,7 +361,8 @@ def create_interactive_shell(
         shell_cls: Type[InteractiveShellType],
         timeout: float = SETTINGS.timeout,
         privileged: bool = False,
-        eal_parameters: EalParameters | str | None = None,
+        eal_parameters: EalParameters | None = None,
+        app_parameters: str = "",
     ) -> InteractiveShellType:
         """Factory method for creating a handler for an interactive session.
 
@@ -376,19 +377,22 @@ def create_interactive_shell(
             eal_parameters: List of EAL parameters to use to launch the app. If this
                 isn't provided or an empty string is passed, it will default to calling
                 create_eal_parameters().
+            app_parameters: Additional arguments to pass into the application on the
+                command-line.
         Returns:
             Instance of the desired interactive application.
         """
-        if not eal_parameters:
-            eal_parameters = self.create_eal_parameters()
-
-        # We need to append the build directory for DPDK apps
+        # We need to append the build directory and add EAL parameters for DPDK apps
         if shell_cls.dpdk_app:
+            if not eal_parameters:
+                eal_parameters = self.create_eal_parameters()
+            app_parameters = f"{eal_parameters} -- {app_parameters}"
+
             shell_cls.path = self.main_session.join_remote_path(
                 self.remote_dpdk_build_dir, shell_cls.path
             )
 
-        return super().create_interactive_shell(shell_cls, timeout, privileged, str(eal_parameters))
+        return super().create_interactive_shell(shell_cls, timeout, privileged, app_parameters)
 
     def bind_ports_to_driver(self, for_dpdk: bool = True) -> None:
         """Bind all ports on the SUT to a driver.
-- 
2.43.0

Re: [PATCH v4 2/7] dts: allow passing parameters into interactive apps

[Juraj Linkeš]

We should also update the subject and the body based on our previous
discussion. I don't think they properly describe the change, as we're
also updating the method's behavior of DPDK apps.

On Mon, Dec 18, 2023 at 7:13 PM <jspewock@iol.unh.edu> wrote:
>
> From: Jeremy Spewock <jspewock@iol.unh.edu>
>
> Modified interactive applications to allow for the ability to pass
> parameters into the app on start up. Also modified the way EAL
> parameters are handled so that the trailing "--" separator is added be
> default after all EAL parameters.
>
> Signed-off-by: Jeremy Spewock <jspewock@iol.unh.edu>
> ---
>  .../remote_session/remote/testpmd_shell.py       |  2 +-
>  dts/framework/testbed_model/sut_node.py          | 16 ++++++++++------
>  2 files changed, 11 insertions(+), 7 deletions(-)
>
> diff --git a/dts/framework/remote_session/remote/testpmd_shell.py b/dts/framework/remote_session/remote/testpmd_shell.py
> index b5e4cba9b3..369807a33e 100644
> --- a/dts/framework/remote_session/remote/testpmd_shell.py
> +++ b/dts/framework/remote_session/remote/testpmd_shell.py
> @@ -62,7 +62,7 @@ class TestPmdShell(InteractiveShell):
>
>      def _start_application(self, get_privileged_command: Callable[[str], str] | None) -> None:
>          """See "_start_application" in InteractiveShell."""
> -        self._app_args += " -- -i"
> +        self._app_args += " -i"
>          super()._start_application(get_privileged_command)
>
>      def start(self, verify: bool = True) -> None:
> diff --git a/dts/framework/testbed_model/sut_node.py b/dts/framework/testbed_model/sut_node.py
> index 7f75043bd3..9c92232d9e 100644
> --- a/dts/framework/testbed_model/sut_node.py
> +++ b/dts/framework/testbed_model/sut_node.py
> @@ -361,7 +361,8 @@ def create_interactive_shell(
>          shell_cls: Type[InteractiveShellType],
>          timeout: float = SETTINGS.timeout,
>          privileged: bool = False,
> -        eal_parameters: EalParameters | str | None = None,
> +        eal_parameters: EalParameters | None = None,
> +        app_parameters: str = "",
>      ) -> InteractiveShellType:
>          """Factory method for creating a handler for an interactive session.
>
> @@ -376,19 +377,22 @@ def create_interactive_shell(
>              eal_parameters: List of EAL parameters to use to launch the app. If this
>                  isn't provided or an empty string is passed, it will default to calling
>                  create_eal_parameters().
> +            app_parameters: Additional arguments to pass into the application on the
> +                command-line.
>          Returns:
>              Instance of the desired interactive application.
>          """
> -        if not eal_parameters:
> -            eal_parameters = self.create_eal_parameters()
> -
> -        # We need to append the build directory for DPDK apps
> +        # We need to append the build directory and add EAL parameters for DPDK apps
>          if shell_cls.dpdk_app:
> +            if not eal_parameters:
> +                eal_parameters = self.create_eal_parameters()
> +            app_parameters = f"{eal_parameters} -- {app_parameters}"
> +
>              shell_cls.path = self.main_session.join_remote_path(
>                  self.remote_dpdk_build_dir, shell_cls.path
>              )
>
> -        return super().create_interactive_shell(shell_cls, timeout, privileged, str(eal_parameters))
> +        return super().create_interactive_shell(shell_cls, timeout, privileged, app_parameters)
>
>      def bind_ports_to_driver(self, for_dpdk: bool = True) -> None:
>          """Bind all ports on the SUT to a driver.
> --
> 2.43.0
>

[PATCH v4 3/7] dts: add optional packet filtering to scapy sniffer

[jspewock]

From: Jeremy Spewock <jspewock@iol.unh.edu>

Added the options to filter out LLDP and ARP packets when
sniffing for packets with scapy. This was done using BPF filters to
ensure that the noise these packets provide does not interfere with test
cases.

Signed-off-by: Jeremy Spewock <jspewock@iol.unh.edu>
---
 dts/framework/test_suite.py                   | 14 ++++++++--
 .../capturing_traffic_generator.py            | 22 ++++++++++++++-
 dts/framework/testbed_model/scapy.py          | 28 ++++++++++++++++++-
 dts/framework/testbed_model/tg_node.py        | 12 ++++++--
 4 files changed, 70 insertions(+), 6 deletions(-)

diff --git a/dts/framework/test_suite.py b/dts/framework/test_suite.py
index 4a7907ec33..6dfa570041 100644
--- a/dts/framework/test_suite.py
+++ b/dts/framework/test_suite.py
@@ -27,6 +27,7 @@
 from .settings import SETTINGS
 from .test_result import BuildTargetResult, Result, TestCaseResult, TestSuiteResult
 from .testbed_model import SutNode, TGNode
+from .testbed_model.capturing_traffic_generator import PacketFilteringConfig
 from .testbed_model.hw.port import Port, PortLink
 from .utils import get_packet_summaries
 
@@ -149,7 +150,12 @@ def configure_testbed_ipv4(self, restore: bool = False) -> None:
     def _configure_ipv4_forwarding(self, enable: bool) -> None:
         self.sut_node.configure_ipv4_forwarding(enable)
 
-    def send_packet_and_capture(self, packet: Packet, duration: float = 1) -> list[Packet]:
+    def send_packet_and_capture(
+        self,
+        packet: Packet,
+        filter_config: PacketFilteringConfig = PacketFilteringConfig(),
+        duration: float = 1,
+    ) -> list[Packet]:
         """
         Send a packet through the appropriate interface and
         receive on the appropriate interface.
@@ -158,7 +164,11 @@ def send_packet_and_capture(self, packet: Packet, duration: float = 1) -> list[P
         """
         packet = self._adjust_addresses(packet)
         return self.tg_node.send_packet_and_capture(
-            packet, self._tg_port_egress, self._tg_port_ingress, duration
+            packet,
+            self._tg_port_egress,
+            self._tg_port_ingress,
+            filter_config,
+            duration,
         )
 
     def get_expected_packet(self, packet: Packet) -> Packet:
diff --git a/dts/framework/testbed_model/capturing_traffic_generator.py b/dts/framework/testbed_model/capturing_traffic_generator.py
index e6512061d7..c40b030fe4 100644
--- a/dts/framework/testbed_model/capturing_traffic_generator.py
+++ b/dts/framework/testbed_model/capturing_traffic_generator.py
@@ -11,6 +11,7 @@
 
 import uuid
 from abc import abstractmethod
+from dataclasses import dataclass
 
 import scapy.utils  # type: ignore[import]
 from scapy.packet import Packet  # type: ignore[import]
@@ -29,6 +30,19 @@ def _get_default_capture_name() -> str:
     return str(uuid.uuid4())
 
 
+@dataclass(slots=True)
+class PacketFilteringConfig:
+    """The supported filtering options for :class:`CapturingTrafficGenerator`.
+
+    Attributes:
+        no_lldp: If :data:`True`, LLDP packets will be filtered out when capturing.
+        no_arp: If :data:`True`, ARP packets will be filtered out when capturing.
+    """
+
+    no_lldp: bool = True
+    no_arp: bool = True
+
+
 class CapturingTrafficGenerator(TrafficGenerator):
     """Capture packets after sending traffic.
 
@@ -51,6 +65,7 @@ def send_packet_and_capture(
         packet: Packet,
         send_port: Port,
         receive_port: Port,
+        filter_config: PacketFilteringConfig,
         duration: float,
         capture_name: str = _get_default_capture_name(),
     ) -> list[Packet]:
@@ -64,6 +79,7 @@ def send_packet_and_capture(
             packet: The packet to send.
             send_port: The egress port on the TG node.
             receive_port: The ingress port in the TG node.
+            filter_config: Filters to apply when capturing packets.
             duration: Capture traffic for this amount of time after sending the packet.
             capture_name: The name of the .pcap file where to store the capture.
 
@@ -71,7 +87,7 @@ def send_packet_and_capture(
              A list of received packets. May be empty if no packets are captured.
         """
         return self.send_packets_and_capture(
-            [packet], send_port, receive_port, duration, capture_name
+            [packet], send_port, receive_port, filter_config, duration, capture_name
         )
 
     def send_packets_and_capture(
@@ -79,6 +95,7 @@ def send_packets_and_capture(
         packets: list[Packet],
         send_port: Port,
         receive_port: Port,
+        filter_config: PacketFilteringConfig,
         duration: float,
         capture_name: str = _get_default_capture_name(),
     ) -> list[Packet]:
@@ -92,6 +109,7 @@ def send_packets_and_capture(
             packets: The packets to send.
             send_port: The egress port on the TG node.
             receive_port: The ingress port in the TG node.
+            filter_config: Filters to apply when capturing packets.
             duration: Capture traffic for this amount of time after sending the packets.
             capture_name: The name of the .pcap file where to store the capture.
 
@@ -106,6 +124,7 @@ def send_packets_and_capture(
             packets,
             send_port,
             receive_port,
+            filter_config,
             duration,
         )
 
@@ -119,6 +138,7 @@ def _send_packets_and_capture(
         packets: list[Packet],
         send_port: Port,
         receive_port: Port,
+        filter_config: PacketFilteringConfig,
         duration: float,
     ) -> list[Packet]:
         """
diff --git a/dts/framework/testbed_model/scapy.py b/dts/framework/testbed_model/scapy.py
index 9083e92b3d..94b0af7c6f 100644
--- a/dts/framework/testbed_model/scapy.py
+++ b/dts/framework/testbed_model/scapy.py
@@ -30,6 +30,7 @@
 
 from .capturing_traffic_generator import (
     CapturingTrafficGenerator,
+    PacketFilteringConfig,
     _get_default_capture_name,
 )
 from .hw.port import Port
@@ -69,6 +70,7 @@ def scapy_send_packets_and_capture(
     send_iface: str,
     recv_iface: str,
     duration: float,
+    sniff_filter: str,
 ) -> list[bytes]:
     """RPC function to send and capture packets.
 
@@ -90,6 +92,7 @@ def scapy_send_packets_and_capture(
         iface=recv_iface,
         store=True,
         started_callback=lambda *args: scapy.all.sendp(scapy_packets, iface=send_iface),
+        filter=sniff_filter,
     )
     sniffer.start()
     time.sleep(duration)
@@ -249,16 +252,38 @@ def _send_packets(self, packets: list[Packet], port: Port) -> None:
         packets = [packet.build() for packet in packets]
         self.rpc_server_proxy.scapy_send_packets(packets, port.logical_name)
 
+    def _create_packet_filter(self, filter_config: PacketFilteringConfig) -> str:
+        """Combines filter settings from `filter_config` into a BPF that scapy can use.
+
+        Scapy allows for the use of Berkeley Packet Filters (BPFs) to filter what packets are
+        collected based on various attributes of the packet.
+
+        Args:
+            filter_config: Config class that specifies which filters should be applied.
+
+        Returns:
+            A string representing the combination of BPF filters to be passed to scapy. For
+            example:
+
+            "ether[12:2] != 0x88cc && ether[12:2] != 0x0806"
+        """
+        bpf_filter: list[str] = []
+        if filter_config.no_arp:
+            bpf_filter.append("ether[12:2] != 0x0806")
+        if filter_config.no_lldp:
+            bpf_filter.append("ether[12:2] != 0x88cc")
+        return " && ".join(bpf_filter)
+
     def _send_packets_and_capture(
         self,
         packets: list[Packet],
         send_port: Port,
         receive_port: Port,
+        filter_config: PacketFilteringConfig,
         duration: float,
         capture_name: str = _get_default_capture_name(),
     ) -> list[Packet]:
         binary_packets = [packet.build() for packet in packets]
-
         xmlrpc_packets: list[
             xmlrpc.client.Binary
         ] = self.rpc_server_proxy.scapy_send_packets_and_capture(
@@ -266,6 +291,7 @@ def _send_packets_and_capture(
             send_port.logical_name,
             receive_port.logical_name,
             duration,
+            self._create_packet_filter(filter_config),
         )  # type: ignore[assignment]
 
         scapy_packets = [Ether(packet.data) for packet in xmlrpc_packets]
diff --git a/dts/framework/testbed_model/tg_node.py b/dts/framework/testbed_model/tg_node.py
index 79a55663b5..475dc2968d 100644
--- a/dts/framework/testbed_model/tg_node.py
+++ b/dts/framework/testbed_model/tg_node.py
@@ -23,7 +23,10 @@
 )
 from framework.exception import ConfigurationError
 
-from .capturing_traffic_generator import CapturingTrafficGenerator
+from .capturing_traffic_generator import (
+    CapturingTrafficGenerator,
+    PacketFilteringConfig,
+)
 from .hw.port import Port
 from .node import Node
 
@@ -53,6 +56,7 @@ def send_packet_and_capture(
         packet: Packet,
         send_port: Port,
         receive_port: Port,
+        filter_config: PacketFilteringConfig = PacketFilteringConfig(),
         duration: float = 1,
     ) -> list[Packet]:
         """Send a packet, return received traffic.
@@ -71,7 +75,11 @@ def send_packet_and_capture(
              A list of received packets. May be empty if no packets are captured.
         """
         return self.traffic_generator.send_packet_and_capture(
-            packet, send_port, receive_port, duration
+            packet,
+            send_port,
+            receive_port,
+            filter_config,
+            duration,
         )
 
     def close(self) -> None:
-- 
2.43.0

Re: [PATCH v4 3/7] dts: add optional packet filtering to scapy sniffer

[Juraj Linkeš]

Reviewed-by: Juraj Linkeš <juraj.linkes@pantheon.tech>

On Mon, Dec 18, 2023 at 7:13 PM <jspewock@iol.unh.edu> wrote:
>
> From: Jeremy Spewock <jspewock@iol.unh.edu>
>
> Added the options to filter out LLDP and ARP packets when
> sniffing for packets with scapy. This was done using BPF filters to
> ensure that the noise these packets provide does not interfere with test
> cases.
>
> Signed-off-by: Jeremy Spewock <jspewock@iol.unh.edu>
> ---
>  dts/framework/test_suite.py                   | 14 ++++++++--
>  .../capturing_traffic_generator.py            | 22 ++++++++++++++-
>  dts/framework/testbed_model/scapy.py          | 28 ++++++++++++++++++-
>  dts/framework/testbed_model/tg_node.py        | 12 ++++++--
>  4 files changed, 70 insertions(+), 6 deletions(-)
>
> diff --git a/dts/framework/test_suite.py b/dts/framework/test_suite.py
> index 4a7907ec33..6dfa570041 100644
> --- a/dts/framework/test_suite.py
> +++ b/dts/framework/test_suite.py
> @@ -27,6 +27,7 @@
>  from .settings import SETTINGS
>  from .test_result import BuildTargetResult, Result, TestCaseResult, TestSuiteResult
>  from .testbed_model import SutNode, TGNode
> +from .testbed_model.capturing_traffic_generator import PacketFilteringConfig
>  from .testbed_model.hw.port import Port, PortLink
>  from .utils import get_packet_summaries
>
> @@ -149,7 +150,12 @@ def configure_testbed_ipv4(self, restore: bool = False) -> None:
>      def _configure_ipv4_forwarding(self, enable: bool) -> None:
>          self.sut_node.configure_ipv4_forwarding(enable)
>
> -    def send_packet_and_capture(self, packet: Packet, duration: float = 1) -> list[Packet]:
> +    def send_packet_and_capture(
> +        self,
> +        packet: Packet,
> +        filter_config: PacketFilteringConfig = PacketFilteringConfig(),
> +        duration: float = 1,
> +    ) -> list[Packet]:
>          """
>          Send a packet through the appropriate interface and
>          receive on the appropriate interface.
> @@ -158,7 +164,11 @@ def send_packet_and_capture(self, packet: Packet, duration: float = 1) -> list[P
>          """
>          packet = self._adjust_addresses(packet)
>          return self.tg_node.send_packet_and_capture(
> -            packet, self._tg_port_egress, self._tg_port_ingress, duration
> +            packet,
> +            self._tg_port_egress,
> +            self._tg_port_ingress,
> +            filter_config,
> +            duration,
>          )
>
>      def get_expected_packet(self, packet: Packet) -> Packet:
> diff --git a/dts/framework/testbed_model/capturing_traffic_generator.py b/dts/framework/testbed_model/capturing_traffic_generator.py
> index e6512061d7..c40b030fe4 100644
> --- a/dts/framework/testbed_model/capturing_traffic_generator.py
> +++ b/dts/framework/testbed_model/capturing_traffic_generator.py
> @@ -11,6 +11,7 @@
>
>  import uuid
>  from abc import abstractmethod
> +from dataclasses import dataclass
>
>  import scapy.utils  # type: ignore[import]
>  from scapy.packet import Packet  # type: ignore[import]
> @@ -29,6 +30,19 @@ def _get_default_capture_name() -> str:
>      return str(uuid.uuid4())
>
>
> +@dataclass(slots=True)
> +class PacketFilteringConfig:
> +    """The supported filtering options for :class:`CapturingTrafficGenerator`.
> +
> +    Attributes:
> +        no_lldp: If :data:`True`, LLDP packets will be filtered out when capturing.
> +        no_arp: If :data:`True`, ARP packets will be filtered out when capturing.
> +    """
> +
> +    no_lldp: bool = True
> +    no_arp: bool = True
> +
> +
>  class CapturingTrafficGenerator(TrafficGenerator):
>      """Capture packets after sending traffic.
>
> @@ -51,6 +65,7 @@ def send_packet_and_capture(
>          packet: Packet,
>          send_port: Port,
>          receive_port: Port,
> +        filter_config: PacketFilteringConfig,
>          duration: float,
>          capture_name: str = _get_default_capture_name(),
>      ) -> list[Packet]:
> @@ -64,6 +79,7 @@ def send_packet_and_capture(
>              packet: The packet to send.
>              send_port: The egress port on the TG node.
>              receive_port: The ingress port in the TG node.
> +            filter_config: Filters to apply when capturing packets.
>              duration: Capture traffic for this amount of time after sending the packet.
>              capture_name: The name of the .pcap file where to store the capture.
>
> @@ -71,7 +87,7 @@ def send_packet_and_capture(
>               A list of received packets. May be empty if no packets are captured.
>          """
>          return self.send_packets_and_capture(
> -            [packet], send_port, receive_port, duration, capture_name
> +            [packet], send_port, receive_port, filter_config, duration, capture_name
>          )
>
>      def send_packets_and_capture(
> @@ -79,6 +95,7 @@ def send_packets_and_capture(
>          packets: list[Packet],
>          send_port: Port,
>          receive_port: Port,
> +        filter_config: PacketFilteringConfig,
>          duration: float,
>          capture_name: str = _get_default_capture_name(),
>      ) -> list[Packet]:
> @@ -92,6 +109,7 @@ def send_packets_and_capture(
>              packets: The packets to send.
>              send_port: The egress port on the TG node.
>              receive_port: The ingress port in the TG node.
> +            filter_config: Filters to apply when capturing packets.
>              duration: Capture traffic for this amount of time after sending the packets.
>              capture_name: The name of the .pcap file where to store the capture.
>
> @@ -106,6 +124,7 @@ def send_packets_and_capture(
>              packets,
>              send_port,
>              receive_port,
> +            filter_config,
>              duration,
>          )
>
> @@ -119,6 +138,7 @@ def _send_packets_and_capture(
>          packets: list[Packet],
>          send_port: Port,
>          receive_port: Port,
> +        filter_config: PacketFilteringConfig,
>          duration: float,
>      ) -> list[Packet]:
>          """
> diff --git a/dts/framework/testbed_model/scapy.py b/dts/framework/testbed_model/scapy.py
> index 9083e92b3d..94b0af7c6f 100644
> --- a/dts/framework/testbed_model/scapy.py
> +++ b/dts/framework/testbed_model/scapy.py
> @@ -30,6 +30,7 @@
>
>  from .capturing_traffic_generator import (
>      CapturingTrafficGenerator,
> +    PacketFilteringConfig,
>      _get_default_capture_name,
>  )
>  from .hw.port import Port
> @@ -69,6 +70,7 @@ def scapy_send_packets_and_capture(
>      send_iface: str,
>      recv_iface: str,
>      duration: float,
> +    sniff_filter: str,
>  ) -> list[bytes]:
>      """RPC function to send and capture packets.
>
> @@ -90,6 +92,7 @@ def scapy_send_packets_and_capture(
>          iface=recv_iface,
>          store=True,
>          started_callback=lambda *args: scapy.all.sendp(scapy_packets, iface=send_iface),
> +        filter=sniff_filter,
>      )
>      sniffer.start()
>      time.sleep(duration)
> @@ -249,16 +252,38 @@ def _send_packets(self, packets: list[Packet], port: Port) -> None:
>          packets = [packet.build() for packet in packets]
>          self.rpc_server_proxy.scapy_send_packets(packets, port.logical_name)
>
> +    def _create_packet_filter(self, filter_config: PacketFilteringConfig) -> str:
> +        """Combines filter settings from `filter_config` into a BPF that scapy can use.
> +
> +        Scapy allows for the use of Berkeley Packet Filters (BPFs) to filter what packets are
> +        collected based on various attributes of the packet.
> +
> +        Args:
> +            filter_config: Config class that specifies which filters should be applied.
> +
> +        Returns:
> +            A string representing the combination of BPF filters to be passed to scapy. For
> +            example:
> +
> +            "ether[12:2] != 0x88cc && ether[12:2] != 0x0806"
> +        """
> +        bpf_filter: list[str] = []
> +        if filter_config.no_arp:
> +            bpf_filter.append("ether[12:2] != 0x0806")
> +        if filter_config.no_lldp:
> +            bpf_filter.append("ether[12:2] != 0x88cc")
> +        return " && ".join(bpf_filter)
> +
>      def _send_packets_and_capture(
>          self,
>          packets: list[Packet],
>          send_port: Port,
>          receive_port: Port,
> +        filter_config: PacketFilteringConfig,
>          duration: float,
>          capture_name: str = _get_default_capture_name(),
>      ) -> list[Packet]:
>          binary_packets = [packet.build() for packet in packets]
> -
>          xmlrpc_packets: list[
>              xmlrpc.client.Binary
>          ] = self.rpc_server_proxy.scapy_send_packets_and_capture(
> @@ -266,6 +291,7 @@ def _send_packets_and_capture(
>              send_port.logical_name,
>              receive_port.logical_name,
>              duration,
> +            self._create_packet_filter(filter_config),
>          )  # type: ignore[assignment]
>
>          scapy_packets = [Ether(packet.data) for packet in xmlrpc_packets]
> diff --git a/dts/framework/testbed_model/tg_node.py b/dts/framework/testbed_model/tg_node.py
> index 79a55663b5..475dc2968d 100644
> --- a/dts/framework/testbed_model/tg_node.py
> +++ b/dts/framework/testbed_model/tg_node.py
> @@ -23,7 +23,10 @@
>  )
>  from framework.exception import ConfigurationError
>
> -from .capturing_traffic_generator import CapturingTrafficGenerator
> +from .capturing_traffic_generator import (
> +    CapturingTrafficGenerator,
> +    PacketFilteringConfig,
> +)
>  from .hw.port import Port
>  from .node import Node
>
> @@ -53,6 +56,7 @@ def send_packet_and_capture(
>          packet: Packet,
>          send_port: Port,
>          receive_port: Port,
> +        filter_config: PacketFilteringConfig = PacketFilteringConfig(),
>          duration: float = 1,
>      ) -> list[Packet]:
>          """Send a packet, return received traffic.
> @@ -71,7 +75,11 @@ def send_packet_and_capture(
>               A list of received packets. May be empty if no packets are captured.
>          """
>          return self.traffic_generator.send_packet_and_capture(
> -            packet, send_port, receive_port, duration
> +            packet,
> +            send_port,
> +            receive_port,
> +            filter_config,
> +            duration,
>          )
>
>      def close(self) -> None:
> --
> 2.43.0
>

[PATCH v4 4/7] dts: add pci addresses to EAL parameters

[jspewock]

From: Jeremy Spewock <jspewock@iol.unh.edu>

Added allow list to the EAL parameters created in DTS to ensure that
only the relevant PCI devices are considered when launching DPDK
applications.

Signed-off-by: Jeremy Spewock <jspewock@iol.unh.edu>
---
 dts/framework/testbed_model/sut_node.py | 11 +++++++++++
 1 file changed, 11 insertions(+)

diff --git a/dts/framework/testbed_model/sut_node.py b/dts/framework/testbed_model/sut_node.py
index 9c92232d9e..77caea2fc9 100644
--- a/dts/framework/testbed_model/sut_node.py
+++ b/dts/framework/testbed_model/sut_node.py
@@ -20,6 +20,7 @@
 from framework.utils import MesonArgs
 
 from .hw import LogicalCoreCount, LogicalCoreList, VirtualDevice
+from .hw.port import Port
 from .node import Node
 
 
@@ -31,6 +32,7 @@ def __init__(
         prefix: str,
         no_pci: bool,
         vdevs: list[VirtualDevice],
+        ports: list[Port],
         other_eal_param: str,
     ):
         """
@@ -46,6 +48,7 @@ def __init__(
                             VirtualDevice('net_ring0'),
                             VirtualDevice('net_ring1')
                         ]
+        :param ports: the list of ports to allow.
         :param other_eal_param: user defined DPDK eal parameters, eg:
                         other_eal_param='--single-file-segments'
         """
@@ -56,6 +59,7 @@ def __init__(
             self._prefix = f"--file-prefix={prefix}"
         self._no_pci = "--no-pci" if no_pci else ""
         self._vdevs = " ".join(f"--vdev {vdev}" for vdev in vdevs)
+        self._ports = " ".join(f"-a {port.pci}" for port in ports)
         self._other_eal_param = other_eal_param
 
     def __str__(self) -> str:
@@ -65,6 +69,7 @@ def __str__(self) -> str:
             f"{self._prefix} "
             f"{self._no_pci} "
             f"{self._vdevs} "
+            f"{self._ports} "
             f"{self._other_eal_param}"
         )
 
@@ -294,6 +299,7 @@ def create_eal_parameters(
         append_prefix_timestamp: bool = True,
         no_pci: bool = False,
         vdevs: list[VirtualDevice] = None,
+        ports: list[Port] | None = None,
         other_eal_param: str = "",
     ) -> "EalParameters":
         """
@@ -317,6 +323,7 @@ def create_eal_parameters(
                             VirtualDevice('net_ring0'),
                             VirtualDevice('net_ring1')
                         ]
+        :param ports: the list of ports to allow.
         :param other_eal_param: user defined DPDK eal parameters, eg:
                         other_eal_param='--single-file-segments'
         :return: eal param string, eg:
@@ -334,12 +341,16 @@ def create_eal_parameters(
         if vdevs is None:
             vdevs = []
 
+        if ports is None:
+            ports = self.ports
+
         return EalParameters(
             lcore_list=lcore_list,
             memory_channels=self.config.memory_channels,
             prefix=prefix,
             no_pci=no_pci,
             vdevs=vdevs,
+            ports=ports,
             other_eal_param=other_eal_param,
         )
 
-- 
2.43.0

Re: [PATCH v4 4/7] dts: add pci addresses to EAL parameters

[Juraj Linkeš]

Reviewed-by: Juraj Linkeš <juraj.linkes@pantheon.tech>

On Mon, Dec 18, 2023 at 7:13 PM <jspewock@iol.unh.edu> wrote:
>
> From: Jeremy Spewock <jspewock@iol.unh.edu>
>
> Added allow list to the EAL parameters created in DTS to ensure that
> only the relevant PCI devices are considered when launching DPDK
> applications.
>
> Signed-off-by: Jeremy Spewock <jspewock@iol.unh.edu>
> ---
>  dts/framework/testbed_model/sut_node.py | 11 +++++++++++
>  1 file changed, 11 insertions(+)
>
> diff --git a/dts/framework/testbed_model/sut_node.py b/dts/framework/testbed_model/sut_node.py
> index 9c92232d9e..77caea2fc9 100644
> --- a/dts/framework/testbed_model/sut_node.py
> +++ b/dts/framework/testbed_model/sut_node.py
> @@ -20,6 +20,7 @@
>  from framework.utils import MesonArgs
>
>  from .hw import LogicalCoreCount, LogicalCoreList, VirtualDevice
> +from .hw.port import Port
>  from .node import Node
>
>
> @@ -31,6 +32,7 @@ def __init__(
>          prefix: str,
>          no_pci: bool,
>          vdevs: list[VirtualDevice],
> +        ports: list[Port],
>          other_eal_param: str,
>      ):
>          """
> @@ -46,6 +48,7 @@ def __init__(
>                              VirtualDevice('net_ring0'),
>                              VirtualDevice('net_ring1')
>                          ]
> +        :param ports: the list of ports to allow.
>          :param other_eal_param: user defined DPDK eal parameters, eg:
>                          other_eal_param='--single-file-segments'
>          """
> @@ -56,6 +59,7 @@ def __init__(
>              self._prefix = f"--file-prefix={prefix}"
>          self._no_pci = "--no-pci" if no_pci else ""
>          self._vdevs = " ".join(f"--vdev {vdev}" for vdev in vdevs)
> +        self._ports = " ".join(f"-a {port.pci}" for port in ports)
>          self._other_eal_param = other_eal_param
>
>      def __str__(self) -> str:
> @@ -65,6 +69,7 @@ def __str__(self) -> str:
>              f"{self._prefix} "
>              f"{self._no_pci} "
>              f"{self._vdevs} "
> +            f"{self._ports} "
>              f"{self._other_eal_param}"
>          )
>
> @@ -294,6 +299,7 @@ def create_eal_parameters(
>          append_prefix_timestamp: bool = True,
>          no_pci: bool = False,
>          vdevs: list[VirtualDevice] = None,
> +        ports: list[Port] | None = None,
>          other_eal_param: str = "",
>      ) -> "EalParameters":
>          """
> @@ -317,6 +323,7 @@ def create_eal_parameters(
>                              VirtualDevice('net_ring0'),
>                              VirtualDevice('net_ring1')
>                          ]
> +        :param ports: the list of ports to allow.
>          :param other_eal_param: user defined DPDK eal parameters, eg:
>                          other_eal_param='--single-file-segments'
>          :return: eal param string, eg:
> @@ -334,12 +341,16 @@ def create_eal_parameters(
>          if vdevs is None:
>              vdevs = []
>
> +        if ports is None:
> +            ports = self.ports
> +
>          return EalParameters(
>              lcore_list=lcore_list,
>              memory_channels=self.config.memory_channels,
>              prefix=prefix,
>              no_pci=no_pci,
>              vdevs=vdevs,
> +            ports=ports,
>              other_eal_param=other_eal_param,
>          )
>
> --
> 2.43.0
>

[PATCH v4 5/7] dts: allow configuring MTU of ports

[jspewock]

From: Jeremy Spewock <jspewock@iol.unh.edu>

Adds methods in both os_session and linux session to allow for setting
MTU of port interfaces in an OS agnostic way.

Signed-off-by: Jeremy Spewock <jspewock@iol.unh.edu>
---
 dts/framework/remote_session/linux_session.py | 8 ++++++++
 dts/framework/remote_session/os_session.py    | 9 +++++++++
 2 files changed, 17 insertions(+)

diff --git a/dts/framework/remote_session/linux_session.py b/dts/framework/remote_session/linux_session.py
index fd877fbfae..aaa4d57a36 100644
--- a/dts/framework/remote_session/linux_session.py
+++ b/dts/framework/remote_session/linux_session.py
@@ -177,6 +177,14 @@ def configure_port_ip_address(
             verify=True,
         )
 
+    def configure_port_mtu(self, mtu: int, port: Port) -> None:
+        """Overrides :meth:`~.os_session.OSSession.configure_port_mtu`."""
+        self.send_command(
+            f"ip link set dev {port.logical_name} mtu {mtu}",
+            privileged=True,
+            verify=True,
+        )
+
     def configure_ipv4_forwarding(self, enable: bool) -> None:
         state = 1 if enable else 0
         self.send_command(f"sysctl -w net.ipv4.ip_forward={state}", privileged=True)
diff --git a/dts/framework/remote_session/os_session.py b/dts/framework/remote_session/os_session.py
index 8a709eac1c..cd073f5774 100644
--- a/dts/framework/remote_session/os_session.py
+++ b/dts/framework/remote_session/os_session.py
@@ -277,6 +277,15 @@ def configure_port_ip_address(
         Configure (add or delete) an IP address of the input port.
         """
 
+    @abstractmethod
+    def configure_port_mtu(self, mtu: int, port: Port) -> None:
+        """Configure `mtu` on `port`.
+
+        Args:
+            mtu: Desired MTU value.
+            port: Port to set `mtu` on.
+        """
+
     @abstractmethod
     def configure_ipv4_forwarding(self, enable: bool) -> None:
         """
-- 
2.43.0

Re: [PATCH v4 5/7] dts: allow configuring MTU of ports

[Juraj Linkeš]

On Mon, Dec 18, 2023 at 7:13 PM <jspewock@iol.unh.edu> wrote:
>
> From: Jeremy Spewock <jspewock@iol.unh.edu>
>
> Adds methods in both os_session and linux session to allow for setting
> MTU of port interfaces in an OS agnostic way.
>

The previous two commit messages had a little bit of an explanation,
but this one is missing one. Something like why a test case/suite
needs to set the MTU.

> Signed-off-by: Jeremy Spewock <jspewock@iol.unh.edu>
> ---
>  dts/framework/remote_session/linux_session.py | 8 ++++++++
>  dts/framework/remote_session/os_session.py    | 9 +++++++++
>  2 files changed, 17 insertions(+)
>
> diff --git a/dts/framework/remote_session/linux_session.py b/dts/framework/remote_session/linux_session.py
> index fd877fbfae..aaa4d57a36 100644
> --- a/dts/framework/remote_session/linux_session.py
> +++ b/dts/framework/remote_session/linux_session.py
> @@ -177,6 +177,14 @@ def configure_port_ip_address(
>              verify=True,
>          )
>
> +    def configure_port_mtu(self, mtu: int, port: Port) -> None:
> +        """Overrides :meth:`~.os_session.OSSession.configure_port_mtu`."""
> +        self.send_command(
> +            f"ip link set dev {port.logical_name} mtu {mtu}",
> +            privileged=True,
> +            verify=True,
> +        )
> +
>      def configure_ipv4_forwarding(self, enable: bool) -> None:
>          state = 1 if enable else 0
>          self.send_command(f"sysctl -w net.ipv4.ip_forward={state}", privileged=True)
> diff --git a/dts/framework/remote_session/os_session.py b/dts/framework/remote_session/os_session.py
> index 8a709eac1c..cd073f5774 100644
> --- a/dts/framework/remote_session/os_session.py
> +++ b/dts/framework/remote_session/os_session.py
> @@ -277,6 +277,15 @@ def configure_port_ip_address(
>          Configure (add or delete) an IP address of the input port.
>          """
>
> +    @abstractmethod
> +    def configure_port_mtu(self, mtu: int, port: Port) -> None:
> +        """Configure `mtu` on `port`.
> +
> +        Args:
> +            mtu: Desired MTU value.
> +            port: Port to set `mtu` on.
> +        """
> +
>      @abstractmethod
>      def configure_ipv4_forwarding(self, enable: bool) -> None:
>          """
> --
> 2.43.0
>

[PATCH v4 6/7] dts: add scatter to the yaml schema

[jspewock]

From: Jeremy Spewock <jspewock@iol.unh.edu>

Allow for scatter to be specified in the configuration file.

Signed-off-by: Jeremy Spewock <jspewock@iol.unh.edu>
---
 dts/framework/config/conf_yaml_schema.json | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)

diff --git a/dts/framework/config/conf_yaml_schema.json b/dts/framework/config/conf_yaml_schema.json
index 84e45fe3c2..e6dc50ca7f 100644
--- a/dts/framework/config/conf_yaml_schema.json
+++ b/dts/framework/config/conf_yaml_schema.json
@@ -186,7 +186,8 @@
       "type": "string",
       "enum": [
         "hello_world",
-        "os_udp"
+        "os_udp",
+        "pmd_buffer_scatter"
       ]
     },
     "test_target": {
-- 
2.43.0

Re: [PATCH v4 6/7] dts: add scatter to the yaml schema

[Juraj Linkeš]

Reviewed-by: Juraj Linkeš <juraj.linkes@pantheon.tech>

On Mon, Dec 18, 2023 at 7:13 PM <jspewock@iol.unh.edu> wrote:
>
> From: Jeremy Spewock <jspewock@iol.unh.edu>
>
> Allow for scatter to be specified in the configuration file.
>
> Signed-off-by: Jeremy Spewock <jspewock@iol.unh.edu>
> ---
>  dts/framework/config/conf_yaml_schema.json | 3 ++-
>  1 file changed, 2 insertions(+), 1 deletion(-)
>
> diff --git a/dts/framework/config/conf_yaml_schema.json b/dts/framework/config/conf_yaml_schema.json
> index 84e45fe3c2..e6dc50ca7f 100644
> --- a/dts/framework/config/conf_yaml_schema.json
> +++ b/dts/framework/config/conf_yaml_schema.json
> @@ -186,7 +186,8 @@
>        "type": "string",
>        "enum": [
>          "hello_world",
> -        "os_udp"
> +        "os_udp",
> +        "pmd_buffer_scatter"
>        ]
>      },
>      "test_target": {
> --
> 2.43.0
>

[PATCH v4 7/7] dts: add scatter test suite

[jspewock]

From: Jeremy Spewock <jspewock@iol.unh.edu>

This test suite provides testing the support of scattered packets by
Poll Mode Drivers using testpmd. It incorporates 5 different test cases
which test the sending and receiving of packets with lengths that are
less than the mbuf data buffer size, the same as the mbuf data buffer
size, and the mbuf data buffer size plus 1, 4, and 5. The goal of this
test suite is to align with the existing dts test plan for scattered
packets within DTS.

Signed-off-by: Jeremy Spewock <jspewock@iol.unh.edu>
---
 dts/tests/TestSuite_pmd_buffer_scatter.py | 105 ++++++++++++++++++++++
 1 file changed, 105 insertions(+)
 create mode 100644 dts/tests/TestSuite_pmd_buffer_scatter.py

diff --git a/dts/tests/TestSuite_pmd_buffer_scatter.py b/dts/tests/TestSuite_pmd_buffer_scatter.py
new file mode 100644
index 0000000000..8e2a32a1aa
--- /dev/null
+++ b/dts/tests/TestSuite_pmd_buffer_scatter.py
@@ -0,0 +1,105 @@
+# SPDX-License-Identifier: BSD-3-Clause
+# Copyright(c) 2023 University of New Hampshire
+
+"""Multi-segment packet scattering testing suite.
+
+Configure the Rx queues to have mbuf data buffers whose sizes are smaller than the maximum packet
+size (currently set to 2048 to fit a full 1512-byte ethernet frame) and send a total of 5 packets
+with lengths less than, equal to, and greater than the mbuf size (CRC included).
+"""
+import struct
+
+from scapy.layers.inet import IP  # type: ignore[import]
+from scapy.layers.l2 import Ether  # type: ignore[import]
+from scapy.packet import Raw  # type: ignore[import]
+from scapy.utils import hexstr  # type: ignore[import]
+
+from framework.remote_session.remote.testpmd_shell import (
+    TestPmdForwardingModes,
+    TestPmdShell,
+)
+from framework.test_suite import TestSuite
+
+
+class PmdBufferScatter(TestSuite):
+    """DPDK packet scattering test suite.
+
+    Attributes:
+        mbsize: The size to se the mbuf to be.
+    """
+
+    mbsize: int
+
+    def set_up_suite(self) -> None:
+        self.verify(
+            len(self._port_links) > 1,
+            "Must have at least two port links to run scatter",
+        )
+
+        self.tg_node.main_session.configure_port_mtu(9000, self._tg_port_egress)
+        self.tg_node.main_session.configure_port_mtu(9000, self._tg_port_ingress)
+
+    def scatter_pktgen_send_packet(self, pktsize: int) -> str:
+        """Generate and send packet to the SUT.
+
+        Functional test for scatter packets.
+
+        Args:
+            pktsize: Size of the packet to generate and send.
+        """
+        packet = Ether() / IP() / Raw()
+        packet.getlayer(2).load = ""
+        payload_len = pktsize - len(packet) - 4
+        payload = ["58"] * payload_len
+        # pack the payload
+        for X_in_hex in payload:
+            packet.load += struct.pack("=B", int("%s%s" % (X_in_hex[0], X_in_hex[1]), 16))
+        load = hexstr(packet.getlayer(2), onlyhex=1)
+        received_packets = self.send_packet_and_capture(packet)
+        self.verify(len(received_packets) > 0, "Did not receive any packets.")
+        load = hexstr(received_packets[0].getlayer(2), onlyhex=1)
+
+        return load
+
+    def pmd_scatter(self) -> None:
+        """Testpmd support of receiving and sending scattered multi-segment packets.
+
+        Support for scattered packets is shown by sending 5 packets of differing length
+        where the length of the packet is calculated by taking mbuf-size + an offset. The
+        offsets used in the test case are -1, 0, 1, 4, 5 respectively.
+
+        Test:
+            Start testpmd and run functional test with preset mbsize.
+        """
+        testpmd = self.sut_node.create_interactive_shell(
+            TestPmdShell,
+            app_parameters=(
+                "--mbcache=200 "
+                f"--mbuf-size={self.mbsize} "
+                "--max-pkt-len=9000 "
+                "--port-topology=paired "
+                "--tx-offloads=0x00008000"
+            ),
+            privileged=True,
+        )
+        testpmd.set_forward_mode(TestPmdForwardingModes.mac)
+        testpmd.start()
+        link_is_up = testpmd.wait_link_status_up(0) and testpmd.wait_link_status_up(1)
+        self.verify(link_is_up, "Links never came up in TestPMD.")
+
+        for offset in [-1, 0, 1, 4, 5]:
+            recv_payload = self.scatter_pktgen_send_packet(self.mbsize + offset)
+            self.verify(
+                ("58 " * 8).strip() in recv_payload,
+                "Received packet had incorrect payload",
+            )
+        testpmd.stop()
+
+    def test_scatter_mbuf_2048(self) -> None:
+        """Run :func:`~PmdBufferScatter.pmd_scatter` function after setting the `mbsize` to 2048."""
+        self.mbsize = 2048
+        self.pmd_scatter()
+
+    def tear_down_suite(self) -> None:
+        self.tg_node.main_session.configure_port_mtu(1500, self._tg_port_egress)
+        self.tg_node.main_session.configure_port_mtu(1500, self._tg_port_ingress)
-- 
2.43.0

Re: [PATCH v4 7/7] dts: add scatter test suite

[Juraj Linkeš]

Should we use the full name (pmd_buffer_scatter) in the subject? I
lean towards the full name.

On Mon, Dec 18, 2023 at 7:13 PM <jspewock@iol.unh.edu> wrote:
>
> From: Jeremy Spewock <jspewock@iol.unh.edu>
>
> This test suite provides testing the support of scattered packets by
> Poll Mode Drivers using testpmd. It incorporates 5 different test cases
> which test the sending and receiving of packets with lengths that are
> less than the mbuf data buffer size, the same as the mbuf data buffer
> size, and the mbuf data buffer size plus 1, 4, and 5. The goal of this
> test suite is to align with the existing dts test plan for scattered
> packets within DTS.
>

Again, we need to describe why we're adding this commit. In the case
of test suites, the why is obvious, so we should give a good high
level description of what the tests test (something like the test
suite tests the x feature by doing y, y being the salient part of the
tests). The original test plan is actually pretty good, so we can
extract the rationale from it.

> Signed-off-by: Jeremy Spewock <jspewock@iol.unh.edu>
> ---
>  dts/tests/TestSuite_pmd_buffer_scatter.py | 105 ++++++++++++++++++++++
>  1 file changed, 105 insertions(+)
>  create mode 100644 dts/tests/TestSuite_pmd_buffer_scatter.py
>
> diff --git a/dts/tests/TestSuite_pmd_buffer_scatter.py b/dts/tests/TestSuite_pmd_buffer_scatter.py
> new file mode 100644
> index 0000000000..8e2a32a1aa
> --- /dev/null
> +++ b/dts/tests/TestSuite_pmd_buffer_scatter.py
> @@ -0,0 +1,105 @@
> +# SPDX-License-Identifier: BSD-3-Clause
> +# Copyright(c) 2023 University of New Hampshire
> +
> +"""Multi-segment packet scattering testing suite.
> +
> +Configure the Rx queues to have mbuf data buffers whose sizes are smaller than the maximum packet
> +size (currently set to 2048 to fit a full 1512-byte ethernet frame) and send a total of 5 packets
> +with lengths less than, equal to, and greater than the mbuf size (CRC included).
> +"""

Let's expand this. I'll point to the original test plan again, let's
use some of it here. I think it makes sense to make this docstring a
kind of a test plan with high level description.

> +import struct
> +
> +from scapy.layers.inet import IP  # type: ignore[import]
> +from scapy.layers.l2 import Ether  # type: ignore[import]
> +from scapy.packet import Raw  # type: ignore[import]
> +from scapy.utils import hexstr  # type: ignore[import]
> +
> +from framework.remote_session.remote.testpmd_shell import (
> +    TestPmdForwardingModes,
> +    TestPmdShell,
> +)
> +from framework.test_suite import TestSuite
> +
> +
> +class PmdBufferScatter(TestSuite):
> +    """DPDK packet scattering test suite.
> +

And here we could add some more specifics.

I'd like to utilize the original test plans and a split like this
makes sense at a first glance.

> +    Attributes:
> +        mbsize: The size to se the mbuf to be.
> +    """
> +
> +    mbsize: int
> +
> +    def set_up_suite(self) -> None:
> +        self.verify(
> +            len(self._port_links) > 1,
> +            "Must have at least two port links to run scatter",
> +        )
> +
> +        self.tg_node.main_session.configure_port_mtu(9000, self._tg_port_egress)
> +        self.tg_node.main_session.configure_port_mtu(9000, self._tg_port_ingress)
> +
> +    def scatter_pktgen_send_packet(self, pktsize: int) -> str:
> +        """Generate and send packet to the SUT.
> +
> +        Functional test for scatter packets.
> +
> +        Args:
> +            pktsize: Size of the packet to generate and send.
> +        """
> +        packet = Ether() / IP() / Raw()
> +        packet.getlayer(2).load = ""
> +        payload_len = pktsize - len(packet) - 4
> +        payload = ["58"] * payload_len
> +        # pack the payload
> +        for X_in_hex in payload:
> +            packet.load += struct.pack("=B", int("%s%s" % (X_in_hex[0], X_in_hex[1]), 16))
> +        load = hexstr(packet.getlayer(2), onlyhex=1)
> +        received_packets = self.send_packet_and_capture(packet)
> +        self.verify(len(received_packets) > 0, "Did not receive any packets.")
> +        load = hexstr(received_packets[0].getlayer(2), onlyhex=1)
> +
> +        return load
> +
> +    def pmd_scatter(self) -> None:
> +        """Testpmd support of receiving and sending scattered multi-segment packets.
> +
> +        Support for scattered packets is shown by sending 5 packets of differing length
> +        where the length of the packet is calculated by taking mbuf-size + an offset. The
> +        offsets used in the test case are -1, 0, 1, 4, 5 respectively.
> +
> +        Test:
> +            Start testpmd and run functional test with preset mbsize.
> +        """
> +        testpmd = self.sut_node.create_interactive_shell(
> +            TestPmdShell,
> +            app_parameters=(
> +                "--mbcache=200 "
> +                f"--mbuf-size={self.mbsize} "
> +                "--max-pkt-len=9000 "
> +                "--port-topology=paired "
> +                "--tx-offloads=0x00008000"
> +            ),
> +            privileged=True,
> +        )
> +        testpmd.set_forward_mode(TestPmdForwardingModes.mac)
> +        testpmd.start()
> +        link_is_up = testpmd.wait_link_status_up(0) and testpmd.wait_link_status_up(1)

These two calls should probably be inside testpmd.start(). Looks like
we're always going to be doing this.

Also, this assumes there will be two ports. Ideally, the shell itself
will know which ports to check (that should be in EAL parameters), but
that may require a bigger refactor (unless we just parse back the -a
options, which could be permissible).

> +        self.verify(link_is_up, "Links never came up in TestPMD.")
> +
> +        for offset in [-1, 0, 1, 4, 5]:
> +            recv_payload = self.scatter_pktgen_send_packet(self.mbsize + offset)
> +            self.verify(
> +                ("58 " * 8).strip() in recv_payload,
> +                "Received packet had incorrect payload",
> +            )

This is still just one test case. We should probably leave it as is
(i.e. not split into 5 test cases), but we should add the information
about the offset/test case into the message so that we know what
actually failed right away.

> +        testpmd.stop()
> +
> +    def test_scatter_mbuf_2048(self) -> None:
> +        """Run :func:`~PmdBufferScatter.pmd_scatter` function after setting the `mbsize` to 2048."""
> +        self.mbsize = 2048
> +        self.pmd_scatter()

Let's just pass 2048 to the method, I don't see a reason for the
instance variable.

> +
> +    def tear_down_suite(self) -> None:
> +        self.tg_node.main_session.configure_port_mtu(1500, self._tg_port_egress)
> +        self.tg_node.main_session.configure_port_mtu(1500, self._tg_port_ingress)
> --
> 2.43.0
>

Re: [PATCH v4 7/7] dts: add scatter test suite

[Jeremy Spewock]

[-- Attachment #1: Type: text/plain, Size: 5974 bytes --]

On Tue, Dec 19, 2023 at 12:29 PM Juraj Linkeš <juraj.linkes@pantheon.tech>
wrote:

> Should we use the full name (pmd_buffer_scatter) in the subject? I
> lean towards the full name.
>
> On Mon, Dec 18, 2023 at 7:13 PM <jspewock@iol.unh.edu> wrote:
> >
> > From: Jeremy Spewock <jspewock@iol.unh.edu>
> >
> > This test suite provides testing the support of scattered packets by
> > Poll Mode Drivers using testpmd. It incorporates 5 different test cases
> > which test the sending and receiving of packets with lengths that are
> > less than the mbuf data buffer size, the same as the mbuf data buffer
> > size, and the mbuf data buffer size plus 1, 4, and 5. The goal of this
> > test suite is to align with the existing dts test plan for scattered
> > packets within DTS.
> >
>
> Again, we need to describe why we're adding this commit. In the case
> of test suites, the why is obvious, so we should give a good high
> level description of what the tests test (something like the test
> suite tests the x feature by doing y, y being the salient part of the
> tests). The original test plan is actually pretty good, so we can
> extract the rationale from it.
>

This is a good point, I'll pull more from the test plan to improve this.


>
> > Signed-off-by: Jeremy Spewock <jspewock@iol.unh.edu>
> > ---
> >  dts/tests/TestSuite_pmd_buffer_scatter.py | 105 ++++++++++++++++++++++
> >  1 file changed, 105 insertions(+)
> >  create mode 100644 dts/tests/TestSuite_pmd_buffer_scatter.py
> >
> > diff --git a/dts/tests/TestSuite_pmd_buffer_scatter.py
> b/dts/tests/TestSuite_pmd_buffer_scatter.py
> > new file mode 100644
> > index 0000000000..8e2a32a1aa
> > --- /dev/null
> > +++ b/dts/tests/TestSuite_pmd_buffer_scatter.py
> > @@ -0,0 +1,105 @@
> > +# SPDX-License-Identifier: BSD-3-Clause
> > +# Copyright(c) 2023 University of New Hampshire
> > +
> > +"""Multi-segment packet scattering testing suite.
> > +
> > +Configure the Rx queues to have mbuf data buffers whose sizes are
> smaller than the maximum packet
> > +size (currently set to 2048 to fit a full 1512-byte ethernet frame) and
> send a total of 5 packets
> > +with lengths less than, equal to, and greater than the mbuf size (CRC
> included).
> > +"""
>
> Let's expand this. I'll point to the original test plan again, let's
> use some of it here. I think it makes sense to make this docstring a
> kind of a test plan with high level description.
>

Sounds good, I'll expand this too.


>
> > +import struct
> > +
> > +from scapy.layers.inet import IP  # type: ignore[import]
> > +from scapy.layers.l2 import Ether  # type: ignore[import]
> > +from scapy.packet import Raw  # type: ignore[import]
> > +from scapy.utils import hexstr  # type: ignore[import]
> > +
> > +from framework.remote_session.remote.testpmd_shell import (
> > +    TestPmdForwardingModes,
> > +    TestPmdShell,
> > +)
> > +from framework.test_suite import TestSuite
> > +
> > +
> > +class PmdBufferScatter(TestSuite):
> > +    """DPDK packet scattering test suite.
> > +
>
> And here we could add some more specifics.
>

> I'd like to utilize the original test plans and a split like this
> makes sense at a first glance.
>

I'll add more here in the next version as well. I agree that using the
original test plans will help make this more descriptive and better for the
documentation.


> > +        testpmd.set_forward_mode(TestPmdForwardingModes.mac)
> > +        testpmd.start()
> > +        link_is_up = testpmd.wait_link_status_up(0) and
> testpmd.wait_link_status_up(1)
>
> These two calls should probably be inside testpmd.start(). Looks like
> we're always going to be doing this.
>
> Also, this assumes there will be two ports. Ideally, the shell itself
> will know which ports to check (that should be in EAL parameters), but
> that may require a bigger refactor (unless we just parse back the -a
> options, which could be permissible).
>

Collecting the number of ports should be easy enough from the original args
and then I can just verify ports 0-num are up so I agree that this is a
simple way to change this that adds a good amount of value.

While I don't believe the link status is actually directly related to
starting testpmd, I think that if we are going to start forwarding we still
are also always going to want to be sure that the links are up and we can
properly forward the packets. I'll add this to the verification check in
the start method.


>
> > +        self.verify(link_is_up, "Links never came up in TestPMD.")
> > +
> > +        for offset in [-1, 0, 1, 4, 5]:
> > +            recv_payload = self.scatter_pktgen_send_packet(self.mbsize
> + offset)
> > +            self.verify(
> > +                ("58 " * 8).strip() in recv_payload,
> > +                "Received packet had incorrect payload",
> > +            )
>
> This is still just one test case. We should probably leave it as is
> (i.e. not split into 5 test cases), but we should add the information
> about the offset/test case into the message so that we know what
> actually failed right away.
>

Good catch, never hurts to be more descriptive.


>
> > +        testpmd.stop()
> > +
> > +    def test_scatter_mbuf_2048(self) -> None:
> > +        """Run :func:`~PmdBufferScatter.pmd_scatter` function after
> setting the `mbsize` to 2048."""
> > +        self.mbsize = 2048
> > +        self.pmd_scatter()
>
> Let's just pass 2048 to the method, I don't see a reason for the
> instance variable.
>

Sure, that makes sense to me as well, I'll change this in the next version.


>
> > +
> > +    def tear_down_suite(self) -> None:
> > +        self.tg_node.main_session.configure_port_mtu(1500,
> self._tg_port_egress)
> > +        self.tg_node.main_session.configure_port_mtu(1500,
> self._tg_port_ingress)
> > --
> > 2.43.0
> >
>

Thank you for the feedback!

[-- Attachment #2: Type: text/html, Size: 8959 bytes --]

Re: [PATCH v4 7/7] dts: add scatter test suite

[Juraj Linkeš]

On Thu, Dec 21, 2023 at 10:47 PM Jeremy Spewock <jspewock@iol.unh.edu> wrote:
>
>
>
> On Tue, Dec 19, 2023 at 12:29 PM Juraj Linkeš <juraj.linkes@pantheon.tech> wrote:
>>
>> Should we use the full name (pmd_buffer_scatter) in the subject? I
>> lean towards the full name.
>>
>> On Mon, Dec 18, 2023 at 7:13 PM <jspewock@iol.unh.edu> wrote:
>> >
>> > From: Jeremy Spewock <jspewock@iol.unh.edu>
>> >
>> > This test suite provides testing the support of scattered packets by
>> > Poll Mode Drivers using testpmd. It incorporates 5 different test cases
>> > which test the sending and receiving of packets with lengths that are
>> > less than the mbuf data buffer size, the same as the mbuf data buffer
>> > size, and the mbuf data buffer size plus 1, 4, and 5. The goal of this
>> > test suite is to align with the existing dts test plan for scattered
>> > packets within DTS.
>> >
>>
>> Again, we need to describe why we're adding this commit. In the case
>> of test suites, the why is obvious, so we should give a good high
>> level description of what the tests test (something like the test
>> suite tests the x feature by doing y, y being the salient part of the
>> tests). The original test plan is actually pretty good, so we can
>> extract the rationale from it.
>
>
> This is a good point, I'll pull more from the test plan to improve this.
>
>>
>>
>> > Signed-off-by: Jeremy Spewock <jspewock@iol.unh.edu>
>> > ---
>> >  dts/tests/TestSuite_pmd_buffer_scatter.py | 105 ++++++++++++++++++++++
>> >  1 file changed, 105 insertions(+)
>> >  create mode 100644 dts/tests/TestSuite_pmd_buffer_scatter.py
>> >
>> > diff --git a/dts/tests/TestSuite_pmd_buffer_scatter.py b/dts/tests/TestSuite_pmd_buffer_scatter.py
>> > new file mode 100644
>> > index 0000000000..8e2a32a1aa
>> > --- /dev/null
>> > +++ b/dts/tests/TestSuite_pmd_buffer_scatter.py
>> > @@ -0,0 +1,105 @@
>> > +# SPDX-License-Identifier: BSD-3-Clause
>> > +# Copyright(c) 2023 University of New Hampshire
>> > +
>> > +"""Multi-segment packet scattering testing suite.
>> > +
>> > +Configure the Rx queues to have mbuf data buffers whose sizes are smaller than the maximum packet
>> > +size (currently set to 2048 to fit a full 1512-byte ethernet frame) and send a total of 5 packets
>> > +with lengths less than, equal to, and greater than the mbuf size (CRC included).
>> > +"""
>>
>> Let's expand this. I'll point to the original test plan again, let's
>> use some of it here. I think it makes sense to make this docstring a
>> kind of a test plan with high level description.
>
>
> Sounds good, I'll expand this too.
>
>>
>>
>> > +import struct
>> > +
>> > +from scapy.layers.inet import IP  # type: ignore[import]
>> > +from scapy.layers.l2 import Ether  # type: ignore[import]
>> > +from scapy.packet import Raw  # type: ignore[import]
>> > +from scapy.utils import hexstr  # type: ignore[import]
>> > +
>> > +from framework.remote_session.remote.testpmd_shell import (
>> > +    TestPmdForwardingModes,
>> > +    TestPmdShell,
>> > +)
>> > +from framework.test_suite import TestSuite
>> > +
>> > +
>> > +class PmdBufferScatter(TestSuite):
>> > +    """DPDK packet scattering test suite.
>> > +
>>
>> And here we could add some more specifics.
>>
>>
>> I'd like to utilize the original test plans and a split like this
>> makes sense at a first glance.
>
>
> I'll add more here in the next version as well. I agree that using the original test plans will help make this more descriptive and better for the documentation.
>
>>
>> > +        testpmd.set_forward_mode(TestPmdForwardingModes.mac)
>> > +        testpmd.start()
>> > +        link_is_up = testpmd.wait_link_status_up(0) and testpmd.wait_link_status_up(1)
>>
>> These two calls should probably be inside testpmd.start(). Looks like
>> we're always going to be doing this.
>>
>> Also, this assumes there will be two ports. Ideally, the shell itself
>> will know which ports to check (that should be in EAL parameters), but
>> that may require a bigger refactor (unless we just parse back the -a
>> options, which could be permissible).
>
>
> Collecting the number of ports should be easy enough from the original args and then I can just verify ports 0-num are up so I agree that this is a simple way to change this that adds a good amount of value.
>
> While I don't believe the link status is actually directly related to starting testpmd, I think that if we are going to start forwarding we still are also always going to want to be sure that the links are up and we can properly forward the packets. I'll add this to the verification check in the start method.
>

Right, we don't necessarily need to verify port status when starting
testpmd (that could be optional though, we could use that in a smoke
test). We should always check it when enabling forwarding (and if we
ever need to not do that we can add an option for that later).

>>
>>
>> > +        self.verify(link_is_up, "Links never came up in TestPMD.")
>> > +
>> > +        for offset in [-1, 0, 1, 4, 5]:
>> > +            recv_payload = self.scatter_pktgen_send_packet(self.mbsize + offset)
>> > +            self.verify(
>> > +                ("58 " * 8).strip() in recv_payload,
>> > +                "Received packet had incorrect payload",
>> > +            )
>>
>> This is still just one test case. We should probably leave it as is
>> (i.e. not split into 5 test cases), but we should add the information
>> about the offset/test case into the message so that we know what
>> actually failed right away.
>
>
> Good catch, never hurts to be more descriptive.
>
>>
>>
>> > +        testpmd.stop()
>> > +
>> > +    def test_scatter_mbuf_2048(self) -> None:
>> > +        """Run :func:`~PmdBufferScatter.pmd_scatter` function after setting the `mbsize` to 2048."""
>> > +        self.mbsize = 2048
>> > +        self.pmd_scatter()
>>
>> Let's just pass 2048 to the method, I don't see a reason for the
>> instance variable.
>
>
> Sure, that makes sense to me as well, I'll change this in the next version.
>
>>
>>
>> > +
>> > +    def tear_down_suite(self) -> None:
>> > +        self.tg_node.main_session.configure_port_mtu(1500, self._tg_port_egress)
>> > +        self.tg_node.main_session.configure_port_mtu(1500, self._tg_port_ingress)
>> > --
>> > 2.43.0
>> >
>
>
> Thank you for the feedback!

[PATCH v5 0/7] dts: Port scatter suite over

[jspewock]

From: Jeremy Spewock <jspewock@iol.unh.edu>

v5:

Addressed comments and made changes to files where appropriate. Notably,
added things such as verification to testpmd methods and a flag which
hides messages about "link state change events" in testpmd as such
messages changed the expected format of the terminal and cause
collecting output to be unreliable. Link statuses however are verified
manually to account for this.

Most other changes surrounded modification of commit message
descriptions and documentation in docstrings.

Jeremy Spewock (7):
  dts: add startup verification and forwarding modes to testpmd shell
  dts: limit EAL parameters to DPDK apps and add parameters to all apps
  dts: add optional packet filtering to scapy sniffer
  dts: add pci addresses to EAL parameters
  dts: allow configuring MTU of ports
  dts: add scatter to the yaml schema
  dts: add pmd_buffer_scatter test suite

 dts/framework/config/conf_yaml_schema.json    |   3 +-
 dts/framework/exception.py                    |   4 +
 dts/framework/remote_session/testpmd_shell.py | 148 +++++++++++++++++-
 dts/framework/test_suite.py                   |  15 +-
 dts/framework/testbed_model/linux_session.py  |   8 +
 dts/framework/testbed_model/os_session.py     |   9 ++
 dts/framework/testbed_model/sut_node.py       |  28 +++-
 dts/framework/testbed_model/tg_node.py        |  14 +-
 .../traffic_generator/__init__.py             |   5 +-
 .../capturing_traffic_generator.py            |  22 ++-
 .../testbed_model/traffic_generator/scapy.py  |  28 +++-
 dts/tests/TestSuite_pmd_buffer_scatter.py     | 115 ++++++++++++++
 12 files changed, 384 insertions(+), 15 deletions(-)
 create mode 100644 dts/tests/TestSuite_pmd_buffer_scatter.py

-- 
2.43.0

[PATCH v5 1/7] dts: add startup verification and forwarding modes to testpmd shell

[jspewock]

From: Jeremy Spewock <jspewock@iol.unh.edu>

Added commonly used methods in testpmd such as starting and stopping
packet forwarding, changing foward modes, and verifying link status of
ports so that developers can configure testpmd and start forwaring
through the provided class rather than sending commands to the testpmd
session directly.

Signed-off-by: Jeremy Spewock <jspewock@iol.unh.edu>
---
 dts/framework/exception.py                    |   7 +
 dts/framework/remote_session/testpmd_shell.py | 149 +++++++++++++++++-
 2 files changed, 155 insertions(+), 1 deletion(-)

diff --git a/dts/framework/exception.py b/dts/framework/exception.py
index 658eee2c38..cce1e0231a 100644
--- a/dts/framework/exception.py
+++ b/dts/framework/exception.py
@@ -146,6 +146,13 @@ def __str__(self) -> str:
         return f"Command {self.command} returned a non-zero exit code: {self._command_return_code}"
 
 
+class InteractiveCommandExecutionError(DTSError):
+    """An unsuccessful execution of a remote command in an interactive environment."""
+
+    #:
+    severity: ClassVar[ErrorSeverity] = ErrorSeverity.REMOTE_CMD_EXEC_ERR
+
+
 class RemoteDirectoryExistsError(DTSError):
     """A directory that exists on a remote node."""
 
diff --git a/dts/framework/remote_session/testpmd_shell.py b/dts/framework/remote_session/testpmd_shell.py
index 0184cc2e71..f310705fac 100644
--- a/dts/framework/remote_session/testpmd_shell.py
+++ b/dts/framework/remote_session/testpmd_shell.py
@@ -15,9 +15,15 @@
     testpmd_shell.close()
 """
 
+import time
+from enum import auto
 from pathlib import PurePath
 from typing import Callable, ClassVar
 
+from framework.exception import InteractiveCommandExecutionError
+from framework.settings import SETTINGS
+from framework.utils import StrEnum
+
 from .interactive_shell import InteractiveShell
 
 
@@ -43,14 +49,51 @@ def __str__(self) -> str:
         return self.pci_address
 
 
+class TestPmdForwardingModes(StrEnum):
+    r"""The supported packet forwarding modes for :class:`~TestPmdShell`\s."""
+
+    #:
+    io = auto()
+    #:
+    mac = auto()
+    #:
+    macswap = auto()
+    #:
+    flowgen = auto()
+    #:
+    rxonly = auto()
+    #:
+    txonly = auto()
+    #:
+    csum = auto()
+    #:
+    icmpecho = auto()
+    #:
+    ieee1588 = auto()
+    #:
+    noisy = auto()
+    #:
+    fivetswap = "5tswap"
+    #:
+    shared_rxq = "shared-rxq"
+    #:
+    recycle_mbufs = auto()
+
+
 class TestPmdShell(InteractiveShell):
     """Testpmd interactive shell.
 
     The testpmd shell users should never use
     the :meth:`~.interactive_shell.InteractiveShell.send_command` method directly, but rather
     call specialized methods. If there isn't one that satisfies a need, it should be added.
+
+    Attributes:
+        number_of_ports: The number of ports which were allowed on the command-line when testpmd
+            was started.
     """
 
+    number_of_ports: int
+
     #: The path to the testpmd executable.
     path: ClassVar[PurePath] = PurePath("app", "dpdk-testpmd")
 
@@ -65,9 +108,66 @@ class TestPmdShell(InteractiveShell):
     _command_extra_chars: ClassVar[str] = "\n"
 
     def _start_application(self, get_privileged_command: Callable[[str], str] | None) -> None:
-        self._app_args += " -- -i"
+        """Overrides :meth:`~.interactive_shell._start_application`.
+
+        Add flags for starting testpmd in interactive mode and disabling messages for link state
+        change events before starting the application. Link state is verified before starting
+        packet forwarding and the messages create unexpected newlines in the terminal which
+        complicates output collection.
+
+        Also find the number of pci addresses which were allowed on the command line when the app
+        was started.
+        """
+        self._app_args += " -- -i --mask-event intr_lsc"
+        self.number_of_ports = self._app_args.count("-a ")
         super()._start_application(get_privileged_command)
 
+    def start(self, verify: bool = True) -> None:
+        """Start packet forwarding with the current configuration.
+
+        Args:
+            verify: If :data:`True` , a second start command will be sent in an attempt to verify
+                packet forwarding started as expected.
+
+        Raises:
+            InteractiveCommandExecutionError: If `verify` is :data:`True` and forwarding fails to
+                start or ports fail to come up.
+        """
+        self.send_command("start")
+        if verify:
+            # If forwarding was already started, sending "start" again should tell us
+            start_cmd_output = self.send_command("start")
+            if "Packet forwarding already started" not in start_cmd_output:
+                self._logger.debug(f"Failed to start packet forwarding: \n{start_cmd_output}")
+                raise InteractiveCommandExecutionError("Testpmd failed to start packet forwarding.")
+
+            for port_id in range(self.number_of_ports):
+                if not self.wait_link_status_up(port_id):
+                    raise InteractiveCommandExecutionError(
+                        "Not all ports came up after starting packet forwarding in testpmd."
+                    )
+
+    def stop(self, verify: bool = True) -> None:
+        """Stop packet forwarding.
+
+        Args:
+            verify: If :data:`True` , the output of the stop command is scanned to verify that
+                forwarding was stopped successfully or not started. If neither is found, it is
+                considered an error.
+
+        Raises:
+            InteractiveCommandExecutionError: If `verify` is :data:`True` and the command to stop
+                forwarding results in an error.
+        """
+        stop_cmd_output = self.send_command("stop")
+        if verify:
+            if (
+                "Done." not in stop_cmd_output
+                and "Packet forwarding not started" not in stop_cmd_output
+            ):
+                self._logger.debug(f"Failed to stop packet forwarding: \n{stop_cmd_output}")
+                raise InteractiveCommandExecutionError("Testpmd failed to stop packet forwarding.")
+
     def get_devices(self) -> list[TestPmdDevice]:
         """Get a list of device names that are known to testpmd.
 
@@ -82,3 +182,50 @@ def get_devices(self) -> list[TestPmdDevice]:
             if "device name:" in line.lower():
                 dev_list.append(TestPmdDevice(line))
         return dev_list
+
+    def wait_link_status_up(self, port_id: int, timeout=SETTINGS.timeout) -> bool:
+        """Wait until the link status on the given port is "up".
+
+        Arguments:
+            port_id: Port to check the link status on.
+            timeout: Time to wait for the link to come up. The default value for this
+                argument is set using the :option:`-t, --timeout` command-line argument
+                or the :envvar:`DTS_TIMEOUT` environment variable.
+
+        Returns:
+            Whether the link came up in time or not.
+        """
+        time_to_stop = time.time() + timeout
+        port_info: str = ""
+        while time.time() < time_to_stop:
+            port_info = self.send_command(f"show port info {port_id}")
+            if "Link status: up" in port_info:
+                break
+            time.sleep(0.5)
+        else:
+            self._logger.error(f"The link for port {port_id} did not come up in the given timeout.")
+        return "Link status: up" in port_info
+
+    def set_forward_mode(self, mode: TestPmdForwardingModes, verify: bool = True):
+        """Set packet forwarding mode.
+
+        Args:
+            mode: The forwarding mode to use.
+            verify: If :data:`True` the output of the command will be scanned in an attempt to
+                verify that the forwarding mode was set to `mode` properly.
+
+        Raises:
+            InteractiveCommandExecutionError: If `verify` is :data:`True` and forwarding mode fails
+                to update.
+        """
+        set_fwd_output = self.send_command(f"set fwd {mode.value}")
+        if f"Set {mode.value} packet forwarding mode" not in set_fwd_output:
+            self._logger.debug(f"Failed to set fwd mode to {mode.value}:\n{set_fwd_output}")
+            raise InteractiveCommandExecutionError(
+                f"Test pmd failed to set fwd mode to {mode.value}"
+            )
+
+    def close(self) -> None:
+        """Overrides :meth:`~.interactive_shell.close`."""
+        self.send_command("quit", "")
+        return super().close()
-- 
2.43.0

[PATCH v5 2/7] dts: limit EAL parameters to DPDK apps and add parameters to all apps

[jspewock]

From: Jeremy Spewock <jspewock@iol.unh.edu>

Changed the factory method for creating interactive apps in the SUT Node
so that EAL parameters would only be passed into DPDK apps since
non-DPDK apps wouldn't be able to process them. Also modified
interactive apps to allow for the ability to pass parameters into the
app on startup so that the applications can be started with certain
configuration steps passed on the command line.

Signed-off-by: Jeremy Spewock <jspewock@iol.unh.edu>
---

I ended up reverting part of this back to making the argument for
eal_parameters allowed to be a string. This was because it was casuing
mypy errors where the method signatures of sut_node did not match with
that of node.

 dts/framework/remote_session/testpmd_shell.py |  2 +-
 dts/framework/testbed_model/sut_node.py       | 14 +++++++++-----
 2 files changed, 10 insertions(+), 6 deletions(-)

diff --git a/dts/framework/remote_session/testpmd_shell.py b/dts/framework/remote_session/testpmd_shell.py
index f310705fac..8f40e8f40e 100644
--- a/dts/framework/remote_session/testpmd_shell.py
+++ b/dts/framework/remote_session/testpmd_shell.py
@@ -118,7 +118,7 @@ def _start_application(self, get_privileged_command: Callable[[str], str] | None
         Also find the number of pci addresses which were allowed on the command line when the app
         was started.
         """
-        self._app_args += " -- -i --mask-event intr_lsc"
+        self._app_args += " -i --mask-event intr_lsc"
         self.number_of_ports = self._app_args.count("-a ")
         super()._start_application(get_privileged_command)
 
diff --git a/dts/framework/testbed_model/sut_node.py b/dts/framework/testbed_model/sut_node.py
index c4acea38d1..4df18bc183 100644
--- a/dts/framework/testbed_model/sut_node.py
+++ b/dts/framework/testbed_model/sut_node.py
@@ -431,6 +431,7 @@ def create_interactive_shell(
         timeout: float = SETTINGS.timeout,
         privileged: bool = False,
         eal_parameters: EalParameters | str | None = None,
+        app_parameters: str = "",
     ) -> InteractiveShellType:
         """Extend the factory for interactive session handlers.
 
@@ -449,20 +450,23 @@ def create_interactive_shell(
             eal_parameters: List of EAL parameters to use to launch the app. If this
                 isn't provided or an empty string is passed, it will default to calling
                 :meth:`create_eal_parameters`.
+            app_parameters: Additional arguments to pass into the application on the
+                command-line.
 
         Returns:
             An instance of the desired interactive application shell.
         """
-        if not eal_parameters:
-            eal_parameters = self.create_eal_parameters()
-
-        # We need to append the build directory for DPDK apps
+        # We need to append the build directory and add EAL parameters for DPDK apps
         if shell_cls.dpdk_app:
+            if not eal_parameters:
+                eal_parameters = self.create_eal_parameters()
+            app_parameters = f"{eal_parameters} -- {app_parameters}"
+
             shell_cls.path = self.main_session.join_remote_path(
                 self.remote_dpdk_build_dir, shell_cls.path
             )
 
-        return super().create_interactive_shell(shell_cls, timeout, privileged, str(eal_parameters))
+        return super().create_interactive_shell(shell_cls, timeout, privileged, app_parameters)
 
     def bind_ports_to_driver(self, for_dpdk: bool = True) -> None:
         """Bind all ports on the SUT to a driver.
-- 
2.43.0

[PATCH v5 3/7] dts: add optional packet filtering to scapy sniffer

[jspewock]

From: Jeremy Spewock <jspewock@iol.unh.edu>

Added the options to filter out LLDP and ARP packets when
sniffing for packets with scapy. This was done using BPF filters to
ensure that the noise these packets provide does not interfere with test
cases.

Signed-off-by: Jeremy Spewock <jspewock@iol.unh.edu>
---
 dts/framework/test_suite.py                   | 15 +++++++++--
 dts/framework/testbed_model/tg_node.py        | 14 ++++++++--
 .../traffic_generator/__init__.py             |  7 ++++-
 .../capturing_traffic_generator.py            | 22 ++++++++++++++-
 .../testbed_model/traffic_generator/scapy.py  | 27 +++++++++++++++++++
 5 files changed, 79 insertions(+), 6 deletions(-)

diff --git a/dts/framework/test_suite.py b/dts/framework/test_suite.py
index dfb391ffbd..ffea917690 100644
--- a/dts/framework/test_suite.py
+++ b/dts/framework/test_suite.py
@@ -38,6 +38,7 @@
 from .settings import SETTINGS
 from .test_result import BuildTargetResult, Result, TestCaseResult, TestSuiteResult
 from .testbed_model import Port, PortLink, SutNode, TGNode
+from .testbed_model.traffic_generator import PacketFilteringConfig
 from .utils import get_packet_summaries
 
 
@@ -208,7 +209,12 @@ def configure_testbed_ipv4(self, restore: bool = False) -> None:
     def _configure_ipv4_forwarding(self, enable: bool) -> None:
         self.sut_node.configure_ipv4_forwarding(enable)
 
-    def send_packet_and_capture(self, packet: Packet, duration: float = 1) -> list[Packet]:
+    def send_packet_and_capture(
+        self,
+        packet: Packet,
+        filter_config: PacketFilteringConfig = PacketFilteringConfig(),
+        duration: float = 1,
+    ) -> list[Packet]:
         """Send and receive `packet` using the associated TG.
 
         Send `packet` through the appropriate interface and receive on the appropriate interface.
@@ -216,6 +222,7 @@ def send_packet_and_capture(self, packet: Packet, duration: float = 1) -> list[P
 
         Args:
             packet: The packet to send.
+            filter_config: The filter to use when capturing packets.
             duration: Capture traffic for this amount of time after sending `packet`.
 
         Returns:
@@ -223,7 +230,11 @@ def send_packet_and_capture(self, packet: Packet, duration: float = 1) -> list[P
         """
         packet = self._adjust_addresses(packet)
         return self.tg_node.send_packet_and_capture(
-            packet, self._tg_port_egress, self._tg_port_ingress, duration
+            packet,
+            self._tg_port_egress,
+            self._tg_port_ingress,
+            filter_config,
+            duration,
         )
 
     def get_expected_packet(self, packet: Packet) -> Packet:
diff --git a/dts/framework/testbed_model/tg_node.py b/dts/framework/testbed_model/tg_node.py
index f269d4c585..d3206e87e0 100644
--- a/dts/framework/testbed_model/tg_node.py
+++ b/dts/framework/testbed_model/tg_node.py
@@ -15,7 +15,11 @@
 
 from .node import Node
 from .port import Port
-from .traffic_generator import CapturingTrafficGenerator, create_traffic_generator
+from .traffic_generator import (
+    CapturingTrafficGenerator,
+    PacketFilteringConfig,
+    create_traffic_generator,
+)
 
 
 class TGNode(Node):
@@ -53,6 +57,7 @@ def send_packet_and_capture(
         packet: Packet,
         send_port: Port,
         receive_port: Port,
+        filter_config: PacketFilteringConfig = PacketFilteringConfig(),
         duration: float = 1,
     ) -> list[Packet]:
         """Send `packet`, return received traffic.
@@ -65,13 +70,18 @@ def send_packet_and_capture(
             packet: The packet to send.
             send_port: The egress port on the TG node.
             receive_port: The ingress port in the TG node.
+            filter_config: The filter to use when capturing packets.
             duration: Capture traffic for this amount of time after sending `packet`.
 
         Returns:
              A list of received packets. May be empty if no packets are captured.
         """
         return self.traffic_generator.send_packet_and_capture(
-            packet, send_port, receive_port, duration
+            packet,
+            send_port,
+            receive_port,
+            filter_config,
+            duration,
         )
 
     def close(self) -> None:
diff --git a/dts/framework/testbed_model/traffic_generator/__init__.py b/dts/framework/testbed_model/traffic_generator/__init__.py
index 11e2bd7d97..0eaf0355cd 100644
--- a/dts/framework/testbed_model/traffic_generator/__init__.py
+++ b/dts/framework/testbed_model/traffic_generator/__init__.py
@@ -14,11 +14,16 @@
 and a capturing traffic generator is required.
 """
 
+# pylama:ignore=W0611
+
 from framework.config import ScapyTrafficGeneratorConfig, TrafficGeneratorType
 from framework.exception import ConfigurationError
 from framework.testbed_model.node import Node
 
-from .capturing_traffic_generator import CapturingTrafficGenerator
+from .capturing_traffic_generator import (
+    CapturingTrafficGenerator,
+    PacketFilteringConfig,
+)
 from .scapy import ScapyTrafficGenerator
 
 
diff --git a/dts/framework/testbed_model/traffic_generator/capturing_traffic_generator.py b/dts/framework/testbed_model/traffic_generator/capturing_traffic_generator.py
index 0246590333..c1c9facedd 100644
--- a/dts/framework/testbed_model/traffic_generator/capturing_traffic_generator.py
+++ b/dts/framework/testbed_model/traffic_generator/capturing_traffic_generator.py
@@ -11,6 +11,7 @@
 
 import uuid
 from abc import abstractmethod
+from dataclasses import dataclass
 
 import scapy.utils  # type: ignore[import]
 from scapy.packet import Packet  # type: ignore[import]
@@ -26,6 +27,19 @@ def _get_default_capture_name() -> str:
     return str(uuid.uuid4())
 
 
+@dataclass(slots=True)
+class PacketFilteringConfig:
+    """The supported filtering options for :class:`CapturingTrafficGenerator`.
+
+    Attributes:
+        no_lldp: If :data:`True`, LLDP packets will be filtered out when capturing.
+        no_arp: If :data:`True`, ARP packets will be filtered out when capturing.
+    """
+
+    no_lldp: bool = True
+    no_arp: bool = True
+
+
 class CapturingTrafficGenerator(TrafficGenerator):
     """Capture packets after sending traffic.
 
@@ -54,6 +68,7 @@ def send_packet_and_capture(
         packet: Packet,
         send_port: Port,
         receive_port: Port,
+        filter_config: PacketFilteringConfig,
         duration: float,
         capture_name: str = _get_default_capture_name(),
     ) -> list[Packet]:
@@ -68,6 +83,7 @@ def send_packet_and_capture(
             packet: The packet to send.
             send_port: The egress port on the TG node.
             receive_port: The ingress port in the TG node.
+            filter_config: Filters to apply when capturing packets.
             duration: Capture traffic for this amount of time after sending the packet.
             capture_name: The name of the .pcap file where to store the capture.
 
@@ -75,7 +91,7 @@ def send_packet_and_capture(
              The received packets. May be empty if no packets are captured.
         """
         return self.send_packets_and_capture(
-            [packet], send_port, receive_port, duration, capture_name
+            [packet], send_port, receive_port, filter_config, duration, capture_name
         )
 
     def send_packets_and_capture(
@@ -83,6 +99,7 @@ def send_packets_and_capture(
         packets: list[Packet],
         send_port: Port,
         receive_port: Port,
+        filter_config: PacketFilteringConfig,
         duration: float,
         capture_name: str = _get_default_capture_name(),
     ) -> list[Packet]:
@@ -99,6 +116,7 @@ def send_packets_and_capture(
             packets: The packets to send.
             send_port: The egress port on the TG node.
             receive_port: The ingress port in the TG node.
+            filter_config: Filters to apply when capturing packets.
             duration: Capture traffic for this amount of time after sending the packets.
             capture_name: The name of the .pcap file where to store the capture.
 
@@ -113,6 +131,7 @@ def send_packets_and_capture(
             packets,
             send_port,
             receive_port,
+            filter_config,
             duration,
         )
 
@@ -126,6 +145,7 @@ def _send_packets_and_capture(
         packets: list[Packet],
         send_port: Port,
         receive_port: Port,
+        filter_config: PacketFilteringConfig,
         duration: float,
     ) -> list[Packet]:
         """The implementation of :method:`send_packets_and_capture`.
diff --git a/dts/framework/testbed_model/traffic_generator/scapy.py b/dts/framework/testbed_model/traffic_generator/scapy.py
index 5b60f66237..505de0be94 100644
--- a/dts/framework/testbed_model/traffic_generator/scapy.py
+++ b/dts/framework/testbed_model/traffic_generator/scapy.py
@@ -32,6 +32,7 @@
 
 from .capturing_traffic_generator import (
     CapturingTrafficGenerator,
+    PacketFilteringConfig,
     _get_default_capture_name,
 )
 
@@ -69,6 +70,7 @@ def scapy_send_packets_and_capture(
     send_iface: str,
     recv_iface: str,
     duration: float,
+    sniff_filter: str,
 ) -> list[bytes]:
     """The RPC function to send and capture packets.
 
@@ -90,6 +92,7 @@ def scapy_send_packets_and_capture(
         iface=recv_iface,
         store=True,
         started_callback=lambda *args: scapy.all.sendp(scapy_packets, iface=send_iface),
+        filter=sniff_filter,
     )
     sniffer.start()
     time.sleep(duration)
@@ -260,11 +263,34 @@ def _send_packets(self, packets: list[Packet], port: Port) -> None:
         packets = [packet.build() for packet in packets]
         self.rpc_server_proxy.scapy_send_packets(packets, port.logical_name)
 
+    def _create_packet_filter(self, filter_config: PacketFilteringConfig) -> str:
+        """Combines filter settings from `filter_config` into a BPF that scapy can use.
+
+        Scapy allows for the use of Berkeley Packet Filters (BPFs) to filter what packets are
+        collected based on various attributes of the packet.
+
+        Args:
+            filter_config: Config class that specifies which filters should be applied.
+
+        Returns:
+            A string representing the combination of BPF filters to be passed to scapy. For
+            example:
+
+            "ether[12:2] != 0x88cc && ether[12:2] != 0x0806"
+        """
+        bpf_filter: list[str] = []
+        if filter_config.no_arp:
+            bpf_filter.append("ether[12:2] != 0x0806")
+        if filter_config.no_lldp:
+            bpf_filter.append("ether[12:2] != 0x88cc")
+        return " && ".join(bpf_filter)
+
     def _send_packets_and_capture(
         self,
         packets: list[Packet],
         send_port: Port,
         receive_port: Port,
+        filter_config: PacketFilteringConfig,
         duration: float,
         capture_name: str = _get_default_capture_name(),
     ) -> list[Packet]:
@@ -277,6 +303,7 @@ def _send_packets_and_capture(
             send_port.logical_name,
             receive_port.logical_name,
             duration,
+            self._create_packet_filter(filter_config),
         )  # type: ignore[assignment]
 
         scapy_packets = [Ether(packet.data) for packet in xmlrpc_packets]
-- 
2.43.0

[PATCH v5 4/7] dts: add pci addresses to EAL parameters

[jspewock]

From: Jeremy Spewock <jspewock@iol.unh.edu>

Added allow list to the EAL parameters created in DTS to ensure that
only the relevant PCI devices are considered when launching DPDK
applications.

Signed-off-by: Jeremy Spewock <jspewock@iol.unh.edu>
---
 dts/framework/testbed_model/sut_node.py | 12 ++++++++++++
 1 file changed, 12 insertions(+)

diff --git a/dts/framework/testbed_model/sut_node.py b/dts/framework/testbed_model/sut_node.py
index 4df18bc183..cc894fb07d 100644
--- a/dts/framework/testbed_model/sut_node.py
+++ b/dts/framework/testbed_model/sut_node.py
@@ -30,6 +30,7 @@
 from .cpu import LogicalCoreCount, LogicalCoreList
 from .node import Node
 from .os_session import InteractiveShellType, OSSession
+from .port import Port
 from .virtual_device import VirtualDevice
 
 
@@ -46,6 +47,7 @@ def __init__(
         prefix: str,
         no_pci: bool,
         vdevs: list[VirtualDevice],
+        ports: list[Port],
         other_eal_param: str,
     ):
         """Initialize the parameters according to inputs.
@@ -63,6 +65,7 @@ def __init__(
                     VirtualDevice('net_ring0'),
                     VirtualDevice('net_ring1')
                 ]
+            ports: The list of ports to allow.
             other_eal_param: user defined DPDK EAL parameters, e.g.:
                 ``other_eal_param='--single-file-segments'``
         """
@@ -73,6 +76,7 @@ def __init__(
             self._prefix = f"--file-prefix={prefix}"
         self._no_pci = "--no-pci" if no_pci else ""
         self._vdevs = " ".join(f"--vdev {vdev}" for vdev in vdevs)
+        self._ports = " ".join(f"-a {port.pci}" for port in ports)
         self._other_eal_param = other_eal_param
 
     def __str__(self) -> str:
@@ -83,6 +87,7 @@ def __str__(self) -> str:
             f"{self._prefix} "
             f"{self._no_pci} "
             f"{self._vdevs} "
+            f"{self._ports} "
             f"{self._other_eal_param}"
         )
 
@@ -347,6 +352,7 @@ def create_eal_parameters(
         append_prefix_timestamp: bool = True,
         no_pci: bool = False,
         vdevs: list[VirtualDevice] | None = None,
+        ports: list[Port] | None = None,
         other_eal_param: str = "",
     ) -> "EalParameters":
         """Compose the EAL parameters.
@@ -370,6 +376,8 @@ def create_eal_parameters(
                     VirtualDevice('net_ring0'),
                     VirtualDevice('net_ring1')
                 ]
+            ports: The list of ports to allow. If :data:`None`, all ports listed in `self.ports`
+                will be allowed.
             other_eal_param: user defined DPDK EAL parameters, e.g.:
                 ``other_eal_param='--single-file-segments'``.
 
@@ -388,12 +396,16 @@ def create_eal_parameters(
         if vdevs is None:
             vdevs = []
 
+        if ports is None:
+            ports = self.ports
+
         return EalParameters(
             lcore_list=lcore_list,
             memory_channels=self.config.memory_channels,
             prefix=prefix,
             no_pci=no_pci,
             vdevs=vdevs,
+            ports=ports,
             other_eal_param=other_eal_param,
         )
 
-- 
2.43.0

[PATCH v5 5/7] dts: allow configuring MTU of ports

[jspewock]

From: Jeremy Spewock <jspewock@iol.unh.edu>

Adds methods in both os_session and linux session to allow for setting
MTU of port interfaces so that suites that require the sending and
receiving of packets of a specific size, or the rejection of packets
over a certain size, can configure this maximum as needed.

Signed-off-by: Jeremy Spewock <jspewock@iol.unh.edu>
---
 dts/framework/testbed_model/linux_session.py | 8 ++++++++
 dts/framework/testbed_model/os_session.py    | 9 +++++++++
 2 files changed, 17 insertions(+)

diff --git a/dts/framework/testbed_model/linux_session.py b/dts/framework/testbed_model/linux_session.py
index 0ab59cef85..5d24030c3d 100644
--- a/dts/framework/testbed_model/linux_session.py
+++ b/dts/framework/testbed_model/linux_session.py
@@ -198,6 +198,14 @@ def configure_port_ip_address(
             verify=True,
         )
 
+    def configure_port_mtu(self, mtu: int, port: Port) -> None:
+        """Overrides :meth:`~.os_session.OSSession.configure_port_mtu`."""
+        self.send_command(
+            f"ip link set dev {port.logical_name} mtu {mtu}",
+            privileged=True,
+            verify=True,
+        )
+
     def configure_ipv4_forwarding(self, enable: bool) -> None:
         """Overrides :meth:`~.os_session.OSSession.configure_ipv4_forwarding`."""
         state = 1 if enable else 0
diff --git a/dts/framework/testbed_model/os_session.py b/dts/framework/testbed_model/os_session.py
index ac6bb5e112..e42f4d752a 100644
--- a/dts/framework/testbed_model/os_session.py
+++ b/dts/framework/testbed_model/os_session.py
@@ -413,6 +413,15 @@ def configure_port_ip_address(
             delete: If :data:`True`, remove the IP address, otherwise configure it.
         """
 
+    @abstractmethod
+    def configure_port_mtu(self, mtu: int, port: Port) -> None:
+        """Configure `mtu` on `port`.
+
+        Args:
+            mtu: Desired MTU value.
+            port: Port to set `mtu` on.
+        """
+
     @abstractmethod
     def configure_ipv4_forwarding(self, enable: bool) -> None:
         """Enable IPv4 forwarding in the operating system.
-- 
2.43.0

[PATCH v5 6/7] dts: add scatter to the yaml schema

[jspewock]

From: Jeremy Spewock <jspewock@iol.unh.edu>

Allow for scatter to be specified in the configuration file.

Signed-off-by: Jeremy Spewock <jspewock@iol.unh.edu>
---
 dts/framework/config/conf_yaml_schema.json | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)

diff --git a/dts/framework/config/conf_yaml_schema.json b/dts/framework/config/conf_yaml_schema.json
index 84e45fe3c2..e6dc50ca7f 100644
--- a/dts/framework/config/conf_yaml_schema.json
+++ b/dts/framework/config/conf_yaml_schema.json
@@ -186,7 +186,8 @@
       "type": "string",
       "enum": [
         "hello_world",
-        "os_udp"
+        "os_udp",
+        "pmd_buffer_scatter"
       ]
     },
     "test_target": {
-- 
2.43.0

[PATCH v5 7/7] dts: add pmd_buffer_scatter test suite

[jspewock]

From: Jeremy Spewock <jspewock@iol.unh.edu>

This test suite provides testing of the support of scattered packets by
Poll Mode Drivers using testpmd, verifying the ability to receive and
transmit scattered multi-segment packets made up of multiple
non-contiguous memory buffers. This is tested through 5 different cases
in which the length of the packets sent are less than the mbuf size,
equal to the mbuf size, and 1, 4, and 5 bytes greater than the mbuf size
in order to show both the CRC and the packet data are capable of
existing in the first, second, or both buffers.

Naturally, if the PMD is capable of forwarding scattered packets which
it receives as input, this shows it is capable of both receiving and
transmitting scattered packets.

Signed-off-by: Jeremy Spewock <jspewock@iol.unh.edu>
---
 dts/tests/TestSuite_pmd_buffer_scatter.py | 126 ++++++++++++++++++++++
 1 file changed, 126 insertions(+)
 create mode 100644 dts/tests/TestSuite_pmd_buffer_scatter.py

diff --git a/dts/tests/TestSuite_pmd_buffer_scatter.py b/dts/tests/TestSuite_pmd_buffer_scatter.py
new file mode 100644
index 0000000000..8838c3404f
--- /dev/null
+++ b/dts/tests/TestSuite_pmd_buffer_scatter.py
@@ -0,0 +1,126 @@
+# SPDX-License-Identifier: BSD-3-Clause
+# Copyright(c) 2023-2024 University of New Hampshire
+
+"""Multi-segment packet scattering testing suite.
+
+This testing suite tests the support of transmitting and receiving scattered packets. This is shown
+by the Poll Mode Driver being able to forward scattered multi-segment packets composed of multiple
+non-contiguous memory buffers. To ensure the receipt of scattered packets, the DMA rings of the
+port's RX queues must be configured with mbuf data buffers whose size is less than the maximum
+length.
+
+If it is the case that the Poll Mode Driver can forward scattered packets which it receives, then
+this suffices to show the Poll Mode Driver is capable of both receiving and transmitting scattered
+packets.
+"""
+import struct
+
+from scapy.layers.inet import IP  # type: ignore[import]
+from scapy.layers.l2 import Ether  # type: ignore[import]
+from scapy.packet import Raw  # type: ignore[import]
+from scapy.utils import hexstr  # type: ignore[import]
+
+from framework.remote_session.testpmd_shell import TestPmdForwardingModes, TestPmdShell
+from framework.test_suite import TestSuite
+
+
+class PmdBufferScatter(TestSuite):
+    """DPDK PMD packet scattering test suite.
+
+    Configure the Rx queues to have mbuf data buffers whose sizes are smaller than the maximum
+    packet size. Specifically, set mbuf data buffers to have a size of 2048 to fit a full 1512-byte
+    (CRC included) ethernet frame in a mono-segment packet. The testing of scattered packets is
+    done by sending a packet whose length is greater than the size of the configured size of mbuf
+    data buffers. There are a total of 5 packets sent within test cases which have lengths less
+    than, equal to, and greater than the mbuf size. There are multiple packets sent with lengths
+    greater than the mbuf size in order to test cases such as:
+
+    1. A single byte of the CRC being in a second buffer while the remaining 3 bytes are stored in
+        the first buffer alongside packet data.
+    2. The entire CRC being stored in a second buffer while all of the packet data is stored in the
+        first.
+    3. Most of the packet data being stored in the first buffer and a single byte of packet data
+        stored in a second buffer alongside the CRC.
+    """
+
+    def set_up_suite(self) -> None:
+        """Set up the test suite.
+
+        Setup:
+            Verify they we have at least 2 port links in the current execution and increase the MTU
+            of both ports on the tg_node to 9000 to support larger packet sizes.
+        """
+        self.verify(
+            len(self._port_links) > 1,
+            "Must have at least two port links to run scatter",
+        )
+
+        self.tg_node.main_session.configure_port_mtu(9000, self._tg_port_egress)
+        self.tg_node.main_session.configure_port_mtu(9000, self._tg_port_ingress)
+
+    def scatter_pktgen_send_packet(self, pktsize: int) -> str:
+        """Generate and send packet to the SUT.
+
+        Functional test for scatter packets.
+
+        Args:
+            pktsize: Size of the packet to generate and send.
+        """
+        packet = Ether() / IP() / Raw()
+        packet.getlayer(2).load = ""
+        payload_len = pktsize - len(packet) - 4
+        payload = ["58"] * payload_len
+        # pack the payload
+        for X_in_hex in payload:
+            packet.load += struct.pack("=B", int("%s%s" % (X_in_hex[0], X_in_hex[1]), 16))
+        received_packets = self.send_packet_and_capture(packet)
+        self.verify(len(received_packets) > 0, "Did not receive any packets.")
+        load = hexstr(received_packets[0].getlayer(2), onlyhex=1)
+
+        return load
+
+    def pmd_scatter(self, mbsize: int) -> None:
+        """Testpmd support of receiving and sending scattered multi-segment packets.
+
+        Support for scattered packets is shown by sending 5 packets of differing length
+        where the length of the packet is calculated by taking mbuf-size + an offset. The
+        offsets used in the test case are -1, 0, 1, 4, 5 respectively.
+
+        Test:
+            Start testpmd and run functional test with preset mbsize.
+        """
+        testpmd = self.sut_node.create_interactive_shell(
+            TestPmdShell,
+            app_parameters=(
+                "--mbcache=200 "
+                f"--mbuf-size={mbsize} "
+                "--max-pkt-len=9000 "
+                "--port-topology=paired "
+                "--tx-offloads=0x00008000"
+            ),
+            privileged=True,
+        )
+        testpmd.set_forward_mode(TestPmdForwardingModes.mac)
+        testpmd.start()
+
+        for offset in [-1, 0, 1, 4, 5]:
+            recv_payload = self.scatter_pktgen_send_packet(mbsize + offset)
+            self._logger.debug(f"Payload of scattered packet after forwarding: \n{recv_payload}")
+            self.verify(
+                ("58 " * 8).strip() in recv_payload,
+                f"Payload of scattered packet did not match expected payload with offset {offset}.",
+            )
+        testpmd.stop()
+
+    def test_scatter_mbuf_2048(self) -> None:
+        """Run :func:`~PmdBufferScatter.pmd_scatter` function with `mbsize` set to 2048."""
+        self.pmd_scatter(mbsize=2048)
+
+    def tear_down_suite(self) -> None:
+        """Tear down the test suite.
+
+        Teardown:
+            Set the MTU of the tg_node back to a more standard size of 1500
+        """
+        self.tg_node.main_session.configure_port_mtu(1500, self._tg_port_egress)
+        self.tg_node.main_session.configure_port_mtu(1500, self._tg_port_ingress)
-- 
2.43.0

[PATCH v6 0/7] dts: Port scatter suite over

[jspewock]

From: Jeremy Spewock <jspewock@iol.unh.edu>

v6:

Fixed spelling mistake that caused checkpatch failure.

Jeremy Spewock (7):
  dts: add startup verification and forwarding modes to testpmd shell
  dts: limit EAL parameters to DPDK apps and add parameters to all apps
  dts: add optional packet filtering to scapy sniffer
  dts: add pci addresses to EAL parameters
  dts: allow configuring MTU of ports
  dts: add scatter to the yaml schema
  dts: add pmd_buffer_scatter test suite

 dts/framework/config/conf_yaml_schema.json    |   3 +-
 dts/framework/exception.py                    |   7 +
 dts/framework/remote_session/testpmd_shell.py | 149 +++++++++++++++++-
 dts/framework/test_suite.py                   |  15 +-
 dts/framework/testbed_model/linux_session.py  |   8 +
 dts/framework/testbed_model/os_session.py     |   9 ++
 dts/framework/testbed_model/sut_node.py       |  26 ++-
 dts/framework/testbed_model/tg_node.py        |  14 +-
 .../traffic_generator/__init__.py             |   7 +-
 .../capturing_traffic_generator.py            |  22 ++-
 .../testbed_model/traffic_generator/scapy.py  |  27 ++++
 dts/tests/TestSuite_pmd_buffer_scatter.py     | 126 +++++++++++++++
 12 files changed, 400 insertions(+), 13 deletions(-)
 create mode 100644 dts/tests/TestSuite_pmd_buffer_scatter.py

-- 
2.43.0

[PATCH v6 1/7] dts: add startup verification and forwarding modes to testpmd shell

[jspewock]

From: Jeremy Spewock <jspewock@iol.unh.edu>

Added commonly used methods in testpmd such as starting and stopping
packet forwarding, changing forward modes, and verifying link status of
ports so that developers can configure testpmd and start forwarding
through the provided class rather than sending commands to the testpmd
session directly.

Signed-off-by: Jeremy Spewock <jspewock@iol.unh.edu>
---
 dts/framework/exception.py                    |   7 +
 dts/framework/remote_session/testpmd_shell.py | 149 +++++++++++++++++-
 2 files changed, 155 insertions(+), 1 deletion(-)

diff --git a/dts/framework/exception.py b/dts/framework/exception.py
index 658eee2c38..cce1e0231a 100644
--- a/dts/framework/exception.py
+++ b/dts/framework/exception.py
@@ -146,6 +146,13 @@ def __str__(self) -> str:
         return f"Command {self.command} returned a non-zero exit code: {self._command_return_code}"
 
 
+class InteractiveCommandExecutionError(DTSError):
+    """An unsuccessful execution of a remote command in an interactive environment."""
+
+    #:
+    severity: ClassVar[ErrorSeverity] = ErrorSeverity.REMOTE_CMD_EXEC_ERR
+
+
 class RemoteDirectoryExistsError(DTSError):
     """A directory that exists on a remote node."""
 
diff --git a/dts/framework/remote_session/testpmd_shell.py b/dts/framework/remote_session/testpmd_shell.py
index 0184cc2e71..f310705fac 100644
--- a/dts/framework/remote_session/testpmd_shell.py
+++ b/dts/framework/remote_session/testpmd_shell.py
@@ -15,9 +15,15 @@
     testpmd_shell.close()
 """
 
+import time
+from enum import auto
 from pathlib import PurePath
 from typing import Callable, ClassVar
 
+from framework.exception import InteractiveCommandExecutionError
+from framework.settings import SETTINGS
+from framework.utils import StrEnum
+
 from .interactive_shell import InteractiveShell
 
 
@@ -43,14 +49,51 @@ def __str__(self) -> str:
         return self.pci_address
 
 
+class TestPmdForwardingModes(StrEnum):
+    r"""The supported packet forwarding modes for :class:`~TestPmdShell`\s."""
+
+    #:
+    io = auto()
+    #:
+    mac = auto()
+    #:
+    macswap = auto()
+    #:
+    flowgen = auto()
+    #:
+    rxonly = auto()
+    #:
+    txonly = auto()
+    #:
+    csum = auto()
+    #:
+    icmpecho = auto()
+    #:
+    ieee1588 = auto()
+    #:
+    noisy = auto()
+    #:
+    fivetswap = "5tswap"
+    #:
+    shared_rxq = "shared-rxq"
+    #:
+    recycle_mbufs = auto()
+
+
 class TestPmdShell(InteractiveShell):
     """Testpmd interactive shell.
 
     The testpmd shell users should never use
     the :meth:`~.interactive_shell.InteractiveShell.send_command` method directly, but rather
     call specialized methods. If there isn't one that satisfies a need, it should be added.
+
+    Attributes:
+        number_of_ports: The number of ports which were allowed on the command-line when testpmd
+            was started.
     """
 
+    number_of_ports: int
+
     #: The path to the testpmd executable.
     path: ClassVar[PurePath] = PurePath("app", "dpdk-testpmd")
 
@@ -65,9 +108,66 @@ class TestPmdShell(InteractiveShell):
     _command_extra_chars: ClassVar[str] = "\n"
 
     def _start_application(self, get_privileged_command: Callable[[str], str] | None) -> None:
-        self._app_args += " -- -i"
+        """Overrides :meth:`~.interactive_shell._start_application`.
+
+        Add flags for starting testpmd in interactive mode and disabling messages for link state
+        change events before starting the application. Link state is verified before starting
+        packet forwarding and the messages create unexpected newlines in the terminal which
+        complicates output collection.
+
+        Also find the number of pci addresses which were allowed on the command line when the app
+        was started.
+        """
+        self._app_args += " -- -i --mask-event intr_lsc"
+        self.number_of_ports = self._app_args.count("-a ")
         super()._start_application(get_privileged_command)
 
+    def start(self, verify: bool = True) -> None:
+        """Start packet forwarding with the current configuration.
+
+        Args:
+            verify: If :data:`True` , a second start command will be sent in an attempt to verify
+                packet forwarding started as expected.
+
+        Raises:
+            InteractiveCommandExecutionError: If `verify` is :data:`True` and forwarding fails to
+                start or ports fail to come up.
+        """
+        self.send_command("start")
+        if verify:
+            # If forwarding was already started, sending "start" again should tell us
+            start_cmd_output = self.send_command("start")
+            if "Packet forwarding already started" not in start_cmd_output:
+                self._logger.debug(f"Failed to start packet forwarding: \n{start_cmd_output}")
+                raise InteractiveCommandExecutionError("Testpmd failed to start packet forwarding.")
+
+            for port_id in range(self.number_of_ports):
+                if not self.wait_link_status_up(port_id):
+                    raise InteractiveCommandExecutionError(
+                        "Not all ports came up after starting packet forwarding in testpmd."
+                    )
+
+    def stop(self, verify: bool = True) -> None:
+        """Stop packet forwarding.
+
+        Args:
+            verify: If :data:`True` , the output of the stop command is scanned to verify that
+                forwarding was stopped successfully or not started. If neither is found, it is
+                considered an error.
+
+        Raises:
+            InteractiveCommandExecutionError: If `verify` is :data:`True` and the command to stop
+                forwarding results in an error.
+        """
+        stop_cmd_output = self.send_command("stop")
+        if verify:
+            if (
+                "Done." not in stop_cmd_output
+                and "Packet forwarding not started" not in stop_cmd_output
+            ):
+                self._logger.debug(f"Failed to stop packet forwarding: \n{stop_cmd_output}")
+                raise InteractiveCommandExecutionError("Testpmd failed to stop packet forwarding.")
+
     def get_devices(self) -> list[TestPmdDevice]:
         """Get a list of device names that are known to testpmd.
 
@@ -82,3 +182,50 @@ def get_devices(self) -> list[TestPmdDevice]:
             if "device name:" in line.lower():
                 dev_list.append(TestPmdDevice(line))
         return dev_list
+
+    def wait_link_status_up(self, port_id: int, timeout=SETTINGS.timeout) -> bool:
+        """Wait until the link status on the given port is "up".
+
+        Arguments:
+            port_id: Port to check the link status on.
+            timeout: Time to wait for the link to come up. The default value for this
+                argument is set using the :option:`-t, --timeout` command-line argument
+                or the :envvar:`DTS_TIMEOUT` environment variable.
+
+        Returns:
+            Whether the link came up in time or not.
+        """
+        time_to_stop = time.time() + timeout
+        port_info: str = ""
+        while time.time() < time_to_stop:
+            port_info = self.send_command(f"show port info {port_id}")
+            if "Link status: up" in port_info:
+                break
+            time.sleep(0.5)
+        else:
+            self._logger.error(f"The link for port {port_id} did not come up in the given timeout.")
+        return "Link status: up" in port_info
+
+    def set_forward_mode(self, mode: TestPmdForwardingModes, verify: bool = True):
+        """Set packet forwarding mode.
+
+        Args:
+            mode: The forwarding mode to use.
+            verify: If :data:`True` the output of the command will be scanned in an attempt to
+                verify that the forwarding mode was set to `mode` properly.
+
+        Raises:
+            InteractiveCommandExecutionError: If `verify` is :data:`True` and forwarding mode fails
+                to update.
+        """
+        set_fwd_output = self.send_command(f"set fwd {mode.value}")
+        if f"Set {mode.value} packet forwarding mode" not in set_fwd_output:
+            self._logger.debug(f"Failed to set fwd mode to {mode.value}:\n{set_fwd_output}")
+            raise InteractiveCommandExecutionError(
+                f"Test pmd failed to set fwd mode to {mode.value}"
+            )
+
+    def close(self) -> None:
+        """Overrides :meth:`~.interactive_shell.close`."""
+        self.send_command("quit", "")
+        return super().close()
-- 
2.43.0

Re: [PATCH v6 1/7] dts: add startup verification and forwarding modes to testpmd shell

[Juraj Linkeš]

On Wed, Jan 3, 2024 at 11:33 PM <jspewock@iol.unh.edu> wrote:
>
> From: Jeremy Spewock <jspewock@iol.unh.edu>
>
> Added commonly used methods in testpmd such as starting and stopping
> packet forwarding, changing forward modes, and verifying link status of
> ports so that developers can configure testpmd and start forwarding
> through the provided class rather than sending commands to the testpmd
> session directly.
>
> Signed-off-by: Jeremy Spewock <jspewock@iol.unh.edu>
> ---
>  dts/framework/exception.py                    |   7 +
>  dts/framework/remote_session/testpmd_shell.py | 149 +++++++++++++++++-
>  2 files changed, 155 insertions(+), 1 deletion(-)
>
> diff --git a/dts/framework/exception.py b/dts/framework/exception.py
> index 658eee2c38..cce1e0231a 100644
> --- a/dts/framework/exception.py
> +++ b/dts/framework/exception.py
<snip>
> @@ -65,9 +108,66 @@ class TestPmdShell(InteractiveShell):
>      _command_extra_chars: ClassVar[str] = "\n"
>
>      def _start_application(self, get_privileged_command: Callable[[str], str] | None) -> None:
> -        self._app_args += " -- -i"
> +        """Overrides :meth:`~.interactive_shell._start_application`.
> +
> +        Add flags for starting testpmd in interactive mode and disabling messages for link state
> +        change events before starting the application. Link state is verified before starting
> +        packet forwarding and the messages create unexpected newlines in the terminal which
> +        complicates output collection.
> +

We should adjust the collection so that it can handle the newlines.
Also, can you explain exactly why we are disabling the initial link
state messages?

> +        Also find the number of pci addresses which were allowed on the command line when the app
> +        was started.
> +        """
> +        self._app_args += " -- -i --mask-event intr_lsc"
> +        self.number_of_ports = self._app_args.count("-a ")
>          super()._start_application(get_privileged_command)
>
> +    def start(self, verify: bool = True) -> None:
> +        """Start packet forwarding with the current configuration.
> +
> +        Args:
> +            verify: If :data:`True` , a second start command will be sent in an attempt to verify
> +                packet forwarding started as expected.
> +
> +        Raises:
> +            InteractiveCommandExecutionError: If `verify` is :data:`True` and forwarding fails to
> +                start or ports fail to come up.
> +        """
> +        self.send_command("start")
> +        if verify:
> +            # If forwarding was already started, sending "start" again should tell us
> +            start_cmd_output = self.send_command("start")
> +            if "Packet forwarding already started" not in start_cmd_output:
> +                self._logger.debug(f"Failed to start packet forwarding: \n{start_cmd_output}")
> +                raise InteractiveCommandExecutionError("Testpmd failed to start packet forwarding.")
> +
> +            for port_id in range(self.number_of_ports):
> +                if not self.wait_link_status_up(port_id):
> +                    raise InteractiveCommandExecutionError(
> +                        "Not all ports came up after starting packet forwarding in testpmd."
> +                    )
> +
> +    def stop(self, verify: bool = True) -> None:
> +        """Stop packet forwarding.
> +
> +        Args:
> +            verify: If :data:`True` , the output of the stop command is scanned to verify that
> +                forwarding was stopped successfully or not started. If neither is found, it is
> +                considered an error.
> +
> +        Raises:
> +            InteractiveCommandExecutionError: If `verify` is :data:`True` and the command to stop
> +                forwarding results in an error.
> +        """
> +        stop_cmd_output = self.send_command("stop")
> +        if verify:
> +            if (
> +                "Done." not in stop_cmd_output
> +                and "Packet forwarding not started" not in stop_cmd_output
> +            ):

I want to make sure I understand this condition. When none of these
appear, it's an error. When just "Done." appears, we successfully
stopped ongoing forwarding and when "Packet forwarding not started"
appears, we're trying to stop forwarding that didn't start (or isn't
ongoing - it could've stopped in the meantime)?
I'm thinking about false failures here (Is there a string that would
indicate a failure even if one of the strings is printed?) - we're
basically looking at "not success" instead of looking for strings
telling us about a failure explicitly. Does the stop command not
produce such output? Or do we not know all of the failure strings or
is looking for the above two strings sufficient to rule out false
failures?

> +                self._logger.debug(f"Failed to stop packet forwarding: \n{stop_cmd_output}")
> +                raise InteractiveCommandExecutionError("Testpmd failed to stop packet forwarding.")
> +
>      def get_devices(self) -> list[TestPmdDevice]:
>          """Get a list of device names that are known to testpmd.
>
> @@ -82,3 +182,50 @@ def get_devices(self) -> list[TestPmdDevice]:
>              if "device name:" in line.lower():
>                  dev_list.append(TestPmdDevice(line))
>          return dev_list
> +
> +    def wait_link_status_up(self, port_id: int, timeout=SETTINGS.timeout) -> bool:
> +        """Wait until the link status on the given port is "up".
> +
> +        Arguments:
> +            port_id: Port to check the link status on.
> +            timeout: Time to wait for the link to come up. The default value for this
> +                argument is set using the :option:`-t, --timeout` command-line argument
> +                or the :envvar:`DTS_TIMEOUT` environment variable.
> +

This really should be "may be modified", as it is optional.

> +        Returns:
> +            Whether the link came up in time or not.
> +        """
> +        time_to_stop = time.time() + timeout
> +        port_info: str = ""
> +        while time.time() < time_to_stop:
> +            port_info = self.send_command(f"show port info {port_id}")
> +            if "Link status: up" in port_info:
> +                break
> +            time.sleep(0.5)
> +        else:
> +            self._logger.error(f"The link for port {port_id} did not come up in the given timeout.")
> +        return "Link status: up" in port_info
> +
> +    def set_forward_mode(self, mode: TestPmdForwardingModes, verify: bool = True):
> +        """Set packet forwarding mode.
> +
> +        Args:
> +            mode: The forwarding mode to use.
> +            verify: If :data:`True` the output of the command will be scanned in an attempt to
> +                verify that the forwarding mode was set to `mode` properly.
> +
> +        Raises:
> +            InteractiveCommandExecutionError: If `verify` is :data:`True` and forwarding mode fails

I think there should be a definite article here - the forwarding mode.

> +                to update.
> +        """
> +        set_fwd_output = self.send_command(f"set fwd {mode.value}")
> +        if f"Set {mode.value} packet forwarding mode" not in set_fwd_output:
> +            self._logger.debug(f"Failed to set fwd mode to {mode.value}:\n{set_fwd_output}")
> +            raise InteractiveCommandExecutionError(
> +                f"Test pmd failed to set fwd mode to {mode.value}"
> +            )
> +
> +    def close(self) -> None:
> +        """Overrides :meth:`~.interactive_shell.close`."""
> +        self.send_command("quit", "")
> +        return super().close()
> --
> 2.43.0
>

Re: [PATCH v6 1/7] dts: add startup verification and forwarding modes to testpmd shell

[Jeremy Spewock]

[-- Attachment #1: Type: text/plain, Size: 10792 bytes --]

On Mon, Jan 8, 2024 at 6:35 AM Juraj Linkeš <juraj.linkes@pantheon.tech>
wrote:

> On Wed, Jan 3, 2024 at 11:33 PM <jspewock@iol.unh.edu> wrote:
> >
> > From: Jeremy Spewock <jspewock@iol.unh.edu>
> >
> > Added commonly used methods in testpmd such as starting and stopping
> > packet forwarding, changing forward modes, and verifying link status of
> > ports so that developers can configure testpmd and start forwarding
> > through the provided class rather than sending commands to the testpmd
> > session directly.
> >
> > Signed-off-by: Jeremy Spewock <jspewock@iol.unh.edu>
> > ---
> >  dts/framework/exception.py                    |   7 +
> >  dts/framework/remote_session/testpmd_shell.py | 149 +++++++++++++++++-
> >  2 files changed, 155 insertions(+), 1 deletion(-)
> >
> > diff --git a/dts/framework/exception.py b/dts/framework/exception.py
> > index 658eee2c38..cce1e0231a 100644
> > --- a/dts/framework/exception.py
> > +++ b/dts/framework/exception.py
> <snip>
> > @@ -65,9 +108,66 @@ class TestPmdShell(InteractiveShell):
> >      _command_extra_chars: ClassVar[str] = "\n"
> >
> >      def _start_application(self, get_privileged_command:
> Callable[[str], str] | None) -> None:
> > -        self._app_args += " -- -i"
> > +        """Overrides :meth:`~.interactive_shell._start_application`.
> > +
> > +        Add flags for starting testpmd in interactive mode and
> disabling messages for link state
> > +        change events before starting the application. Link state is
> verified before starting
> > +        packet forwarding and the messages create unexpected newlines
> in the terminal which
> > +        complicates output collection.
> > +
>
> We should adjust the collection so that it can handle the newlines.
> Also, can you explain exactly why we are disabling the initial link
> state messages?
>

The problem really comes from the newlines causing the prompt to exist in
the buffer before any command is sent. So, what ends up happening is after
starting the application these link state change events happen at some
point, and they cause an empty "testpmd>" line to exist in the buffer and
the next time you send a command it will stop as soon as it encounters that
line. An additional issue with this prompt is it is put in the buffer
before the link state change event occurs, and there is no prompt that
appears after the event messages, just an empty line. This makes it much
more difficult to detect when the link state change event occurs and
consume it because the event isn't captured the next time you collect
output, all that is consumed is a line containing the prompt.. So, this
makes you essentially one command's worth of output behind because the next
time you send a command you will consume what you were supposed to get from
the last command where you stopped early, and this causes false positives
for things like the link state detection method and failures in output
verification.

This puts you in a position where the only way you can really detect that
one of these events happened is either assuming that only getting an empty
prompt means one of these events happened, or trying to consume output
twice and looking ahead to see if one of these events happened. However,
because we wouldn't be doing anything with these events and we verify link
status before starting anyway, it seemed like the less complex but still
functional solution would just be to mask these events.


>
> > +        Also find the number of pci addresses which were allowed on the
> command line when the app
> > +        was started.
> > +        """
> > +        self._app_args += " -- -i --mask-event intr_lsc"
> > +        self.number_of_ports = self._app_args.count("-a ")
> >          super()._start_application(get_privileged_command)
> >
> > +    def start(self, verify: bool = True) -> None:
> > +        """Start packet forwarding with the current configuration.
> > +
> > +        Args:
> > +            verify: If :data:`True` , a second start command will be
> sent in an attempt to verify
> > +                packet forwarding started as expected.
> > +
> > +        Raises:
> > +            InteractiveCommandExecutionError: If `verify` is
> :data:`True` and forwarding fails to
> > +                start or ports fail to come up.
> > +        """
> > +        self.send_command("start")
> > +        if verify:
> > +            # If forwarding was already started, sending "start" again
> should tell us
> > +            start_cmd_output = self.send_command("start")
> > +            if "Packet forwarding already started" not in
> start_cmd_output:
> > +                self._logger.debug(f"Failed to start packet forwarding:
> \n{start_cmd_output}")
> > +                raise InteractiveCommandExecutionError("Testpmd failed
> to start packet forwarding.")
> > +
> > +            for port_id in range(self.number_of_ports):
> > +                if not self.wait_link_status_up(port_id):
> > +                    raise InteractiveCommandExecutionError(
> > +                        "Not all ports came up after starting packet
> forwarding in testpmd."
> > +                    )
> > +
> > +    def stop(self, verify: bool = True) -> None:
> > +        """Stop packet forwarding.
> > +
> > +        Args:
> > +            verify: If :data:`True` , the output of the stop command is
> scanned to verify that
> > +                forwarding was stopped successfully or not started. If
> neither is found, it is
> > +                considered an error.
> > +
> > +        Raises:
> > +            InteractiveCommandExecutionError: If `verify` is
> :data:`True` and the command to stop
> > +                forwarding results in an error.
> > +        """
> > +        stop_cmd_output = self.send_command("stop")
> > +        if verify:
> > +            if (
> > +                "Done." not in stop_cmd_output
> > +                and "Packet forwarding not started" not in
> stop_cmd_output
> > +            ):
>
> I want to make sure I understand this condition. When none of these
> appear, it's an error. When just "Done." appears, we successfully
> stopped ongoing forwarding and when "Packet forwarding not started"
> appears, we're trying to stop forwarding that didn't start (or isn't
> ongoing - it could've stopped in the meantime)?
> I'm thinking about false failures here (Is there a string that would
> indicate a failure even if one of the strings is printed?) - we're
> basically looking at "not success" instead of looking for strings
> telling us about a failure explicitly. Does the stop command not
> produce such output? Or do we not know all of the failure strings or
> is looking for the above two strings sufficient to rule out false
> failures?
>

You are correct that essentially what I am looking for here is if we
succeeded and else, it's a failure. When I looked through some of the
source code for testpmd from the method stop_packet_forwarding, I didn't
see any explicit error messages other than displaying that there was an
error printing statistics. So this was something where I both didn't know
the error messages but it doesn't look like there are any that are
explicitly printed. In the case of false failures however, the strings I am
detecting are always encountered in our two success cases (forwarding not
currently started and successfully stopped). The "Done." message does also
get printed in the case of statistics failing to print for a core as well.


>
> > +                self._logger.debug(f"Failed to stop packet forwarding:
> \n{stop_cmd_output}")
> > +                raise InteractiveCommandExecutionError("Testpmd failed
> to stop packet forwarding.")
> > +
> >      def get_devices(self) -> list[TestPmdDevice]:
> >          """Get a list of device names that are known to testpmd.
> >
> > @@ -82,3 +182,50 @@ def get_devices(self) -> list[TestPmdDevice]:
> >              if "device name:" in line.lower():
> >                  dev_list.append(TestPmdDevice(line))
> >          return dev_list
> > +
> > +    def wait_link_status_up(self, port_id: int,
> timeout=SETTINGS.timeout) -> bool:
> > +        """Wait until the link status on the given port is "up".
> > +
> > +        Arguments:
> > +            port_id: Port to check the link status on.
> > +            timeout: Time to wait for the link to come up. The default
> value for this
> > +                argument is set using the :option:`-t, --timeout`
> command-line argument
> > +                or the :envvar:`DTS_TIMEOUT` environment variable.
> > +
>
> This really should be "may be modified", as it is optional.
>

Good catch, I'll edit this.


>
> > +        Returns:
> > +            Whether the link came up in time or not.
> > +        """
> > +        time_to_stop = time.time() + timeout
> > +        port_info: str = ""
> > +        while time.time() < time_to_stop:
> > +            port_info = self.send_command(f"show port info {port_id}")
> > +            if "Link status: up" in port_info:
> > +                break
> > +            time.sleep(0.5)
> > +        else:
> > +            self._logger.error(f"The link for port {port_id} did not
> come up in the given timeout.")
> > +        return "Link status: up" in port_info
> > +
> > +    def set_forward_mode(self, mode: TestPmdForwardingModes, verify:
> bool = True):
> > +        """Set packet forwarding mode.
> > +
> > +        Args:
> > +            mode: The forwarding mode to use.
> > +            verify: If :data:`True` the output of the command will be
> scanned in an attempt to
> > +                verify that the forwarding mode was set to `mode`
> properly.
> > +
> > +        Raises:
> > +            InteractiveCommandExecutionError: If `verify` is
> :data:`True` and forwarding mode fails
>
> I think there should be a definite article here - the forwarding mode.
>

Good catch, I'll change this too.


>
> > +                to update.
> > +        """
> > +        set_fwd_output = self.send_command(f"set fwd {mode.value}")
> > +        if f"Set {mode.value} packet forwarding mode" not in
> set_fwd_output:
> > +            self._logger.debug(f"Failed to set fwd mode to
> {mode.value}:\n{set_fwd_output}")
> > +            raise InteractiveCommandExecutionError(
> > +                f"Test pmd failed to set fwd mode to {mode.value}"
> > +            )
> > +
> > +    def close(self) -> None:
> > +        """Overrides :meth:`~.interactive_shell.close`."""
> > +        self.send_command("quit", "")
> > +        return super().close()
> > --
> > 2.43.0
> >
>

[-- Attachment #2: Type: text/html, Size: 14191 bytes --]

Re: [PATCH v6 1/7] dts: add startup verification and forwarding modes to testpmd shell

[Juraj Linkeš]

On Mon, Jan 8, 2024 at 5:36 PM Jeremy Spewock <jspewock@iol.unh.edu> wrote:
>
>
>
> On Mon, Jan 8, 2024 at 6:35 AM Juraj Linkeš <juraj.linkes@pantheon.tech> wrote:
>>
>> On Wed, Jan 3, 2024 at 11:33 PM <jspewock@iol.unh.edu> wrote:
>> >
>> > From: Jeremy Spewock <jspewock@iol.unh.edu>
>> >
>> > Added commonly used methods in testpmd such as starting and stopping
>> > packet forwarding, changing forward modes, and verifying link status of
>> > ports so that developers can configure testpmd and start forwarding
>> > through the provided class rather than sending commands to the testpmd
>> > session directly.
>> >
>> > Signed-off-by: Jeremy Spewock <jspewock@iol.unh.edu>
>> > ---
>> >  dts/framework/exception.py                    |   7 +
>> >  dts/framework/remote_session/testpmd_shell.py | 149 +++++++++++++++++-
>> >  2 files changed, 155 insertions(+), 1 deletion(-)
>> >
>> > diff --git a/dts/framework/exception.py b/dts/framework/exception.py
>> > index 658eee2c38..cce1e0231a 100644
>> > --- a/dts/framework/exception.py
>> > +++ b/dts/framework/exception.py
>> <snip>
>> > @@ -65,9 +108,66 @@ class TestPmdShell(InteractiveShell):
>> >      _command_extra_chars: ClassVar[str] = "\n"
>> >
>> >      def _start_application(self, get_privileged_command: Callable[[str], str] | None) -> None:
>> > -        self._app_args += " -- -i"
>> > +        """Overrides :meth:`~.interactive_shell._start_application`.
>> > +
>> > +        Add flags for starting testpmd in interactive mode and disabling messages for link state
>> > +        change events before starting the application. Link state is verified before starting
>> > +        packet forwarding and the messages create unexpected newlines in the terminal which
>> > +        complicates output collection.
>> > +
>>
>> We should adjust the collection so that it can handle the newlines.
>> Also, can you explain exactly why we are disabling the initial link
>> state messages?
>
>
> The problem really comes from the newlines causing the prompt to exist in the buffer before any command is sent. So, what ends up happening is after starting the application these link state change events happen at some point, and they cause an empty "testpmd>" line to exist in the buffer and the next time you send a command it will stop as soon as it encounters that line.

These buffer issues keep cropping up. We should think about making
this more robust. Can we flush the buffer before sending a new command
(because any previous output is irrelevant)? This probably won't fix
all the problems, but it sounds like it could help. Maybe it could
help with the scapy docstring issue we're seen in the past as well.

In this patch though, we should just make this functional (I
understand disabling the messages achieves that) and address the
buffer issues in a separate patch.

> An additional issue with this prompt is it is put in the buffer before the link state change event occurs, and there is no prompt that appears after the event messages, just an empty line. This makes it much more difficult to detect when the link state change event occurs and consume it because the event isn't captured the next time you collect output, all that is consumed is a line containing the prompt.. So, this makes you essentially one command's worth of output behind because the next time you send a command you will consume what you were supposed to get from the last command where you stopped early, and this causes false positives for things like the link state detection method and failures in output verification.
> This puts you in a position where the only way you can really detect that one of these events happened is either assuming that only getting an empty prompt means one of these events happened, or trying to consume output twice and looking ahead to see if one of these events happened. However, because we wouldn't be doing anything with these events and we verify link status before starting anyway, it seemed like the less complex but still functional solution would just be to mask these events.

Right, these are basically random events, which makes it hard to
collect (but not impossible). Checking the status explicitly is way
better. Thanks for the explanation.

>
>>
>>
>> > +        Also find the number of pci addresses which were allowed on the command line when the app
>> > +        was started.
>> > +        """
>> > +        self._app_args += " -- -i --mask-event intr_lsc"
>> > +        self.number_of_ports = self._app_args.count("-a ")
>> >          super()._start_application(get_privileged_command)
>> >
>> > +    def start(self, verify: bool = True) -> None:
>> > +        """Start packet forwarding with the current configuration.
>> > +
>> > +        Args:
>> > +            verify: If :data:`True` , a second start command will be sent in an attempt to verify
>> > +                packet forwarding started as expected.
>> > +
>> > +        Raises:
>> > +            InteractiveCommandExecutionError: If `verify` is :data:`True` and forwarding fails to
>> > +                start or ports fail to come up.
>> > +        """
>> > +        self.send_command("start")
>> > +        if verify:
>> > +            # If forwarding was already started, sending "start" again should tell us
>> > +            start_cmd_output = self.send_command("start")
>> > +            if "Packet forwarding already started" not in start_cmd_output:
>> > +                self._logger.debug(f"Failed to start packet forwarding: \n{start_cmd_output}")
>> > +                raise InteractiveCommandExecutionError("Testpmd failed to start packet forwarding.")
>> > +
>> > +            for port_id in range(self.number_of_ports):
>> > +                if not self.wait_link_status_up(port_id):
>> > +                    raise InteractiveCommandExecutionError(
>> > +                        "Not all ports came up after starting packet forwarding in testpmd."
>> > +                    )
>> > +
>> > +    def stop(self, verify: bool = True) -> None:
>> > +        """Stop packet forwarding.
>> > +
>> > +        Args:
>> > +            verify: If :data:`True` , the output of the stop command is scanned to verify that
>> > +                forwarding was stopped successfully or not started. If neither is found, it is
>> > +                considered an error.
>> > +
>> > +        Raises:
>> > +            InteractiveCommandExecutionError: If `verify` is :data:`True` and the command to stop
>> > +                forwarding results in an error.
>> > +        """
>> > +        stop_cmd_output = self.send_command("stop")
>> > +        if verify:
>> > +            if (
>> > +                "Done." not in stop_cmd_output
>> > +                and "Packet forwarding not started" not in stop_cmd_output
>> > +            ):
>>
>> I want to make sure I understand this condition. When none of these
>> appear, it's an error. When just "Done." appears, we successfully
>> stopped ongoing forwarding and when "Packet forwarding not started"
>> appears, we're trying to stop forwarding that didn't start (or isn't
>> ongoing - it could've stopped in the meantime)?
>> I'm thinking about false failures here (Is there a string that would
>> indicate a failure even if one of the strings is printed?) - we're
>> basically looking at "not success" instead of looking for strings
>> telling us about a failure explicitly. Does the stop command not
>> produce such output? Or do we not know all of the failure strings or
>> is looking for the above two strings sufficient to rule out false
>> failures?
>
>
> You are correct that essentially what I am looking for here is if we succeeded and else, it's a failure. When I looked through some of the source code for testpmd from the method stop_packet_forwarding, I didn't see any explicit error messages other than displaying that there was an error printing statistics. So this was something where I both didn't know the error messages but it doesn't look like there are any that are explicitly printed. In the case of false failures however, the strings I am detecting are always encountered in our two success cases (forwarding not currently started and successfully stopped). The "Done." message does also get printed in the case of statistics failing to print for a core as well.

Ok, seems like the check is robust enough.

Re: [PATCH v6 1/7] dts: add startup verification and forwarding modes to testpmd shell

[Jeremy Spewock]

[-- Attachment #1: Type: text/plain, Size: 10133 bytes --]

On Tue, Jan 9, 2024 at 6:55 AM Juraj Linkeš <juraj.linkes@pantheon.tech>
wrote:

> On Mon, Jan 8, 2024 at 5:36 PM Jeremy Spewock <jspewock@iol.unh.edu>
> wrote:
> >
> >
> >
> > On Mon, Jan 8, 2024 at 6:35 AM Juraj Linkeš <juraj.linkes@pantheon.tech>
> wrote:
> >>
> >> On Wed, Jan 3, 2024 at 11:33 PM <jspewock@iol.unh.edu> wrote:
> >> >
> >> > From: Jeremy Spewock <jspewock@iol.unh.edu>
> >> >
> >> > Added commonly used methods in testpmd such as starting and stopping
> >> > packet forwarding, changing forward modes, and verifying link status
> of
> >> > ports so that developers can configure testpmd and start forwarding
> >> > through the provided class rather than sending commands to the testpmd
> >> > session directly.
> >> >
> >> > Signed-off-by: Jeremy Spewock <jspewock@iol.unh.edu>
> >> > ---
> >> >  dts/framework/exception.py                    |   7 +
> >> >  dts/framework/remote_session/testpmd_shell.py | 149
> +++++++++++++++++-
> >> >  2 files changed, 155 insertions(+), 1 deletion(-)
> >> >
> >> > diff --git a/dts/framework/exception.py b/dts/framework/exception.py
> >> > index 658eee2c38..cce1e0231a 100644
> >> > --- a/dts/framework/exception.py
> >> > +++ b/dts/framework/exception.py
> >> <snip>
> >> > @@ -65,9 +108,66 @@ class TestPmdShell(InteractiveShell):
> >> >      _command_extra_chars: ClassVar[str] = "\n"
> >> >
> >> >      def _start_application(self, get_privileged_command:
> Callable[[str], str] | None) -> None:
> >> > -        self._app_args += " -- -i"
> >> > +        """Overrides :meth:`~.interactive_shell._start_application`.
> >> > +
> >> > +        Add flags for starting testpmd in interactive mode and
> disabling messages for link state
> >> > +        change events before starting the application. Link state is
> verified before starting
> >> > +        packet forwarding and the messages create unexpected
> newlines in the terminal which
> >> > +        complicates output collection.
> >> > +
> >>
> >> We should adjust the collection so that it can handle the newlines.
> >> Also, can you explain exactly why we are disabling the initial link
> >> state messages?
> >
> >
> > The problem really comes from the newlines causing the prompt to exist
> in the buffer before any command is sent. So, what ends up happening is
> after starting the application these link state change events happen at
> some point, and they cause an empty "testpmd>" line to exist in the buffer
> and the next time you send a command it will stop as soon as it encounters
> that line.
>
> These buffer issues keep cropping up. We should think about making
> this more robust. Can we flush the buffer before sending a new command
> (because any previous output is irrelevant)? This probably won't fix
> all the problems, but it sounds like it could help. Maybe it could
>

I definitely think this would be something that is useful because it allows
us to at least make some more helpful assumptions like the buffer only
contains the output after you sent your command. When I was first looking
into making the interactive shells I was looking for ways to do this and
the only way I could really find for flushing the buffer is just consuming
everything that's in it. One way to do this is just to keep collecting
output until you reach the timeout which occurs when there is nothing left.
Waiting for a timeout has its flaws as well, but I believe it was the
cleanest way to make sure the buffer was empty. Overall though, I agree
this would be something good to look into and I'd happily explore expanding
this and also looking into things we discussed earlier like changing the
delimiter on the buffer.


> help with the scapy docstring issue we're seen in the past as well.
>

I'm not sure it would do much for this issue because this stemmed more from
the multiline command I believe. However, making output collection more
robust does also make the opportunity to fix these issues by doing things
like removing empty lines before sending as a sanitizing step.


> In this patch though, we should just make this functional (I
> understand disabling the messages achieves that) and address the
> buffer issues in a separate patch.
>
> > An additional issue with this prompt is it is put in the buffer before
> the link state change event occurs, and there is no prompt that appears
> after the event messages, just an empty line. This makes it much more
> difficult to detect when the link state change event occurs and consume it
> because the event isn't captured the next time you collect output, all that
> is consumed is a line containing the prompt.. So, this makes you
> essentially one command's worth of output behind because the next time you
> send a command you will consume what you were supposed to get from the last
> command where you stopped early, and this causes false positives for things
> like the link state detection method and failures in output verification.
> > This puts you in a position where the only way you can really detect
> that one of these events happened is either assuming that only getting an
> empty prompt means one of these events happened, or trying to consume
> output twice and looking ahead to see if one of these events happened.
> However, because we wouldn't be doing anything with these events and we
> verify link status before starting anyway, it seemed like the less complex
> but still functional solution would just be to mask these events.
>
> Right, these are basically random events, which makes it hard to
> collect (but not impossible). Checking the status explicitly is way
> better. Thanks for the explanation.
>
> >
> >>
> >>
> >> > +        Also find the number of pci addresses which were allowed on
> the command line when the app
> >> > +        was started.
> >> > +        """
> >> > +        self._app_args += " -- -i --mask-event intr_lsc"
> >> > +        self.number_of_ports = self._app_args.count("-a ")
> >> >          super()._start_application(get_privileged_command)
> >> >
> >> > +    def start(self, verify: bool = True) -> None:
> >> > +        """Start packet forwarding with the current configuration.
> >> > +
> >> > +        Args:
> >> > +            verify: If :data:`True` , a second start command will be
> sent in an attempt to verify
> >> > +                packet forwarding started as expected.
> >> > +
> >> > +        Raises:
> >> > +            InteractiveCommandExecutionError: If `verify` is
> :data:`True` and forwarding fails to
> >> > +                start or ports fail to come up.
> >> > +        """
> >> > +        self.send_command("start")
> >> > +        if verify:
> >> > +            # If forwarding was already started, sending "start"
> again should tell us
> >> > +            start_cmd_output = self.send_command("start")
> >> > +            if "Packet forwarding already started" not in
> start_cmd_output:
> >> > +                self._logger.debug(f"Failed to start packet
> forwarding: \n{start_cmd_output}")
> >> > +                raise InteractiveCommandExecutionError("Testpmd
> failed to start packet forwarding.")
> >> > +
> >> > +            for port_id in range(self.number_of_ports):
> >> > +                if not self.wait_link_status_up(port_id):
> >> > +                    raise InteractiveCommandExecutionError(
> >> > +                        "Not all ports came up after starting packet
> forwarding in testpmd."
> >> > +                    )
> >> > +
> >> > +    def stop(self, verify: bool = True) -> None:
> >> > +        """Stop packet forwarding.
> >> > +
> >> > +        Args:
> >> > +            verify: If :data:`True` , the output of the stop command
> is scanned to verify that
> >> > +                forwarding was stopped successfully or not started.
> If neither is found, it is
> >> > +                considered an error.
> >> > +
> >> > +        Raises:
> >> > +            InteractiveCommandExecutionError: If `verify` is
> :data:`True` and the command to stop
> >> > +                forwarding results in an error.
> >> > +        """
> >> > +        stop_cmd_output = self.send_command("stop")
> >> > +        if verify:
> >> > +            if (
> >> > +                "Done." not in stop_cmd_output
> >> > +                and "Packet forwarding not started" not in
> stop_cmd_output
> >> > +            ):
> >>
> >> I want to make sure I understand this condition. When none of these
> >> appear, it's an error. When just "Done." appears, we successfully
> >> stopped ongoing forwarding and when "Packet forwarding not started"
> >> appears, we're trying to stop forwarding that didn't start (or isn't
> >> ongoing - it could've stopped in the meantime)?
> >> I'm thinking about false failures here (Is there a string that would
> >> indicate a failure even if one of the strings is printed?) - we're
> >> basically looking at "not success" instead of looking for strings
> >> telling us about a failure explicitly. Does the stop command not
> >> produce such output? Or do we not know all of the failure strings or
> >> is looking for the above two strings sufficient to rule out false
> >> failures?
> >
> >
> > You are correct that essentially what I am looking for here is if we
> succeeded and else, it's a failure. When I looked through some of the
> source code for testpmd from the method stop_packet_forwarding, I didn't
> see any explicit error messages other than displaying that there was an
> error printing statistics. So this was something where I both didn't know
> the error messages but it doesn't look like there are any that are
> explicitly printed. In the case of false failures however, the strings I am
> detecting are always encountered in our two success cases (forwarding not
> currently started and successfully stopped). The "Done." message does also
> get printed in the case of statistics failing to print for a core as well.
>
> Ok, seems like the check is robust enough.
>

[-- Attachment #2: Type: text/html, Size: 12889 bytes --]

[PATCH v6 2/7] dts: limit EAL parameters to DPDK apps and add parameters to all apps

[jspewock]

From: Jeremy Spewock <jspewock@iol.unh.edu>

Changed the factory method for creating interactive apps in the SUT Node
so that EAL parameters would only be passed into DPDK apps since
non-DPDK apps wouldn't be able to process them. Also modified
interactive apps to allow for the ability to pass parameters into the
app on startup so that the applications can be started with certain
configuration steps passed on the command line.

Signed-off-by: Jeremy Spewock <jspewock@iol.unh.edu>
---

I ended up reverting part of this back to making the argument for
eal_parameters allowed to be a string. This was because it was casuing
mypy errors where the method signatures of sut_node did not match with
that of node.

 dts/framework/remote_session/testpmd_shell.py |  2 +-
 dts/framework/testbed_model/sut_node.py       | 14 +++++++++-----
 2 files changed, 10 insertions(+), 6 deletions(-)

diff --git a/dts/framework/remote_session/testpmd_shell.py b/dts/framework/remote_session/testpmd_shell.py
index f310705fac..8f40e8f40e 100644
--- a/dts/framework/remote_session/testpmd_shell.py
+++ b/dts/framework/remote_session/testpmd_shell.py
@@ -118,7 +118,7 @@ def _start_application(self, get_privileged_command: Callable[[str], str] | None
         Also find the number of pci addresses which were allowed on the command line when the app
         was started.
         """
-        self._app_args += " -- -i --mask-event intr_lsc"
+        self._app_args += " -i --mask-event intr_lsc"
         self.number_of_ports = self._app_args.count("-a ")
         super()._start_application(get_privileged_command)
 
diff --git a/dts/framework/testbed_model/sut_node.py b/dts/framework/testbed_model/sut_node.py
index c4acea38d1..4df18bc183 100644
--- a/dts/framework/testbed_model/sut_node.py
+++ b/dts/framework/testbed_model/sut_node.py
@@ -431,6 +431,7 @@ def create_interactive_shell(
         timeout: float = SETTINGS.timeout,
         privileged: bool = False,
         eal_parameters: EalParameters | str | None = None,
+        app_parameters: str = "",
     ) -> InteractiveShellType:
         """Extend the factory for interactive session handlers.
 
@@ -449,20 +450,23 @@ def create_interactive_shell(
             eal_parameters: List of EAL parameters to use to launch the app. If this
                 isn't provided or an empty string is passed, it will default to calling
                 :meth:`create_eal_parameters`.
+            app_parameters: Additional arguments to pass into the application on the
+                command-line.
 
         Returns:
             An instance of the desired interactive application shell.
         """
-        if not eal_parameters:
-            eal_parameters = self.create_eal_parameters()
-
-        # We need to append the build directory for DPDK apps
+        # We need to append the build directory and add EAL parameters for DPDK apps
         if shell_cls.dpdk_app:
+            if not eal_parameters:
+                eal_parameters = self.create_eal_parameters()
+            app_parameters = f"{eal_parameters} -- {app_parameters}"
+
             shell_cls.path = self.main_session.join_remote_path(
                 self.remote_dpdk_build_dir, shell_cls.path
             )
 
-        return super().create_interactive_shell(shell_cls, timeout, privileged, str(eal_parameters))
+        return super().create_interactive_shell(shell_cls, timeout, privileged, app_parameters)
 
     def bind_ports_to_driver(self, for_dpdk: bool = True) -> None:
         """Bind all ports on the SUT to a driver.
-- 
2.43.0

Re: [PATCH v6 2/7] dts: limit EAL parameters to DPDK apps and add parameters to all apps

[Juraj Linkeš]

On Wed, Jan 3, 2024 at 11:33 PM <jspewock@iol.unh.edu> wrote:
>
> From: Jeremy Spewock <jspewock@iol.unh.edu>
>
> Changed the factory method for creating interactive apps in the SUT Node
> so that EAL parameters would only be passed into DPDK apps since
> non-DPDK apps wouldn't be able to process them. Also modified
> interactive apps to allow for the ability to pass parameters into the
> app on startup so that the applications can be started with certain
> configuration steps passed on the command line.
>
> Signed-off-by: Jeremy Spewock <jspewock@iol.unh.edu>
> ---
>
> I ended up reverting part of this back to making the argument for
> eal_parameters allowed to be a string. This was because it was casuing
> mypy errors where the method signatures of sut_node did not match with
> that of node.
>

This is because the signatures don't actually match :-)

The eal_parameters parameter is added on not top of what's in the base
methods. I suggest we move eal_parameters to the end and then we don't
need to allow str for eal_parameters.

>  dts/framework/remote_session/testpmd_shell.py |  2 +-
>  dts/framework/testbed_model/sut_node.py       | 14 +++++++++-----
>  2 files changed, 10 insertions(+), 6 deletions(-)
>
> diff --git a/dts/framework/remote_session/testpmd_shell.py b/dts/framework/remote_session/testpmd_shell.py
> index f310705fac..8f40e8f40e 100644
> --- a/dts/framework/remote_session/testpmd_shell.py
> +++ b/dts/framework/remote_session/testpmd_shell.py
> @@ -118,7 +118,7 @@ def _start_application(self, get_privileged_command: Callable[[str], str] | None
>          Also find the number of pci addresses which were allowed on the command line when the app
>          was started.
>          """
> -        self._app_args += " -- -i --mask-event intr_lsc"
> +        self._app_args += " -i --mask-event intr_lsc"
>          self.number_of_ports = self._app_args.count("-a ")
>          super()._start_application(get_privileged_command)
>
> diff --git a/dts/framework/testbed_model/sut_node.py b/dts/framework/testbed_model/sut_node.py
> index c4acea38d1..4df18bc183 100644
> --- a/dts/framework/testbed_model/sut_node.py
> +++ b/dts/framework/testbed_model/sut_node.py
> @@ -431,6 +431,7 @@ def create_interactive_shell(
>          timeout: float = SETTINGS.timeout,
>          privileged: bool = False,
>          eal_parameters: EalParameters | str | None = None,
> +        app_parameters: str = "",

What I meant above is we should move app_parameters before
eal_parameters and then we can remove the str type of eal_parameters.

>      ) -> InteractiveShellType:
>          """Extend the factory for interactive session handlers.
>
> @@ -449,20 +450,23 @@ def create_interactive_shell(
>              eal_parameters: List of EAL parameters to use to launch the app. If this
>                  isn't provided or an empty string is passed, it will default to calling
>                  :meth:`create_eal_parameters`.
> +            app_parameters: Additional arguments to pass into the application on the
> +                command-line.
>
>          Returns:
>              An instance of the desired interactive application shell.
>          """
> -        if not eal_parameters:
> -            eal_parameters = self.create_eal_parameters()
> -
> -        # We need to append the build directory for DPDK apps
> +        # We need to append the build directory and add EAL parameters for DPDK apps
>          if shell_cls.dpdk_app:
> +            if not eal_parameters:
> +                eal_parameters = self.create_eal_parameters()
> +            app_parameters = f"{eal_parameters} -- {app_parameters}"
> +
>              shell_cls.path = self.main_session.join_remote_path(
>                  self.remote_dpdk_build_dir, shell_cls.path
>              )
>
> -        return super().create_interactive_shell(shell_cls, timeout, privileged, str(eal_parameters))
> +        return super().create_interactive_shell(shell_cls, timeout, privileged, app_parameters)
>
>      def bind_ports_to_driver(self, for_dpdk: bool = True) -> None:
>          """Bind all ports on the SUT to a driver.
> --
> 2.43.0
>

Re: [PATCH v6 2/7] dts: limit EAL parameters to DPDK apps and add parameters to all apps

[Jeremy Spewock]

[-- Attachment #1: Type: text/plain, Size: 4602 bytes --]

On Mon, Jan 8, 2024 at 6:52 AM Juraj Linkeš <juraj.linkes@pantheon.tech>
wrote:

> On Wed, Jan 3, 2024 at 11:33 PM <jspewock@iol.unh.edu> wrote:
> >
> > From: Jeremy Spewock <jspewock@iol.unh.edu>
> >
> > Changed the factory method for creating interactive apps in the SUT Node
> > so that EAL parameters would only be passed into DPDK apps since
> > non-DPDK apps wouldn't be able to process them. Also modified
> > interactive apps to allow for the ability to pass parameters into the
> > app on startup so that the applications can be started with certain
> > configuration steps passed on the command line.
> >
> > Signed-off-by: Jeremy Spewock <jspewock@iol.unh.edu>
> > ---
> >
> > I ended up reverting part of this back to making the argument for
> > eal_parameters allowed to be a string. This was because it was casuing
> > mypy errors where the method signatures of sut_node did not match with
> > that of node.
> >
>
> This is because the signatures don't actually match :-)
>
> The eal_parameters parameter is added on not top of what's in the base
> methods. I suggest we move eal_parameters to the end and then we don't
> need to allow str for eal_parameters.
>
> >  dts/framework/remote_session/testpmd_shell.py |  2 +-
> >  dts/framework/testbed_model/sut_node.py       | 14 +++++++++-----
> >  2 files changed, 10 insertions(+), 6 deletions(-)
> >
> > diff --git a/dts/framework/remote_session/testpmd_shell.py
> b/dts/framework/remote_session/testpmd_shell.py
> > index f310705fac..8f40e8f40e 100644
> > --- a/dts/framework/remote_session/testpmd_shell.py
> > +++ b/dts/framework/remote_session/testpmd_shell.py
> > @@ -118,7 +118,7 @@ def _start_application(self, get_privileged_command:
> Callable[[str], str] | None
> >          Also find the number of pci addresses which were allowed on the
> command line when the app
> >          was started.
> >          """
> > -        self._app_args += " -- -i --mask-event intr_lsc"
> > +        self._app_args += " -i --mask-event intr_lsc"
> >          self.number_of_ports = self._app_args.count("-a ")
> >          super()._start_application(get_privileged_command)
> >
> > diff --git a/dts/framework/testbed_model/sut_node.py
> b/dts/framework/testbed_model/sut_node.py
> > index c4acea38d1..4df18bc183 100644
> > --- a/dts/framework/testbed_model/sut_node.py
> > +++ b/dts/framework/testbed_model/sut_node.py
> > @@ -431,6 +431,7 @@ def create_interactive_shell(
> >          timeout: float = SETTINGS.timeout,
> >          privileged: bool = False,
> >          eal_parameters: EalParameters | str | None = None,
> > +        app_parameters: str = "",
>
> What I meant above is we should move app_parameters before
> eal_parameters and then we can remove the str type of eal_parameters.
>

Sounds good to me, I'll move these around in the next version.


>
> >      ) -> InteractiveShellType:
> >          """Extend the factory for interactive session handlers.
> >
> > @@ -449,20 +450,23 @@ def create_interactive_shell(
> >              eal_parameters: List of EAL parameters to use to launch the
> app. If this
> >                  isn't provided or an empty string is passed, it will
> default to calling
> >                  :meth:`create_eal_parameters`.
> > +            app_parameters: Additional arguments to pass into the
> application on the
> > +                command-line.
> >
> >          Returns:
> >              An instance of the desired interactive application shell.
> >          """
> > -        if not eal_parameters:
> > -            eal_parameters = self.create_eal_parameters()
> > -
> > -        # We need to append the build directory for DPDK apps
> > +        # We need to append the build directory and add EAL parameters
> for DPDK apps
> >          if shell_cls.dpdk_app:
> > +            if not eal_parameters:
> > +                eal_parameters = self.create_eal_parameters()
> > +            app_parameters = f"{eal_parameters} -- {app_parameters}"
> > +
> >              shell_cls.path = self.main_session.join_remote_path(
> >                  self.remote_dpdk_build_dir, shell_cls.path
> >              )
> >
> > -        return super().create_interactive_shell(shell_cls, timeout,
> privileged, str(eal_parameters))
> > +        return super().create_interactive_shell(shell_cls, timeout,
> privileged, app_parameters)
> >
> >      def bind_ports_to_driver(self, for_dpdk: bool = True) -> None:
> >          """Bind all ports on the SUT to a driver.
> > --
> > 2.43.0
> >
>

[-- Attachment #2: Type: text/html, Size: 6099 bytes --]

[PATCH v6 3/7] dts: add optional packet filtering to scapy sniffer

[jspewock]

From: Jeremy Spewock <jspewock@iol.unh.edu>

Added the options to filter out LLDP and ARP packets when
sniffing for packets with scapy. This was done using BPF filters to
ensure that the noise these packets provide does not interfere with test
cases.

Signed-off-by: Jeremy Spewock <jspewock@iol.unh.edu>
---
 dts/framework/test_suite.py                   | 15 +++++++++--
 dts/framework/testbed_model/tg_node.py        | 14 ++++++++--
 .../traffic_generator/__init__.py             |  7 ++++-
 .../capturing_traffic_generator.py            | 22 ++++++++++++++-
 .../testbed_model/traffic_generator/scapy.py  | 27 +++++++++++++++++++
 5 files changed, 79 insertions(+), 6 deletions(-)

diff --git a/dts/framework/test_suite.py b/dts/framework/test_suite.py
index dfb391ffbd..ffea917690 100644
--- a/dts/framework/test_suite.py
+++ b/dts/framework/test_suite.py
@@ -38,6 +38,7 @@
 from .settings import SETTINGS
 from .test_result import BuildTargetResult, Result, TestCaseResult, TestSuiteResult
 from .testbed_model import Port, PortLink, SutNode, TGNode
+from .testbed_model.traffic_generator import PacketFilteringConfig
 from .utils import get_packet_summaries
 
 
@@ -208,7 +209,12 @@ def configure_testbed_ipv4(self, restore: bool = False) -> None:
     def _configure_ipv4_forwarding(self, enable: bool) -> None:
         self.sut_node.configure_ipv4_forwarding(enable)
 
-    def send_packet_and_capture(self, packet: Packet, duration: float = 1) -> list[Packet]:
+    def send_packet_and_capture(
+        self,
+        packet: Packet,
+        filter_config: PacketFilteringConfig = PacketFilteringConfig(),
+        duration: float = 1,
+    ) -> list[Packet]:
         """Send and receive `packet` using the associated TG.
 
         Send `packet` through the appropriate interface and receive on the appropriate interface.
@@ -216,6 +222,7 @@ def send_packet_and_capture(self, packet: Packet, duration: float = 1) -> list[P
 
         Args:
             packet: The packet to send.
+            filter_config: The filter to use when capturing packets.
             duration: Capture traffic for this amount of time after sending `packet`.
 
         Returns:
@@ -223,7 +230,11 @@ def send_packet_and_capture(self, packet: Packet, duration: float = 1) -> list[P
         """
         packet = self._adjust_addresses(packet)
         return self.tg_node.send_packet_and_capture(
-            packet, self._tg_port_egress, self._tg_port_ingress, duration
+            packet,
+            self._tg_port_egress,
+            self._tg_port_ingress,
+            filter_config,
+            duration,
         )
 
     def get_expected_packet(self, packet: Packet) -> Packet:
diff --git a/dts/framework/testbed_model/tg_node.py b/dts/framework/testbed_model/tg_node.py
index f269d4c585..d3206e87e0 100644
--- a/dts/framework/testbed_model/tg_node.py
+++ b/dts/framework/testbed_model/tg_node.py
@@ -15,7 +15,11 @@
 
 from .node import Node
 from .port import Port
-from .traffic_generator import CapturingTrafficGenerator, create_traffic_generator
+from .traffic_generator import (
+    CapturingTrafficGenerator,
+    PacketFilteringConfig,
+    create_traffic_generator,
+)
 
 
 class TGNode(Node):
@@ -53,6 +57,7 @@ def send_packet_and_capture(
         packet: Packet,
         send_port: Port,
         receive_port: Port,
+        filter_config: PacketFilteringConfig = PacketFilteringConfig(),
         duration: float = 1,
     ) -> list[Packet]:
         """Send `packet`, return received traffic.
@@ -65,13 +70,18 @@ def send_packet_and_capture(
             packet: The packet to send.
             send_port: The egress port on the TG node.
             receive_port: The ingress port in the TG node.
+            filter_config: The filter to use when capturing packets.
             duration: Capture traffic for this amount of time after sending `packet`.
 
         Returns:
              A list of received packets. May be empty if no packets are captured.
         """
         return self.traffic_generator.send_packet_and_capture(
-            packet, send_port, receive_port, duration
+            packet,
+            send_port,
+            receive_port,
+            filter_config,
+            duration,
         )
 
     def close(self) -> None:
diff --git a/dts/framework/testbed_model/traffic_generator/__init__.py b/dts/framework/testbed_model/traffic_generator/__init__.py
index 11e2bd7d97..0eaf0355cd 100644
--- a/dts/framework/testbed_model/traffic_generator/__init__.py
+++ b/dts/framework/testbed_model/traffic_generator/__init__.py
@@ -14,11 +14,16 @@
 and a capturing traffic generator is required.
 """
 
+# pylama:ignore=W0611
+
 from framework.config import ScapyTrafficGeneratorConfig, TrafficGeneratorType
 from framework.exception import ConfigurationError
 from framework.testbed_model.node import Node
 
-from .capturing_traffic_generator import CapturingTrafficGenerator
+from .capturing_traffic_generator import (
+    CapturingTrafficGenerator,
+    PacketFilteringConfig,
+)
 from .scapy import ScapyTrafficGenerator
 
 
diff --git a/dts/framework/testbed_model/traffic_generator/capturing_traffic_generator.py b/dts/framework/testbed_model/traffic_generator/capturing_traffic_generator.py
index 0246590333..c1c9facedd 100644
--- a/dts/framework/testbed_model/traffic_generator/capturing_traffic_generator.py
+++ b/dts/framework/testbed_model/traffic_generator/capturing_traffic_generator.py
@@ -11,6 +11,7 @@
 
 import uuid
 from abc import abstractmethod
+from dataclasses import dataclass
 
 import scapy.utils  # type: ignore[import]
 from scapy.packet import Packet  # type: ignore[import]
@@ -26,6 +27,19 @@ def _get_default_capture_name() -> str:
     return str(uuid.uuid4())
 
 
+@dataclass(slots=True)
+class PacketFilteringConfig:
+    """The supported filtering options for :class:`CapturingTrafficGenerator`.
+
+    Attributes:
+        no_lldp: If :data:`True`, LLDP packets will be filtered out when capturing.
+        no_arp: If :data:`True`, ARP packets will be filtered out when capturing.
+    """
+
+    no_lldp: bool = True
+    no_arp: bool = True
+
+
 class CapturingTrafficGenerator(TrafficGenerator):
     """Capture packets after sending traffic.
 
@@ -54,6 +68,7 @@ def send_packet_and_capture(
         packet: Packet,
         send_port: Port,
         receive_port: Port,
+        filter_config: PacketFilteringConfig,
         duration: float,
         capture_name: str = _get_default_capture_name(),
     ) -> list[Packet]:
@@ -68,6 +83,7 @@ def send_packet_and_capture(
             packet: The packet to send.
             send_port: The egress port on the TG node.
             receive_port: The ingress port in the TG node.
+            filter_config: Filters to apply when capturing packets.
             duration: Capture traffic for this amount of time after sending the packet.
             capture_name: The name of the .pcap file where to store the capture.
 
@@ -75,7 +91,7 @@ def send_packet_and_capture(
              The received packets. May be empty if no packets are captured.
         """
         return self.send_packets_and_capture(
-            [packet], send_port, receive_port, duration, capture_name
+            [packet], send_port, receive_port, filter_config, duration, capture_name
         )
 
     def send_packets_and_capture(
@@ -83,6 +99,7 @@ def send_packets_and_capture(
         packets: list[Packet],
         send_port: Port,
         receive_port: Port,
+        filter_config: PacketFilteringConfig,
         duration: float,
         capture_name: str = _get_default_capture_name(),
     ) -> list[Packet]:
@@ -99,6 +116,7 @@ def send_packets_and_capture(
             packets: The packets to send.
             send_port: The egress port on the TG node.
             receive_port: The ingress port in the TG node.
+            filter_config: Filters to apply when capturing packets.
             duration: Capture traffic for this amount of time after sending the packets.
             capture_name: The name of the .pcap file where to store the capture.
 
@@ -113,6 +131,7 @@ def send_packets_and_capture(
             packets,
             send_port,
             receive_port,
+            filter_config,
             duration,
         )
 
@@ -126,6 +145,7 @@ def _send_packets_and_capture(
         packets: list[Packet],
         send_port: Port,
         receive_port: Port,
+        filter_config: PacketFilteringConfig,
         duration: float,
     ) -> list[Packet]:
         """The implementation of :method:`send_packets_and_capture`.
diff --git a/dts/framework/testbed_model/traffic_generator/scapy.py b/dts/framework/testbed_model/traffic_generator/scapy.py
index 5b60f66237..505de0be94 100644
--- a/dts/framework/testbed_model/traffic_generator/scapy.py
+++ b/dts/framework/testbed_model/traffic_generator/scapy.py
@@ -32,6 +32,7 @@
 
 from .capturing_traffic_generator import (
     CapturingTrafficGenerator,
+    PacketFilteringConfig,
     _get_default_capture_name,
 )
 
@@ -69,6 +70,7 @@ def scapy_send_packets_and_capture(
     send_iface: str,
     recv_iface: str,
     duration: float,
+    sniff_filter: str,
 ) -> list[bytes]:
     """The RPC function to send and capture packets.
 
@@ -90,6 +92,7 @@ def scapy_send_packets_and_capture(
         iface=recv_iface,
         store=True,
         started_callback=lambda *args: scapy.all.sendp(scapy_packets, iface=send_iface),
+        filter=sniff_filter,
     )
     sniffer.start()
     time.sleep(duration)
@@ -260,11 +263,34 @@ def _send_packets(self, packets: list[Packet], port: Port) -> None:
         packets = [packet.build() for packet in packets]
         self.rpc_server_proxy.scapy_send_packets(packets, port.logical_name)
 
+    def _create_packet_filter(self, filter_config: PacketFilteringConfig) -> str:
+        """Combines filter settings from `filter_config` into a BPF that scapy can use.
+
+        Scapy allows for the use of Berkeley Packet Filters (BPFs) to filter what packets are
+        collected based on various attributes of the packet.
+
+        Args:
+            filter_config: Config class that specifies which filters should be applied.
+
+        Returns:
+            A string representing the combination of BPF filters to be passed to scapy. For
+            example:
+
+            "ether[12:2] != 0x88cc && ether[12:2] != 0x0806"
+        """
+        bpf_filter: list[str] = []
+        if filter_config.no_arp:
+            bpf_filter.append("ether[12:2] != 0x0806")
+        if filter_config.no_lldp:
+            bpf_filter.append("ether[12:2] != 0x88cc")
+        return " && ".join(bpf_filter)
+
     def _send_packets_and_capture(
         self,
         packets: list[Packet],
         send_port: Port,
         receive_port: Port,
+        filter_config: PacketFilteringConfig,
         duration: float,
         capture_name: str = _get_default_capture_name(),
     ) -> list[Packet]:
@@ -277,6 +303,7 @@ def _send_packets_and_capture(
             send_port.logical_name,
             receive_port.logical_name,
             duration,
+            self._create_packet_filter(filter_config),
         )  # type: ignore[assignment]
 
         scapy_packets = [Ether(packet.data) for packet in xmlrpc_packets]
-- 
2.43.0

Re: [PATCH v6 3/7] dts: add optional packet filtering to scapy sniffer

[Juraj Linkeš]

On Wed, Jan 3, 2024 at 11:33 PM <jspewock@iol.unh.edu> wrote:
>
> From: Jeremy Spewock <jspewock@iol.unh.edu>
>
> Added the options to filter out LLDP and ARP packets when
> sniffing for packets with scapy. This was done using BPF filters to
> ensure that the noise these packets provide does not interfere with test
> cases.
>
> Signed-off-by: Jeremy Spewock <jspewock@iol.unh.edu>
> ---
>  dts/framework/test_suite.py                   | 15 +++++++++--
>  dts/framework/testbed_model/tg_node.py        | 14 ++++++++--
>  .../traffic_generator/__init__.py             |  7 ++++-
>  .../capturing_traffic_generator.py            | 22 ++++++++++++++-
>  .../testbed_model/traffic_generator/scapy.py  | 27 +++++++++++++++++++
>  5 files changed, 79 insertions(+), 6 deletions(-)
>

<snip>

> diff --git a/dts/framework/testbed_model/traffic_generator/capturing_traffic_generator.py b/dts/framework/testbed_model/traffic_generator/capturing_traffic_generator.py
> index 0246590333..c1c9facedd 100644
> --- a/dts/framework/testbed_model/traffic_generator/capturing_traffic_generator.py
> +++ b/dts/framework/testbed_model/traffic_generator/capturing_traffic_generator.py
<snip>
> @@ -26,6 +27,19 @@ def _get_default_capture_name() -> str:
>      return str(uuid.uuid4())
>
>
> +@dataclass(slots=True)

This should also be frozen. If we need a different filter, it's better
to create a new object I think.

> +class PacketFilteringConfig:
> +    """The supported filtering options for :class:`CapturingTrafficGenerator`.
> +
> +    Attributes:
> +        no_lldp: If :data:`True`, LLDP packets will be filtered out when capturing.
> +        no_arp: If :data:`True`, ARP packets will be filtered out when capturing.
> +    """
> +
> +    no_lldp: bool = True
> +    no_arp: bool = True
> +
> +
>  class CapturingTrafficGenerator(TrafficGenerator):
>      """Capture packets after sending traffic.
>
<snip>
> diff --git a/dts/framework/testbed_model/traffic_generator/scapy.py b/dts/framework/testbed_model/traffic_generator/scapy.py
> index 5b60f66237..505de0be94 100644
> --- a/dts/framework/testbed_model/traffic_generator/scapy.py
> +++ b/dts/framework/testbed_model/traffic_generator/scapy.py
<snip>
> @@ -260,11 +263,34 @@ def _send_packets(self, packets: list[Packet], port: Port) -> None:
>          packets = [packet.build() for packet in packets]
>          self.rpc_server_proxy.scapy_send_packets(packets, port.logical_name)
>
> +    def _create_packet_filter(self, filter_config: PacketFilteringConfig) -> str:
> +        """Combines filter settings from `filter_config` into a BPF that scapy can use.
> +
> +        Scapy allows for the use of Berkeley Packet Filters (BPFs) to filter what packets are
> +        collected based on various attributes of the packet.
> +
> +        Args:
> +            filter_config: Config class that specifies which filters should be applied.
> +
> +        Returns:
> +            A string representing the combination of BPF filters to be passed to scapy. For
> +            example:
> +
> +            "ether[12:2] != 0x88cc && ether[12:2] != 0x0806"
> +        """
> +        bpf_filter: list[str] = []

The type hint here is not needed, so let's make this consistent with
the rest of the code - we don't specify local type hints if they're
not necessary.

> +        if filter_config.no_arp:
> +            bpf_filter.append("ether[12:2] != 0x0806")
> +        if filter_config.no_lldp:
> +            bpf_filter.append("ether[12:2] != 0x88cc")
> +        return " && ".join(bpf_filter)
> +
>      def _send_packets_and_capture(
>          self,
>          packets: list[Packet],
>          send_port: Port,
>          receive_port: Port,
> +        filter_config: PacketFilteringConfig,
>          duration: float,
>          capture_name: str = _get_default_capture_name(),
>      ) -> list[Packet]:

Re: [PATCH v6 3/7] dts: add optional packet filtering to scapy sniffer

[Jeremy Spewock]

[-- Attachment #1: Type: text/plain, Size: 4398 bytes --]

On Mon, Jan 8, 2024 at 7:01 AM Juraj Linkeš <juraj.linkes@pantheon.tech>
wrote:

> On Wed, Jan 3, 2024 at 11:33 PM <jspewock@iol.unh.edu> wrote:
> >
> > From: Jeremy Spewock <jspewock@iol.unh.edu>
> >
> > Added the options to filter out LLDP and ARP packets when
> > sniffing for packets with scapy. This was done using BPF filters to
> > ensure that the noise these packets provide does not interfere with test
> > cases.
> >
> > Signed-off-by: Jeremy Spewock <jspewock@iol.unh.edu>
> > ---
> >  dts/framework/test_suite.py                   | 15 +++++++++--
> >  dts/framework/testbed_model/tg_node.py        | 14 ++++++++--
> >  .../traffic_generator/__init__.py             |  7 ++++-
> >  .../capturing_traffic_generator.py            | 22 ++++++++++++++-
> >  .../testbed_model/traffic_generator/scapy.py  | 27 +++++++++++++++++++
> >  5 files changed, 79 insertions(+), 6 deletions(-)
> >
>
> <snip>
>
> > diff --git
> a/dts/framework/testbed_model/traffic_generator/capturing_traffic_generator.py
> b/dts/framework/testbed_model/traffic_generator/capturing_traffic_generator.py
> > index 0246590333..c1c9facedd 100644
> > ---
> a/dts/framework/testbed_model/traffic_generator/capturing_traffic_generator.py
> > +++
> b/dts/framework/testbed_model/traffic_generator/capturing_traffic_generator.py
> <snip>
> > @@ -26,6 +27,19 @@ def _get_default_capture_name() -> str:
> >      return str(uuid.uuid4())
> >
> >
> > +@dataclass(slots=True)
>
> This should also be frozen. If we need a different filter, it's better
> to create a new object I think.
>
> > +class PacketFilteringConfig:
> > +    """The supported filtering options for
> :class:`CapturingTrafficGenerator`.
> > +
> > +    Attributes:
> > +        no_lldp: If :data:`True`, LLDP packets will be filtered out
> when capturing.
> > +        no_arp: If :data:`True`, ARP packets will be filtered out when
> capturing.
> > +    """
> > +
> > +    no_lldp: bool = True
> > +    no_arp: bool = True
> > +
> > +
> >  class CapturingTrafficGenerator(TrafficGenerator):
> >      """Capture packets after sending traffic.
> >
> <snip>
> > diff --git a/dts/framework/testbed_model/traffic_generator/scapy.py
> b/dts/framework/testbed_model/traffic_generator/scapy.py
> > index 5b60f66237..505de0be94 100644
> > --- a/dts/framework/testbed_model/traffic_generator/scapy.py
> > +++ b/dts/framework/testbed_model/traffic_generator/scapy.py
> <snip>
> > @@ -260,11 +263,34 @@ def _send_packets(self, packets: list[Packet],
> port: Port) -> None:
> >          packets = [packet.build() for packet in packets]
> >          self.rpc_server_proxy.scapy_send_packets(packets,
> port.logical_name)
> >
> > +    def _create_packet_filter(self, filter_config:
> PacketFilteringConfig) -> str:
> > +        """Combines filter settings from `filter_config` into a BPF
> that scapy can use.
> > +
> > +        Scapy allows for the use of Berkeley Packet Filters (BPFs) to
> filter what packets are
> > +        collected based on various attributes of the packet.
> > +
> > +        Args:
> > +            filter_config: Config class that specifies which filters
> should be applied.
> > +
> > +        Returns:
> > +            A string representing the combination of BPF filters to be
> passed to scapy. For
> > +            example:
> > +
> > +            "ether[12:2] != 0x88cc && ether[12:2] != 0x0806"
> > +        """
> > +        bpf_filter: list[str] = []
>
> The type hint here is not needed, so let's make this consistent with
> the rest of the code - we don't specify local type hints if they're
> not necessary.
>

Good catch, this might have been left over from when I was experimenting
with formatting the filters, I'll remove it.


>
> > +        if filter_config.no_arp:
> > +            bpf_filter.append("ether[12:2] != 0x0806")
> > +        if filter_config.no_lldp:
> > +            bpf_filter.append("ether[12:2] != 0x88cc")
> > +        return " && ".join(bpf_filter)
> > +
> >      def _send_packets_and_capture(
> >          self,
> >          packets: list[Packet],
> >          send_port: Port,
> >          receive_port: Port,
> > +        filter_config: PacketFilteringConfig,
> >          duration: float,
> >          capture_name: str = _get_default_capture_name(),
> >      ) -> list[Packet]:
>

[-- Attachment #2: Type: text/html, Size: 5922 bytes --]

Re: [PATCH v6 3/7] dts: add optional packet filtering to scapy sniffer

[Jeremy Spewock]

[-- Attachment #1: Type: text/plain, Size: 1798 bytes --]

On Mon, Jan 8, 2024 at 11:39 AM Jeremy Spewock <jspewock@iol.unh.edu> wrote:

>
>
> On Mon, Jan 8, 2024 at 7:01 AM Juraj Linkeš <juraj.linkes@pantheon.tech>
> wrote:
>
>> On Wed, Jan 3, 2024 at 11:33 PM <jspewock@iol.unh.edu> wrote:
>> >
>> > From: Jeremy Spewock <jspewock@iol.unh.edu>
>> >
>> > Added the options to filter out LLDP and ARP packets when
>> > sniffing for packets with scapy. This was done using BPF filters to
>> > ensure that the noise these packets provide does not interfere with test
>> > cases.
>> >
>> > Signed-off-by: Jeremy Spewock <jspewock@iol.unh.edu>
>> > ---
>> >  dts/framework/test_suite.py                   | 15 +++++++++--
>> >  dts/framework/testbed_model/tg_node.py        | 14 ++++++++--
>> >  .../traffic_generator/__init__.py             |  7 ++++-
>> >  .../capturing_traffic_generator.py            | 22 ++++++++++++++-
>> >  .../testbed_model/traffic_generator/scapy.py  | 27 +++++++++++++++++++
>> >  5 files changed, 79 insertions(+), 6 deletions(-)
>> >
>>
>> <snip>
>>
>> > diff --git
>> a/dts/framework/testbed_model/traffic_generator/capturing_traffic_generator.py
>> b/dts/framework/testbed_model/traffic_generator/capturing_traffic_generator.py
>> > index 0246590333..c1c9facedd 100644
>> > ---
>> a/dts/framework/testbed_model/traffic_generator/capturing_traffic_generator.py
>> > +++
>> b/dts/framework/testbed_model/traffic_generator/capturing_traffic_generator.py
>> <snip>
>> > @@ -26,6 +27,19 @@ def _get_default_capture_name() -> str:
>> >      return str(uuid.uuid4())
>> >
>> >
>> > +@dataclass(slots=True)
>>
>> This should also be frozen. If we need a different filter, it's better
>> to create a new object I think.
>>
>> This is also a good point, I'll make this change as well.

[-- Attachment #2: Type: text/html, Size: 2905 bytes --]

[PATCH v6 4/7] dts: add pci addresses to EAL parameters

[jspewock]

From: Jeremy Spewock <jspewock@iol.unh.edu>

Added allow list to the EAL parameters created in DTS to ensure that
only the relevant PCI devices are considered when launching DPDK
applications.

Signed-off-by: Jeremy Spewock <jspewock@iol.unh.edu>
---
 dts/framework/testbed_model/sut_node.py | 12 ++++++++++++
 1 file changed, 12 insertions(+)

diff --git a/dts/framework/testbed_model/sut_node.py b/dts/framework/testbed_model/sut_node.py
index 4df18bc183..cc894fb07d 100644
--- a/dts/framework/testbed_model/sut_node.py
+++ b/dts/framework/testbed_model/sut_node.py
@@ -30,6 +30,7 @@
 from .cpu import LogicalCoreCount, LogicalCoreList
 from .node import Node
 from .os_session import InteractiveShellType, OSSession
+from .port import Port
 from .virtual_device import VirtualDevice
 
 
@@ -46,6 +47,7 @@ def __init__(
         prefix: str,
         no_pci: bool,
         vdevs: list[VirtualDevice],
+        ports: list[Port],
         other_eal_param: str,
     ):
         """Initialize the parameters according to inputs.
@@ -63,6 +65,7 @@ def __init__(
                     VirtualDevice('net_ring0'),
                     VirtualDevice('net_ring1')
                 ]
+            ports: The list of ports to allow.
             other_eal_param: user defined DPDK EAL parameters, e.g.:
                 ``other_eal_param='--single-file-segments'``
         """
@@ -73,6 +76,7 @@ def __init__(
             self._prefix = f"--file-prefix={prefix}"
         self._no_pci = "--no-pci" if no_pci else ""
         self._vdevs = " ".join(f"--vdev {vdev}" for vdev in vdevs)
+        self._ports = " ".join(f"-a {port.pci}" for port in ports)
         self._other_eal_param = other_eal_param
 
     def __str__(self) -> str:
@@ -83,6 +87,7 @@ def __str__(self) -> str:
             f"{self._prefix} "
             f"{self._no_pci} "
             f"{self._vdevs} "
+            f"{self._ports} "
             f"{self._other_eal_param}"
         )
 
@@ -347,6 +352,7 @@ def create_eal_parameters(
         append_prefix_timestamp: bool = True,
         no_pci: bool = False,
         vdevs: list[VirtualDevice] | None = None,
+        ports: list[Port] | None = None,
         other_eal_param: str = "",
     ) -> "EalParameters":
         """Compose the EAL parameters.
@@ -370,6 +376,8 @@ def create_eal_parameters(
                     VirtualDevice('net_ring0'),
                     VirtualDevice('net_ring1')
                 ]
+            ports: The list of ports to allow. If :data:`None`, all ports listed in `self.ports`
+                will be allowed.
             other_eal_param: user defined DPDK EAL parameters, e.g.:
                 ``other_eal_param='--single-file-segments'``.
 
@@ -388,12 +396,16 @@ def create_eal_parameters(
         if vdevs is None:
             vdevs = []
 
+        if ports is None:
+            ports = self.ports
+
         return EalParameters(
             lcore_list=lcore_list,
             memory_channels=self.config.memory_channels,
             prefix=prefix,
             no_pci=no_pci,
             vdevs=vdevs,
+            ports=ports,
             other_eal_param=other_eal_param,
         )
 
-- 
2.43.0

Re: [PATCH v6 4/7] dts: add pci addresses to EAL parameters

[Juraj Linkeš]

Reviewed-by: Juraj Linkeš <juraj.linkes@pantheon.tech>

On Wed, Jan 3, 2024 at 11:33 PM <jspewock@iol.unh.edu> wrote:
>
> From: Jeremy Spewock <jspewock@iol.unh.edu>
>
> Added allow list to the EAL parameters created in DTS to ensure that
> only the relevant PCI devices are considered when launching DPDK
> applications.
>
> Signed-off-by: Jeremy Spewock <jspewock@iol.unh.edu>
> ---
>  dts/framework/testbed_model/sut_node.py | 12 ++++++++++++
>  1 file changed, 12 insertions(+)
>
> diff --git a/dts/framework/testbed_model/sut_node.py b/dts/framework/testbed_model/sut_node.py
> index 4df18bc183..cc894fb07d 100644
> --- a/dts/framework/testbed_model/sut_node.py
> +++ b/dts/framework/testbed_model/sut_node.py
> @@ -30,6 +30,7 @@
>  from .cpu import LogicalCoreCount, LogicalCoreList
>  from .node import Node
>  from .os_session import InteractiveShellType, OSSession
> +from .port import Port
>  from .virtual_device import VirtualDevice
>
>
> @@ -46,6 +47,7 @@ def __init__(
>          prefix: str,
>          no_pci: bool,
>          vdevs: list[VirtualDevice],
> +        ports: list[Port],
>          other_eal_param: str,
>      ):
>          """Initialize the parameters according to inputs.
> @@ -63,6 +65,7 @@ def __init__(
>                      VirtualDevice('net_ring0'),
>                      VirtualDevice('net_ring1')
>                  ]
> +            ports: The list of ports to allow.
>              other_eal_param: user defined DPDK EAL parameters, e.g.:
>                  ``other_eal_param='--single-file-segments'``
>          """
> @@ -73,6 +76,7 @@ def __init__(
>              self._prefix = f"--file-prefix={prefix}"
>          self._no_pci = "--no-pci" if no_pci else ""
>          self._vdevs = " ".join(f"--vdev {vdev}" for vdev in vdevs)
> +        self._ports = " ".join(f"-a {port.pci}" for port in ports)
>          self._other_eal_param = other_eal_param
>
>      def __str__(self) -> str:
> @@ -83,6 +87,7 @@ def __str__(self) -> str:
>              f"{self._prefix} "
>              f"{self._no_pci} "
>              f"{self._vdevs} "
> +            f"{self._ports} "
>              f"{self._other_eal_param}"
>          )
>
> @@ -347,6 +352,7 @@ def create_eal_parameters(
>          append_prefix_timestamp: bool = True,
>          no_pci: bool = False,
>          vdevs: list[VirtualDevice] | None = None,
> +        ports: list[Port] | None = None,
>          other_eal_param: str = "",
>      ) -> "EalParameters":
>          """Compose the EAL parameters.
> @@ -370,6 +376,8 @@ def create_eal_parameters(
>                      VirtualDevice('net_ring0'),
>                      VirtualDevice('net_ring1')
>                  ]
> +            ports: The list of ports to allow. If :data:`None`, all ports listed in `self.ports`
> +                will be allowed.
>              other_eal_param: user defined DPDK EAL parameters, e.g.:
>                  ``other_eal_param='--single-file-segments'``.
>
> @@ -388,12 +396,16 @@ def create_eal_parameters(
>          if vdevs is None:
>              vdevs = []
>
> +        if ports is None:
> +            ports = self.ports
> +
>          return EalParameters(
>              lcore_list=lcore_list,
>              memory_channels=self.config.memory_channels,
>              prefix=prefix,
>              no_pci=no_pci,
>              vdevs=vdevs,
> +            ports=ports,
>              other_eal_param=other_eal_param,
>          )
>
> --
> 2.43.0
>

[PATCH v6 5/7] dts: allow configuring MTU of ports

[jspewock]

From: Jeremy Spewock <jspewock@iol.unh.edu>

Adds methods in both os_session and linux session to allow for setting
MTU of port interfaces so that suites that require the sending and
receiving of packets of a specific size, or the rejection of packets
over a certain size, can configure this maximum as needed.

Signed-off-by: Jeremy Spewock <jspewock@iol.unh.edu>
---
 dts/framework/testbed_model/linux_session.py | 8 ++++++++
 dts/framework/testbed_model/os_session.py    | 9 +++++++++
 2 files changed, 17 insertions(+)

diff --git a/dts/framework/testbed_model/linux_session.py b/dts/framework/testbed_model/linux_session.py
index 0ab59cef85..5d24030c3d 100644
--- a/dts/framework/testbed_model/linux_session.py
+++ b/dts/framework/testbed_model/linux_session.py
@@ -198,6 +198,14 @@ def configure_port_ip_address(
             verify=True,
         )
 
+    def configure_port_mtu(self, mtu: int, port: Port) -> None:
+        """Overrides :meth:`~.os_session.OSSession.configure_port_mtu`."""
+        self.send_command(
+            f"ip link set dev {port.logical_name} mtu {mtu}",
+            privileged=True,
+            verify=True,
+        )
+
     def configure_ipv4_forwarding(self, enable: bool) -> None:
         """Overrides :meth:`~.os_session.OSSession.configure_ipv4_forwarding`."""
         state = 1 if enable else 0
diff --git a/dts/framework/testbed_model/os_session.py b/dts/framework/testbed_model/os_session.py
index ac6bb5e112..e42f4d752a 100644
--- a/dts/framework/testbed_model/os_session.py
+++ b/dts/framework/testbed_model/os_session.py
@@ -413,6 +413,15 @@ def configure_port_ip_address(
             delete: If :data:`True`, remove the IP address, otherwise configure it.
         """
 
+    @abstractmethod
+    def configure_port_mtu(self, mtu: int, port: Port) -> None:
+        """Configure `mtu` on `port`.
+
+        Args:
+            mtu: Desired MTU value.
+            port: Port to set `mtu` on.
+        """
+
     @abstractmethod
     def configure_ipv4_forwarding(self, enable: bool) -> None:
         """Enable IPv4 forwarding in the operating system.
-- 
2.43.0

Re: [PATCH v6 5/7] dts: allow configuring MTU of ports

[Juraj Linkeš]

Reviewed-by: Juraj Linkeš <juraj.linkes@pantheon.tech>

On Wed, Jan 3, 2024 at 11:33 PM <jspewock@iol.unh.edu> wrote:
>
> From: Jeremy Spewock <jspewock@iol.unh.edu>
>
> Adds methods in both os_session and linux session to allow for setting
> MTU of port interfaces so that suites that require the sending and
> receiving of packets of a specific size, or the rejection of packets
> over a certain size, can configure this maximum as needed.
>
> Signed-off-by: Jeremy Spewock <jspewock@iol.unh.edu>

[PATCH v6 6/7] dts: add scatter to the yaml schema

[jspewock]

From: Jeremy Spewock <jspewock@iol.unh.edu>

Allow for scatter to be specified in the configuration file.

Signed-off-by: Jeremy Spewock <jspewock@iol.unh.edu>
---
 dts/framework/config/conf_yaml_schema.json | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)

diff --git a/dts/framework/config/conf_yaml_schema.json b/dts/framework/config/conf_yaml_schema.json
index 84e45fe3c2..e6dc50ca7f 100644
--- a/dts/framework/config/conf_yaml_schema.json
+++ b/dts/framework/config/conf_yaml_schema.json
@@ -186,7 +186,8 @@
       "type": "string",
       "enum": [
         "hello_world",
-        "os_udp"
+        "os_udp",
+        "pmd_buffer_scatter"
       ]
     },
     "test_target": {
-- 
2.43.0

Re: [PATCH v6 6/7] dts: add scatter to the yaml schema

[Juraj Linkeš]

Reviewed-by: Juraj Linkeš <juraj.linkes@pantheon.tech>

On Wed, Jan 3, 2024 at 11:33 PM <jspewock@iol.unh.edu> wrote:
>
> From: Jeremy Spewock <jspewock@iol.unh.edu>
>
> Allow for scatter to be specified in the configuration file.
>
> Signed-off-by: Jeremy Spewock <jspewock@iol.unh.edu>

[PATCH v6 7/7] dts: add pmd_buffer_scatter test suite

[jspewock]

From: Jeremy Spewock <jspewock@iol.unh.edu>

This test suite provides testing of the support of scattered packets by
Poll Mode Drivers using testpmd, verifying the ability to receive and
transmit scattered multi-segment packets made up of multiple
non-contiguous memory buffers. This is tested through 5 different cases
in which the length of the packets sent are less than the mbuf size,
equal to the mbuf size, and 1, 4, and 5 bytes greater than the mbuf size
in order to show both the CRC and the packet data are capable of
existing in the first, second, or both buffers.

Naturally, if the PMD is capable of forwarding scattered packets which
it receives as input, this shows it is capable of both receiving and
transmitting scattered packets.

Signed-off-by: Jeremy Spewock <jspewock@iol.unh.edu>
---
 dts/tests/TestSuite_pmd_buffer_scatter.py | 126 ++++++++++++++++++++++
 1 file changed, 126 insertions(+)
 create mode 100644 dts/tests/TestSuite_pmd_buffer_scatter.py

diff --git a/dts/tests/TestSuite_pmd_buffer_scatter.py b/dts/tests/TestSuite_pmd_buffer_scatter.py
new file mode 100644
index 0000000000..8838c3404f
--- /dev/null
+++ b/dts/tests/TestSuite_pmd_buffer_scatter.py
@@ -0,0 +1,126 @@
+# SPDX-License-Identifier: BSD-3-Clause
+# Copyright(c) 2023-2024 University of New Hampshire
+
+"""Multi-segment packet scattering testing suite.
+
+This testing suite tests the support of transmitting and receiving scattered packets. This is shown
+by the Poll Mode Driver being able to forward scattered multi-segment packets composed of multiple
+non-contiguous memory buffers. To ensure the receipt of scattered packets, the DMA rings of the
+port's RX queues must be configured with mbuf data buffers whose size is less than the maximum
+length.
+
+If it is the case that the Poll Mode Driver can forward scattered packets which it receives, then
+this suffices to show the Poll Mode Driver is capable of both receiving and transmitting scattered
+packets.
+"""
+import struct
+
+from scapy.layers.inet import IP  # type: ignore[import]
+from scapy.layers.l2 import Ether  # type: ignore[import]
+from scapy.packet import Raw  # type: ignore[import]
+from scapy.utils import hexstr  # type: ignore[import]
+
+from framework.remote_session.testpmd_shell import TestPmdForwardingModes, TestPmdShell
+from framework.test_suite import TestSuite
+
+
+class PmdBufferScatter(TestSuite):
+    """DPDK PMD packet scattering test suite.
+
+    Configure the Rx queues to have mbuf data buffers whose sizes are smaller than the maximum
+    packet size. Specifically, set mbuf data buffers to have a size of 2048 to fit a full 1512-byte
+    (CRC included) ethernet frame in a mono-segment packet. The testing of scattered packets is
+    done by sending a packet whose length is greater than the size of the configured size of mbuf
+    data buffers. There are a total of 5 packets sent within test cases which have lengths less
+    than, equal to, and greater than the mbuf size. There are multiple packets sent with lengths
+    greater than the mbuf size in order to test cases such as:
+
+    1. A single byte of the CRC being in a second buffer while the remaining 3 bytes are stored in
+        the first buffer alongside packet data.
+    2. The entire CRC being stored in a second buffer while all of the packet data is stored in the
+        first.
+    3. Most of the packet data being stored in the first buffer and a single byte of packet data
+        stored in a second buffer alongside the CRC.
+    """
+
+    def set_up_suite(self) -> None:
+        """Set up the test suite.
+
+        Setup:
+            Verify they we have at least 2 port links in the current execution and increase the MTU
+            of both ports on the tg_node to 9000 to support larger packet sizes.
+        """
+        self.verify(
+            len(self._port_links) > 1,
+            "Must have at least two port links to run scatter",
+        )
+
+        self.tg_node.main_session.configure_port_mtu(9000, self._tg_port_egress)
+        self.tg_node.main_session.configure_port_mtu(9000, self._tg_port_ingress)
+
+    def scatter_pktgen_send_packet(self, pktsize: int) -> str:
+        """Generate and send packet to the SUT.
+
+        Functional test for scatter packets.
+
+        Args:
+            pktsize: Size of the packet to generate and send.
+        """
+        packet = Ether() / IP() / Raw()
+        packet.getlayer(2).load = ""
+        payload_len = pktsize - len(packet) - 4
+        payload = ["58"] * payload_len
+        # pack the payload
+        for X_in_hex in payload:
+            packet.load += struct.pack("=B", int("%s%s" % (X_in_hex[0], X_in_hex[1]), 16))
+        received_packets = self.send_packet_and_capture(packet)
+        self.verify(len(received_packets) > 0, "Did not receive any packets.")
+        load = hexstr(received_packets[0].getlayer(2), onlyhex=1)
+
+        return load
+
+    def pmd_scatter(self, mbsize: int) -> None:
+        """Testpmd support of receiving and sending scattered multi-segment packets.
+
+        Support for scattered packets is shown by sending 5 packets of differing length
+        where the length of the packet is calculated by taking mbuf-size + an offset. The
+        offsets used in the test case are -1, 0, 1, 4, 5 respectively.
+
+        Test:
+            Start testpmd and run functional test with preset mbsize.
+        """
+        testpmd = self.sut_node.create_interactive_shell(
+            TestPmdShell,
+            app_parameters=(
+                "--mbcache=200 "
+                f"--mbuf-size={mbsize} "
+                "--max-pkt-len=9000 "
+                "--port-topology=paired "
+                "--tx-offloads=0x00008000"
+            ),
+            privileged=True,
+        )
+        testpmd.set_forward_mode(TestPmdForwardingModes.mac)
+        testpmd.start()
+
+        for offset in [-1, 0, 1, 4, 5]:
+            recv_payload = self.scatter_pktgen_send_packet(mbsize + offset)
+            self._logger.debug(f"Payload of scattered packet after forwarding: \n{recv_payload}")
+            self.verify(
+                ("58 " * 8).strip() in recv_payload,
+                f"Payload of scattered packet did not match expected payload with offset {offset}.",
+            )
+        testpmd.stop()
+
+    def test_scatter_mbuf_2048(self) -> None:
+        """Run :func:`~PmdBufferScatter.pmd_scatter` function with `mbsize` set to 2048."""
+        self.pmd_scatter(mbsize=2048)
+
+    def tear_down_suite(self) -> None:
+        """Tear down the test suite.
+
+        Teardown:
+            Set the MTU of the tg_node back to a more standard size of 1500
+        """
+        self.tg_node.main_session.configure_port_mtu(1500, self._tg_port_egress)
+        self.tg_node.main_session.configure_port_mtu(1500, self._tg_port_ingress)
-- 
2.43.0

Re: [PATCH v6 7/7] dts: add pmd_buffer_scatter test suite

[Juraj Linkeš]

On Wed, Jan 3, 2024 at 11:33 PM <jspewock@iol.unh.edu> wrote:
>
> From: Jeremy Spewock <jspewock@iol.unh.edu>
>
> This test suite provides testing of the support of scattered packets by
> Poll Mode Drivers using testpmd, verifying the ability to receive and
> transmit scattered multi-segment packets made up of multiple
> non-contiguous memory buffers. This is tested through 5 different cases
> in which the length of the packets sent are less than the mbuf size,
> equal to the mbuf size, and 1, 4, and 5 bytes greater than the mbuf size
> in order to show both the CRC and the packet data are capable of
> existing in the first, second, or both buffers.
>
> Naturally, if the PMD is capable of forwarding scattered packets which
> it receives as input, this shows it is capable of both receiving and
> transmitting scattered packets.
>
> Signed-off-by: Jeremy Spewock <jspewock@iol.unh.edu>
> ---
>  dts/tests/TestSuite_pmd_buffer_scatter.py | 126 ++++++++++++++++++++++
>  1 file changed, 126 insertions(+)
>  create mode 100644 dts/tests/TestSuite_pmd_buffer_scatter.py
>
> diff --git a/dts/tests/TestSuite_pmd_buffer_scatter.py b/dts/tests/TestSuite_pmd_buffer_scatter.py
> new file mode 100644
> index 0000000000..8838c3404f
> --- /dev/null
> +++ b/dts/tests/TestSuite_pmd_buffer_scatter.py
> @@ -0,0 +1,126 @@
> +# SPDX-License-Identifier: BSD-3-Clause
> +# Copyright(c) 2023-2024 University of New Hampshire
> +
> +"""Multi-segment packet scattering testing suite.

Test suite - I guess this is a copy-paste?

> +
> +This testing suite tests the support of transmitting and receiving scattered packets. This is shown
> +by the Poll Mode Driver being able to forward scattered multi-segment packets composed of multiple
> +non-contiguous memory buffers. To ensure the receipt of scattered packets, the DMA rings of the
> +port's RX queues must be configured with mbuf data buffers whose size is less than the maximum
> +length.
> +
> +If it is the case that the Poll Mode Driver can forward scattered packets which it receives, then
> +this suffices to show the Poll Mode Driver is capable of both receiving and transmitting scattered
> +packets.
> +"""

We have a newline between the docstring and the import everywhere.

> +import struct
> +
> +from scapy.layers.inet import IP  # type: ignore[import]
> +from scapy.layers.l2 import Ether  # type: ignore[import]
> +from scapy.packet import Raw  # type: ignore[import]
> +from scapy.utils import hexstr  # type: ignore[import]
> +
> +from framework.remote_session.testpmd_shell import TestPmdForwardingModes, TestPmdShell
> +from framework.test_suite import TestSuite
> +
> +
> +class PmdBufferScatter(TestSuite):
> +    """DPDK PMD packet scattering test suite.
> +
> +    Configure the Rx queues to have mbuf data buffers whose sizes are smaller than the maximum
> +    packet size. Specifically, set mbuf data buffers to have a size of 2048 to fit a full 1512-byte
> +    (CRC included) ethernet frame in a mono-segment packet. The testing of scattered packets is
> +    done by sending a packet whose length is greater than the size of the configured size of mbuf
> +    data buffers. There are a total of 5 packets sent within test cases which have lengths less
> +    than, equal to, and greater than the mbuf size. There are multiple packets sent with lengths
> +    greater than the mbuf size in order to test cases such as:
> +
> +    1. A single byte of the CRC being in a second buffer while the remaining 3 bytes are stored in
> +        the first buffer alongside packet data.
> +    2. The entire CRC being stored in a second buffer while all of the packet data is stored in the
> +        first.
> +    3. Most of the packet data being stored in the first buffer and a single byte of packet data
> +        stored in a second buffer alongside the CRC.
> +    """
> +
> +    def set_up_suite(self) -> None:
> +        """Set up the test suite.
> +
> +        Setup:
> +            Verify they we have at least 2 port links in the current execution and increase the MTU

Typo - they.

> +            of both ports on the tg_node to 9000 to support larger packet sizes.

The description should be code agnostic, so let's use traffic
generator node instead of tg_node.

> +        """
> +        self.verify(
> +            len(self._port_links) > 1,
> +            "Must have at least two port links to run scatter",

I'd like this to be at least "Must have at least two port links to run
the scatter test suite" so that it's immediately obvious where this
comes from. I'm also debating which of these is better: "Must have at
least" or "There must be at least", but I'm not sure.

> +        )
> +
> +        self.tg_node.main_session.configure_port_mtu(9000, self._tg_port_egress)
> +        self.tg_node.main_session.configure_port_mtu(9000, self._tg_port_ingress)
> +
> +    def scatter_pktgen_send_packet(self, pktsize: int) -> str:
> +        """Generate and send packet to the SUT.

send a packet

But this also captures a packet, so let's mention that.

> +
> +        Functional test for scatter packets.

This is just part of the test. The actual test is the pmd_scatter
method with test cases being the callers of pmd_scatter.
We should improve this. We mentioned a packet, so let's describe it.

> +
> +        Args:
> +            pktsize: Size of the packet to generate and send.
> +        """
> +        packet = Ether() / IP() / Raw()
> +        packet.getlayer(2).load = ""
> +        payload_len = pktsize - len(packet) - 4
> +        payload = ["58"] * payload_len
> +        # pack the payload
> +        for X_in_hex in payload:
> +            packet.load += struct.pack("=B", int("%s%s" % (X_in_hex[0], X_in_hex[1]), 16))
> +        received_packets = self.send_packet_and_capture(packet)
> +        self.verify(len(received_packets) > 0, "Did not receive any packets.")
> +        load = hexstr(received_packets[0].getlayer(2), onlyhex=1)
> +
> +        return load
> +
> +    def pmd_scatter(self, mbsize: int) -> None:
> +        """Testpmd support of receiving and sending scattered multi-segment packets.
> +
> +        Support for scattered packets is shown by sending 5 packets of differing length
> +        where the length of the packet is calculated by taking mbuf-size + an offset. The
> +        offsets used in the test case are -1, 0, 1, 4, 5 respectively.
> +

In the test.

> +        Test:
> +            Start testpmd and run functional test with preset mbsize.
> +        """
> +        testpmd = self.sut_node.create_interactive_shell(
> +            TestPmdShell,
> +            app_parameters=(
> +                "--mbcache=200 "
> +                f"--mbuf-size={mbsize} "
> +                "--max-pkt-len=9000 "
> +                "--port-topology=paired "
> +                "--tx-offloads=0x00008000"
> +            ),
> +            privileged=True,
> +        )
> +        testpmd.set_forward_mode(TestPmdForwardingModes.mac)
> +        testpmd.start()
> +
> +        for offset in [-1, 0, 1, 4, 5]:
> +            recv_payload = self.scatter_pktgen_send_packet(mbsize + offset)
> +            self._logger.debug(f"Payload of scattered packet after forwarding: \n{recv_payload}")
> +            self.verify(
> +                ("58 " * 8).strip() in recv_payload,
> +                f"Payload of scattered packet did not match expected payload with offset {offset}.",
> +            )
> +        testpmd.stop()
> +
> +    def test_scatter_mbuf_2048(self) -> None:
> +        """Run :func:`~PmdBufferScatter.pmd_scatter` function with `mbsize` set to 2048."""

This would probably read better as "Run the pmd_scatter test"

> +        self.pmd_scatter(mbsize=2048)
> +
> +    def tear_down_suite(self) -> None:
> +        """Tear down the test suite.
> +
> +        Teardown:
> +            Set the MTU of the tg_node back to a more standard size of 1500
> +        """
> +        self.tg_node.main_session.configure_port_mtu(1500, self._tg_port_egress)
> +        self.tg_node.main_session.configure_port_mtu(1500, self._tg_port_ingress)
> --
> 2.43.0
>

Re: [PATCH v6 7/7] dts: add pmd_buffer_scatter test suite

[Jeremy Spewock]

[-- Attachment #1: Type: text/plain, Size: 9412 bytes --]

On Mon, Jan 8, 2024 at 10:47 AM Juraj Linkeš <juraj.linkes@pantheon.tech>
wrote:

> On Wed, Jan 3, 2024 at 11:33 PM <jspewock@iol.unh.edu> wrote:
> >
> > From: Jeremy Spewock <jspewock@iol.unh.edu>
> >
> > This test suite provides testing of the support of scattered packets by
> > Poll Mode Drivers using testpmd, verifying the ability to receive and
> > transmit scattered multi-segment packets made up of multiple
> > non-contiguous memory buffers. This is tested through 5 different cases
> > in which the length of the packets sent are less than the mbuf size,
> > equal to the mbuf size, and 1, 4, and 5 bytes greater than the mbuf size
> > in order to show both the CRC and the packet data are capable of
> > existing in the first, second, or both buffers.
> >
> > Naturally, if the PMD is capable of forwarding scattered packets which
> > it receives as input, this shows it is capable of both receiving and
> > transmitting scattered packets.
> >
> > Signed-off-by: Jeremy Spewock <jspewock@iol.unh.edu>
> > ---
> >  dts/tests/TestSuite_pmd_buffer_scatter.py | 126 ++++++++++++++++++++++
> >  1 file changed, 126 insertions(+)
> >  create mode 100644 dts/tests/TestSuite_pmd_buffer_scatter.py
> >
> > diff --git a/dts/tests/TestSuite_pmd_buffer_scatter.py
> b/dts/tests/TestSuite_pmd_buffer_scatter.py
> > new file mode 100644
> > index 0000000000..8838c3404f
> > --- /dev/null
> > +++ b/dts/tests/TestSuite_pmd_buffer_scatter.py
> > @@ -0,0 +1,126 @@
> > +# SPDX-License-Identifier: BSD-3-Clause
> > +# Copyright(c) 2023-2024 University of New Hampshire
> > +
> > +"""Multi-segment packet scattering testing suite.
>
> Test suite - I guess this is a copy-paste?
>

Good catch, it probably was.


> > +
> > +This testing suite tests the support of transmitting and receiving
> scattered packets. This is shown
> > +by the Poll Mode Driver being able to forward scattered multi-segment
> packets composed of multiple
> > +non-contiguous memory buffers. To ensure the receipt of scattered
> packets, the DMA rings of the
> > +port's RX queues must be configured with mbuf data buffers whose size
> is less than the maximum
> > +length.
> > +
> > +If it is the case that the Poll Mode Driver can forward scattered
> packets which it receives, then
> > +this suffices to show the Poll Mode Driver is capable of both receiving
> and transmitting scattered
> > +packets.
> > +"""
>
> We have a newline between the docstring and the import everywhere.
>

You're right, I must have missed it here, I'll add that.


> > +import struct
> > +
> > +from scapy.layers.inet import IP  # type: ignore[import]
> > +from scapy.layers.l2 import Ether  # type: ignore[import]
> > +from scapy.packet import Raw  # type: ignore[import]
> > +from scapy.utils import hexstr  # type: ignore[import]
> > +
> > +from framework.remote_session.testpmd_shell import
> TestPmdForwardingModes, TestPmdShell
> > +from framework.test_suite import TestSuite
> > +
> > +
> > +class PmdBufferScatter(TestSuite):
> > +    """DPDK PMD packet scattering test suite.
> > +
> > +    Configure the Rx queues to have mbuf data buffers whose sizes are
> smaller than the maximum
> > +    packet size. Specifically, set mbuf data buffers to have a size of
> 2048 to fit a full 1512-byte
> > +    (CRC included) ethernet frame in a mono-segment packet. The testing
> of scattered packets is
> > +    done by sending a packet whose length is greater than the size of
> the configured size of mbuf
> > +    data buffers. There are a total of 5 packets sent within test cases
> which have lengths less
> > +    than, equal to, and greater than the mbuf size. There are multiple
> packets sent with lengths
> > +    greater than the mbuf size in order to test cases such as:
> > +
> > +    1. A single byte of the CRC being in a second buffer while the
> remaining 3 bytes are stored in
> > +        the first buffer alongside packet data.
> > +    2. The entire CRC being stored in a second buffer while all of the
> packet data is stored in the
> > +        first.
> > +    3. Most of the packet data being stored in the first buffer and a
> single byte of packet data
> > +        stored in a second buffer alongside the CRC.
> > +    """
> > +
> > +    def set_up_suite(self) -> None:
> > +        """Set up the test suite.
> > +
> > +        Setup:
> > +            Verify they we have at least 2 port links in the current
> execution and increase the MTU
>
> Typo - they.
>

Oops, thank you!


>
> > +            of both ports on the tg_node to 9000 to support larger
> packet sizes.
>
> The description should be code agnostic, so let's use traffic
> generator node instead of tg_node.
>

Good point, I'll update this.


>
> > +        """
> > +        self.verify(
> > +            len(self._port_links) > 1,
> > +            "Must have at least two port links to run scatter",
>
> I'd like this to be at least "Must have at least two port links to run
> the scatter test suite" so that it's immediately obvious where this
> comes from. I'm also debating which of these is better: "Must have at
> least" or "There must be at least", but I'm not sure.
>

I think reading it over that "There must be at least" sounds better, I'll
update this as well.


>
> > +        )
> > +
> > +        self.tg_node.main_session.configure_port_mtu(9000,
> self._tg_port_egress)
> > +        self.tg_node.main_session.configure_port_mtu(9000,
> self._tg_port_ingress)
> > +
> > +    def scatter_pktgen_send_packet(self, pktsize: int) -> str:
> > +        """Generate and send packet to the SUT.
>
> send a packet
>
> But this also captures a packet, so let's mention that.
>

Good point, I'll add this.


>
> > +
> > +        Functional test for scatter packets.
>
> This is just part of the test. The actual test is the pmd_scatter
> method with test cases being the callers of pmd_scatter.
> We should improve this. We mentioned a packet, so let's describe it.
>

This definitely could be expanded, this likely just came from pulling from
old DTS or before the docstrings were expanded. Good catch!


>
> > +
> > +        Args:
> > +            pktsize: Size of the packet to generate and send.
> > +        """
> > +        packet = Ether() / IP() / Raw()
> > +        packet.getlayer(2).load = ""
> > +        payload_len = pktsize - len(packet) - 4
> > +        payload = ["58"] * payload_len
> > +        # pack the payload
> > +        for X_in_hex in payload:
> > +            packet.load += struct.pack("=B", int("%s%s" % (X_in_hex[0],
> X_in_hex[1]), 16))
> > +        received_packets = self.send_packet_and_capture(packet)
> > +        self.verify(len(received_packets) > 0, "Did not receive any
> packets.")
> > +        load = hexstr(received_packets[0].getlayer(2), onlyhex=1)
> > +
> > +        return load
> > +
> > +    def pmd_scatter(self, mbsize: int) -> None:
> > +        """Testpmd support of receiving and sending scattered
> multi-segment packets.
> > +
> > +        Support for scattered packets is shown by sending 5 packets of
> differing length
> > +        where the length of the packet is calculated by taking
> mbuf-size + an offset. The
> > +        offsets used in the test case are -1, 0, 1, 4, 5 respectively.
> > +
>
> In the test.
>

Good point, I'll change this.


>
> > +        Test:
> > +            Start testpmd and run functional test with preset mbsize.
> > +        """
> > +        testpmd = self.sut_node.create_interactive_shell(
> > +            TestPmdShell,
> > +            app_parameters=(
> > +                "--mbcache=200 "
> > +                f"--mbuf-size={mbsize} "
> > +                "--max-pkt-len=9000 "
> > +                "--port-topology=paired "
> > +                "--tx-offloads=0x00008000"
> > +            ),
> > +            privileged=True,
> > +        )
> > +        testpmd.set_forward_mode(TestPmdForwardingModes.mac)
> > +        testpmd.start()
> > +
> > +        for offset in [-1, 0, 1, 4, 5]:
> > +            recv_payload = self.scatter_pktgen_send_packet(mbsize +
> offset)
> > +            self._logger.debug(f"Payload of scattered packet after
> forwarding: \n{recv_payload}")
> > +            self.verify(
> > +                ("58 " * 8).strip() in recv_payload,
> > +                f"Payload of scattered packet did not match expected
> payload with offset {offset}.",
> > +            )
> > +        testpmd.stop()
> > +
> > +    def test_scatter_mbuf_2048(self) -> None:
> > +        """Run :func:`~PmdBufferScatter.pmd_scatter` function with
> `mbsize` set to 2048."""
>
> This would probably read better as "Run the pmd_scatter test"
>

I agree, I'll update this.


>
> > +        self.pmd_scatter(mbsize=2048)
> > +
> > +    def tear_down_suite(self) -> None:
> > +        """Tear down the test suite.
> > +
> > +        Teardown:
> > +            Set the MTU of the tg_node back to a more standard size of
> 1500
> > +        """
> > +        self.tg_node.main_session.configure_port_mtu(1500,
> self._tg_port_egress)
> > +        self.tg_node.main_session.configure_port_mtu(1500,
> self._tg_port_ingress)
> > --
> > 2.43.0
> >
>

Thank you again for the thorough feedback!

[-- Attachment #2: Type: text/html, Size: 13749 bytes --]

[PATCH v7 0/7] dts: Port scatter suite over

[jspewock]

From: Jeremy Spewock <jspewock@iol.unh.edu>

v7:

Addressed comments and made appropriate changes. Most changes had to do
with grammatical/spelling errors and expanding upon documentation. Also
fixed the typing of eal parameters when creating interactive shells
through the SUT node.

Jeremy Spewock (7):
  dts: add startup verification and forwarding modes to testpmd shell
  dts: limit EAL parameters to DPDK apps and add parameters to all apps
  dts: add optional packet filtering to scapy sniffer
  dts: add pci addresses to EAL parameters
  dts: allow configuring MTU of ports
  dts: add scatter to the yaml schema
  dts: add pmd_buffer_scatter test suite

 dts/framework/config/conf_yaml_schema.json    |   3 +-
 dts/framework/exception.py                    |   7 +
 dts/framework/remote_session/testpmd_shell.py | 149 +++++++++++++++++-
 dts/framework/test_suite.py                   |  15 +-
 dts/framework/testbed_model/linux_session.py  |   8 +
 dts/framework/testbed_model/os_session.py     |   9 ++
 dts/framework/testbed_model/sut_node.py       |  28 +++-
 dts/framework/testbed_model/tg_node.py        |  14 +-
 .../traffic_generator/__init__.py             |   7 +-
 .../capturing_traffic_generator.py            |  22 ++-
 .../testbed_model/traffic_generator/scapy.py  |  27 ++++
 dts/tests/TestSuite_pmd_buffer_scatter.py     | 132 ++++++++++++++++
 12 files changed, 407 insertions(+), 14 deletions(-)
 create mode 100644 dts/tests/TestSuite_pmd_buffer_scatter.py

-- 
2.43.0

[PATCH v7 1/7] dts: add startup verification and forwarding modes to testpmd shell

[jspewock]

From: Jeremy Spewock <jspewock@iol.unh.edu>

Added commonly used methods in testpmd such as starting and stopping
packet forwarding, changing forward modes, and verifying link status of
ports so that developers can configure testpmd and start forwarding
through the provided class rather than sending commands to the testpmd
session directly.

Signed-off-by: Jeremy Spewock <jspewock@iol.unh.edu>
---
 dts/framework/exception.py                    |   7 +
 dts/framework/remote_session/testpmd_shell.py | 149 +++++++++++++++++-
 2 files changed, 155 insertions(+), 1 deletion(-)

diff --git a/dts/framework/exception.py b/dts/framework/exception.py
index 658eee2c38..cce1e0231a 100644
--- a/dts/framework/exception.py
+++ b/dts/framework/exception.py
@@ -146,6 +146,13 @@ def __str__(self) -> str:
         return f"Command {self.command} returned a non-zero exit code: {self._command_return_code}"
 
 
+class InteractiveCommandExecutionError(DTSError):
+    """An unsuccessful execution of a remote command in an interactive environment."""
+
+    #:
+    severity: ClassVar[ErrorSeverity] = ErrorSeverity.REMOTE_CMD_EXEC_ERR
+
+
 class RemoteDirectoryExistsError(DTSError):
     """A directory that exists on a remote node."""
 
diff --git a/dts/framework/remote_session/testpmd_shell.py b/dts/framework/remote_session/testpmd_shell.py
index 0184cc2e71..11c5c7f93c 100644
--- a/dts/framework/remote_session/testpmd_shell.py
+++ b/dts/framework/remote_session/testpmd_shell.py
@@ -15,9 +15,15 @@
     testpmd_shell.close()
 """
 
+import time
+from enum import auto
 from pathlib import PurePath
 from typing import Callable, ClassVar
 
+from framework.exception import InteractiveCommandExecutionError
+from framework.settings import SETTINGS
+from framework.utils import StrEnum
+
 from .interactive_shell import InteractiveShell
 
 
@@ -43,14 +49,51 @@ def __str__(self) -> str:
         return self.pci_address
 
 
+class TestPmdForwardingModes(StrEnum):
+    r"""The supported packet forwarding modes for :class:`~TestPmdShell`\s."""
+
+    #:
+    io = auto()
+    #:
+    mac = auto()
+    #:
+    macswap = auto()
+    #:
+    flowgen = auto()
+    #:
+    rxonly = auto()
+    #:
+    txonly = auto()
+    #:
+    csum = auto()
+    #:
+    icmpecho = auto()
+    #:
+    ieee1588 = auto()
+    #:
+    noisy = auto()
+    #:
+    fivetswap = "5tswap"
+    #:
+    shared_rxq = "shared-rxq"
+    #:
+    recycle_mbufs = auto()
+
+
 class TestPmdShell(InteractiveShell):
     """Testpmd interactive shell.
 
     The testpmd shell users should never use
     the :meth:`~.interactive_shell.InteractiveShell.send_command` method directly, but rather
     call specialized methods. If there isn't one that satisfies a need, it should be added.
+
+    Attributes:
+        number_of_ports: The number of ports which were allowed on the command-line when testpmd
+            was started.
     """
 
+    number_of_ports: int
+
     #: The path to the testpmd executable.
     path: ClassVar[PurePath] = PurePath("app", "dpdk-testpmd")
 
@@ -65,9 +108,66 @@ class TestPmdShell(InteractiveShell):
     _command_extra_chars: ClassVar[str] = "\n"
 
     def _start_application(self, get_privileged_command: Callable[[str], str] | None) -> None:
-        self._app_args += " -- -i"
+        """Overrides :meth:`~.interactive_shell._start_application`.
+
+        Add flags for starting testpmd in interactive mode and disabling messages for link state
+        change events before starting the application. Link state is verified before starting
+        packet forwarding and the messages create unexpected newlines in the terminal which
+        complicates output collection.
+
+        Also find the number of pci addresses which were allowed on the command line when the app
+        was started.
+        """
+        self._app_args += " -- -i --mask-event intr_lsc"
+        self.number_of_ports = self._app_args.count("-a ")
         super()._start_application(get_privileged_command)
 
+    def start(self, verify: bool = True) -> None:
+        """Start packet forwarding with the current configuration.
+
+        Args:
+            verify: If :data:`True` , a second start command will be sent in an attempt to verify
+                packet forwarding started as expected.
+
+        Raises:
+            InteractiveCommandExecutionError: If `verify` is :data:`True` and forwarding fails to
+                start or ports fail to come up.
+        """
+        self.send_command("start")
+        if verify:
+            # If forwarding was already started, sending "start" again should tell us
+            start_cmd_output = self.send_command("start")
+            if "Packet forwarding already started" not in start_cmd_output:
+                self._logger.debug(f"Failed to start packet forwarding: \n{start_cmd_output}")
+                raise InteractiveCommandExecutionError("Testpmd failed to start packet forwarding.")
+
+            for port_id in range(self.number_of_ports):
+                if not self.wait_link_status_up(port_id):
+                    raise InteractiveCommandExecutionError(
+                        "Not all ports came up after starting packet forwarding in testpmd."
+                    )
+
+    def stop(self, verify: bool = True) -> None:
+        """Stop packet forwarding.
+
+        Args:
+            verify: If :data:`True` , the output of the stop command is scanned to verify that
+                forwarding was stopped successfully or not started. If neither is found, it is
+                considered an error.
+
+        Raises:
+            InteractiveCommandExecutionError: If `verify` is :data:`True` and the command to stop
+                forwarding results in an error.
+        """
+        stop_cmd_output = self.send_command("stop")
+        if verify:
+            if (
+                "Done." not in stop_cmd_output
+                and "Packet forwarding not started" not in stop_cmd_output
+            ):
+                self._logger.debug(f"Failed to stop packet forwarding: \n{stop_cmd_output}")
+                raise InteractiveCommandExecutionError("Testpmd failed to stop packet forwarding.")
+
     def get_devices(self) -> list[TestPmdDevice]:
         """Get a list of device names that are known to testpmd.
 
@@ -82,3 +182,50 @@ def get_devices(self) -> list[TestPmdDevice]:
             if "device name:" in line.lower():
                 dev_list.append(TestPmdDevice(line))
         return dev_list
+
+    def wait_link_status_up(self, port_id: int, timeout=SETTINGS.timeout) -> bool:
+        """Wait until the link status on the given port is "up".
+
+        Arguments:
+            port_id: Port to check the link status on.
+            timeout: Time to wait for the link to come up. The default value for this
+                argument may be modified using the :option:`-t, --timeout` command-line argument
+                or the :envvar:`DTS_TIMEOUT` environment variable.
+
+        Returns:
+            Whether the link came up in time or not.
+        """
+        time_to_stop = time.time() + timeout
+        port_info: str = ""
+        while time.time() < time_to_stop:
+            port_info = self.send_command(f"show port info {port_id}")
+            if "Link status: up" in port_info:
+                break
+            time.sleep(0.5)
+        else:
+            self._logger.error(f"The link for port {port_id} did not come up in the given timeout.")
+        return "Link status: up" in port_info
+
+    def set_forward_mode(self, mode: TestPmdForwardingModes, verify: bool = True):
+        """Set packet forwarding mode.
+
+        Args:
+            mode: The forwarding mode to use.
+            verify: If :data:`True` the output of the command will be scanned in an attempt to
+                verify that the forwarding mode was set to `mode` properly.
+
+        Raises:
+            InteractiveCommandExecutionError: If `verify` is :data:`True` and the forwarding mode
+                fails to update.
+        """
+        set_fwd_output = self.send_command(f"set fwd {mode.value}")
+        if f"Set {mode.value} packet forwarding mode" not in set_fwd_output:
+            self._logger.debug(f"Failed to set fwd mode to {mode.value}:\n{set_fwd_output}")
+            raise InteractiveCommandExecutionError(
+                f"Test pmd failed to set fwd mode to {mode.value}"
+            )
+
+    def close(self) -> None:
+        """Overrides :meth:`~.interactive_shell.close`."""
+        self.send_command("quit", "")
+        return super().close()
-- 
2.43.0

Re: [PATCH v7 1/7] dts: add startup verification and forwarding modes to testpmd shell

[Juraj Linkeš]

> diff --git a/dts/framework/remote_session/testpmd_shell.py b/dts/framework/remote_session/testpmd_shell.py
> index 0184cc2e71..11c5c7f93c 100644
> --- a/dts/framework/remote_session/testpmd_shell.py
> +++ b/dts/framework/remote_session/testpmd_shell.py
<snip>
> @@ -82,3 +182,50 @@ def get_devices(self) -> list[TestPmdDevice]:
>              if "device name:" in line.lower():
>                  dev_list.append(TestPmdDevice(line))
>          return dev_list
> +
> +    def wait_link_status_up(self, port_id: int, timeout=SETTINGS.timeout) -> bool:
> +        """Wait until the link status on the given port is "up".
> +
> +        Arguments:
> +            port_id: Port to check the link status on.
> +            timeout: Time to wait for the link to come up. The default value for this
> +                argument may be modified using the :option:`-t, --timeout` command-line argument

This should be just :option:`--timeout`, otherwise Sphinx complains:
WARNING: unknown option: '-t, --timeout'

> +                or the :envvar:`DTS_TIMEOUT` environment variable.
> +
> +        Returns:
> +            Whether the link came up in time or not.
> +        """

Re: [PATCH v7 1/7] dts: add startup verification and forwarding modes to testpmd shell

[Jeremy Spewock]

[-- Attachment #1: Type: text/plain, Size: 1382 bytes --]

Good to know, I'll fix this.

On Wed, Jan 10, 2024 at 8:18 AM Juraj Linkeš <juraj.linkes@pantheon.tech>
wrote:

> > diff --git a/dts/framework/remote_session/testpmd_shell.py
> b/dts/framework/remote_session/testpmd_shell.py
> > index 0184cc2e71..11c5c7f93c 100644
> > --- a/dts/framework/remote_session/testpmd_shell.py
> > +++ b/dts/framework/remote_session/testpmd_shell.py
> <snip>
> > @@ -82,3 +182,50 @@ def get_devices(self) -> list[TestPmdDevice]:
> >              if "device name:" in line.lower():
> >                  dev_list.append(TestPmdDevice(line))
> >          return dev_list
> > +
> > +    def wait_link_status_up(self, port_id: int,
> timeout=SETTINGS.timeout) -> bool:
> > +        """Wait until the link status on the given port is "up".
> > +
> > +        Arguments:
> > +            port_id: Port to check the link status on.
> > +            timeout: Time to wait for the link to come up. The default
> value for this
> > +                argument may be modified using the :option:`-t,
> --timeout` command-line argument
>
> This should be just :option:`--timeout`, otherwise Sphinx complains:
> WARNING: unknown option: '-t, --timeout'
>
> > +                or the :envvar:`DTS_TIMEOUT` environment variable.
> > +
> > +        Returns:
> > +            Whether the link came up in time or not.
> > +        """
>

[-- Attachment #2: Type: text/html, Size: 1934 bytes --]

[PATCH v7 2/7] dts: limit EAL parameters to DPDK apps and add parameters to all apps

[jspewock]

From: Jeremy Spewock <jspewock@iol.unh.edu>

Changed the factory method for creating interactive apps in the SUT Node
so that EAL parameters would only be passed into DPDK apps since
non-DPDK apps wouldn't be able to process them. Also modified
interactive apps to allow for the ability to pass parameters into the
app on startup so that the applications can be started with certain
configuration steps passed on the command line.

Signed-off-by: Jeremy Spewock <jspewock@iol.unh.edu>
---
 dts/framework/remote_session/testpmd_shell.py |  2 +-
 dts/framework/testbed_model/sut_node.py       | 16 ++++++++++------
 2 files changed, 11 insertions(+), 7 deletions(-)

diff --git a/dts/framework/remote_session/testpmd_shell.py b/dts/framework/remote_session/testpmd_shell.py
index 11c5c7f93c..8a9578913b 100644
--- a/dts/framework/remote_session/testpmd_shell.py
+++ b/dts/framework/remote_session/testpmd_shell.py
@@ -118,7 +118,7 @@ def _start_application(self, get_privileged_command: Callable[[str], str] | None
         Also find the number of pci addresses which were allowed on the command line when the app
         was started.
         """
-        self._app_args += " -- -i --mask-event intr_lsc"
+        self._app_args += " -i --mask-event intr_lsc"
         self.number_of_ports = self._app_args.count("-a ")
         super()._start_application(get_privileged_command)
 
diff --git a/dts/framework/testbed_model/sut_node.py b/dts/framework/testbed_model/sut_node.py
index c4acea38d1..909394e756 100644
--- a/dts/framework/testbed_model/sut_node.py
+++ b/dts/framework/testbed_model/sut_node.py
@@ -430,7 +430,8 @@ def create_interactive_shell(
         shell_cls: Type[InteractiveShellType],
         timeout: float = SETTINGS.timeout,
         privileged: bool = False,
-        eal_parameters: EalParameters | str | None = None,
+        app_parameters: str = "",
+        eal_parameters: EalParameters | None = None,
     ) -> InteractiveShellType:
         """Extend the factory for interactive session handlers.
 
@@ -449,20 +450,23 @@ def create_interactive_shell(
             eal_parameters: List of EAL parameters to use to launch the app. If this
                 isn't provided or an empty string is passed, it will default to calling
                 :meth:`create_eal_parameters`.
+            app_parameters: Additional arguments to pass into the application on the
+                command-line.
 
         Returns:
             An instance of the desired interactive application shell.
         """
-        if not eal_parameters:
-            eal_parameters = self.create_eal_parameters()
-
-        # We need to append the build directory for DPDK apps
+        # We need to append the build directory and add EAL parameters for DPDK apps
         if shell_cls.dpdk_app:
+            if not eal_parameters:
+                eal_parameters = self.create_eal_parameters()
+            app_parameters = f"{eal_parameters} -- {app_parameters}"
+
             shell_cls.path = self.main_session.join_remote_path(
                 self.remote_dpdk_build_dir, shell_cls.path
             )
 
-        return super().create_interactive_shell(shell_cls, timeout, privileged, str(eal_parameters))
+        return super().create_interactive_shell(shell_cls, timeout, privileged, app_parameters)
 
     def bind_ports_to_driver(self, for_dpdk: bool = True) -> None:
         """Bind all ports on the SUT to a driver.
-- 
2.43.0

[PATCH v7 3/7] dts: add optional packet filtering to scapy sniffer

[jspewock]

From: Jeremy Spewock <jspewock@iol.unh.edu>

Added the options to filter out LLDP and ARP packets when
sniffing for packets with scapy. This was done using BPF filters to
ensure that the noise these packets provide does not interfere with test
cases.

Signed-off-by: Jeremy Spewock <jspewock@iol.unh.edu>
---
 dts/framework/test_suite.py                   | 15 +++++++++--
 dts/framework/testbed_model/tg_node.py        | 14 ++++++++--
 .../traffic_generator/__init__.py             |  7 ++++-
 .../capturing_traffic_generator.py            | 22 ++++++++++++++-
 .../testbed_model/traffic_generator/scapy.py  | 27 +++++++++++++++++++
 5 files changed, 79 insertions(+), 6 deletions(-)

diff --git a/dts/framework/test_suite.py b/dts/framework/test_suite.py
index dfb391ffbd..ffea917690 100644
--- a/dts/framework/test_suite.py
+++ b/dts/framework/test_suite.py
@@ -38,6 +38,7 @@
 from .settings import SETTINGS
 from .test_result import BuildTargetResult, Result, TestCaseResult, TestSuiteResult
 from .testbed_model import Port, PortLink, SutNode, TGNode
+from .testbed_model.traffic_generator import PacketFilteringConfig
 from .utils import get_packet_summaries
 
 
@@ -208,7 +209,12 @@ def configure_testbed_ipv4(self, restore: bool = False) -> None:
     def _configure_ipv4_forwarding(self, enable: bool) -> None:
         self.sut_node.configure_ipv4_forwarding(enable)
 
-    def send_packet_and_capture(self, packet: Packet, duration: float = 1) -> list[Packet]:
+    def send_packet_and_capture(
+        self,
+        packet: Packet,
+        filter_config: PacketFilteringConfig = PacketFilteringConfig(),
+        duration: float = 1,
+    ) -> list[Packet]:
         """Send and receive `packet` using the associated TG.
 
         Send `packet` through the appropriate interface and receive on the appropriate interface.
@@ -216,6 +222,7 @@ def send_packet_and_capture(self, packet: Packet, duration: float = 1) -> list[P
 
         Args:
             packet: The packet to send.
+            filter_config: The filter to use when capturing packets.
             duration: Capture traffic for this amount of time after sending `packet`.
 
         Returns:
@@ -223,7 +230,11 @@ def send_packet_and_capture(self, packet: Packet, duration: float = 1) -> list[P
         """
         packet = self._adjust_addresses(packet)
         return self.tg_node.send_packet_and_capture(
-            packet, self._tg_port_egress, self._tg_port_ingress, duration
+            packet,
+            self._tg_port_egress,
+            self._tg_port_ingress,
+            filter_config,
+            duration,
         )
 
     def get_expected_packet(self, packet: Packet) -> Packet:
diff --git a/dts/framework/testbed_model/tg_node.py b/dts/framework/testbed_model/tg_node.py
index f269d4c585..d3206e87e0 100644
--- a/dts/framework/testbed_model/tg_node.py
+++ b/dts/framework/testbed_model/tg_node.py
@@ -15,7 +15,11 @@
 
 from .node import Node
 from .port import Port
-from .traffic_generator import CapturingTrafficGenerator, create_traffic_generator
+from .traffic_generator import (
+    CapturingTrafficGenerator,
+    PacketFilteringConfig,
+    create_traffic_generator,
+)
 
 
 class TGNode(Node):
@@ -53,6 +57,7 @@ def send_packet_and_capture(
         packet: Packet,
         send_port: Port,
         receive_port: Port,
+        filter_config: PacketFilteringConfig = PacketFilteringConfig(),
         duration: float = 1,
     ) -> list[Packet]:
         """Send `packet`, return received traffic.
@@ -65,13 +70,18 @@ def send_packet_and_capture(
             packet: The packet to send.
             send_port: The egress port on the TG node.
             receive_port: The ingress port in the TG node.
+            filter_config: The filter to use when capturing packets.
             duration: Capture traffic for this amount of time after sending `packet`.
 
         Returns:
              A list of received packets. May be empty if no packets are captured.
         """
         return self.traffic_generator.send_packet_and_capture(
-            packet, send_port, receive_port, duration
+            packet,
+            send_port,
+            receive_port,
+            filter_config,
+            duration,
         )
 
     def close(self) -> None:
diff --git a/dts/framework/testbed_model/traffic_generator/__init__.py b/dts/framework/testbed_model/traffic_generator/__init__.py
index 11e2bd7d97..0eaf0355cd 100644
--- a/dts/framework/testbed_model/traffic_generator/__init__.py
+++ b/dts/framework/testbed_model/traffic_generator/__init__.py
@@ -14,11 +14,16 @@
 and a capturing traffic generator is required.
 """
 
+# pylama:ignore=W0611
+
 from framework.config import ScapyTrafficGeneratorConfig, TrafficGeneratorType
 from framework.exception import ConfigurationError
 from framework.testbed_model.node import Node
 
-from .capturing_traffic_generator import CapturingTrafficGenerator
+from .capturing_traffic_generator import (
+    CapturingTrafficGenerator,
+    PacketFilteringConfig,
+)
 from .scapy import ScapyTrafficGenerator
 
 
diff --git a/dts/framework/testbed_model/traffic_generator/capturing_traffic_generator.py b/dts/framework/testbed_model/traffic_generator/capturing_traffic_generator.py
index 0246590333..e5a1560e90 100644
--- a/dts/framework/testbed_model/traffic_generator/capturing_traffic_generator.py
+++ b/dts/framework/testbed_model/traffic_generator/capturing_traffic_generator.py
@@ -11,6 +11,7 @@
 
 import uuid
 from abc import abstractmethod
+from dataclasses import dataclass
 
 import scapy.utils  # type: ignore[import]
 from scapy.packet import Packet  # type: ignore[import]
@@ -26,6 +27,19 @@ def _get_default_capture_name() -> str:
     return str(uuid.uuid4())
 
 
+@dataclass(slots=True, frozen=True)
+class PacketFilteringConfig:
+    """The supported filtering options for :class:`CapturingTrafficGenerator`.
+
+    Attributes:
+        no_lldp: If :data:`True`, LLDP packets will be filtered out when capturing.
+        no_arp: If :data:`True`, ARP packets will be filtered out when capturing.
+    """
+
+    no_lldp: bool = True
+    no_arp: bool = True
+
+
 class CapturingTrafficGenerator(TrafficGenerator):
     """Capture packets after sending traffic.
 
@@ -54,6 +68,7 @@ def send_packet_and_capture(
         packet: Packet,
         send_port: Port,
         receive_port: Port,
+        filter_config: PacketFilteringConfig,
         duration: float,
         capture_name: str = _get_default_capture_name(),
     ) -> list[Packet]:
@@ -68,6 +83,7 @@ def send_packet_and_capture(
             packet: The packet to send.
             send_port: The egress port on the TG node.
             receive_port: The ingress port in the TG node.
+            filter_config: Filters to apply when capturing packets.
             duration: Capture traffic for this amount of time after sending the packet.
             capture_name: The name of the .pcap file where to store the capture.
 
@@ -75,7 +91,7 @@ def send_packet_and_capture(
              The received packets. May be empty if no packets are captured.
         """
         return self.send_packets_and_capture(
-            [packet], send_port, receive_port, duration, capture_name
+            [packet], send_port, receive_port, filter_config, duration, capture_name
         )
 
     def send_packets_and_capture(
@@ -83,6 +99,7 @@ def send_packets_and_capture(
         packets: list[Packet],
         send_port: Port,
         receive_port: Port,
+        filter_config: PacketFilteringConfig,
         duration: float,
         capture_name: str = _get_default_capture_name(),
     ) -> list[Packet]:
@@ -99,6 +116,7 @@ def send_packets_and_capture(
             packets: The packets to send.
             send_port: The egress port on the TG node.
             receive_port: The ingress port in the TG node.
+            filter_config: Filters to apply when capturing packets.
             duration: Capture traffic for this amount of time after sending the packets.
             capture_name: The name of the .pcap file where to store the capture.
 
@@ -113,6 +131,7 @@ def send_packets_and_capture(
             packets,
             send_port,
             receive_port,
+            filter_config,
             duration,
         )
 
@@ -126,6 +145,7 @@ def _send_packets_and_capture(
         packets: list[Packet],
         send_port: Port,
         receive_port: Port,
+        filter_config: PacketFilteringConfig,
         duration: float,
     ) -> list[Packet]:
         """The implementation of :method:`send_packets_and_capture`.
diff --git a/dts/framework/testbed_model/traffic_generator/scapy.py b/dts/framework/testbed_model/traffic_generator/scapy.py
index 5b60f66237..df3069d516 100644
--- a/dts/framework/testbed_model/traffic_generator/scapy.py
+++ b/dts/framework/testbed_model/traffic_generator/scapy.py
@@ -32,6 +32,7 @@
 
 from .capturing_traffic_generator import (
     CapturingTrafficGenerator,
+    PacketFilteringConfig,
     _get_default_capture_name,
 )
 
@@ -69,6 +70,7 @@ def scapy_send_packets_and_capture(
     send_iface: str,
     recv_iface: str,
     duration: float,
+    sniff_filter: str,
 ) -> list[bytes]:
     """The RPC function to send and capture packets.
 
@@ -90,6 +92,7 @@ def scapy_send_packets_and_capture(
         iface=recv_iface,
         store=True,
         started_callback=lambda *args: scapy.all.sendp(scapy_packets, iface=send_iface),
+        filter=sniff_filter,
     )
     sniffer.start()
     time.sleep(duration)
@@ -260,11 +263,34 @@ def _send_packets(self, packets: list[Packet], port: Port) -> None:
         packets = [packet.build() for packet in packets]
         self.rpc_server_proxy.scapy_send_packets(packets, port.logical_name)
 
+    def _create_packet_filter(self, filter_config: PacketFilteringConfig) -> str:
+        """Combines filter settings from `filter_config` into a BPF that scapy can use.
+
+        Scapy allows for the use of Berkeley Packet Filters (BPFs) to filter what packets are
+        collected based on various attributes of the packet.
+
+        Args:
+            filter_config: Config class that specifies which filters should be applied.
+
+        Returns:
+            A string representing the combination of BPF filters to be passed to scapy. For
+            example:
+
+            "ether[12:2] != 0x88cc && ether[12:2] != 0x0806"
+        """
+        bpf_filter = []
+        if filter_config.no_arp:
+            bpf_filter.append("ether[12:2] != 0x0806")
+        if filter_config.no_lldp:
+            bpf_filter.append("ether[12:2] != 0x88cc")
+        return " && ".join(bpf_filter)
+
     def _send_packets_and_capture(
         self,
         packets: list[Packet],
         send_port: Port,
         receive_port: Port,
+        filter_config: PacketFilteringConfig,
         duration: float,
         capture_name: str = _get_default_capture_name(),
     ) -> list[Packet]:
@@ -277,6 +303,7 @@ def _send_packets_and_capture(
             send_port.logical_name,
             receive_port.logical_name,
             duration,
+            self._create_packet_filter(filter_config),
         )  # type: ignore[assignment]
 
         scapy_packets = [Ether(packet.data) for packet in xmlrpc_packets]
-- 
2.43.0

[PATCH v7 4/7] dts: add pci addresses to EAL parameters

[jspewock]

From: Jeremy Spewock <jspewock@iol.unh.edu>

Added allow list to the EAL parameters created in DTS to ensure that
only the relevant PCI devices are considered when launching DPDK
applications.

Signed-off-by: Jeremy Spewock <jspewock@iol.unh.edu>
---
 dts/framework/testbed_model/sut_node.py | 12 ++++++++++++
 1 file changed, 12 insertions(+)

diff --git a/dts/framework/testbed_model/sut_node.py b/dts/framework/testbed_model/sut_node.py
index 909394e756..97aa26d419 100644
--- a/dts/framework/testbed_model/sut_node.py
+++ b/dts/framework/testbed_model/sut_node.py
@@ -30,6 +30,7 @@
 from .cpu import LogicalCoreCount, LogicalCoreList
 from .node import Node
 from .os_session import InteractiveShellType, OSSession
+from .port import Port
 from .virtual_device import VirtualDevice
 
 
@@ -46,6 +47,7 @@ def __init__(
         prefix: str,
         no_pci: bool,
         vdevs: list[VirtualDevice],
+        ports: list[Port],
         other_eal_param: str,
     ):
         """Initialize the parameters according to inputs.
@@ -63,6 +65,7 @@ def __init__(
                     VirtualDevice('net_ring0'),
                     VirtualDevice('net_ring1')
                 ]
+            ports: The list of ports to allow.
             other_eal_param: user defined DPDK EAL parameters, e.g.:
                 ``other_eal_param='--single-file-segments'``
         """
@@ -73,6 +76,7 @@ def __init__(
             self._prefix = f"--file-prefix={prefix}"
         self._no_pci = "--no-pci" if no_pci else ""
         self._vdevs = " ".join(f"--vdev {vdev}" for vdev in vdevs)
+        self._ports = " ".join(f"-a {port.pci}" for port in ports)
         self._other_eal_param = other_eal_param
 
     def __str__(self) -> str:
@@ -83,6 +87,7 @@ def __str__(self) -> str:
             f"{self._prefix} "
             f"{self._no_pci} "
             f"{self._vdevs} "
+            f"{self._ports} "
             f"{self._other_eal_param}"
         )
 
@@ -347,6 +352,7 @@ def create_eal_parameters(
         append_prefix_timestamp: bool = True,
         no_pci: bool = False,
         vdevs: list[VirtualDevice] | None = None,
+        ports: list[Port] | None = None,
         other_eal_param: str = "",
     ) -> "EalParameters":
         """Compose the EAL parameters.
@@ -370,6 +376,8 @@ def create_eal_parameters(
                     VirtualDevice('net_ring0'),
                     VirtualDevice('net_ring1')
                 ]
+            ports: The list of ports to allow. If :data:`None`, all ports listed in `self.ports`
+                will be allowed.
             other_eal_param: user defined DPDK EAL parameters, e.g.:
                 ``other_eal_param='--single-file-segments'``.
 
@@ -388,12 +396,16 @@ def create_eal_parameters(
         if vdevs is None:
             vdevs = []
 
+        if ports is None:
+            ports = self.ports
+
         return EalParameters(
             lcore_list=lcore_list,
             memory_channels=self.config.memory_channels,
             prefix=prefix,
             no_pci=no_pci,
             vdevs=vdevs,
+            ports=ports,
             other_eal_param=other_eal_param,
         )
 
-- 
2.43.0

[PATCH v7 5/7] dts: allow configuring MTU of ports

[jspewock]

From: Jeremy Spewock <jspewock@iol.unh.edu>

Adds methods in both os_session and linux session to allow for setting
MTU of port interfaces so that suites that require the sending and
receiving of packets of a specific size, or the rejection of packets
over a certain size, can configure this maximum as needed.

Signed-off-by: Jeremy Spewock <jspewock@iol.unh.edu>
---
 dts/framework/testbed_model/linux_session.py | 8 ++++++++
 dts/framework/testbed_model/os_session.py    | 9 +++++++++
 2 files changed, 17 insertions(+)

diff --git a/dts/framework/testbed_model/linux_session.py b/dts/framework/testbed_model/linux_session.py
index 0ab59cef85..5d24030c3d 100644
--- a/dts/framework/testbed_model/linux_session.py
+++ b/dts/framework/testbed_model/linux_session.py
@@ -198,6 +198,14 @@ def configure_port_ip_address(
             verify=True,
         )
 
+    def configure_port_mtu(self, mtu: int, port: Port) -> None:
+        """Overrides :meth:`~.os_session.OSSession.configure_port_mtu`."""
+        self.send_command(
+            f"ip link set dev {port.logical_name} mtu {mtu}",
+            privileged=True,
+            verify=True,
+        )
+
     def configure_ipv4_forwarding(self, enable: bool) -> None:
         """Overrides :meth:`~.os_session.OSSession.configure_ipv4_forwarding`."""
         state = 1 if enable else 0
diff --git a/dts/framework/testbed_model/os_session.py b/dts/framework/testbed_model/os_session.py
index ac6bb5e112..e42f4d752a 100644
--- a/dts/framework/testbed_model/os_session.py
+++ b/dts/framework/testbed_model/os_session.py
@@ -413,6 +413,15 @@ def configure_port_ip_address(
             delete: If :data:`True`, remove the IP address, otherwise configure it.
         """
 
+    @abstractmethod
+    def configure_port_mtu(self, mtu: int, port: Port) -> None:
+        """Configure `mtu` on `port`.
+
+        Args:
+            mtu: Desired MTU value.
+            port: Port to set `mtu` on.
+        """
+
     @abstractmethod
     def configure_ipv4_forwarding(self, enable: bool) -> None:
         """Enable IPv4 forwarding in the operating system.
-- 
2.43.0

[PATCH v7 6/7] dts: add scatter to the yaml schema

[jspewock]

From: Jeremy Spewock <jspewock@iol.unh.edu>

Allow for scatter to be specified in the configuration file.

Signed-off-by: Jeremy Spewock <jspewock@iol.unh.edu>
---
 dts/framework/config/conf_yaml_schema.json | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)

diff --git a/dts/framework/config/conf_yaml_schema.json b/dts/framework/config/conf_yaml_schema.json
index 84e45fe3c2..e6dc50ca7f 100644
--- a/dts/framework/config/conf_yaml_schema.json
+++ b/dts/framework/config/conf_yaml_schema.json
@@ -186,7 +186,8 @@
       "type": "string",
       "enum": [
         "hello_world",
-        "os_udp"
+        "os_udp",
+        "pmd_buffer_scatter"
       ]
     },
     "test_target": {
-- 
2.43.0

[PATCH v7 7/7] dts: add pmd_buffer_scatter test suite

[jspewock]

From: Jeremy Spewock <jspewock@iol.unh.edu>

This test suite provides testing of the support of scattered packets by
Poll Mode Drivers using testpmd, verifying the ability to receive and
transmit scattered multi-segment packets made up of multiple
non-contiguous memory buffers. This is tested through 5 different cases
in which the length of the packets sent are less than the mbuf size,
equal to the mbuf size, and 1, 4, and 5 bytes greater than the mbuf size
in order to show both the CRC and the packet data are capable of
existing in the first, second, or both buffers.

Naturally, if the PMD is capable of forwarding scattered packets which
it receives as input, this shows it is capable of both receiving and
transmitting scattered packets.

Signed-off-by: Jeremy Spewock <jspewock@iol.unh.edu>
---
 dts/tests/TestSuite_pmd_buffer_scatter.py | 132 ++++++++++++++++++++++
 1 file changed, 132 insertions(+)
 create mode 100644 dts/tests/TestSuite_pmd_buffer_scatter.py

diff --git a/dts/tests/TestSuite_pmd_buffer_scatter.py b/dts/tests/TestSuite_pmd_buffer_scatter.py
new file mode 100644
index 0000000000..9ce1bacabf
--- /dev/null
+++ b/dts/tests/TestSuite_pmd_buffer_scatter.py
@@ -0,0 +1,132 @@
+# SPDX-License-Identifier: BSD-3-Clause
+# Copyright(c) 2023-2024 University of New Hampshire
+
+"""Multi-segment packet scattering testing suite.
+
+This testing suite tests the support of transmitting and receiving scattered packets. This is shown
+by the Poll Mode Driver being able to forward scattered multi-segment packets composed of multiple
+non-contiguous memory buffers. To ensure the receipt of scattered packets, the DMA rings of the
+port's RX queues must be configured with mbuf data buffers whose size is less than the maximum
+length.
+
+If it is the case that the Poll Mode Driver can forward scattered packets which it receives, then
+this suffices to show the Poll Mode Driver is capable of both receiving and transmitting scattered
+packets.
+"""
+
+import struct
+
+from scapy.layers.inet import IP  # type: ignore[import]
+from scapy.layers.l2 import Ether  # type: ignore[import]
+from scapy.packet import Raw  # type: ignore[import]
+from scapy.utils import hexstr  # type: ignore[import]
+
+from framework.remote_session.testpmd_shell import TestPmdForwardingModes, TestPmdShell
+from framework.test_suite import TestSuite
+
+
+class PmdBufferScatter(TestSuite):
+    """DPDK PMD packet scattering test suite.
+
+    Configure the Rx queues to have mbuf data buffers whose sizes are smaller than the maximum
+    packet size. Specifically, set mbuf data buffers to have a size of 2048 to fit a full 1512-byte
+    (CRC included) ethernet frame in a mono-segment packet. The testing of scattered packets is
+    done by sending a packet whose length is greater than the size of the configured size of mbuf
+    data buffers. There are a total of 5 packets sent within test cases which have lengths less
+    than, equal to, and greater than the mbuf size. There are multiple packets sent with lengths
+    greater than the mbuf size in order to test cases such as:
+
+    1. A single byte of the CRC being in a second buffer while the remaining 3 bytes are stored in
+        the first buffer alongside packet data.
+    2. The entire CRC being stored in a second buffer while all of the packet data is stored in the
+        first.
+    3. Most of the packet data being stored in the first buffer and a single byte of packet data
+        stored in a second buffer alongside the CRC.
+    """
+
+    def set_up_suite(self) -> None:
+        """Set up the test suite.
+
+        Setup:
+            Verify that we have at least 2 port links in the current execution and increase the MTU
+            of both ports on the traffic generator to 9000 to support larger packet sizes.
+        """
+        self.verify(
+            len(self._port_links) > 1,
+            "There must be at least two port links to run the scatter test suite",
+        )
+
+        self.tg_node.main_session.configure_port_mtu(9000, self._tg_port_egress)
+        self.tg_node.main_session.configure_port_mtu(9000, self._tg_port_ingress)
+
+    def scatter_pktgen_send_packet(self, pktsize: int) -> str:
+        """Generate and send a packet to the SUT then capture what is forwarded back.
+
+        Generate an IP packet of a specific length and send it to the SUT, then capture the
+        resulting received packet and extract its payload. The desired length of the packet is met
+        by packing its payload with the letter "X" in hexadecimal.
+
+        Args:
+            pktsize: Size of the packet to generate and send.
+
+        Returns:
+            The payload of the received packet as a string.
+        """
+        packet = Ether() / IP() / Raw()
+        packet.getlayer(2).load = ""
+        payload_len = pktsize - len(packet) - 4
+        payload = ["58"] * payload_len
+        # pack the payload
+        for X_in_hex in payload:
+            packet.load += struct.pack("=B", int("%s%s" % (X_in_hex[0], X_in_hex[1]), 16))
+        received_packets = self.send_packet_and_capture(packet)
+        self.verify(len(received_packets) > 0, "Did not receive any packets.")
+        load = hexstr(received_packets[0].getlayer(2), onlyhex=1)
+
+        return load
+
+    def pmd_scatter(self, mbsize: int) -> None:
+        """Testpmd support of receiving and sending scattered multi-segment packets.
+
+        Support for scattered packets is shown by sending 5 packets of differing length
+        where the length of the packet is calculated by taking mbuf-size + an offset. The
+        offsets used in the test are -1, 0, 1, 4, 5 respectively.
+
+        Test:
+            Start testpmd and run functional test with preset mbsize.
+        """
+        testpmd = self.sut_node.create_interactive_shell(
+            TestPmdShell,
+            app_parameters=(
+                "--mbcache=200 "
+                f"--mbuf-size={mbsize} "
+                "--max-pkt-len=9000 "
+                "--port-topology=paired "
+                "--tx-offloads=0x00008000"
+            ),
+            privileged=True,
+        )
+        testpmd.set_forward_mode(TestPmdForwardingModes.mac)
+        testpmd.start()
+
+        for offset in [-1, 0, 1, 4, 5]:
+            recv_payload = self.scatter_pktgen_send_packet(mbsize + offset)
+            self._logger.debug(f"Payload of scattered packet after forwarding: \n{recv_payload}")
+            self.verify(
+                ("58 " * 8).strip() in recv_payload,
+                f"Payload of scattered packet did not match expected payload with offset {offset}.",
+            )
+        testpmd.stop()
+
+    def test_scatter_mbuf_2048(self) -> None:
+        """Run the pmd_scatter test with `mbsize` set to 2048."""
+        self.pmd_scatter(mbsize=2048)
+
+    def tear_down_suite(self) -> None:
+        """Tear down the test suite.
+
+        Teardown:
+            Set the MTU of the tg_node back to a more standard size of 1500
+        """
+        self.tg_node.main_session.configure_port_mtu(1500, self._tg_port_egress)
+        self.tg_node.main_session.configure_port_mtu(1500, self._tg_port_ingress)
-- 
2.43.0

Re: [PATCH v7 7/7] dts: add pmd_buffer_scatter test suite

[Juraj Linkeš]

<snip>

> +    def test_scatter_mbuf_2048(self) -> None:
> +        """Run the pmd_scatter test with `mbsize` set to 2048."""

I've used a shortcut, I liked the reference :-). Looking at it, it
could be simplified using :meth:
"""Run the :meth:`pmd_scatter` test with `mbsize` set to 2048."""

Re: [PATCH v7 7/7] dts: add pmd_buffer_scatter test suite

[Jeremy Spewock]

[-- Attachment #1: Type: text/plain, Size: 464 bytes --]

On Wed, Jan 10, 2024 at 8:16 AM Juraj Linkeš <juraj.linkes@pantheon.tech>
wrote:

> <snip>
>
> > +    def test_scatter_mbuf_2048(self) -> None:
> > +        """Run the pmd_scatter test with `mbsize` set to 2048."""
>
> I've used a shortcut, I liked the reference :-). Looking at it, it
> could be simplified using :meth:
> """Run the :meth:`pmd_scatter` test with `mbsize` set to 2048."""
>

My mistake, I took it too literally. I'll update this.

[-- Attachment #2: Type: text/html, Size: 983 bytes --]

Re: [PATCH v7 0/7] dts: Port scatter suite over

[Juraj Linkeš]

I only had two minor comments in two of the patches.
Other than that. for the whole series:
Reviewed-by: Juraj Linkeš <juraj.linkes@pantheon.tech>


On Tue, Jan 9, 2024 at 4:36 PM <jspewock@iol.unh.edu> wrote:
>
> From: Jeremy Spewock <jspewock@iol.unh.edu>
>
> v7:
>
> Addressed comments and made appropriate changes. Most changes had to do
> with grammatical/spelling errors and expanding upon documentation. Also
> fixed the typing of eal parameters when creating interactive shells
> through the SUT node.
>
> Jeremy Spewock (7):
>   dts: add startup verification and forwarding modes to testpmd shell
>   dts: limit EAL parameters to DPDK apps and add parameters to all apps
>   dts: add optional packet filtering to scapy sniffer
>   dts: add pci addresses to EAL parameters
>   dts: allow configuring MTU of ports
>   dts: add scatter to the yaml schema
>   dts: add pmd_buffer_scatter test suite
>
>  dts/framework/config/conf_yaml_schema.json    |   3 +-
>  dts/framework/exception.py                    |   7 +
>  dts/framework/remote_session/testpmd_shell.py | 149 +++++++++++++++++-
>  dts/framework/test_suite.py                   |  15 +-
>  dts/framework/testbed_model/linux_session.py  |   8 +
>  dts/framework/testbed_model/os_session.py     |   9 ++
>  dts/framework/testbed_model/sut_node.py       |  28 +++-
>  dts/framework/testbed_model/tg_node.py        |  14 +-
>  .../traffic_generator/__init__.py             |   7 +-
>  .../capturing_traffic_generator.py            |  22 ++-
>  .../testbed_model/traffic_generator/scapy.py  |  27 ++++
>  dts/tests/TestSuite_pmd_buffer_scatter.py     | 132 ++++++++++++++++
>  12 files changed, 407 insertions(+), 14 deletions(-)
>  create mode 100644 dts/tests/TestSuite_pmd_buffer_scatter.py
>
> --
> 2.43.0
>

[PATCH v8 0/7] dts: Port scatter suite over

[jspewock]

From: Jeremy Spewock <jspewock@iol.unh.edu>

v8:

Address comments by making minor docstring adjustments.

Jeremy Spewock (7):
  dts: add startup verification and forwarding modes to testpmd shell
  dts: limit EAL parameters to DPDK apps and add parameters to all apps
  dts: add optional packet filtering to scapy sniffer
  dts: add pci addresses to EAL parameters
  dts: allow configuring MTU of ports
  dts: add scatter to the yaml schema
  dts: add pmd_buffer_scatter test suite

 dts/framework/config/conf_yaml_schema.json    |   3 +-
 dts/framework/exception.py                    |   7 +
 dts/framework/remote_session/testpmd_shell.py | 149 +++++++++++++++++-
 dts/framework/test_suite.py                   |  15 +-
 dts/framework/testbed_model/linux_session.py  |   8 +
 dts/framework/testbed_model/os_session.py     |   9 ++
 dts/framework/testbed_model/sut_node.py       |  28 +++-
 dts/framework/testbed_model/tg_node.py        |  14 +-
 .../traffic_generator/__init__.py             |   7 +-
 .../capturing_traffic_generator.py            |  22 ++-
 .../testbed_model/traffic_generator/scapy.py  |  27 ++++
 dts/tests/TestSuite_pmd_buffer_scatter.py     | 132 ++++++++++++++++
 12 files changed, 407 insertions(+), 14 deletions(-)
 create mode 100644 dts/tests/TestSuite_pmd_buffer_scatter.py

-- 
2.43.0

[PATCH v8 1/7] dts: add startup verification and forwarding modes to testpmd shell

[jspewock]

From: Jeremy Spewock <jspewock@iol.unh.edu>

Added commonly used methods in testpmd such as starting and stopping
packet forwarding, changing forward modes, and verifying link status of
ports so that developers can configure testpmd and start forwarding
through the provided class rather than sending commands to the testpmd
session directly.

Signed-off-by: Jeremy Spewock <jspewock@iol.unh.edu>
---
 dts/framework/exception.py                    |   7 +
 dts/framework/remote_session/testpmd_shell.py | 149 +++++++++++++++++-
 2 files changed, 155 insertions(+), 1 deletion(-)

diff --git a/dts/framework/exception.py b/dts/framework/exception.py
index 658eee2c38..cce1e0231a 100644
--- a/dts/framework/exception.py
+++ b/dts/framework/exception.py
@@ -146,6 +146,13 @@ def __str__(self) -> str:
         return f"Command {self.command} returned a non-zero exit code: {self._command_return_code}"
 
 
+class InteractiveCommandExecutionError(DTSError):
+    """An unsuccessful execution of a remote command in an interactive environment."""
+
+    #:
+    severity: ClassVar[ErrorSeverity] = ErrorSeverity.REMOTE_CMD_EXEC_ERR
+
+
 class RemoteDirectoryExistsError(DTSError):
     """A directory that exists on a remote node."""
 
diff --git a/dts/framework/remote_session/testpmd_shell.py b/dts/framework/remote_session/testpmd_shell.py
index 0184cc2e71..3a66907394 100644
--- a/dts/framework/remote_session/testpmd_shell.py
+++ b/dts/framework/remote_session/testpmd_shell.py
@@ -15,9 +15,15 @@
     testpmd_shell.close()
 """
 
+import time
+from enum import auto
 from pathlib import PurePath
 from typing import Callable, ClassVar
 
+from framework.exception import InteractiveCommandExecutionError
+from framework.settings import SETTINGS
+from framework.utils import StrEnum
+
 from .interactive_shell import InteractiveShell
 
 
@@ -43,14 +49,51 @@ def __str__(self) -> str:
         return self.pci_address
 
 
+class TestPmdForwardingModes(StrEnum):
+    r"""The supported packet forwarding modes for :class:`~TestPmdShell`\s."""
+
+    #:
+    io = auto()
+    #:
+    mac = auto()
+    #:
+    macswap = auto()
+    #:
+    flowgen = auto()
+    #:
+    rxonly = auto()
+    #:
+    txonly = auto()
+    #:
+    csum = auto()
+    #:
+    icmpecho = auto()
+    #:
+    ieee1588 = auto()
+    #:
+    noisy = auto()
+    #:
+    fivetswap = "5tswap"
+    #:
+    shared_rxq = "shared-rxq"
+    #:
+    recycle_mbufs = auto()
+
+
 class TestPmdShell(InteractiveShell):
     """Testpmd interactive shell.
 
     The testpmd shell users should never use
     the :meth:`~.interactive_shell.InteractiveShell.send_command` method directly, but rather
     call specialized methods. If there isn't one that satisfies a need, it should be added.
+
+    Attributes:
+        number_of_ports: The number of ports which were allowed on the command-line when testpmd
+            was started.
     """
 
+    number_of_ports: int
+
     #: The path to the testpmd executable.
     path: ClassVar[PurePath] = PurePath("app", "dpdk-testpmd")
 
@@ -65,9 +108,66 @@ class TestPmdShell(InteractiveShell):
     _command_extra_chars: ClassVar[str] = "\n"
 
     def _start_application(self, get_privileged_command: Callable[[str], str] | None) -> None:
-        self._app_args += " -- -i"
+        """Overrides :meth:`~.interactive_shell._start_application`.
+
+        Add flags for starting testpmd in interactive mode and disabling messages for link state
+        change events before starting the application. Link state is verified before starting
+        packet forwarding and the messages create unexpected newlines in the terminal which
+        complicates output collection.
+
+        Also find the number of pci addresses which were allowed on the command line when the app
+        was started.
+        """
+        self._app_args += " -- -i --mask-event intr_lsc"
+        self.number_of_ports = self._app_args.count("-a ")
         super()._start_application(get_privileged_command)
 
+    def start(self, verify: bool = True) -> None:
+        """Start packet forwarding with the current configuration.
+
+        Args:
+            verify: If :data:`True` , a second start command will be sent in an attempt to verify
+                packet forwarding started as expected.
+
+        Raises:
+            InteractiveCommandExecutionError: If `verify` is :data:`True` and forwarding fails to
+                start or ports fail to come up.
+        """
+        self.send_command("start")
+        if verify:
+            # If forwarding was already started, sending "start" again should tell us
+            start_cmd_output = self.send_command("start")
+            if "Packet forwarding already started" not in start_cmd_output:
+                self._logger.debug(f"Failed to start packet forwarding: \n{start_cmd_output}")
+                raise InteractiveCommandExecutionError("Testpmd failed to start packet forwarding.")
+
+            for port_id in range(self.number_of_ports):
+                if not self.wait_link_status_up(port_id):
+                    raise InteractiveCommandExecutionError(
+                        "Not all ports came up after starting packet forwarding in testpmd."
+                    )
+
+    def stop(self, verify: bool = True) -> None:
+        """Stop packet forwarding.
+
+        Args:
+            verify: If :data:`True` , the output of the stop command is scanned to verify that
+                forwarding was stopped successfully or not started. If neither is found, it is
+                considered an error.
+
+        Raises:
+            InteractiveCommandExecutionError: If `verify` is :data:`True` and the command to stop
+                forwarding results in an error.
+        """
+        stop_cmd_output = self.send_command("stop")
+        if verify:
+            if (
+                "Done." not in stop_cmd_output
+                and "Packet forwarding not started" not in stop_cmd_output
+            ):
+                self._logger.debug(f"Failed to stop packet forwarding: \n{stop_cmd_output}")
+                raise InteractiveCommandExecutionError("Testpmd failed to stop packet forwarding.")
+
     def get_devices(self) -> list[TestPmdDevice]:
         """Get a list of device names that are known to testpmd.
 
@@ -82,3 +182,50 @@ def get_devices(self) -> list[TestPmdDevice]:
             if "device name:" in line.lower():
                 dev_list.append(TestPmdDevice(line))
         return dev_list
+
+    def wait_link_status_up(self, port_id: int, timeout=SETTINGS.timeout) -> bool:
+        """Wait until the link status on the given port is "up".
+
+        Arguments:
+            port_id: Port to check the link status on.
+            timeout: Time to wait for the link to come up. The default value for this
+                argument may be modified using the :option:`--timeout` command-line argument
+                or the :envvar:`DTS_TIMEOUT` environment variable.
+
+        Returns:
+            Whether the link came up in time or not.
+        """
+        time_to_stop = time.time() + timeout
+        port_info: str = ""
+        while time.time() < time_to_stop:
+            port_info = self.send_command(f"show port info {port_id}")
+            if "Link status: up" in port_info:
+                break
+            time.sleep(0.5)
+        else:
+            self._logger.error(f"The link for port {port_id} did not come up in the given timeout.")
+        return "Link status: up" in port_info
+
+    def set_forward_mode(self, mode: TestPmdForwardingModes, verify: bool = True):
+        """Set packet forwarding mode.
+
+        Args:
+            mode: The forwarding mode to use.
+            verify: If :data:`True` the output of the command will be scanned in an attempt to
+                verify that the forwarding mode was set to `mode` properly.
+
+        Raises:
+            InteractiveCommandExecutionError: If `verify` is :data:`True` and the forwarding mode
+                fails to update.
+        """
+        set_fwd_output = self.send_command(f"set fwd {mode.value}")
+        if f"Set {mode.value} packet forwarding mode" not in set_fwd_output:
+            self._logger.debug(f"Failed to set fwd mode to {mode.value}:\n{set_fwd_output}")
+            raise InteractiveCommandExecutionError(
+                f"Test pmd failed to set fwd mode to {mode.value}"
+            )
+
+    def close(self) -> None:
+        """Overrides :meth:`~.interactive_shell.close`."""
+        self.send_command("quit", "")
+        return super().close()
-- 
2.43.0

[PATCH v8 2/7] dts: limit EAL parameters to DPDK apps and add parameters to all apps

[jspewock]

From: Jeremy Spewock <jspewock@iol.unh.edu>

Changed the factory method for creating interactive apps in the SUT Node
so that EAL parameters would only be passed into DPDK apps since
non-DPDK apps wouldn't be able to process them. Also modified
interactive apps to allow for the ability to pass parameters into the
app on startup so that the applications can be started with certain
configuration steps passed on the command line.

Signed-off-by: Jeremy Spewock <jspewock@iol.unh.edu>
---
 dts/framework/remote_session/testpmd_shell.py |  2 +-
 dts/framework/testbed_model/sut_node.py       | 16 ++++++++++------
 2 files changed, 11 insertions(+), 7 deletions(-)

diff --git a/dts/framework/remote_session/testpmd_shell.py b/dts/framework/remote_session/testpmd_shell.py
index 3a66907394..cb2ab6bd00 100644
--- a/dts/framework/remote_session/testpmd_shell.py
+++ b/dts/framework/remote_session/testpmd_shell.py
@@ -118,7 +118,7 @@ def _start_application(self, get_privileged_command: Callable[[str], str] | None
         Also find the number of pci addresses which were allowed on the command line when the app
         was started.
         """
-        self._app_args += " -- -i --mask-event intr_lsc"
+        self._app_args += " -i --mask-event intr_lsc"
         self.number_of_ports = self._app_args.count("-a ")
         super()._start_application(get_privileged_command)
 
diff --git a/dts/framework/testbed_model/sut_node.py b/dts/framework/testbed_model/sut_node.py
index c4acea38d1..909394e756 100644
--- a/dts/framework/testbed_model/sut_node.py
+++ b/dts/framework/testbed_model/sut_node.py
@@ -430,7 +430,8 @@ def create_interactive_shell(
         shell_cls: Type[InteractiveShellType],
         timeout: float = SETTINGS.timeout,
         privileged: bool = False,
-        eal_parameters: EalParameters | str | None = None,
+        app_parameters: str = "",
+        eal_parameters: EalParameters | None = None,
     ) -> InteractiveShellType:
         """Extend the factory for interactive session handlers.
 
@@ -449,20 +450,23 @@ def create_interactive_shell(
             eal_parameters: List of EAL parameters to use to launch the app. If this
                 isn't provided or an empty string is passed, it will default to calling
                 :meth:`create_eal_parameters`.
+            app_parameters: Additional arguments to pass into the application on the
+                command-line.
 
         Returns:
             An instance of the desired interactive application shell.
         """
-        if not eal_parameters:
-            eal_parameters = self.create_eal_parameters()
-
-        # We need to append the build directory for DPDK apps
+        # We need to append the build directory and add EAL parameters for DPDK apps
         if shell_cls.dpdk_app:
+            if not eal_parameters:
+                eal_parameters = self.create_eal_parameters()
+            app_parameters = f"{eal_parameters} -- {app_parameters}"
+
             shell_cls.path = self.main_session.join_remote_path(
                 self.remote_dpdk_build_dir, shell_cls.path
             )
 
-        return super().create_interactive_shell(shell_cls, timeout, privileged, str(eal_parameters))
+        return super().create_interactive_shell(shell_cls, timeout, privileged, app_parameters)
 
     def bind_ports_to_driver(self, for_dpdk: bool = True) -> None:
         """Bind all ports on the SUT to a driver.
-- 
2.43.0

[PATCH v8 3/7] dts: add optional packet filtering to scapy sniffer

[jspewock]

From: Jeremy Spewock <jspewock@iol.unh.edu>

Added the options to filter out LLDP and ARP packets when
sniffing for packets with scapy. This was done using BPF filters to
ensure that the noise these packets provide does not interfere with test
cases.

Signed-off-by: Jeremy Spewock <jspewock@iol.unh.edu>
---
 dts/framework/test_suite.py                   | 15 +++++++++--
 dts/framework/testbed_model/tg_node.py        | 14 ++++++++--
 .../traffic_generator/__init__.py             |  7 ++++-
 .../capturing_traffic_generator.py            | 22 ++++++++++++++-
 .../testbed_model/traffic_generator/scapy.py  | 27 +++++++++++++++++++
 5 files changed, 79 insertions(+), 6 deletions(-)

diff --git a/dts/framework/test_suite.py b/dts/framework/test_suite.py
index dfb391ffbd..ffea917690 100644
--- a/dts/framework/test_suite.py
+++ b/dts/framework/test_suite.py
@@ -38,6 +38,7 @@
 from .settings import SETTINGS
 from .test_result import BuildTargetResult, Result, TestCaseResult, TestSuiteResult
 from .testbed_model import Port, PortLink, SutNode, TGNode
+from .testbed_model.traffic_generator import PacketFilteringConfig
 from .utils import get_packet_summaries
 
 
@@ -208,7 +209,12 @@ def configure_testbed_ipv4(self, restore: bool = False) -> None:
     def _configure_ipv4_forwarding(self, enable: bool) -> None:
         self.sut_node.configure_ipv4_forwarding(enable)
 
-    def send_packet_and_capture(self, packet: Packet, duration: float = 1) -> list[Packet]:
+    def send_packet_and_capture(
+        self,
+        packet: Packet,
+        filter_config: PacketFilteringConfig = PacketFilteringConfig(),
+        duration: float = 1,
+    ) -> list[Packet]:
         """Send and receive `packet` using the associated TG.
 
         Send `packet` through the appropriate interface and receive on the appropriate interface.
@@ -216,6 +222,7 @@ def send_packet_and_capture(self, packet: Packet, duration: float = 1) -> list[P
 
         Args:
             packet: The packet to send.
+            filter_config: The filter to use when capturing packets.
             duration: Capture traffic for this amount of time after sending `packet`.
 
         Returns:
@@ -223,7 +230,11 @@ def send_packet_and_capture(self, packet: Packet, duration: float = 1) -> list[P
         """
         packet = self._adjust_addresses(packet)
         return self.tg_node.send_packet_and_capture(
-            packet, self._tg_port_egress, self._tg_port_ingress, duration
+            packet,
+            self._tg_port_egress,
+            self._tg_port_ingress,
+            filter_config,
+            duration,
         )
 
     def get_expected_packet(self, packet: Packet) -> Packet:
diff --git a/dts/framework/testbed_model/tg_node.py b/dts/framework/testbed_model/tg_node.py
index f269d4c585..d3206e87e0 100644
--- a/dts/framework/testbed_model/tg_node.py
+++ b/dts/framework/testbed_model/tg_node.py
@@ -15,7 +15,11 @@
 
 from .node import Node
 from .port import Port
-from .traffic_generator import CapturingTrafficGenerator, create_traffic_generator
+from .traffic_generator import (
+    CapturingTrafficGenerator,
+    PacketFilteringConfig,
+    create_traffic_generator,
+)
 
 
 class TGNode(Node):
@@ -53,6 +57,7 @@ def send_packet_and_capture(
         packet: Packet,
         send_port: Port,
         receive_port: Port,
+        filter_config: PacketFilteringConfig = PacketFilteringConfig(),
         duration: float = 1,
     ) -> list[Packet]:
         """Send `packet`, return received traffic.
@@ -65,13 +70,18 @@ def send_packet_and_capture(
             packet: The packet to send.
             send_port: The egress port on the TG node.
             receive_port: The ingress port in the TG node.
+            filter_config: The filter to use when capturing packets.
             duration: Capture traffic for this amount of time after sending `packet`.
 
         Returns:
              A list of received packets. May be empty if no packets are captured.
         """
         return self.traffic_generator.send_packet_and_capture(
-            packet, send_port, receive_port, duration
+            packet,
+            send_port,
+            receive_port,
+            filter_config,
+            duration,
         )
 
     def close(self) -> None:
diff --git a/dts/framework/testbed_model/traffic_generator/__init__.py b/dts/framework/testbed_model/traffic_generator/__init__.py
index 11e2bd7d97..0eaf0355cd 100644
--- a/dts/framework/testbed_model/traffic_generator/__init__.py
+++ b/dts/framework/testbed_model/traffic_generator/__init__.py
@@ -14,11 +14,16 @@
 and a capturing traffic generator is required.
 """
 
+# pylama:ignore=W0611
+
 from framework.config import ScapyTrafficGeneratorConfig, TrafficGeneratorType
 from framework.exception import ConfigurationError
 from framework.testbed_model.node import Node
 
-from .capturing_traffic_generator import CapturingTrafficGenerator
+from .capturing_traffic_generator import (
+    CapturingTrafficGenerator,
+    PacketFilteringConfig,
+)
 from .scapy import ScapyTrafficGenerator
 
 
diff --git a/dts/framework/testbed_model/traffic_generator/capturing_traffic_generator.py b/dts/framework/testbed_model/traffic_generator/capturing_traffic_generator.py
index 0246590333..e5a1560e90 100644
--- a/dts/framework/testbed_model/traffic_generator/capturing_traffic_generator.py
+++ b/dts/framework/testbed_model/traffic_generator/capturing_traffic_generator.py
@@ -11,6 +11,7 @@
 
 import uuid
 from abc import abstractmethod
+from dataclasses import dataclass
 
 import scapy.utils  # type: ignore[import]
 from scapy.packet import Packet  # type: ignore[import]
@@ -26,6 +27,19 @@ def _get_default_capture_name() -> str:
     return str(uuid.uuid4())
 
 
+@dataclass(slots=True, frozen=True)
+class PacketFilteringConfig:
+    """The supported filtering options for :class:`CapturingTrafficGenerator`.
+
+    Attributes:
+        no_lldp: If :data:`True`, LLDP packets will be filtered out when capturing.
+        no_arp: If :data:`True`, ARP packets will be filtered out when capturing.
+    """
+
+    no_lldp: bool = True
+    no_arp: bool = True
+
+
 class CapturingTrafficGenerator(TrafficGenerator):
     """Capture packets after sending traffic.
 
@@ -54,6 +68,7 @@ def send_packet_and_capture(
         packet: Packet,
         send_port: Port,
         receive_port: Port,
+        filter_config: PacketFilteringConfig,
         duration: float,
         capture_name: str = _get_default_capture_name(),
     ) -> list[Packet]:
@@ -68,6 +83,7 @@ def send_packet_and_capture(
             packet: The packet to send.
             send_port: The egress port on the TG node.
             receive_port: The ingress port in the TG node.
+            filter_config: Filters to apply when capturing packets.
             duration: Capture traffic for this amount of time after sending the packet.
             capture_name: The name of the .pcap file where to store the capture.
 
@@ -75,7 +91,7 @@ def send_packet_and_capture(
              The received packets. May be empty if no packets are captured.
         """
         return self.send_packets_and_capture(
-            [packet], send_port, receive_port, duration, capture_name
+            [packet], send_port, receive_port, filter_config, duration, capture_name
         )
 
     def send_packets_and_capture(
@@ -83,6 +99,7 @@ def send_packets_and_capture(
         packets: list[Packet],
         send_port: Port,
         receive_port: Port,
+        filter_config: PacketFilteringConfig,
         duration: float,
         capture_name: str = _get_default_capture_name(),
     ) -> list[Packet]:
@@ -99,6 +116,7 @@ def send_packets_and_capture(
             packets: The packets to send.
             send_port: The egress port on the TG node.
             receive_port: The ingress port in the TG node.
+            filter_config: Filters to apply when capturing packets.
             duration: Capture traffic for this amount of time after sending the packets.
             capture_name: The name of the .pcap file where to store the capture.
 
@@ -113,6 +131,7 @@ def send_packets_and_capture(
             packets,
             send_port,
             receive_port,
+            filter_config,
             duration,
         )
 
@@ -126,6 +145,7 @@ def _send_packets_and_capture(
         packets: list[Packet],
         send_port: Port,
         receive_port: Port,
+        filter_config: PacketFilteringConfig,
         duration: float,
     ) -> list[Packet]:
         """The implementation of :method:`send_packets_and_capture`.
diff --git a/dts/framework/testbed_model/traffic_generator/scapy.py b/dts/framework/testbed_model/traffic_generator/scapy.py
index 5b60f66237..df3069d516 100644
--- a/dts/framework/testbed_model/traffic_generator/scapy.py
+++ b/dts/framework/testbed_model/traffic_generator/scapy.py
@@ -32,6 +32,7 @@
 
 from .capturing_traffic_generator import (
     CapturingTrafficGenerator,
+    PacketFilteringConfig,
     _get_default_capture_name,
 )
 
@@ -69,6 +70,7 @@ def scapy_send_packets_and_capture(
     send_iface: str,
     recv_iface: str,
     duration: float,
+    sniff_filter: str,
 ) -> list[bytes]:
     """The RPC function to send and capture packets.
 
@@ -90,6 +92,7 @@ def scapy_send_packets_and_capture(
         iface=recv_iface,
         store=True,
         started_callback=lambda *args: scapy.all.sendp(scapy_packets, iface=send_iface),
+        filter=sniff_filter,
     )
     sniffer.start()
     time.sleep(duration)
@@ -260,11 +263,34 @@ def _send_packets(self, packets: list[Packet], port: Port) -> None:
         packets = [packet.build() for packet in packets]
         self.rpc_server_proxy.scapy_send_packets(packets, port.logical_name)
 
+    def _create_packet_filter(self, filter_config: PacketFilteringConfig) -> str:
+        """Combines filter settings from `filter_config` into a BPF that scapy can use.
+
+        Scapy allows for the use of Berkeley Packet Filters (BPFs) to filter what packets are
+        collected based on various attributes of the packet.
+
+        Args:
+            filter_config: Config class that specifies which filters should be applied.
+
+        Returns:
+            A string representing the combination of BPF filters to be passed to scapy. For
+            example:
+
+            "ether[12:2] != 0x88cc && ether[12:2] != 0x0806"
+        """
+        bpf_filter = []
+        if filter_config.no_arp:
+            bpf_filter.append("ether[12:2] != 0x0806")
+        if filter_config.no_lldp:
+            bpf_filter.append("ether[12:2] != 0x88cc")
+        return " && ".join(bpf_filter)
+
     def _send_packets_and_capture(
         self,
         packets: list[Packet],
         send_port: Port,
         receive_port: Port,
+        filter_config: PacketFilteringConfig,
         duration: float,
         capture_name: str = _get_default_capture_name(),
     ) -> list[Packet]:
@@ -277,6 +303,7 @@ def _send_packets_and_capture(
             send_port.logical_name,
             receive_port.logical_name,
             duration,
+            self._create_packet_filter(filter_config),
         )  # type: ignore[assignment]
 
         scapy_packets = [Ether(packet.data) for packet in xmlrpc_packets]
-- 
2.43.0

[PATCH v8 4/7] dts: add pci addresses to EAL parameters

[jspewock]

From: Jeremy Spewock <jspewock@iol.unh.edu>

Added allow list to the EAL parameters created in DTS to ensure that
only the relevant PCI devices are considered when launching DPDK
applications.

Signed-off-by: Jeremy Spewock <jspewock@iol.unh.edu>
---
 dts/framework/testbed_model/sut_node.py | 12 ++++++++++++
 1 file changed, 12 insertions(+)

diff --git a/dts/framework/testbed_model/sut_node.py b/dts/framework/testbed_model/sut_node.py
index 909394e756..97aa26d419 100644
--- a/dts/framework/testbed_model/sut_node.py
+++ b/dts/framework/testbed_model/sut_node.py
@@ -30,6 +30,7 @@
 from .cpu import LogicalCoreCount, LogicalCoreList
 from .node import Node
 from .os_session import InteractiveShellType, OSSession
+from .port import Port
 from .virtual_device import VirtualDevice
 
 
@@ -46,6 +47,7 @@ def __init__(
         prefix: str,
         no_pci: bool,
         vdevs: list[VirtualDevice],
+        ports: list[Port],
         other_eal_param: str,
     ):
         """Initialize the parameters according to inputs.
@@ -63,6 +65,7 @@ def __init__(
                     VirtualDevice('net_ring0'),
                     VirtualDevice('net_ring1')
                 ]
+            ports: The list of ports to allow.
             other_eal_param: user defined DPDK EAL parameters, e.g.:
                 ``other_eal_param='--single-file-segments'``
         """
@@ -73,6 +76,7 @@ def __init__(
             self._prefix = f"--file-prefix={prefix}"
         self._no_pci = "--no-pci" if no_pci else ""
         self._vdevs = " ".join(f"--vdev {vdev}" for vdev in vdevs)
+        self._ports = " ".join(f"-a {port.pci}" for port in ports)
         self._other_eal_param = other_eal_param
 
     def __str__(self) -> str:
@@ -83,6 +87,7 @@ def __str__(self) -> str:
             f"{self._prefix} "
             f"{self._no_pci} "
             f"{self._vdevs} "
+            f"{self._ports} "
             f"{self._other_eal_param}"
         )
 
@@ -347,6 +352,7 @@ def create_eal_parameters(
         append_prefix_timestamp: bool = True,
         no_pci: bool = False,
         vdevs: list[VirtualDevice] | None = None,
+        ports: list[Port] | None = None,
         other_eal_param: str = "",
     ) -> "EalParameters":
         """Compose the EAL parameters.
@@ -370,6 +376,8 @@ def create_eal_parameters(
                     VirtualDevice('net_ring0'),
                     VirtualDevice('net_ring1')
                 ]
+            ports: The list of ports to allow. If :data:`None`, all ports listed in `self.ports`
+                will be allowed.
             other_eal_param: user defined DPDK EAL parameters, e.g.:
                 ``other_eal_param='--single-file-segments'``.
 
@@ -388,12 +396,16 @@ def create_eal_parameters(
         if vdevs is None:
             vdevs = []
 
+        if ports is None:
+            ports = self.ports
+
         return EalParameters(
             lcore_list=lcore_list,
             memory_channels=self.config.memory_channels,
             prefix=prefix,
             no_pci=no_pci,
             vdevs=vdevs,
+            ports=ports,
             other_eal_param=other_eal_param,
         )
 
-- 
2.43.0

[PATCH v8 5/7] dts: allow configuring MTU of ports

[jspewock]

From: Jeremy Spewock <jspewock@iol.unh.edu>

Adds methods in both os_session and linux session to allow for setting
MTU of port interfaces so that suites that require the sending and
receiving of packets of a specific size, or the rejection of packets
over a certain size, can configure this maximum as needed.

Signed-off-by: Jeremy Spewock <jspewock@iol.unh.edu>
---
 dts/framework/testbed_model/linux_session.py | 8 ++++++++
 dts/framework/testbed_model/os_session.py    | 9 +++++++++
 2 files changed, 17 insertions(+)

diff --git a/dts/framework/testbed_model/linux_session.py b/dts/framework/testbed_model/linux_session.py
index 0ab59cef85..5d24030c3d 100644
--- a/dts/framework/testbed_model/linux_session.py
+++ b/dts/framework/testbed_model/linux_session.py
@@ -198,6 +198,14 @@ def configure_port_ip_address(
             verify=True,
         )
 
+    def configure_port_mtu(self, mtu: int, port: Port) -> None:
+        """Overrides :meth:`~.os_session.OSSession.configure_port_mtu`."""
+        self.send_command(
+            f"ip link set dev {port.logical_name} mtu {mtu}",
+            privileged=True,
+            verify=True,
+        )
+
     def configure_ipv4_forwarding(self, enable: bool) -> None:
         """Overrides :meth:`~.os_session.OSSession.configure_ipv4_forwarding`."""
         state = 1 if enable else 0
diff --git a/dts/framework/testbed_model/os_session.py b/dts/framework/testbed_model/os_session.py
index ac6bb5e112..e42f4d752a 100644
--- a/dts/framework/testbed_model/os_session.py
+++ b/dts/framework/testbed_model/os_session.py
@@ -413,6 +413,15 @@ def configure_port_ip_address(
             delete: If :data:`True`, remove the IP address, otherwise configure it.
         """
 
+    @abstractmethod
+    def configure_port_mtu(self, mtu: int, port: Port) -> None:
+        """Configure `mtu` on `port`.
+
+        Args:
+            mtu: Desired MTU value.
+            port: Port to set `mtu` on.
+        """
+
     @abstractmethod
     def configure_ipv4_forwarding(self, enable: bool) -> None:
         """Enable IPv4 forwarding in the operating system.
-- 
2.43.0

[PATCH v8 6/7] dts: add scatter to the yaml schema

[jspewock]

From: Jeremy Spewock <jspewock@iol.unh.edu>

Allow for scatter to be specified in the configuration file.

Signed-off-by: Jeremy Spewock <jspewock@iol.unh.edu>
---
 dts/framework/config/conf_yaml_schema.json | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)

diff --git a/dts/framework/config/conf_yaml_schema.json b/dts/framework/config/conf_yaml_schema.json
index 84e45fe3c2..e6dc50ca7f 100644
--- a/dts/framework/config/conf_yaml_schema.json
+++ b/dts/framework/config/conf_yaml_schema.json
@@ -186,7 +186,8 @@
       "type": "string",
       "enum": [
         "hello_world",
-        "os_udp"
+        "os_udp",
+        "pmd_buffer_scatter"
       ]
     },
     "test_target": {
-- 
2.43.0

[PATCH v8 7/7] dts: add pmd_buffer_scatter test suite

[jspewock]

From: Jeremy Spewock <jspewock@iol.unh.edu>

This test suite provides testing of the support of scattered packets by
Poll Mode Drivers using testpmd, verifying the ability to receive and
transmit scattered multi-segment packets made up of multiple
non-contiguous memory buffers. This is tested through 5 different cases
in which the length of the packets sent are less than the mbuf size,
equal to the mbuf size, and 1, 4, and 5 bytes greater than the mbuf size
in order to show both the CRC and the packet data are capable of
existing in the first, second, or both buffers.

Naturally, if the PMD is capable of forwarding scattered packets which
it receives as input, this shows it is capable of both receiving and
transmitting scattered packets.

Signed-off-by: Jeremy Spewock <jspewock@iol.unh.edu>
---
 dts/tests/TestSuite_pmd_buffer_scatter.py | 132 ++++++++++++++++++++++
 1 file changed, 132 insertions(+)
 create mode 100644 dts/tests/TestSuite_pmd_buffer_scatter.py

diff --git a/dts/tests/TestSuite_pmd_buffer_scatter.py b/dts/tests/TestSuite_pmd_buffer_scatter.py
new file mode 100644
index 0000000000..d7535ac822
--- /dev/null
+++ b/dts/tests/TestSuite_pmd_buffer_scatter.py
@@ -0,0 +1,132 @@
+# SPDX-License-Identifier: BSD-3-Clause
+# Copyright(c) 2023-2024 University of New Hampshire
+
+"""Multi-segment packet scattering testing suite.
+
+This testing suite tests the support of transmitting and receiving scattered packets. This is shown
+by the Poll Mode Driver being able to forward scattered multi-segment packets composed of multiple
+non-contiguous memory buffers. To ensure the receipt of scattered packets, the DMA rings of the
+port's RX queues must be configured with mbuf data buffers whose size is less than the maximum
+length.
+
+If it is the case that the Poll Mode Driver can forward scattered packets which it receives, then
+this suffices to show the Poll Mode Driver is capable of both receiving and transmitting scattered
+packets.
+"""
+
+import struct
+
+from scapy.layers.inet import IP  # type: ignore[import]
+from scapy.layers.l2 import Ether  # type: ignore[import]
+from scapy.packet import Raw  # type: ignore[import]
+from scapy.utils import hexstr  # type: ignore[import]
+
+from framework.remote_session.testpmd_shell import TestPmdForwardingModes, TestPmdShell
+from framework.test_suite import TestSuite
+
+
+class PmdBufferScatter(TestSuite):
+    """DPDK PMD packet scattering test suite.
+
+    Configure the Rx queues to have mbuf data buffers whose sizes are smaller than the maximum
+    packet size. Specifically, set mbuf data buffers to have a size of 2048 to fit a full 1512-byte
+    (CRC included) ethernet frame in a mono-segment packet. The testing of scattered packets is
+    done by sending a packet whose length is greater than the size of the configured size of mbuf
+    data buffers. There are a total of 5 packets sent within test cases which have lengths less
+    than, equal to, and greater than the mbuf size. There are multiple packets sent with lengths
+    greater than the mbuf size in order to test cases such as:
+
+    1. A single byte of the CRC being in a second buffer while the remaining 3 bytes are stored in
+        the first buffer alongside packet data.
+    2. The entire CRC being stored in a second buffer while all of the packet data is stored in the
+        first.
+    3. Most of the packet data being stored in the first buffer and a single byte of packet data
+        stored in a second buffer alongside the CRC.
+    """
+
+    def set_up_suite(self) -> None:
+        """Set up the test suite.
+
+        Setup:
+            Verify that we have at least 2 port links in the current execution and increase the MTU
+            of both ports on the traffic generator to 9000 to support larger packet sizes.
+        """
+        self.verify(
+            len(self._port_links) > 1,
+            "There must be at least two port links to run the scatter test suite",
+        )
+
+        self.tg_node.main_session.configure_port_mtu(9000, self._tg_port_egress)
+        self.tg_node.main_session.configure_port_mtu(9000, self._tg_port_ingress)
+
+    def scatter_pktgen_send_packet(self, pktsize: int) -> str:
+        """Generate and send a packet to the SUT then capture what is forwarded back.
+
+        Generate an IP packet of a specific length and send it to the SUT, then capture the
+        resulting received packet and extract its payload. The desired length of the packet is met
+        by packing its payload with the letter "X" in hexadecimal.
+
+        Args:
+            pktsize: Size of the packet to generate and send.
+
+        Returns:
+            The payload of the received packet as a string.
+        """
+        packet = Ether() / IP() / Raw()
+        packet.getlayer(2).load = ""
+        payload_len = pktsize - len(packet) - 4
+        payload = ["58"] * payload_len
+        # pack the payload
+        for X_in_hex in payload:
+            packet.load += struct.pack("=B", int("%s%s" % (X_in_hex[0], X_in_hex[1]), 16))
+        received_packets = self.send_packet_and_capture(packet)
+        self.verify(len(received_packets) > 0, "Did not receive any packets.")
+        load = hexstr(received_packets[0].getlayer(2), onlyhex=1)
+
+        return load
+
+    def pmd_scatter(self, mbsize: int) -> None:
+        """Testpmd support of receiving and sending scattered multi-segment packets.
+
+        Support for scattered packets is shown by sending 5 packets of differing length
+        where the length of the packet is calculated by taking mbuf-size + an offset. The
+        offsets used in the test are -1, 0, 1, 4, 5 respectively.
+
+        Test:
+            Start testpmd and run functional test with preset mbsize.
+        """
+        testpmd = self.sut_node.create_interactive_shell(
+            TestPmdShell,
+            app_parameters=(
+                "--mbcache=200 "
+                f"--mbuf-size={mbsize} "
+                "--max-pkt-len=9000 "
+                "--port-topology=paired "
+                "--tx-offloads=0x00008000"
+            ),
+            privileged=True,
+        )
+        testpmd.set_forward_mode(TestPmdForwardingModes.mac)
+        testpmd.start()
+
+        for offset in [-1, 0, 1, 4, 5]:
+            recv_payload = self.scatter_pktgen_send_packet(mbsize + offset)
+            self._logger.debug(f"Payload of scattered packet after forwarding: \n{recv_payload}")
+            self.verify(
+                ("58 " * 8).strip() in recv_payload,
+                f"Payload of scattered packet did not match expected payload with offset {offset}.",
+            )
+        testpmd.stop()
+
+    def test_scatter_mbuf_2048(self) -> None:
+        """Run the :meth:`pmd_scatter` test with `mbsize` set to 2048."""
+        self.pmd_scatter(mbsize=2048)
+
+    def tear_down_suite(self) -> None:
+        """Tear down the test suite.
+
+        Teardown:
+            Set the MTU of the tg_node back to a more standard size of 1500
+        """
+        self.tg_node.main_session.configure_port_mtu(1500, self._tg_port_egress)
+        self.tg_node.main_session.configure_port_mtu(1500, self._tg_port_ingress)
-- 
2.43.0

Re: [PATCH v8 0/7] dts: Port scatter suite over

[Juraj Linkeš]

Luca, Yoan or Paul, could please have a look at this patch? I don't
have any more comments, but I'd like to have someone else have a look
at this.

For the whole series:
Reviewed-by: Juraj Linkeš <juraj.linkes@pantheon.tech>

On Wed, Jan 10, 2024 at 3:43 PM <jspewock@iol.unh.edu> wrote:
>
> From: Jeremy Spewock <jspewock@iol.unh.edu>
>
> v8:
>
> Address comments by making minor docstring adjustments.
>
> Jeremy Spewock (7):
>   dts: add startup verification and forwarding modes to testpmd shell
>   dts: limit EAL parameters to DPDK apps and add parameters to all apps
>   dts: add optional packet filtering to scapy sniffer
>   dts: add pci addresses to EAL parameters
>   dts: allow configuring MTU of ports
>   dts: add scatter to the yaml schema
>   dts: add pmd_buffer_scatter test suite
>
>  dts/framework/config/conf_yaml_schema.json    |   3 +-
>  dts/framework/exception.py                    |   7 +
>  dts/framework/remote_session/testpmd_shell.py | 149 +++++++++++++++++-
>  dts/framework/test_suite.py                   |  15 +-
>  dts/framework/testbed_model/linux_session.py  |   8 +
>  dts/framework/testbed_model/os_session.py     |   9 ++
>  dts/framework/testbed_model/sut_node.py       |  28 +++-
>  dts/framework/testbed_model/tg_node.py        |  14 +-
>  .../traffic_generator/__init__.py             |   7 +-
>  .../capturing_traffic_generator.py            |  22 ++-
>  .../testbed_model/traffic_generator/scapy.py  |  27 ++++
>  dts/tests/TestSuite_pmd_buffer_scatter.py     | 132 ++++++++++++++++
>  12 files changed, 407 insertions(+), 14 deletions(-)
>  create mode 100644 dts/tests/TestSuite_pmd_buffer_scatter.py
>
> --
> 2.43.0
>

Re: [PATCH v8 0/7] dts: Port scatter suite over

[Patrick Robb]

[-- Attachment #1: Type: text/plain, Size: 2098 bytes --]

Tested-by: Patrick Robb <probb@iol.unh.edu>

I ran this testsuite with a bnxt_en NIC at the Community Lab.

I also spoke with Jeremy about the state of this patch today. He wants to
add a second testcase to the suite for testing the scattered packets
hardware offload (--enable-scatter flag in testpmd). But, he still has
questions about querying ethdev for capabilities and writing the testcase
around that, so that testcase cannot be submitted for this release. It will
come in as a separate patch.

So, from what I can tell Juraj has completed his review and this is the
final v of this patchseries.

On Wed, Jan 10, 2024 at 9:43 AM <jspewock@iol.unh.edu> wrote:

> From: Jeremy Spewock <jspewock@iol.unh.edu>
>
> v8:
>
> Address comments by making minor docstring adjustments.
>
> Jeremy Spewock (7):
>   dts: add startup verification and forwarding modes to testpmd shell
>   dts: limit EAL parameters to DPDK apps and add parameters to all apps
>   dts: add optional packet filtering to scapy sniffer
>   dts: add pci addresses to EAL parameters
>   dts: allow configuring MTU of ports
>   dts: add scatter to the yaml schema
>   dts: add pmd_buffer_scatter test suite
>
>  dts/framework/config/conf_yaml_schema.json    |   3 +-
>  dts/framework/exception.py                    |   7 +
>  dts/framework/remote_session/testpmd_shell.py | 149 +++++++++++++++++-
>  dts/framework/test_suite.py                   |  15 +-
>  dts/framework/testbed_model/linux_session.py  |   8 +
>  dts/framework/testbed_model/os_session.py     |   9 ++
>  dts/framework/testbed_model/sut_node.py       |  28 +++-
>  dts/framework/testbed_model/tg_node.py        |  14 +-
>  .../traffic_generator/__init__.py             |   7 +-
>  .../capturing_traffic_generator.py            |  22 ++-
>  .../testbed_model/traffic_generator/scapy.py  |  27 ++++
>  dts/tests/TestSuite_pmd_buffer_scatter.py     | 132 ++++++++++++++++
>  12 files changed, 407 insertions(+), 14 deletions(-)
>  create mode 100644 dts/tests/TestSuite_pmd_buffer_scatter.py
>
> --
> 2.43.0
>
>

[-- Attachment #2: Type: text/html, Size: 2728 bytes --]

Re: [PATCH v8 0/7] dts: Port scatter suite over

[Thomas Monjalon]

10/01/2024 15:42, jspewock@iol.unh.edu:
>  dts/framework/config/conf_yaml_schema.json    |   3 +-
>  dts/framework/exception.py                    |   7 +
>  dts/framework/remote_session/testpmd_shell.py | 149 +++++++++++++++++-
>  dts/framework/test_suite.py                   |  15 +-
>  dts/framework/testbed_model/linux_session.py  |   8 +
>  dts/framework/testbed_model/os_session.py     |   9 ++
>  dts/framework/testbed_model/sut_node.py       |  28 +++-
>  dts/framework/testbed_model/tg_node.py        |  14 +-
>  .../traffic_generator/__init__.py             |   7 +-
>  .../capturing_traffic_generator.py            |  22 ++-
>  .../testbed_model/traffic_generator/scapy.py  |  27 ++++
>  dts/tests/TestSuite_pmd_buffer_scatter.py     | 132 ++++++++++++++++
>  12 files changed, 407 insertions(+), 14 deletions(-)
>  create mode 100644 dts/tests/TestSuite_pmd_buffer_scatter.py

Big refactoring was done in other patch series by Juraj.
Please could you check whether a rebase is needed for this series?

Re: [PATCH v8 0/7] dts: Port scatter suite over

[Jeremy Spewock]

On Thu, Mar 7, 2024 at 10:00 AM Thomas Monjalon <thomas@monjalon.net> wrote:
>
> 10/01/2024 15:42, jspewock@iol.unh.edu:
> >  dts/framework/config/conf_yaml_schema.json    |   3 +-
> >  dts/framework/exception.py                    |   7 +
> >  dts/framework/remote_session/testpmd_shell.py | 149 +++++++++++++++++-
> >  dts/framework/test_suite.py                   |  15 +-
> >  dts/framework/testbed_model/linux_session.py  |   8 +
> >  dts/framework/testbed_model/os_session.py     |   9 ++
> >  dts/framework/testbed_model/sut_node.py       |  28 +++-
> >  dts/framework/testbed_model/tg_node.py        |  14 +-
> >  .../traffic_generator/__init__.py             |   7 +-
> >  .../capturing_traffic_generator.py            |  22 ++-
> >  .../testbed_model/traffic_generator/scapy.py  |  27 ++++
> >  dts/tests/TestSuite_pmd_buffer_scatter.py     | 132 ++++++++++++++++
> >  12 files changed, 407 insertions(+), 14 deletions(-)
> >  create mode 100644 dts/tests/TestSuite_pmd_buffer_scatter.py
>
> Big refactoring was done in other patch series by Juraj.
> Please could you check whether a rebase is needed for this series?
>
This series does need a rebase but it looks like it was a clean one.
I'll rebase and give it one more quick test run and then send out the
new version.

[PATCH v9 0/7] dts: Port scatter suite over

[jspewock]

From: Jeremy Spewock <jspewock@iol.unh.edu>

v9:
* rebase series on main
* add "Test" to the name of the test suite class so that it gets properly
  recognized as a suite according to changes from patch on main.

Note that changing the name of a test suite class funtionally doesn't
change anything about the suite. The framework imports the first test
suite class it can find within the module, but the new changes on main
enforce that to be a valid test suite class the name must start with
"Test". So, changing this just allows the framework to "see" the test
suite, it does not modify the test suite itself.

Jeremy Spewock (7):
  dts: add startup verification and forwarding modes to testpmd shell
  dts: limit EAL parameters to DPDK apps and add parameters to all apps
  dts: add optional packet filtering to scapy sniffer
  dts: add pci addresses to EAL parameters
  dts: allow configuring MTU of ports
  dts: add scatter to the yaml schema
  dts: add pmd_buffer_scatter test suite

 dts/framework/config/conf_yaml_schema.json    |   3 +-
 dts/framework/exception.py                    |   7 +
 dts/framework/remote_session/testpmd_shell.py | 149 +++++++++++++++++-
 dts/framework/test_suite.py                   |  15 +-
 dts/framework/testbed_model/linux_session.py  |   8 +
 dts/framework/testbed_model/os_session.py     |   9 ++
 dts/framework/testbed_model/sut_node.py       |  28 +++-
 dts/framework/testbed_model/tg_node.py        |  14 +-
 .../traffic_generator/__init__.py             |   7 +-
 .../capturing_traffic_generator.py            |  22 ++-
 .../testbed_model/traffic_generator/scapy.py  |  27 ++++
 dts/tests/TestSuite_pmd_buffer_scatter.py     | 132 ++++++++++++++++
 12 files changed, 407 insertions(+), 14 deletions(-)
 create mode 100644 dts/tests/TestSuite_pmd_buffer_scatter.py

-- 
2.43.2

[PATCH v9 1/7] dts: add startup verification and forwarding modes to testpmd shell

[jspewock]

From: Jeremy Spewock <jspewock@iol.unh.edu>

Added commonly used methods in testpmd such as starting and stopping
packet forwarding, changing forward modes, and verifying link status of
ports so that developers can configure testpmd and start forwarding
through the provided class rather than sending commands to the testpmd
session directly.

Signed-off-by: Jeremy Spewock <jspewock@iol.unh.edu>
---
 dts/framework/exception.py                    |   7 +
 dts/framework/remote_session/testpmd_shell.py | 149 +++++++++++++++++-
 2 files changed, 155 insertions(+), 1 deletion(-)

diff --git a/dts/framework/exception.py b/dts/framework/exception.py
index 658eee2c38..cce1e0231a 100644
--- a/dts/framework/exception.py
+++ b/dts/framework/exception.py
@@ -146,6 +146,13 @@ def __str__(self) -> str:
         return f"Command {self.command} returned a non-zero exit code: {self._command_return_code}"
 
 
+class InteractiveCommandExecutionError(DTSError):
+    """An unsuccessful execution of a remote command in an interactive environment."""
+
+    #:
+    severity: ClassVar[ErrorSeverity] = ErrorSeverity.REMOTE_CMD_EXEC_ERR
+
+
 class RemoteDirectoryExistsError(DTSError):
     """A directory that exists on a remote node."""
 
diff --git a/dts/framework/remote_session/testpmd_shell.py b/dts/framework/remote_session/testpmd_shell.py
index 0184cc2e71..3a66907394 100644
--- a/dts/framework/remote_session/testpmd_shell.py
+++ b/dts/framework/remote_session/testpmd_shell.py
@@ -15,9 +15,15 @@
     testpmd_shell.close()
 """
 
+import time
+from enum import auto
 from pathlib import PurePath
 from typing import Callable, ClassVar
 
+from framework.exception import InteractiveCommandExecutionError
+from framework.settings import SETTINGS
+from framework.utils import StrEnum
+
 from .interactive_shell import InteractiveShell
 
 
@@ -43,14 +49,51 @@ def __str__(self) -> str:
         return self.pci_address
 
 
+class TestPmdForwardingModes(StrEnum):
+    r"""The supported packet forwarding modes for :class:`~TestPmdShell`\s."""
+
+    #:
+    io = auto()
+    #:
+    mac = auto()
+    #:
+    macswap = auto()
+    #:
+    flowgen = auto()
+    #:
+    rxonly = auto()
+    #:
+    txonly = auto()
+    #:
+    csum = auto()
+    #:
+    icmpecho = auto()
+    #:
+    ieee1588 = auto()
+    #:
+    noisy = auto()
+    #:
+    fivetswap = "5tswap"
+    #:
+    shared_rxq = "shared-rxq"
+    #:
+    recycle_mbufs = auto()
+
+
 class TestPmdShell(InteractiveShell):
     """Testpmd interactive shell.
 
     The testpmd shell users should never use
     the :meth:`~.interactive_shell.InteractiveShell.send_command` method directly, but rather
     call specialized methods. If there isn't one that satisfies a need, it should be added.
+
+    Attributes:
+        number_of_ports: The number of ports which were allowed on the command-line when testpmd
+            was started.
     """
 
+    number_of_ports: int
+
     #: The path to the testpmd executable.
     path: ClassVar[PurePath] = PurePath("app", "dpdk-testpmd")
 
@@ -65,9 +108,66 @@ class TestPmdShell(InteractiveShell):
     _command_extra_chars: ClassVar[str] = "\n"
 
     def _start_application(self, get_privileged_command: Callable[[str], str] | None) -> None:
-        self._app_args += " -- -i"
+        """Overrides :meth:`~.interactive_shell._start_application`.
+
+        Add flags for starting testpmd in interactive mode and disabling messages for link state
+        change events before starting the application. Link state is verified before starting
+        packet forwarding and the messages create unexpected newlines in the terminal which
+        complicates output collection.
+
+        Also find the number of pci addresses which were allowed on the command line when the app
+        was started.
+        """
+        self._app_args += " -- -i --mask-event intr_lsc"
+        self.number_of_ports = self._app_args.count("-a ")
         super()._start_application(get_privileged_command)
 
+    def start(self, verify: bool = True) -> None:
+        """Start packet forwarding with the current configuration.
+
+        Args:
+            verify: If :data:`True` , a second start command will be sent in an attempt to verify
+                packet forwarding started as expected.
+
+        Raises:
+            InteractiveCommandExecutionError: If `verify` is :data:`True` and forwarding fails to
+                start or ports fail to come up.
+        """
+        self.send_command("start")
+        if verify:
+            # If forwarding was already started, sending "start" again should tell us
+            start_cmd_output = self.send_command("start")
+            if "Packet forwarding already started" not in start_cmd_output:
+                self._logger.debug(f"Failed to start packet forwarding: \n{start_cmd_output}")
+                raise InteractiveCommandExecutionError("Testpmd failed to start packet forwarding.")
+
+            for port_id in range(self.number_of_ports):
+                if not self.wait_link_status_up(port_id):
+                    raise InteractiveCommandExecutionError(
+                        "Not all ports came up after starting packet forwarding in testpmd."
+                    )
+
+    def stop(self, verify: bool = True) -> None:
+        """Stop packet forwarding.
+
+        Args:
+            verify: If :data:`True` , the output of the stop command is scanned to verify that
+                forwarding was stopped successfully or not started. If neither is found, it is
+                considered an error.
+
+        Raises:
+            InteractiveCommandExecutionError: If `verify` is :data:`True` and the command to stop
+                forwarding results in an error.
+        """
+        stop_cmd_output = self.send_command("stop")
+        if verify:
+            if (
+                "Done." not in stop_cmd_output
+                and "Packet forwarding not started" not in stop_cmd_output
+            ):
+                self._logger.debug(f"Failed to stop packet forwarding: \n{stop_cmd_output}")
+                raise InteractiveCommandExecutionError("Testpmd failed to stop packet forwarding.")
+
     def get_devices(self) -> list[TestPmdDevice]:
         """Get a list of device names that are known to testpmd.
 
@@ -82,3 +182,50 @@ def get_devices(self) -> list[TestPmdDevice]:
             if "device name:" in line.lower():
                 dev_list.append(TestPmdDevice(line))
         return dev_list
+
+    def wait_link_status_up(self, port_id: int, timeout=SETTINGS.timeout) -> bool:
+        """Wait until the link status on the given port is "up".
+
+        Arguments:
+            port_id: Port to check the link status on.
+            timeout: Time to wait for the link to come up. The default value for this
+                argument may be modified using the :option:`--timeout` command-line argument
+                or the :envvar:`DTS_TIMEOUT` environment variable.
+
+        Returns:
+            Whether the link came up in time or not.
+        """
+        time_to_stop = time.time() + timeout
+        port_info: str = ""
+        while time.time() < time_to_stop:
+            port_info = self.send_command(f"show port info {port_id}")
+            if "Link status: up" in port_info:
+                break
+            time.sleep(0.5)
+        else:
+            self._logger.error(f"The link for port {port_id} did not come up in the given timeout.")
+        return "Link status: up" in port_info
+
+    def set_forward_mode(self, mode: TestPmdForwardingModes, verify: bool = True):
+        """Set packet forwarding mode.
+
+        Args:
+            mode: The forwarding mode to use.
+            verify: If :data:`True` the output of the command will be scanned in an attempt to
+                verify that the forwarding mode was set to `mode` properly.
+
+        Raises:
+            InteractiveCommandExecutionError: If `verify` is :data:`True` and the forwarding mode
+                fails to update.
+        """
+        set_fwd_output = self.send_command(f"set fwd {mode.value}")
+        if f"Set {mode.value} packet forwarding mode" not in set_fwd_output:
+            self._logger.debug(f"Failed to set fwd mode to {mode.value}:\n{set_fwd_output}")
+            raise InteractiveCommandExecutionError(
+                f"Test pmd failed to set fwd mode to {mode.value}"
+            )
+
+    def close(self) -> None:
+        """Overrides :meth:`~.interactive_shell.close`."""
+        self.send_command("quit", "")
+        return super().close()
-- 
2.43.2

[PATCH v9 2/7] dts: limit EAL parameters to DPDK apps and add parameters to all apps

[jspewock]

From: Jeremy Spewock <jspewock@iol.unh.edu>

Changed the factory method for creating interactive apps in the SUT Node
so that EAL parameters would only be passed into DPDK apps since
non-DPDK apps wouldn't be able to process them. Also modified
interactive apps to allow for the ability to pass parameters into the
app on startup so that the applications can be started with certain
configuration steps passed on the command line.

Signed-off-by: Jeremy Spewock <jspewock@iol.unh.edu>
---
 dts/framework/remote_session/testpmd_shell.py |  2 +-
 dts/framework/testbed_model/sut_node.py       | 16 ++++++++++------
 2 files changed, 11 insertions(+), 7 deletions(-)

diff --git a/dts/framework/remote_session/testpmd_shell.py b/dts/framework/remote_session/testpmd_shell.py
index 3a66907394..cb2ab6bd00 100644
--- a/dts/framework/remote_session/testpmd_shell.py
+++ b/dts/framework/remote_session/testpmd_shell.py
@@ -118,7 +118,7 @@ def _start_application(self, get_privileged_command: Callable[[str], str] | None
         Also find the number of pci addresses which were allowed on the command line when the app
         was started.
         """
-        self._app_args += " -- -i --mask-event intr_lsc"
+        self._app_args += " -i --mask-event intr_lsc"
         self.number_of_ports = self._app_args.count("-a ")
         super()._start_application(get_privileged_command)
 
diff --git a/dts/framework/testbed_model/sut_node.py b/dts/framework/testbed_model/sut_node.py
index c4acea38d1..909394e756 100644
--- a/dts/framework/testbed_model/sut_node.py
+++ b/dts/framework/testbed_model/sut_node.py
@@ -430,7 +430,8 @@ def create_interactive_shell(
         shell_cls: Type[InteractiveShellType],
         timeout: float = SETTINGS.timeout,
         privileged: bool = False,
-        eal_parameters: EalParameters | str | None = None,
+        app_parameters: str = "",
+        eal_parameters: EalParameters | None = None,
     ) -> InteractiveShellType:
         """Extend the factory for interactive session handlers.
 
@@ -449,20 +450,23 @@ def create_interactive_shell(
             eal_parameters: List of EAL parameters to use to launch the app. If this
                 isn't provided or an empty string is passed, it will default to calling
                 :meth:`create_eal_parameters`.
+            app_parameters: Additional arguments to pass into the application on the
+                command-line.
 
         Returns:
             An instance of the desired interactive application shell.
         """
-        if not eal_parameters:
-            eal_parameters = self.create_eal_parameters()
-
-        # We need to append the build directory for DPDK apps
+        # We need to append the build directory and add EAL parameters for DPDK apps
         if shell_cls.dpdk_app:
+            if not eal_parameters:
+                eal_parameters = self.create_eal_parameters()
+            app_parameters = f"{eal_parameters} -- {app_parameters}"
+
             shell_cls.path = self.main_session.join_remote_path(
                 self.remote_dpdk_build_dir, shell_cls.path
             )
 
-        return super().create_interactive_shell(shell_cls, timeout, privileged, str(eal_parameters))
+        return super().create_interactive_shell(shell_cls, timeout, privileged, app_parameters)
 
     def bind_ports_to_driver(self, for_dpdk: bool = True) -> None:
         """Bind all ports on the SUT to a driver.
-- 
2.43.2

[PATCH v9 3/7] dts: add optional packet filtering to scapy sniffer

[jspewock]

From: Jeremy Spewock <jspewock@iol.unh.edu>

Added the options to filter out LLDP and ARP packets when
sniffing for packets with scapy. This was done using BPF filters to
ensure that the noise these packets provide does not interfere with test
cases.

Signed-off-by: Jeremy Spewock <jspewock@iol.unh.edu>
---
 dts/framework/test_suite.py                   | 15 +++++++++--
 dts/framework/testbed_model/tg_node.py        | 14 ++++++++--
 .../traffic_generator/__init__.py             |  7 ++++-
 .../capturing_traffic_generator.py            | 22 ++++++++++++++-
 .../testbed_model/traffic_generator/scapy.py  | 27 +++++++++++++++++++
 5 files changed, 79 insertions(+), 6 deletions(-)

diff --git a/dts/framework/test_suite.py b/dts/framework/test_suite.py
index 1957ea7328..9c3b516002 100644
--- a/dts/framework/test_suite.py
+++ b/dts/framework/test_suite.py
@@ -23,6 +23,7 @@
 from .exception import TestCaseVerifyError
 from .logger import DTSLogger, get_dts_logger
 from .testbed_model import Port, PortLink, SutNode, TGNode
+from .testbed_model.traffic_generator import PacketFilteringConfig
 from .utils import get_packet_summaries
 
 
@@ -174,7 +175,12 @@ def configure_testbed_ipv4(self, restore: bool = False) -> None:
     def _configure_ipv4_forwarding(self, enable: bool) -> None:
         self.sut_node.configure_ipv4_forwarding(enable)
 
-    def send_packet_and_capture(self, packet: Packet, duration: float = 1) -> list[Packet]:
+    def send_packet_and_capture(
+        self,
+        packet: Packet,
+        filter_config: PacketFilteringConfig = PacketFilteringConfig(),
+        duration: float = 1,
+    ) -> list[Packet]:
         """Send and receive `packet` using the associated TG.
 
         Send `packet` through the appropriate interface and receive on the appropriate interface.
@@ -182,6 +188,7 @@ def send_packet_and_capture(self, packet: Packet, duration: float = 1) -> list[P
 
         Args:
             packet: The packet to send.
+            filter_config: The filter to use when capturing packets.
             duration: Capture traffic for this amount of time after sending `packet`.
 
         Returns:
@@ -189,7 +196,11 @@ def send_packet_and_capture(self, packet: Packet, duration: float = 1) -> list[P
         """
         packet = self._adjust_addresses(packet)
         return self.tg_node.send_packet_and_capture(
-            packet, self._tg_port_egress, self._tg_port_ingress, duration
+            packet,
+            self._tg_port_egress,
+            self._tg_port_ingress,
+            filter_config,
+            duration,
         )
 
     def get_expected_packet(self, packet: Packet) -> Packet:
diff --git a/dts/framework/testbed_model/tg_node.py b/dts/framework/testbed_model/tg_node.py
index f269d4c585..d3206e87e0 100644
--- a/dts/framework/testbed_model/tg_node.py
+++ b/dts/framework/testbed_model/tg_node.py
@@ -15,7 +15,11 @@
 
 from .node import Node
 from .port import Port
-from .traffic_generator import CapturingTrafficGenerator, create_traffic_generator
+from .traffic_generator import (
+    CapturingTrafficGenerator,
+    PacketFilteringConfig,
+    create_traffic_generator,
+)
 
 
 class TGNode(Node):
@@ -53,6 +57,7 @@ def send_packet_and_capture(
         packet: Packet,
         send_port: Port,
         receive_port: Port,
+        filter_config: PacketFilteringConfig = PacketFilteringConfig(),
         duration: float = 1,
     ) -> list[Packet]:
         """Send `packet`, return received traffic.
@@ -65,13 +70,18 @@ def send_packet_and_capture(
             packet: The packet to send.
             send_port: The egress port on the TG node.
             receive_port: The ingress port in the TG node.
+            filter_config: The filter to use when capturing packets.
             duration: Capture traffic for this amount of time after sending `packet`.
 
         Returns:
              A list of received packets. May be empty if no packets are captured.
         """
         return self.traffic_generator.send_packet_and_capture(
-            packet, send_port, receive_port, duration
+            packet,
+            send_port,
+            receive_port,
+            filter_config,
+            duration,
         )
 
     def close(self) -> None:
diff --git a/dts/framework/testbed_model/traffic_generator/__init__.py b/dts/framework/testbed_model/traffic_generator/__init__.py
index 11e2bd7d97..0eaf0355cd 100644
--- a/dts/framework/testbed_model/traffic_generator/__init__.py
+++ b/dts/framework/testbed_model/traffic_generator/__init__.py
@@ -14,11 +14,16 @@
 and a capturing traffic generator is required.
 """
 
+# pylama:ignore=W0611
+
 from framework.config import ScapyTrafficGeneratorConfig, TrafficGeneratorType
 from framework.exception import ConfigurationError
 from framework.testbed_model.node import Node
 
-from .capturing_traffic_generator import CapturingTrafficGenerator
+from .capturing_traffic_generator import (
+    CapturingTrafficGenerator,
+    PacketFilteringConfig,
+)
 from .scapy import ScapyTrafficGenerator
 
 
diff --git a/dts/framework/testbed_model/traffic_generator/capturing_traffic_generator.py b/dts/framework/testbed_model/traffic_generator/capturing_traffic_generator.py
index 0246590333..e5a1560e90 100644
--- a/dts/framework/testbed_model/traffic_generator/capturing_traffic_generator.py
+++ b/dts/framework/testbed_model/traffic_generator/capturing_traffic_generator.py
@@ -11,6 +11,7 @@
 
 import uuid
 from abc import abstractmethod
+from dataclasses import dataclass
 
 import scapy.utils  # type: ignore[import]
 from scapy.packet import Packet  # type: ignore[import]
@@ -26,6 +27,19 @@ def _get_default_capture_name() -> str:
     return str(uuid.uuid4())
 
 
+@dataclass(slots=True, frozen=True)
+class PacketFilteringConfig:
+    """The supported filtering options for :class:`CapturingTrafficGenerator`.
+
+    Attributes:
+        no_lldp: If :data:`True`, LLDP packets will be filtered out when capturing.
+        no_arp: If :data:`True`, ARP packets will be filtered out when capturing.
+    """
+
+    no_lldp: bool = True
+    no_arp: bool = True
+
+
 class CapturingTrafficGenerator(TrafficGenerator):
     """Capture packets after sending traffic.
 
@@ -54,6 +68,7 @@ def send_packet_and_capture(
         packet: Packet,
         send_port: Port,
         receive_port: Port,
+        filter_config: PacketFilteringConfig,
         duration: float,
         capture_name: str = _get_default_capture_name(),
     ) -> list[Packet]:
@@ -68,6 +83,7 @@ def send_packet_and_capture(
             packet: The packet to send.
             send_port: The egress port on the TG node.
             receive_port: The ingress port in the TG node.
+            filter_config: Filters to apply when capturing packets.
             duration: Capture traffic for this amount of time after sending the packet.
             capture_name: The name of the .pcap file where to store the capture.
 
@@ -75,7 +91,7 @@ def send_packet_and_capture(
              The received packets. May be empty if no packets are captured.
         """
         return self.send_packets_and_capture(
-            [packet], send_port, receive_port, duration, capture_name
+            [packet], send_port, receive_port, filter_config, duration, capture_name
         )
 
     def send_packets_and_capture(
@@ -83,6 +99,7 @@ def send_packets_and_capture(
         packets: list[Packet],
         send_port: Port,
         receive_port: Port,
+        filter_config: PacketFilteringConfig,
         duration: float,
         capture_name: str = _get_default_capture_name(),
     ) -> list[Packet]:
@@ -99,6 +116,7 @@ def send_packets_and_capture(
             packets: The packets to send.
             send_port: The egress port on the TG node.
             receive_port: The ingress port in the TG node.
+            filter_config: Filters to apply when capturing packets.
             duration: Capture traffic for this amount of time after sending the packets.
             capture_name: The name of the .pcap file where to store the capture.
 
@@ -113,6 +131,7 @@ def send_packets_and_capture(
             packets,
             send_port,
             receive_port,
+            filter_config,
             duration,
         )
 
@@ -126,6 +145,7 @@ def _send_packets_and_capture(
         packets: list[Packet],
         send_port: Port,
         receive_port: Port,
+        filter_config: PacketFilteringConfig,
         duration: float,
     ) -> list[Packet]:
         """The implementation of :method:`send_packets_and_capture`.
diff --git a/dts/framework/testbed_model/traffic_generator/scapy.py b/dts/framework/testbed_model/traffic_generator/scapy.py
index 5b60f66237..df3069d516 100644
--- a/dts/framework/testbed_model/traffic_generator/scapy.py
+++ b/dts/framework/testbed_model/traffic_generator/scapy.py
@@ -32,6 +32,7 @@
 
 from .capturing_traffic_generator import (
     CapturingTrafficGenerator,
+    PacketFilteringConfig,
     _get_default_capture_name,
 )
 
@@ -69,6 +70,7 @@ def scapy_send_packets_and_capture(
     send_iface: str,
     recv_iface: str,
     duration: float,
+    sniff_filter: str,
 ) -> list[bytes]:
     """The RPC function to send and capture packets.
 
@@ -90,6 +92,7 @@ def scapy_send_packets_and_capture(
         iface=recv_iface,
         store=True,
         started_callback=lambda *args: scapy.all.sendp(scapy_packets, iface=send_iface),
+        filter=sniff_filter,
     )
     sniffer.start()
     time.sleep(duration)
@@ -260,11 +263,34 @@ def _send_packets(self, packets: list[Packet], port: Port) -> None:
         packets = [packet.build() for packet in packets]
         self.rpc_server_proxy.scapy_send_packets(packets, port.logical_name)
 
+    def _create_packet_filter(self, filter_config: PacketFilteringConfig) -> str:
+        """Combines filter settings from `filter_config` into a BPF that scapy can use.
+
+        Scapy allows for the use of Berkeley Packet Filters (BPFs) to filter what packets are
+        collected based on various attributes of the packet.
+
+        Args:
+            filter_config: Config class that specifies which filters should be applied.
+
+        Returns:
+            A string representing the combination of BPF filters to be passed to scapy. For
+            example:
+
+            "ether[12:2] != 0x88cc && ether[12:2] != 0x0806"
+        """
+        bpf_filter = []
+        if filter_config.no_arp:
+            bpf_filter.append("ether[12:2] != 0x0806")
+        if filter_config.no_lldp:
+            bpf_filter.append("ether[12:2] != 0x88cc")
+        return " && ".join(bpf_filter)
+
     def _send_packets_and_capture(
         self,
         packets: list[Packet],
         send_port: Port,
         receive_port: Port,
+        filter_config: PacketFilteringConfig,
         duration: float,
         capture_name: str = _get_default_capture_name(),
     ) -> list[Packet]:
@@ -277,6 +303,7 @@ def _send_packets_and_capture(
             send_port.logical_name,
             receive_port.logical_name,
             duration,
+            self._create_packet_filter(filter_config),
         )  # type: ignore[assignment]
 
         scapy_packets = [Ether(packet.data) for packet in xmlrpc_packets]
-- 
2.43.2

[PATCH v9 4/7] dts: add pci addresses to EAL parameters

[jspewock]

From: Jeremy Spewock <jspewock@iol.unh.edu>

Added allow list to the EAL parameters created in DTS to ensure that
only the relevant PCI devices are considered when launching DPDK
applications.

Signed-off-by: Jeremy Spewock <jspewock@iol.unh.edu>
---
 dts/framework/testbed_model/sut_node.py | 12 ++++++++++++
 1 file changed, 12 insertions(+)

diff --git a/dts/framework/testbed_model/sut_node.py b/dts/framework/testbed_model/sut_node.py
index 909394e756..97aa26d419 100644
--- a/dts/framework/testbed_model/sut_node.py
+++ b/dts/framework/testbed_model/sut_node.py
@@ -30,6 +30,7 @@
 from .cpu import LogicalCoreCount, LogicalCoreList
 from .node import Node
 from .os_session import InteractiveShellType, OSSession
+from .port import Port
 from .virtual_device import VirtualDevice
 
 
@@ -46,6 +47,7 @@ def __init__(
         prefix: str,
         no_pci: bool,
         vdevs: list[VirtualDevice],
+        ports: list[Port],
         other_eal_param: str,
     ):
         """Initialize the parameters according to inputs.
@@ -63,6 +65,7 @@ def __init__(
                     VirtualDevice('net_ring0'),
                     VirtualDevice('net_ring1')
                 ]
+            ports: The list of ports to allow.
             other_eal_param: user defined DPDK EAL parameters, e.g.:
                 ``other_eal_param='--single-file-segments'``
         """
@@ -73,6 +76,7 @@ def __init__(
             self._prefix = f"--file-prefix={prefix}"
         self._no_pci = "--no-pci" if no_pci else ""
         self._vdevs = " ".join(f"--vdev {vdev}" for vdev in vdevs)
+        self._ports = " ".join(f"-a {port.pci}" for port in ports)
         self._other_eal_param = other_eal_param
 
     def __str__(self) -> str:
@@ -83,6 +87,7 @@ def __str__(self) -> str:
             f"{self._prefix} "
             f"{self._no_pci} "
             f"{self._vdevs} "
+            f"{self._ports} "
             f"{self._other_eal_param}"
         )
 
@@ -347,6 +352,7 @@ def create_eal_parameters(
         append_prefix_timestamp: bool = True,
         no_pci: bool = False,
         vdevs: list[VirtualDevice] | None = None,
+        ports: list[Port] | None = None,
         other_eal_param: str = "",
     ) -> "EalParameters":
         """Compose the EAL parameters.
@@ -370,6 +376,8 @@ def create_eal_parameters(
                     VirtualDevice('net_ring0'),
                     VirtualDevice('net_ring1')
                 ]
+            ports: The list of ports to allow. If :data:`None`, all ports listed in `self.ports`
+                will be allowed.
             other_eal_param: user defined DPDK EAL parameters, e.g.:
                 ``other_eal_param='--single-file-segments'``.
 
@@ -388,12 +396,16 @@ def create_eal_parameters(
         if vdevs is None:
             vdevs = []
 
+        if ports is None:
+            ports = self.ports
+
         return EalParameters(
             lcore_list=lcore_list,
             memory_channels=self.config.memory_channels,
             prefix=prefix,
             no_pci=no_pci,
             vdevs=vdevs,
+            ports=ports,
             other_eal_param=other_eal_param,
         )
 
-- 
2.43.2

[PATCH v9 5/7] dts: allow configuring MTU of ports

[jspewock]

From: Jeremy Spewock <jspewock@iol.unh.edu>

Adds methods in both os_session and linux session to allow for setting
MTU of port interfaces so that suites that require the sending and
receiving of packets of a specific size, or the rejection of packets
over a certain size, can configure this maximum as needed.

Signed-off-by: Jeremy Spewock <jspewock@iol.unh.edu>
---
 dts/framework/testbed_model/linux_session.py | 8 ++++++++
 dts/framework/testbed_model/os_session.py    | 9 +++++++++
 2 files changed, 17 insertions(+)

diff --git a/dts/framework/testbed_model/linux_session.py b/dts/framework/testbed_model/linux_session.py
index 0ab59cef85..5d24030c3d 100644
--- a/dts/framework/testbed_model/linux_session.py
+++ b/dts/framework/testbed_model/linux_session.py
@@ -198,6 +198,14 @@ def configure_port_ip_address(
             verify=True,
         )
 
+    def configure_port_mtu(self, mtu: int, port: Port) -> None:
+        """Overrides :meth:`~.os_session.OSSession.configure_port_mtu`."""
+        self.send_command(
+            f"ip link set dev {port.logical_name} mtu {mtu}",
+            privileged=True,
+            verify=True,
+        )
+
     def configure_ipv4_forwarding(self, enable: bool) -> None:
         """Overrides :meth:`~.os_session.OSSession.configure_ipv4_forwarding`."""
         state = 1 if enable else 0
diff --git a/dts/framework/testbed_model/os_session.py b/dts/framework/testbed_model/os_session.py
index 6983aa4a77..d5bf7e0401 100644
--- a/dts/framework/testbed_model/os_session.py
+++ b/dts/framework/testbed_model/os_session.py
@@ -412,6 +412,15 @@ def configure_port_ip_address(
             delete: If :data:`True`, remove the IP address, otherwise configure it.
         """
 
+    @abstractmethod
+    def configure_port_mtu(self, mtu: int, port: Port) -> None:
+        """Configure `mtu` on `port`.
+
+        Args:
+            mtu: Desired MTU value.
+            port: Port to set `mtu` on.
+        """
+
     @abstractmethod
     def configure_ipv4_forwarding(self, enable: bool) -> None:
         """Enable IPv4 forwarding in the operating system.
-- 
2.43.2

[PATCH v9 6/7] dts: add scatter to the yaml schema

[jspewock]

From: Jeremy Spewock <jspewock@iol.unh.edu>

Allow for scatter to be specified in the configuration file.

Signed-off-by: Jeremy Spewock <jspewock@iol.unh.edu>
---
 dts/framework/config/conf_yaml_schema.json | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)

diff --git a/dts/framework/config/conf_yaml_schema.json b/dts/framework/config/conf_yaml_schema.json
index 051b079fe4..4731f4511d 100644
--- a/dts/framework/config/conf_yaml_schema.json
+++ b/dts/framework/config/conf_yaml_schema.json
@@ -186,7 +186,8 @@
       "type": "string",
       "enum": [
         "hello_world",
-        "os_udp"
+        "os_udp",
+        "pmd_buffer_scatter"
       ]
     },
     "test_target": {
-- 
2.43.2

[PATCH v9 7/7] dts: add pmd_buffer_scatter test suite

[jspewock]

From: Jeremy Spewock <jspewock@iol.unh.edu>

This test suite provides testing of the support of scattered packets by
Poll Mode Drivers using testpmd, verifying the ability to receive and
transmit scattered multi-segment packets made up of multiple
non-contiguous memory buffers. This is tested through 5 different cases
in which the length of the packets sent are less than the mbuf size,
equal to the mbuf size, and 1, 4, and 5 bytes greater than the mbuf size
in order to show both the CRC and the packet data are capable of
existing in the first, second, or both buffers.

Naturally, if the PMD is capable of forwarding scattered packets which
it receives as input, this shows it is capable of both receiving and
transmitting scattered packets.

Signed-off-by: Jeremy Spewock <jspewock@iol.unh.edu>
---
 dts/tests/TestSuite_pmd_buffer_scatter.py | 132 ++++++++++++++++++++++
 1 file changed, 132 insertions(+)
 create mode 100644 dts/tests/TestSuite_pmd_buffer_scatter.py

diff --git a/dts/tests/TestSuite_pmd_buffer_scatter.py b/dts/tests/TestSuite_pmd_buffer_scatter.py
new file mode 100644
index 0000000000..6c035595c2
--- /dev/null
+++ b/dts/tests/TestSuite_pmd_buffer_scatter.py
@@ -0,0 +1,132 @@
+# SPDX-License-Identifier: BSD-3-Clause
+# Copyright(c) 2023-2024 University of New Hampshire
+
+"""Multi-segment packet scattering testing suite.
+
+This testing suite tests the support of transmitting and receiving scattered packets. This is shown
+by the Poll Mode Driver being able to forward scattered multi-segment packets composed of multiple
+non-contiguous memory buffers. To ensure the receipt of scattered packets, the DMA rings of the
+port's RX queues must be configured with mbuf data buffers whose size is less than the maximum
+length.
+
+If it is the case that the Poll Mode Driver can forward scattered packets which it receives, then
+this suffices to show the Poll Mode Driver is capable of both receiving and transmitting scattered
+packets.
+"""
+
+import struct
+
+from scapy.layers.inet import IP  # type: ignore[import]
+from scapy.layers.l2 import Ether  # type: ignore[import]
+from scapy.packet import Raw  # type: ignore[import]
+from scapy.utils import hexstr  # type: ignore[import]
+
+from framework.remote_session.testpmd_shell import TestPmdForwardingModes, TestPmdShell
+from framework.test_suite import TestSuite
+
+
+class TestPmdBufferScatter(TestSuite):
+    """DPDK PMD packet scattering test suite.
+
+    Configure the Rx queues to have mbuf data buffers whose sizes are smaller than the maximum
+    packet size. Specifically, set mbuf data buffers to have a size of 2048 to fit a full 1512-byte
+    (CRC included) ethernet frame in a mono-segment packet. The testing of scattered packets is
+    done by sending a packet whose length is greater than the size of the configured size of mbuf
+    data buffers. There are a total of 5 packets sent within test cases which have lengths less
+    than, equal to, and greater than the mbuf size. There are multiple packets sent with lengths
+    greater than the mbuf size in order to test cases such as:
+
+    1. A single byte of the CRC being in a second buffer while the remaining 3 bytes are stored in
+        the first buffer alongside packet data.
+    2. The entire CRC being stored in a second buffer while all of the packet data is stored in the
+        first.
+    3. Most of the packet data being stored in the first buffer and a single byte of packet data
+        stored in a second buffer alongside the CRC.
+    """
+
+    def set_up_suite(self) -> None:
+        """Set up the test suite.
+
+        Setup:
+            Verify that we have at least 2 port links in the current execution and increase the MTU
+            of both ports on the traffic generator to 9000 to support larger packet sizes.
+        """
+        self.verify(
+            len(self._port_links) > 1,
+            "There must be at least two port links to run the scatter test suite",
+        )
+
+        self.tg_node.main_session.configure_port_mtu(9000, self._tg_port_egress)
+        self.tg_node.main_session.configure_port_mtu(9000, self._tg_port_ingress)
+
+    def scatter_pktgen_send_packet(self, pktsize: int) -> str:
+        """Generate and send a packet to the SUT then capture what is forwarded back.
+
+        Generate an IP packet of a specific length and send it to the SUT, then capture the
+        resulting received packet and extract its payload. The desired length of the packet is met
+        by packing its payload with the letter "X" in hexadecimal.
+
+        Args:
+            pktsize: Size of the packet to generate and send.
+
+        Returns:
+            The payload of the received packet as a string.
+        """
+        packet = Ether() / IP() / Raw()
+        packet.getlayer(2).load = ""
+        payload_len = pktsize - len(packet) - 4
+        payload = ["58"] * payload_len
+        # pack the payload
+        for X_in_hex in payload:
+            packet.load += struct.pack("=B", int("%s%s" % (X_in_hex[0], X_in_hex[1]), 16))
+        received_packets = self.send_packet_and_capture(packet)
+        self.verify(len(received_packets) > 0, "Did not receive any packets.")
+        load = hexstr(received_packets[0].getlayer(2), onlyhex=1)
+
+        return load
+
+    def pmd_scatter(self, mbsize: int) -> None:
+        """Testpmd support of receiving and sending scattered multi-segment packets.
+
+        Support for scattered packets is shown by sending 5 packets of differing length
+        where the length of the packet is calculated by taking mbuf-size + an offset. The
+        offsets used in the test are -1, 0, 1, 4, 5 respectively.
+
+        Test:
+            Start testpmd and run functional test with preset mbsize.
+        """
+        testpmd = self.sut_node.create_interactive_shell(
+            TestPmdShell,
+            app_parameters=(
+                "--mbcache=200 "
+                f"--mbuf-size={mbsize} "
+                "--max-pkt-len=9000 "
+                "--port-topology=paired "
+                "--tx-offloads=0x00008000"
+            ),
+            privileged=True,
+        )
+        testpmd.set_forward_mode(TestPmdForwardingModes.mac)
+        testpmd.start()
+
+        for offset in [-1, 0, 1, 4, 5]:
+            recv_payload = self.scatter_pktgen_send_packet(mbsize + offset)
+            self._logger.debug(f"Payload of scattered packet after forwarding: \n{recv_payload}")
+            self.verify(
+                ("58 " * 8).strip() in recv_payload,
+                f"Payload of scattered packet did not match expected payload with offset {offset}.",
+            )
+        testpmd.stop()
+
+    def test_scatter_mbuf_2048(self) -> None:
+        """Run the :meth:`pmd_scatter` test with `mbsize` set to 2048."""
+        self.pmd_scatter(mbsize=2048)
+
+    def tear_down_suite(self) -> None:
+        """Tear down the test suite.
+
+        Teardown:
+            Set the MTU of the tg_node back to a more standard size of 1500
+        """
+        self.tg_node.main_session.configure_port_mtu(1500, self._tg_port_egress)
+        self.tg_node.main_session.configure_port_mtu(1500, self._tg_port_ingress)
-- 
2.43.2

Re: [PATCH v9 0/7] dts: Port scatter suite over

[Thomas Monjalon]

11/03/2024 16:43, jspewock@iol.unh.edu:
> From: Jeremy Spewock <jspewock@iol.unh.edu>
> 
> v9:
> * rebase series on main
> * add "Test" to the name of the test suite class so that it gets properly
>   recognized as a suite according to changes from patch on main.

Reviewed-by: Juraj Linkeš <juraj.linkes@pantheon.tech>
Tested-by: Patrick Robb <probb@iol.unh.edu>

Applied, thanks.