From mboxrd@z Thu Jan 1 00:00:00 1970
Return-Path: <dev-bounces@dpdk.org>
Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124])
by inbox.dpdk.org (Postfix) with ESMTP id DFADB424CA;
Tue, 11 Jun 2024 13:12:07 +0200 (CEST)
Received: from mails.dpdk.org (localhost [127.0.0.1])
by mails.dpdk.org (Postfix) with ESMTP id 9DEC540263;
Tue, 11 Jun 2024 13:12:07 +0200 (CEST)
Received: from mail-wr1-f41.google.com (mail-wr1-f41.google.com
[209.85.221.41]) by mails.dpdk.org (Postfix) with ESMTP id 2AE134021F
for <dev@dpdk.org>; Tue, 11 Jun 2024 13:12:06 +0200 (CEST)
Received: by mail-wr1-f41.google.com with SMTP id
ffacd0b85a97d-35f27eed98aso945958f8f.2
for <dev@dpdk.org>; Tue, 11 Jun 2024 04:12:06 -0700 (PDT)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
d=pantheon.tech; s=google; t=1718104326; x=1718709126; darn=dpdk.org;
h=content-transfer-encoding:in-reply-to:from:content-language
:references:cc:to:subject:user-agent:mime-version:date:message-id
:from:to:cc:subject:date:message-id:reply-to;
bh=9h8lPndWNsyfUxnYq/xqiWbRxTGeE08J2s/ICrDzG8A=;
b=i6keGXMJw3E5sZ/Ts2t6nSmiHenknZWLEZskoZ8t0s25WOIPbV0UXhOHyb3QLHO4fy
e9GATRg+/Y9/JujH2ONEj3Z3F8kV4UZSW067NpSmARp+NN4OMVS5F79DmR3hgiLFGgIg
Z6A19ZzvNZx02k5WjYFlLGLe0BNFjiQizvFd58jFtrxMtlLhOfFHUpbCQa64oLcijZua
AO06pGLws1FMT77b8kj7/lxZMI7FIHfPGMu98mZ8cw7w0w8yHWqUynB6nKu/Bo/c4D4G
ntjBoIQs/Xc1gxpLnP3DSDoNfNXxAqJIN86NEdZNfmPHMNaWY3466vHXKm1Buig//NH2
v24Q==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
d=1e100.net; s=20230601; t=1718104326; x=1718709126;
h=content-transfer-encoding:in-reply-to:from:content-language
:references:cc:to:subject:user-agent:mime-version:date:message-id
:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
bh=9h8lPndWNsyfUxnYq/xqiWbRxTGeE08J2s/ICrDzG8A=;
b=w/Ivxise9gqrOkgsNYs08Uhrz/XnP93bTYBzo0JYCV24eJ1In4YfAiWFOrWIXaPkAy
+5JB6814SOo/RU+TgMdoNMZ0x9J31Li5qg9xjrPaBRLT4XobI5B5q7Sl9Dsy+0tsm1qY
s4f06o/Nw/gEJkQG1p46NJuGkqdADFVeXHRFPTLB7H2C5hS/zrkg+IAWOxO5o79Sp1y1
oChZ2KWJdD1TpAHJ1bEkqGRrTshzmPCCF2r0fcGwetkz6Ywr6eJpTAcEGT48eLXg/vvh
pN9APlS3unWUa3XcPbCn1lLZMrMBcLFY5/TOJyF2BCBoDfqV2aAwdVCcQguakZct+rta
Dzog==
X-Gm-Message-State: AOJu0Yz2p1kjJdp2a8mqYsomTQv/8zkiHpnbJLG1TuKCRHd2cFmdZgv/
jApk3kXJ8nZnhgw8/LGs9Fq00VnTyQwuSdCjM/irX4EpVx8W/QnoBT7jk6sczdM=
X-Google-Smtp-Source: AGHT+IHCbXTce5+wnjTwVhEIljrdGIZ4JXjTZ4qiPlq1AgXx4U0aSuw76xT0WbRLP2m3ysJ8NCgrfA==
X-Received: by 2002:a5d:4f0c:0:b0:35f:23f8:b295 with SMTP id
ffacd0b85a97d-35f23f8b2d4mr3573502f8f.69.1718104325589;
Tue, 11 Jun 2024 04:12:05 -0700 (PDT)
Received: from [10.12.0.137] (81.89.53.154.host.vnet.sk. [81.89.53.154])
by smtp.gmail.com with ESMTPSA id
ffacd0b85a97d-35f0ed44b29sm9590995f8f.66.2024.06.11.04.12.04
(version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
Tue, 11 Jun 2024 04:12:05 -0700 (PDT)
Message-ID: <4a25d1c0-d793-4503-a943-f3b7fe9749d5@pantheon.tech>
Date: Tue, 11 Jun 2024 13:12:04 +0200
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [RFC PATCH v1 1/2] dts: Add interactive shell for managing Scapy
To: jspewock@iol.unh.edu, Luca.Vizzarro@arm.com, probb@iol.unh.edu,
npratte@iol.unh.edu, paul.szczepanek@arm.com, yoan.picchi@foss.arm.com,
thomas@monjalon.net, wathsala.vithanage@arm.com, Honnappa.Nagarahalli@arm.com
Cc: dev@dpdk.org
References: <20240605175227.7003-1-jspewock@iol.unh.edu>
<20240605175227.7003-2-jspewock@iol.unh.edu>
Content-Language: en-US
From: =?UTF-8?Q?Juraj_Linke=C5=A1?= <juraj.linkes@pantheon.tech>
In-Reply-To: <20240605175227.7003-2-jspewock@iol.unh.edu>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 7bit
X-BeenThere: dev@dpdk.org
X-Mailman-Version: 2.1.29
Precedence: list
List-Id: DPDK patches and discussions <dev.dpdk.org>
List-Unsubscribe: <https://mails.dpdk.org/options/dev>,
<mailto:dev-request@dpdk.org?subject=unsubscribe>
List-Archive: <http://mails.dpdk.org/archives/dev/>
List-Post: <mailto:dev@dpdk.org>
List-Help: <mailto:dev-request@dpdk.org?subject=help>
List-Subscribe: <https://mails.dpdk.org/listinfo/dev>,
<mailto:dev-request@dpdk.org?subject=subscribe>
Errors-To: dev-bounces@dpdk.org
> diff --git a/dts/framework/remote_session/scapy_shell.py b/dts/framework/remote_session/scapy_shell.py
> new file mode 100644
> index 0000000000..fa647dc870
> --- /dev/null
> +++ b/dts/framework/remote_session/scapy_shell.py
> @@ -0,0 +1,175 @@
> +# SPDX-License-Identifier: BSD-3-Clause
> +# Copyright(c) 2024 University of New Hampshire
> +
> +"""Scapy interactive shell."""
> +
> +import re
> +import time
> +from typing import Callable, ClassVar
> +
> +from scapy.compat import base64_bytes # type: ignore[import]
> +from scapy.layers.l2 import Ether # type: ignore[import]
> +from scapy.packet import Packet # type: ignore[import]
> +
> +from framework.testbed_model.port import Port
> +from framework.utils import REGEX_FOR_BASE64_ENCODING
> +
> +from .python_shell import PythonShell
> +
> +
> +class ScapyShell(PythonShell):
> + """Scapy interactive shell.
> +
> + The scapy shell is implemented using a :class:`~.python_shell.PythonShell` and importing
> + everything from the "scapy.all" library. This is done due to formatting issues that occur from
> + the scapy interactive shell attempting to use iPython, which is not compatible with the
> + pseudo-terminal that paramiko creates to manage its channels.
> +
> + This class is used as an underlying session for the scapy traffic generator and shouldn't be
> + used directly inside of test suites. If there isn't a method in
> + :class:`framework.testbed_model.traffic_generator.scapy.ScapyTrafficGenerator` to fulfill a
> + need, one should be added there and implemented here.
> + """
> +
> + #: Name of sniffer to ensure the same is used in all places
> + _sniffer_name: ClassVar[str] = "sniffer"
> + #: Name of variable that points to the list of packets inside the scapy shell.
> + _send_packet_list_name: ClassVar[str] = "packets"
> + #: Padding to add to the start of a line for python syntax compliance.
> + _padding: ClassVar[str] = " " * 4
> +
> + def _start_application(self, get_privileged_command: Callable[[str], str] | None) -> None:
> + """Overrides :meth:`~.interactive_shell._start_application`.
This extends the method and in that case we should mention what the
extension is.
> +
> + Adds a command that imports everything from the scapy library immediately after starting
> + the shell for usage in later calls to the methods of this class.
> +
> + Args:
> + get_privileged_command: A function (but could be any callable) that produces
> + the version of the command with elevated privileges.
> + """
> + super()._start_application(get_privileged_command)
> + self.send_command("from scapy.all import *")
> +
> + def _build_send_packet_list(self, packets: list[Packet]) -> None:
The send in the name evokes that the method sends the packets.
The description in the Args section says "packets to recreate in the
shell" and I like that so I'd put that in the name: _create_packet_list()
> + """Build a list of packets to send later.
> +
> + Gets the string that represents the Python command that was used to create each packet in
Gets the string sounds like that's what the methods returns, as a getter
method would.
> + `packets` and sends these commands into the underlying Python session. The purpose behind
> + doing this is to create a list that is identical to `packets` inside the shell. This method
> + should only be called by methods for sending packets immediately prior to sending. The list
> + of packets will continue to exist in the scope of the shell until subsequent calls to this
> + method, so failure to rebuild the list prior to sending packets could lead to undesired
> + "stale" packets to be sent.
> +
> + Args:
> + packets: The list of packets to recreate in the shell.
> + """
> + self._logger.info("Building a list of packets to send...")
The could be just a regular dot instead of the ellipsis (I don't like
random ellipses as those read as if I was supposed to expect something
and we don't provide a subsequent log that would continue this ellipsis).
> + self.send_command(
> + f"{self._send_packet_list_name} = [{', '.join(map(Packet.command, packets))}]"
> + )
> +
> + def send_packets(self, packets: list[Packet], send_port: Port) -> None:
> + """Send packets without capturing any received traffic.
> +
> + Provides a "fire and forget" method for sending packets for situations when there is no
> + need to collected any received traffic.
Typo: collected
> +
> + Args:
> + packets: The packets to send.
> + send_port: The port to send the packets from.
> + """
> + self._build_send_packet_list(packets)
> + send_command = [
> + "sendp(",
> + f"{self._send_packet_list_name},",
> + f"iface='{send_port.logical_name}',",
> + "realtime=True,",
> + "verbose=True",
> + ")",
> + ]
> + self.send_command(f"\n{self._padding}".join(send_command))
> +
> + def _create_sniffer(
> + self, packets_to_send: list[Packet], send_port: Port, recv_port: Port, filter_config: str
> + ) -> None:
> + """Create an asynchronous sniffer in the shell.
> +
> + A list of packets to send is added to the sniffer inside of a callback function so that
> + they are immediately sent at the time sniffing is started.
> +
> + Args:
> + packets_to_send: A list of packets to send when sniffing is started.
> + send_port: The port to send the packets on when sniffing is started.
> + recv_port: The port to collect the traffic from.
> + filter_config: An optional BPF format filter to use when sniffing for packets. Omitted
> + when set to an empty string.
> + """
> + self._build_send_packet_list(packets_to_send)
> + sniffer_commands = [
> + f"{self._sniffer_name} = AsyncSniffer(",
> + f"iface='{recv_port.logical_name}',",
> + "store=True,",
> + "started_callback=lambda *args: sendp(",
> + f"{self._padding}{self._send_packet_list_name}, iface='{send_port.logical_name}'),",
> + ")",
> + ]
> + if filter_config:
> + sniffer_commands.insert(-1, f"filter='{filter_config}'")
> +
> + self.send_command(f"\n{self._padding}".join(sniffer_commands))
> +
> + def _start_and_stop_sniffing(self, duration: float) -> list[Packet]:
> + """Starts asynchronous sniffer, runs for a set `duration`, then collects received packets.
This should be in imperative to align with the rest of the docstrings.
> + > + This method expects that you have first created an
asynchronous sniffer inside the shell
> + and will fail if you haven't. Received packets are collected by printing the base64
> + encoding of each packet in the shell and then harvesting these encodings using regex to
> + convert back into packet objects.
> +
> + Args:
> + duration: The amount of time in seconds to sniff for received packets.
> +
> + Returns:
> + A list of all packets that were received while the sniffer was running.
> + """
> + sniffed_packets_name = "gathered_packets"
> + self.send_command(f"{self._sniffer_name}.start()")
> + time.sleep(duration)
> + self.send_command(f"{sniffed_packets_name} = {self._sniffer_name}.stop(join=True)")
> + # An extra newline is required here due to the nature of interactive Python shells
> + packet_objects = self.send_command(
These are strings, which are objects, but I'd like to be more explicit,
so maybe packet_strs?
> + f"for pakt in {sniffed_packets_name}: print(bytes_base64(pakt.build()))\n"
> + )
> + # In the string of bytes "b'XXXX'", we only want the contents ("XXXX")
> + list_of_packets_base64 = re.findall(
> + f"^b'({REGEX_FOR_BASE64_ENCODING})'", packet_objects, re.MULTILINE
> + )
> + return [Ether(base64_bytes(pakt)) for pakt in list_of_packets_base64]
> +
> + def send_packets_and_capture(
> + self,
> + packets: list[Packet],
> + send_port: Port,
> + recv_port: Port,
> + filter_config: str,
> + duration: float,
> + ) -> list[Packet]:
> + """Send packets and capture any received traffic.
> +
> + The steps required to collect these packets are creating a sniffer that holds the packets to
> + send then starting and stopping the sniffer.
> +
> + Args:
> + packets: The packets to send.
> + send_port: The port to send the packets from.
> + recv_port: The port to collect received packets from.
> + filter_config: The filter to use while sniffing for packets.
> + duration: The amount of time in seconds to sniff for received packets.
> +
> + Returns:
> + A list of packets received after sending `packets`.
> + """
> + self._create_sniffer(packets, send_port, recv_port, filter_config)
> + return self._start_and_stop_sniffing(duration)