From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 9F56D45484; Mon, 17 Jun 2024 21:45:56 +0200 (CEST) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 2073A402DA; Mon, 17 Jun 2024 21:45:56 +0200 (CEST) Received: from mail-pf1-f170.google.com (mail-pf1-f170.google.com [209.85.210.170]) by mails.dpdk.org (Postfix) with ESMTP id BFBF2402BD for ; Mon, 17 Jun 2024 21:45:53 +0200 (CEST) Received: by mail-pf1-f170.google.com with SMTP id d2e1a72fcca58-7061365d2f3so578627b3a.3 for ; Mon, 17 Jun 2024 12:45:53 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=iol.unh.edu; s=unh-iol; t=1718653553; x=1719258353; darn=dpdk.org; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:from:to:cc:subject:date :message-id:reply-to; bh=s4Um2mIDMMZpck5vpqi1wWV1rzDdVdbfZ8ESPIr8800=; b=axGekXbSYW+NQH3AUbT42nzB6Cdi6a+gvaTiSHCpZeuszvSXtd767kbCCiuvu7N7di GQhl0CTX6iG2MSZ1ahrJchdfd7e8AHWz+PHQ2Of/4BHviqFr9wpBbYiRibGYmjhfYAEe MNiBA3n5LtA1z1PT3IOkfxF8c6fwAZQv6feyo= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1718653553; x=1719258353; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=s4Um2mIDMMZpck5vpqi1wWV1rzDdVdbfZ8ESPIr8800=; b=AfqF1f86Xyaa3C8RVZOqNc5XEJP/ePa5gD5+Vd0EJXYiFRhg8ggBBzuginWHvCtsnT PEs2frQBEQdAggZJH0Kockz4VGd6XKBwZhwWaFWc0L+cOt4kZq2cib0/VYPtlI1AhOHp HU8MTe2LmC0jFj7bwxgv63LbwJ1coH1Wd8QpFPjzyyfB73Wq1qRwOMugV8KnkDKgn3Da 8/h1E0p7Seld3TNRJusmPD/+X7Uel66/1X8v2yzY1XDA7XkjRZPLzKacyAjT2b4LRWFE 1g4L1CfosngmArzqXSyyGs49zotaFSxpk8Pt9rJYKHK5jZXyT0HKme1c3gDj2+vPd1SP NaIQ== X-Forwarded-Encrypted: i=1; AJvYcCWPHRRSKjozIzCdV3A2v/rE8jwiUa3p0BYuJ5rKfzxPBwdim8K9WpM1oK8209rFRc/ZkeRK5fDGshgzO2U= X-Gm-Message-State: AOJu0YydUI6acER5xS4tfxRxfHFib5clQQ4sk8x68Gb5ZyssaYFPURLT xSGANbZIeBgIuhqvMcCLZ0l2OJp6Sr14GFUgs1zscVUC1oeX+PRyE0bYgY/walIY+8aFF/oIuaV YPhAD0EtgSgnwK6L1aCRxx7BHAi+tqVsJWZ2YDQ== X-Google-Smtp-Source: AGHT+IGzWNsBwgUY6H8+cLZnB0+ARVT/y9TCrA6ZlJZT8nI027tbKVIrVDtkiI3gdZ+yE4DP+oEz8sl8vXmNd63AqWE= X-Received: by 2002:a05:6a20:394a:b0:1b8:af57:7bc9 with SMTP id adf61e73a8af0-1bae7f5a27amr10315267637.37.1718653552674; Mon, 17 Jun 2024 12:45:52 -0700 (PDT) MIME-Version: 1.0 References: <20240605175227.7003-1-jspewock@iol.unh.edu> <20240605175227.7003-2-jspewock@iol.unh.edu> <4a25d1c0-d793-4503-a943-f3b7fe9749d5@pantheon.tech> In-Reply-To: <4a25d1c0-d793-4503-a943-f3b7fe9749d5@pantheon.tech> From: Jeremy Spewock Date: Mon, 17 Jun 2024 15:45:41 -0400 Message-ID: Subject: Re: [RFC PATCH v1 1/2] dts: Add interactive shell for managing Scapy To: =?UTF-8?Q?Juraj_Linke=C5=A1?= Cc: Luca.Vizzarro@arm.com, probb@iol.unh.edu, npratte@iol.unh.edu, paul.szczepanek@arm.com, yoan.picchi@foss.arm.com, thomas@monjalon.net, wathsala.vithanage@arm.com, Honnappa.Nagarahalli@arm.com, dev@dpdk.org Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org On Tue, Jun 11, 2024 at 7:12=E2=80=AFAM Juraj Linke=C5=A1 wrote: > > > > diff --git a/dts/framework/remote_session/scapy_shell.py b/dts/framewor= k/remote_session/scapy_shell.py > > new file mode 100644 > > index 0000000000..fa647dc870 > > --- /dev/null > > +++ b/dts/framework/remote_session/scapy_shell.py > > @@ -0,0 +1,175 @@ > > +# SPDX-License-Identifier: BSD-3-Clause > > +# Copyright(c) 2024 University of New Hampshire > > + > > +"""Scapy interactive shell.""" > > + > > +import re > > +import time > > +from typing import Callable, ClassVar > > + > > +from scapy.compat import base64_bytes # type: ignore[import] > > +from scapy.layers.l2 import Ether # type: ignore[import] > > +from scapy.packet import Packet # type: ignore[import] > > + > > +from framework.testbed_model.port import Port > > +from framework.utils import REGEX_FOR_BASE64_ENCODING > > + > > +from .python_shell import PythonShell > > + > > + > > +class ScapyShell(PythonShell): > > + """Scapy interactive shell. > > + > > + The scapy shell is implemented using a :class:`~.python_shell.Pyth= onShell` and importing > > + everything from the "scapy.all" library. This is done due to forma= tting issues that occur from > > + the scapy interactive shell attempting to use iPython, which is no= t compatible with the > > + pseudo-terminal that paramiko creates to manage its channels. > > + > > + This class is used as an underlying session for the scapy traffic = generator and shouldn't be > > + used directly inside of test suites. If there isn't a method in > > + :class:`framework.testbed_model.traffic_generator.scapy.ScapyTraff= icGenerator` to fulfill a > > + need, one should be added there and implemented here. > > + """ > > + > > + #: Name of sniffer to ensure the same is used in all places > > + _sniffer_name: ClassVar[str] =3D "sniffer" > > + #: Name of variable that points to the list of packets inside the = scapy shell. > > + _send_packet_list_name: ClassVar[str] =3D "packets" > > + #: Padding to add to the start of a line for python syntax complia= nce. > > + _padding: ClassVar[str] =3D " " * 4 > > + > > + def _start_application(self, get_privileged_command: Callable[[str= ], str] | None) -> None: > > + """Overrides :meth:`~.interactive_shell._start_application`. > > This extends the method and in that case we should mention what the > extension is. Ack. > > > + > > + Adds a command that imports everything from the scapy library = immediately after starting > > + the shell for usage in later calls to the methods of this clas= s. > > + > > + Args: > > + get_privileged_command: A function (but could be any calla= ble) that produces > > + the version of the command with elevated privileges. > > + """ > > + super()._start_application(get_privileged_command) > > + self.send_command("from scapy.all import *") > > + > > + def _build_send_packet_list(self, packets: list[Packet]) -> None: > > The send in the name evokes that the method sends the packets. > > The description in the Args section says "packets to recreate in the > shell" and I like that so I'd put that in the name: _create_packet_list() > Great point. I was trying too hard to force the idea of building the "list of packets that will be sent" but on its own this just builds a list of packets. > > + """Build a list of packets to send later. > > + > > + Gets the string that represents the Python command that was us= ed to create each packet in > > Gets the string sounds like that's what the methods returns, as a getter > method would. Fair enough, I'll edit this. > > > + `packets` and sends these commands into the underlying Python = session. The purpose behind > > + doing this is to create a list that is identical to `packets` = inside the shell. This method > > + should only be called by methods for sending packets immediate= ly prior to sending. The list > > + of packets will continue to exist in the scope of the shell un= til subsequent calls to this > > + method, so failure to rebuild the list prior to sending packet= s could lead to undesired > > + "stale" packets to be sent. > > + > > + Args: > > + packets: The list of packets to recreate in the shell. > > + """ > > + self._logger.info("Building a list of packets to send...") > > The could be just a regular dot instead of the ellipsis (I don't like > random ellipses as those read as if I was supposed to expect something > and we don't provide a subsequent log that would continue this ellipsis). Ack. > > > + self.send_command( > > + f"{self._send_packet_list_name} =3D [{', '.join(map(Packet= .command, packets))}]" > > + ) > > + > > + def send_packets(self, packets: list[Packet], send_port: Port) -> = None: > > + """Send packets without capturing any received traffic. > > + > > + Provides a "fire and forget" method for sending packets for si= tuations when there is no > > + need to collected any received traffic. > > Typo: collected Good catch! > > > + > > + Args: > > + packets: The packets to send. > > + send_port: The port to send the packets from. > > + """ > > + self._build_send_packet_list(packets) > > + send_command =3D [ > > + "sendp(", > > + f"{self._send_packet_list_name},", > > + f"iface=3D'{send_port.logical_name}',", > > + "realtime=3DTrue,", > > + "verbose=3DTrue", > > + ")", > > + ] > > + self.send_command(f"\n{self._padding}".join(send_command)) > > + > > + def _create_sniffer( > > + self, packets_to_send: list[Packet], send_port: Port, recv_por= t: Port, filter_config: str > > + ) -> None: > > + """Create an asynchronous sniffer in the shell. > > + > > + A list of packets to send is added to the sniffer inside of a = callback function so that > > + they are immediately sent at the time sniffing is started. > > + > > + Args: > > + packets_to_send: A list of packets to send when sniffing i= s started. > > + send_port: The port to send the packets on when sniffing i= s started. > > + recv_port: The port to collect the traffic from. > > + filter_config: An optional BPF format filter to use when s= niffing for packets. Omitted > > + when set to an empty string. > > + """ > > + self._build_send_packet_list(packets_to_send) > > + sniffer_commands =3D [ > > + f"{self._sniffer_name} =3D AsyncSniffer(", > > + f"iface=3D'{recv_port.logical_name}',", > > + "store=3DTrue,", > > + "started_callback=3Dlambda *args: sendp(", > > + f"{self._padding}{self._send_packet_list_name}, iface=3D'{= send_port.logical_name}'),", > > + ")", > > + ] > > + if filter_config: > > + sniffer_commands.insert(-1, f"filter=3D'{filter_config}'") > > + > > + self.send_command(f"\n{self._padding}".join(sniffer_commands)) > > + > > + def _start_and_stop_sniffing(self, duration: float) -> list[Packet= ]: > > + """Starts asynchronous sniffer, runs for a set `duration`, the= n collects received packets. > > This should be in imperative to align with the rest of the docstrings. Ack. > > > + > + This method expects that you have first created an > asynchronous sniffer inside the shell > > + and will fail if you haven't. Received packets are collected b= y printing the base64 > > + encoding of each packet in the shell and then harvesting these= encodings using regex to > > + convert back into packet objects. > > + > > + Args: > > + duration: The amount of time in seconds to sniff for recei= ved packets. > > + > > + Returns: > > + A list of all packets that were received while the sniffer= was running. > > + """ > > + sniffed_packets_name =3D "gathered_packets" > > + self.send_command(f"{self._sniffer_name}.start()") > > + time.sleep(duration) > > + self.send_command(f"{sniffed_packets_name} =3D {self._sniffer_= name}.stop(join=3DTrue)") > > + # An extra newline is required here due to the nature of inter= active Python shells > > + packet_objects =3D self.send_command( > > These are strings, which are objects, but I'd like to be more explicit, > so maybe packet_strs? Good point, that would be more clear. > > > + f"for pakt in {sniffed_packets_name}: print(bytes_base64(p= akt.build()))\n" > > + ) > > + # In the string of bytes "b'XXXX'", we only want the contents = ("XXXX") > > + list_of_packets_base64 =3D re.findall( > > + f"^b'({REGEX_FOR_BASE64_ENCODING})'", packet_objects, re.M= ULTILINE > > + ) > > + return [Ether(base64_bytes(pakt)) for pakt in list_of_packets_= base64] > > + > > + def send_packets_and_capture( > > + self, > > + packets: list[Packet], > > + send_port: Port, > > + recv_port: Port, > > + filter_config: str, > > + duration: float, > > + ) -> list[Packet]: > > + """Send packets and capture any received traffic. > > + > > + The steps required to collect these packets are creating a sni= ffer that holds the packets to > > + send then starting and stopping the sniffer. > > + > > + Args: > > + packets: The packets to send. > > + send_port: The port to send the packets from. > > + recv_port: The port to collect received packets from. > > + filter_config: The filter to use while sniffing for packet= s. > > + duration: The amount of time in seconds to sniff for recei= ved packets. > > + > > + Returns: > > + A list of packets received after sending `packets`. > > + """ > > + self._create_sniffer(packets, send_port, recv_port, filter_con= fig) > > + return self._start_and_stop_sniffing(duration)