From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id DFADB424CA; Tue, 11 Jun 2024 13:12:07 +0200 (CEST) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 9DEC540263; Tue, 11 Jun 2024 13:12:07 +0200 (CEST) Received: from mail-wr1-f41.google.com (mail-wr1-f41.google.com [209.85.221.41]) by mails.dpdk.org (Postfix) with ESMTP id 2AE134021F for ; Tue, 11 Jun 2024 13:12:06 +0200 (CEST) Received: by mail-wr1-f41.google.com with SMTP id ffacd0b85a97d-35f27eed98aso945958f8f.2 for ; Tue, 11 Jun 2024 04:12:06 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=pantheon.tech; s=google; t=1718104326; x=1718709126; darn=dpdk.org; h=content-transfer-encoding:in-reply-to:from:content-language :references:cc:to:subject:user-agent:mime-version:date:message-id :from:to:cc:subject:date:message-id:reply-to; bh=9h8lPndWNsyfUxnYq/xqiWbRxTGeE08J2s/ICrDzG8A=; b=i6keGXMJw3E5sZ/Ts2t6nSmiHenknZWLEZskoZ8t0s25WOIPbV0UXhOHyb3QLHO4fy e9GATRg+/Y9/JujH2ONEj3Z3F8kV4UZSW067NpSmARp+NN4OMVS5F79DmR3hgiLFGgIg Z6A19ZzvNZx02k5WjYFlLGLe0BNFjiQizvFd58jFtrxMtlLhOfFHUpbCQa64oLcijZua AO06pGLws1FMT77b8kj7/lxZMI7FIHfPGMu98mZ8cw7w0w8yHWqUynB6nKu/Bo/c4D4G ntjBoIQs/Xc1gxpLnP3DSDoNfNXxAqJIN86NEdZNfmPHMNaWY3466vHXKm1Buig//NH2 v24Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1718104326; x=1718709126; h=content-transfer-encoding:in-reply-to:from:content-language :references:cc:to:subject:user-agent:mime-version:date:message-id :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=9h8lPndWNsyfUxnYq/xqiWbRxTGeE08J2s/ICrDzG8A=; b=w/Ivxise9gqrOkgsNYs08Uhrz/XnP93bTYBzo0JYCV24eJ1In4YfAiWFOrWIXaPkAy +5JB6814SOo/RU+TgMdoNMZ0x9J31Li5qg9xjrPaBRLT4XobI5B5q7Sl9Dsy+0tsm1qY s4f06o/Nw/gEJkQG1p46NJuGkqdADFVeXHRFPTLB7H2C5hS/zrkg+IAWOxO5o79Sp1y1 oChZ2KWJdD1TpAHJ1bEkqGRrTshzmPCCF2r0fcGwetkz6Ywr6eJpTAcEGT48eLXg/vvh pN9APlS3unWUa3XcPbCn1lLZMrMBcLFY5/TOJyF2BCBoDfqV2aAwdVCcQguakZct+rta Dzog== X-Gm-Message-State: AOJu0Yz2p1kjJdp2a8mqYsomTQv/8zkiHpnbJLG1TuKCRHd2cFmdZgv/ jApk3kXJ8nZnhgw8/LGs9Fq00VnTyQwuSdCjM/irX4EpVx8W/QnoBT7jk6sczdM= X-Google-Smtp-Source: AGHT+IHCbXTce5+wnjTwVhEIljrdGIZ4JXjTZ4qiPlq1AgXx4U0aSuw76xT0WbRLP2m3ysJ8NCgrfA== X-Received: by 2002:a5d:4f0c:0:b0:35f:23f8:b295 with SMTP id ffacd0b85a97d-35f23f8b2d4mr3573502f8f.69.1718104325589; Tue, 11 Jun 2024 04:12:05 -0700 (PDT) Received: from [10.12.0.137] (81.89.53.154.host.vnet.sk. [81.89.53.154]) by smtp.gmail.com with ESMTPSA id ffacd0b85a97d-35f0ed44b29sm9590995f8f.66.2024.06.11.04.12.04 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Tue, 11 Jun 2024 04:12:05 -0700 (PDT) Message-ID: <4a25d1c0-d793-4503-a943-f3b7fe9749d5@pantheon.tech> Date: Tue, 11 Jun 2024 13:12:04 +0200 MIME-Version: 1.0 User-Agent: Mozilla Thunderbird Subject: Re: [RFC PATCH v1 1/2] dts: Add interactive shell for managing Scapy To: jspewock@iol.unh.edu, Luca.Vizzarro@arm.com, probb@iol.unh.edu, npratte@iol.unh.edu, paul.szczepanek@arm.com, yoan.picchi@foss.arm.com, thomas@monjalon.net, wathsala.vithanage@arm.com, Honnappa.Nagarahalli@arm.com Cc: dev@dpdk.org References: <20240605175227.7003-1-jspewock@iol.unh.edu> <20240605175227.7003-2-jspewock@iol.unh.edu> Content-Language: en-US From: =?UTF-8?Q?Juraj_Linke=C5=A1?= In-Reply-To: <20240605175227.7003-2-jspewock@iol.unh.edu> Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org > diff --git a/dts/framework/remote_session/scapy_shell.py b/dts/framework/remote_session/scapy_shell.py > new file mode 100644 > index 0000000000..fa647dc870 > --- /dev/null > +++ b/dts/framework/remote_session/scapy_shell.py > @@ -0,0 +1,175 @@ > +# SPDX-License-Identifier: BSD-3-Clause > +# Copyright(c) 2024 University of New Hampshire > + > +"""Scapy interactive shell.""" > + > +import re > +import time > +from typing import Callable, ClassVar > + > +from scapy.compat import base64_bytes # type: ignore[import] > +from scapy.layers.l2 import Ether # type: ignore[import] > +from scapy.packet import Packet # type: ignore[import] > + > +from framework.testbed_model.port import Port > +from framework.utils import REGEX_FOR_BASE64_ENCODING > + > +from .python_shell import PythonShell > + > + > +class ScapyShell(PythonShell): > + """Scapy interactive shell. > + > + The scapy shell is implemented using a :class:`~.python_shell.PythonShell` and importing > + everything from the "scapy.all" library. This is done due to formatting issues that occur from > + the scapy interactive shell attempting to use iPython, which is not compatible with the > + pseudo-terminal that paramiko creates to manage its channels. > + > + This class is used as an underlying session for the scapy traffic generator and shouldn't be > + used directly inside of test suites. If there isn't a method in > + :class:`framework.testbed_model.traffic_generator.scapy.ScapyTrafficGenerator` to fulfill a > + need, one should be added there and implemented here. > + """ > + > + #: Name of sniffer to ensure the same is used in all places > + _sniffer_name: ClassVar[str] = "sniffer" > + #: Name of variable that points to the list of packets inside the scapy shell. > + _send_packet_list_name: ClassVar[str] = "packets" > + #: Padding to add to the start of a line for python syntax compliance. > + _padding: ClassVar[str] = " " * 4 > + > + def _start_application(self, get_privileged_command: Callable[[str], str] | None) -> None: > + """Overrides :meth:`~.interactive_shell._start_application`. This extends the method and in that case we should mention what the extension is. > + > + Adds a command that imports everything from the scapy library immediately after starting > + the shell for usage in later calls to the methods of this class. > + > + Args: > + get_privileged_command: A function (but could be any callable) that produces > + the version of the command with elevated privileges. > + """ > + super()._start_application(get_privileged_command) > + self.send_command("from scapy.all import *") > + > + def _build_send_packet_list(self, packets: list[Packet]) -> None: The send in the name evokes that the method sends the packets. The description in the Args section says "packets to recreate in the shell" and I like that so I'd put that in the name: _create_packet_list() > + """Build a list of packets to send later. > + > + Gets the string that represents the Python command that was used to create each packet in Gets the string sounds like that's what the methods returns, as a getter method would. > + `packets` and sends these commands into the underlying Python session. The purpose behind > + doing this is to create a list that is identical to `packets` inside the shell. This method > + should only be called by methods for sending packets immediately prior to sending. The list > + of packets will continue to exist in the scope of the shell until subsequent calls to this > + method, so failure to rebuild the list prior to sending packets could lead to undesired > + "stale" packets to be sent. > + > + Args: > + packets: The list of packets to recreate in the shell. > + """ > + self._logger.info("Building a list of packets to send...") The could be just a regular dot instead of the ellipsis (I don't like random ellipses as those read as if I was supposed to expect something and we don't provide a subsequent log that would continue this ellipsis). > + self.send_command( > + f"{self._send_packet_list_name} = [{', '.join(map(Packet.command, packets))}]" > + ) > + > + def send_packets(self, packets: list[Packet], send_port: Port) -> None: > + """Send packets without capturing any received traffic. > + > + Provides a "fire and forget" method for sending packets for situations when there is no > + need to collected any received traffic. Typo: collected > + > + Args: > + packets: The packets to send. > + send_port: The port to send the packets from. > + """ > + self._build_send_packet_list(packets) > + send_command = [ > + "sendp(", > + f"{self._send_packet_list_name},", > + f"iface='{send_port.logical_name}',", > + "realtime=True,", > + "verbose=True", > + ")", > + ] > + self.send_command(f"\n{self._padding}".join(send_command)) > + > + def _create_sniffer( > + self, packets_to_send: list[Packet], send_port: Port, recv_port: Port, filter_config: str > + ) -> None: > + """Create an asynchronous sniffer in the shell. > + > + A list of packets to send is added to the sniffer inside of a callback function so that > + they are immediately sent at the time sniffing is started. > + > + Args: > + packets_to_send: A list of packets to send when sniffing is started. > + send_port: The port to send the packets on when sniffing is started. > + recv_port: The port to collect the traffic from. > + filter_config: An optional BPF format filter to use when sniffing for packets. Omitted > + when set to an empty string. > + """ > + self._build_send_packet_list(packets_to_send) > + sniffer_commands = [ > + f"{self._sniffer_name} = AsyncSniffer(", > + f"iface='{recv_port.logical_name}',", > + "store=True,", > + "started_callback=lambda *args: sendp(", > + f"{self._padding}{self._send_packet_list_name}, iface='{send_port.logical_name}'),", > + ")", > + ] > + if filter_config: > + sniffer_commands.insert(-1, f"filter='{filter_config}'") > + > + self.send_command(f"\n{self._padding}".join(sniffer_commands)) > + > + def _start_and_stop_sniffing(self, duration: float) -> list[Packet]: > + """Starts asynchronous sniffer, runs for a set `duration`, then collects received packets. This should be in imperative to align with the rest of the docstrings. > + > + This method expects that you have first created an asynchronous sniffer inside the shell > + and will fail if you haven't. Received packets are collected by printing the base64 > + encoding of each packet in the shell and then harvesting these encodings using regex to > + convert back into packet objects. > + > + Args: > + duration: The amount of time in seconds to sniff for received packets. > + > + Returns: > + A list of all packets that were received while the sniffer was running. > + """ > + sniffed_packets_name = "gathered_packets" > + self.send_command(f"{self._sniffer_name}.start()") > + time.sleep(duration) > + self.send_command(f"{sniffed_packets_name} = {self._sniffer_name}.stop(join=True)") > + # An extra newline is required here due to the nature of interactive Python shells > + packet_objects = self.send_command( These are strings, which are objects, but I'd like to be more explicit, so maybe packet_strs? > + f"for pakt in {sniffed_packets_name}: print(bytes_base64(pakt.build()))\n" > + ) > + # In the string of bytes "b'XXXX'", we only want the contents ("XXXX") > + list_of_packets_base64 = re.findall( > + f"^b'({REGEX_FOR_BASE64_ENCODING})'", packet_objects, re.MULTILINE > + ) > + return [Ether(base64_bytes(pakt)) for pakt in list_of_packets_base64] > + > + def send_packets_and_capture( > + self, > + packets: list[Packet], > + send_port: Port, > + recv_port: Port, > + filter_config: str, > + duration: float, > + ) -> list[Packet]: > + """Send packets and capture any received traffic. > + > + The steps required to collect these packets are creating a sniffer that holds the packets to > + send then starting and stopping the sniffer. > + > + Args: > + packets: The packets to send. > + send_port: The port to send the packets from. > + recv_port: The port to collect received packets from. > + filter_config: The filter to use while sniffing for packets. > + duration: The amount of time in seconds to sniff for received packets. > + > + Returns: > + A list of packets received after sending `packets`. > + """ > + self._create_sniffer(packets, send_port, recv_port, filter_config) > + return self._start_and_stop_sniffing(duration)