From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id E85EE43382; Wed, 22 Nov 2023 14:18:44 +0100 (CET) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 7C04A402E8; Wed, 22 Nov 2023 14:18:44 +0100 (CET) Received: from mail-ej1-f48.google.com (mail-ej1-f48.google.com [209.85.218.48]) by mails.dpdk.org (Postfix) with ESMTP id BC5164028C for ; Wed, 22 Nov 2023 14:18:42 +0100 (CET) Received: by mail-ej1-f48.google.com with SMTP id a640c23a62f3a-a03a9009572so139530566b.3 for ; Wed, 22 Nov 2023 05:18:42 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=pantheon.tech; s=google; t=1700659122; x=1701263922; darn=dpdk.org; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:from:to:cc:subject:date :message-id:reply-to; bh=4RcOJ/bdqZfAM2UMjkxDL4dOFToO//1xxGSiFqLEvlU=; b=ltvvxqA3gp2bDyB0XjXsveI71qje1kXODzXn1YtEF18Qc54Zh638FNtlQUpJCnKSpx l9xANT7Oh4IMAvabjMHZRNIpPeqcA29xBgRRtp12B8gGErwUP6E7dgPqiceqKnFRVOds pFwj4eWfESc2LhTpZ7nuLv6Aw5SkwQeGfVJI1ULATu3lakH/CmDmsd3hTWkqTbxR6XT9 me4BBCO0k+cscDLzb/Tjym+EX/gETLobNvALw9u9VI2lTQbzwqYTmN04hBQ/k6zvV8Sc DXN8B/si/1SNgwpOAYX6hgPxgHdpqhS9H76BzeR/aIUxO/IzTKT1ogSQEH9cIozUbQJz VlwQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1700659122; x=1701263922; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=4RcOJ/bdqZfAM2UMjkxDL4dOFToO//1xxGSiFqLEvlU=; b=HI0mXqlQURqURBpiE9M4cgOZ/MQxXtg6m1qFaDVEtATLdi0y7NJmc4VD6PXQ2AwGRU t76Bg9N0Cjq/Eb6Z2SxKWUzPiQY5yhm4JBYMRseS3+lOupxuuVtXDmhHq8bQPfnFaUmO 39DvhFeg3HYYy0gvGuDLXDd/6b3RKY7mTVhnBwFisoqMNTWiqPQXc+kVAznpx+gl1gM2 /Aja1M8OP1xtTWvAAY904/hVH9D+FcWwJjBC1HjHoPSZquseljWlHVANN6wv+qYlMN7t mv4Mw64SOGM8ImTjB3IUni43PnUqQLSglXtcxy6a1dlnVY1aQ5j6IsXZLbSacqGtPKMh KjqA== X-Gm-Message-State: AOJu0Yw2oOHknP+xi+DZRSlxrNwi2eTsUew2+L8AFMQ5dOCaiuGJb0IM 3QI9OeXj9+crfLP32jU1e9u8EIcmBsLtr4jM5mJ+sOQZQ4LgP7uxo6A= X-Google-Smtp-Source: AGHT+IG+jBTIm89lZLUacKkM1x2xXxDCDgcBvvMD+ee2LgZqs+ofRkxjI0dIbbSipQdmR3Ns01aaJkqj/tDOUu2AqMA= X-Received: by 2002:a17:906:3e0d:b0:a04:9ed9:1a6a with SMTP id k13-20020a1709063e0d00b00a049ed91a6amr1387432eji.31.1700659122305; Wed, 22 Nov 2023 05:18:42 -0800 (PST) MIME-Version: 1.0 References: <20231108125324.191005-23-juraj.linkes@pantheon.tech> <20231115130959.39420-1-juraj.linkes@pantheon.tech> <20231115130959.39420-21-juraj.linkes@pantheon.tech> In-Reply-To: From: =?UTF-8?Q?Juraj_Linke=C5=A1?= Date: Wed, 22 Nov 2023 14:18:31 +0100 Message-ID: Subject: Re: [PATCH v7 20/21] dts: scapy tg docstring update To: Yoan Picchi Cc: thomas@monjalon.net, Honnappa.Nagarahalli@arm.com, jspewock@iol.unh.edu, probb@iol.unh.edu, paul.szczepanek@arm.com, dev@dpdk.org Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org On Tue, Nov 21, 2023 at 5:33=E2=80=AFPM Yoan Picchi wrote: > > On 11/15/23 13:09, Juraj Linke=C5=A1 wrote: > > Format according to the Google format and PEP257, with slight > > deviations. > > > > Signed-off-by: Juraj Linke=C5=A1 > > --- > > .../testbed_model/traffic_generator/scapy.py | 91 +++++++++++-------= - > > 1 file changed, 54 insertions(+), 37 deletions(-) > > > > diff --git a/dts/framework/testbed_model/traffic_generator/scapy.py b/d= ts/framework/testbed_model/traffic_generator/scapy.py > > index 51864b6e6b..ed4f879925 100644 > > --- a/dts/framework/testbed_model/traffic_generator/scapy.py > > +++ b/dts/framework/testbed_model/traffic_generator/scapy.py > > @@ -2,14 +2,15 @@ > > # Copyright(c) 2022 University of New Hampshire > > # Copyright(c) 2023 PANTHEON.tech s.r.o. > > > > -"""Scapy traffic generator. > > +"""The Scapy traffic generator. > > > > -Traffic generator used for functional testing, implemented using the S= capy library. > > +A traffic generator used for functional testing, implemented with > > +`the Scapy library `_. > > The traffic generator uses an XML-RPC server to run Scapy on the remo= te TG node. > > > > -The XML-RPC server runs in an interactive remote SSH session running P= ython console, > > -where we start the server. The communication with the server is facili= tated with > > -a local server proxy. > > +The traffic generator uses the :mod:`xmlrpc.server` module to run an X= ML-RPC server > > +in an interactive remote Python SSH session. The communication with th= e server is facilitated > > +with a local server proxy from the :mod:`xmlrpc.client` module. > > """ > > > > import inspect > > @@ -69,20 +70,20 @@ def scapy_send_packets_and_capture( > > recv_iface: str, > > duration: float, > > ) -> list[bytes]: > > - """RPC function to send and capture packets. > > + """The RPC function to send and capture packets. > > > > - The function is meant to be executed on the remote TG node. > > + The function is meant to be executed on the remote TG node via the= server proxy. > > > > Args: > > xmlrpc_packets: The packets to send. These need to be convert= ed to > > - xmlrpc.client.Binary before sending to the remote server. > > + :class:`~xmlrpc.client.Binary` objects before sending to t= he remote server. > > The string is not raw and no \s. As per you explanation a few commits > earlier this might cause an issue with the tilda ? > Looking around I see it also happen several time here and also in the > previous commit. > The issue is not with the tilda, but with backticks. When backticks are followed by certain characters (I mentioned alphanumeric characters, but there may be others), the character right after the backtick must be escaped and the raw string results in the right escaping for Sphinx. > > send_iface: The logical name of the egress interface. > > recv_iface: The logical name of the ingress interface. > > duration: Capture for this amount of time, in seconds. > > > > Returns: > > A list of bytes. Each item in the list represents one packet,= which needs > > - to be converted back upon transfer from the remote node. > > + to be converted back upon transfer from the remote node. > > """ > > scapy_packets =3D [scapy.all.Packet(packet.data) for packet in xm= lrpc_packets] > > sniffer =3D scapy.all.AsyncSniffer( > > @@ -98,19 +99,15 @@ def scapy_send_packets_and_capture( > > def scapy_send_packets( > > xmlrpc_packets: list[xmlrpc.client.Binary], send_iface: str > > ) -> None: > > - """RPC function to send packets. > > + """The RPC function to send packets. > > > > - The function is meant to be executed on the remote TG node. > > - It doesn't return anything, only sends packets. > > + The function is meant to be executed on the remote TG node via the= server proxy. > > + It only sends `xmlrpc_packets`, without capturing them. > > > > Args: > > xmlrpc_packets: The packets to send. These need to be convert= ed to > > - xmlrpc.client.Binary before sending to the remote server. > > + :class:`~xmlrpc.client.Binary` objects before sending to t= he remote server. > > send_iface: The logical name of the egress interface. > > - > > - Returns: > > - A list of bytes. Each item in the list represents one packet, = which needs > > - to be converted back upon transfer from the remote node. > > """ > > scapy_packets =3D [scapy.all.Packet(packet.data) for packet in xm= lrpc_packets] > > scapy.all.sendp(scapy_packets, iface=3Dsend_iface, realtime=3DTru= e, verbose=3DTrue) > > @@ -130,11 +127,19 @@ def scapy_send_packets( > > > > > > class QuittableXMLRPCServer(SimpleXMLRPCServer): > > - """Basic XML-RPC server that may be extended > > - by functions serializable by the marshal module. > > + r"""Basic XML-RPC server. > > But you have a raw string here, and I don't see the need why. > There is no need here, I'll remove it. It's a remnant from a much bigger docstring which caused issues when sending the code to the TG node. I've talked to Jeremy and we'll fix it in a separate patch which will introduce the full docstring. > > + > > + The server may be augmented by functions serializable by the :mod:= `marshal` module. > > """ > > > > def __init__(self, *args, **kwargs): > > + """Extend the XML-RPC server initialization. > > + > > + Args: > > + args: The positional arguments that will be passed to the = superclass's constructor. > > + kwargs: The keyword arguments that will be passed to the s= uperclass's constructor. > > + The `allow_none` argument will be set to :data:`True`. > > + """ > > kwargs["allow_none"] =3D True > > super().__init__(*args, **kwargs) > > self.register_introspection_functions() > > @@ -142,13 +147,12 @@ def __init__(self, *args, **kwargs): > > self.register_function(self.add_rpc_function) > > > > def quit(self) -> None: > > + """Quit the server.""" > > self._BaseServer__shutdown_request =3D True > > return None > > > > def add_rpc_function(self, name: str, function_bytes: xmlrpc.clie= nt.Binary) -> None: > > - """Add a function to the server. > > - > > - This is meant to be executed remotely. > > + """Add a function to the server from the local server proxy. > > > > Args: > > name: The name of the function. > > @@ -159,6 +163,11 @@ def add_rpc_function(self, name: str, function_byt= es: xmlrpc.client.Binary) -> N > > self.register_function(function) > > > > def serve_forever(self, poll_interval: float =3D 0.5) -> None: > > + """Extend the superclass method with an additional print. > > + > > + Once executed in the local server proxy, the print gives us a = clear string to expect > > + when starting the server. The print means the function was exe= cuted on the XML-RPC server. > > + """ > > print("XMLRPC OK") > > super().serve_forever(poll_interval) > > > > @@ -166,19 +175,12 @@ def serve_forever(self, poll_interval: float =3D = 0.5) -> None: > > class ScapyTrafficGenerator(CapturingTrafficGenerator): > > """Provides access to scapy functions via an RPC interface. > > > > - The traffic generator first starts an XML-RPC on the remote TG nod= e. > > - Then it populates the server with functions which use the Scapy li= brary > > - to send/receive traffic. > > - > > - Any packets sent to the remote server are first converted to bytes= . > > - They are received as xmlrpc.client.Binary objects on the server si= de. > > - When the server sends the packets back, they are also received as > > - xmlrpc.client.Binary object on the client side, are converted back= to Scapy > > - packets and only then returned from the methods. > > + The class extends the base with remote execution of scapy function= s. > > > > - Arguments: > > - tg_node: The node where the traffic generator resides. > > - config: The user configuration of the traffic generator. > > + Any packets sent to the remote server are first converted to bytes= . They are received as > > + :class:`~xmlrpc.client.Binary` objects on the server side. When th= e server sends the packets > > + back, they are also received as :class:`~xmlrpc.client.Binary` obj= ects on the client side, are > > + converted back to :class:`scapy.packet.Packet` objects and only th= en returned from the methods. > > > > Attributes: > > session: The exclusive interactive remote session created by = the Scapy > > @@ -192,6 +194,22 @@ class ScapyTrafficGenerator(CapturingTrafficGenera= tor): > > _config: ScapyTrafficGeneratorConfig > > > > def __init__(self, tg_node: Node, config: ScapyTrafficGeneratorCo= nfig): > > + """Extend the constructor with Scapy TG specifics. > > + > > + The traffic generator first starts an XML-RPC on the remote `t= g_node`. > > + Then it populates the server with functions which use the Scap= y library > > + to send/receive traffic: > > + > > + * :func:`scapy_send_packets_and_capture` > > + * :func:`scapy_send_packets` > > + > > + To enable verbose logging from the xmlrpc client, use the :opt= ion:`--verbose` > > + command line argument or the :envvar:`DTS_VERBOSE` environment= variable. > > + > > + Args: > > + tg_node: The node where the traffic generator resides. > > + config: The traffic generator's test run configuration. > > + """ > > super().__init__(tg_node, config) > > > > assert ( > > @@ -237,10 +255,8 @@ def _start_xmlrpc_server_in_remote_python(self, li= sten_port: int) -> None: > > [line for line in src.splitlines() if not line.isspace() = and line !=3D ""] > > ) > > > > - spacing =3D "\n" * 4 > > - > > # execute it in the python terminal > > - self.session.send_command(spacing + src + spacing) > > + self.session.send_command(src + "\n") > > self.session.send_command( > > f"server =3D QuittableXMLRPCServer(('0.0.0.0', {listen_po= rt}));" > > f"server.serve_forever()", > > @@ -274,6 +290,7 @@ def _send_packets_and_capture( > > return scapy_packets > > > > def close(self) -> None: > > + """Close the traffic generator.""" > > try: > > self.rpc_server_proxy.quit() > > except ConnectionRefusedError: >