From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 6521E45A2A; Wed, 25 Sep 2024 17:46:33 +0200 (CEST) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 749C3402EF; Wed, 25 Sep 2024 17:46:30 +0200 (CEST) Received: from mail-vs1-f98.google.com (mail-vs1-f98.google.com [209.85.217.98]) by mails.dpdk.org (Postfix) with ESMTP id 513A9402EC for ; Wed, 25 Sep 2024 17:46:28 +0200 (CEST) Received: by mail-vs1-f98.google.com with SMTP id ada2fe7eead31-49bd40d77e5so6953137.0 for ; Wed, 25 Sep 2024 08:46:28 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=iol.unh.edu; s=unh-iol; t=1727279187; x=1727883987; darn=dpdk.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=YmSJ8dk9Q7qvtrpV1m/bfN6uyS62UL+N8jiAmTqQJ9k=; b=ia/shsJpzvZL6Qk+OPMMU7X0roAmHkfLnhW1s+1Bx/pWFy6VG3bhk1l04EKoEEP3A7 RN2KOx10j53vGdAicRjcGzNJqhb7giAYPadxet9W18VRHwtd859vxlyb0uWiNceGCWyl 3LiHt+partvlaOIMxoAD2JNAnj5Z+d6FfszP4= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1727279187; x=1727883987; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=YmSJ8dk9Q7qvtrpV1m/bfN6uyS62UL+N8jiAmTqQJ9k=; b=ck9Pg8sEoGinumwhMZEodgbgBKJlkola5Y9C+JV72cy9mVCd6lqbGdMoQP5MPfnnBw IT6fTusNhOAWNkXNDabDrSS+RKjOfL/rgaa+T28Up5RYcYoxjOZnJjpeEYKu6gKXK5ET iXFXqDVvGdtxE9xCDlFVztJsuLf9Vvicyvve9PHf1YHXF0c9gk00fSEAIpNsepstCwQG Gh5GjuMnRSKHWPOI7Jje2dKH4VzYtleEGDbwzFHalJva3feHGCVqopujIZiB3XKUb4b8 sXjSTXQQN1Wbhd0CctPQ0VhIc4ch4EKZ4qQ+w1ifa6l2xS+F8mDtYwfCBCYZq8a51iYQ RY4A== X-Gm-Message-State: AOJu0YyNUeim0dK+r/uPvEJUOTKLSrQ7uLNQ5APY0Y1ZfxxMK4FW/Jsb /4OuiASEcOvI7B2ZBn5Xok++opJR7Rh8bOrTvLU/YgVri+8dL8/cztxD/boT3ml5vmdpsCHG8kw tRBWp3dFm3ZCk0EDu1ilDyEMvyo/HUAtH X-Google-Smtp-Source: AGHT+IHu4EupEzxUCXgXjKCdJTiR9x/l6UsrLcaZtuTSDLCP4iBWad6SzHfK5Zy7o4b9icAs8exL6LoUE4PR X-Received: by 2002:a05:6102:3e8f:b0:493:de37:b3ef with SMTP id ada2fe7eead31-4a15dcb1385mr3762536137.13.1727279187533; Wed, 25 Sep 2024 08:46:27 -0700 (PDT) Received: from postal.iol.unh.edu (postal.iol.unh.edu. [132.177.123.84]) by smtp-relay.gmail.com with ESMTPS id a1e0cc1a2514c-84e73a9615csm129017241.6.2024.09.25.08.46.27 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Wed, 25 Sep 2024 08:46:27 -0700 (PDT) X-Relaying-Domain: iol.unh.edu Received: from iol.unh.edu (unknown [IPv6:2606:4100:3880:1257::1083]) by postal.iol.unh.edu (Postfix) with ESMTP id 9630B605C351; Wed, 25 Sep 2024 11:46:26 -0400 (EDT) From: jspewock@iol.unh.edu To: Luca.Vizzarro@arm.com, Honnappa.Nagarahalli@arm.com, paul.szczepanek@arm.com, alex.chapman@arm.com, probb@iol.unh.edu, juraj.linkes@pantheon.tech, wathsala.vithanage@arm.com, thomas@monjalon.net, yoan.picchi@foss.arm.com, npratte@iol.unh.edu Cc: dev@dpdk.org, Jeremy Spewock Subject: [PATCH v6 1/1] dts: add text parser for testpmd verbose output Date: Wed, 25 Sep 2024 11:46:12 -0400 Message-ID: <20240925154612.11399-2-jspewock@iol.unh.edu> X-Mailer: git-send-email 2.46.0 In-Reply-To: <20240925154612.11399-1-jspewock@iol.unh.edu> References: <20240729203955.267942-1-jspewock@iol.unh.edu> <20240925154612.11399-1-jspewock@iol.unh.edu> MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org From: Jeremy Spewock Multiple test suites from the old DTS framework rely on being able to consume and interpret the verbose output of testpmd. The new framework doesn't have an elegant way for handling the verbose output, but test suites are starting to be written that rely on it. This patch creates a TextParser class that can be used to extract the verbose information from any testpmd output and also adjusts the `stop` method of the shell to return all output that it collected. Signed-off-by: Jeremy Spewock --- dts/framework/remote_session/testpmd_shell.py | 537 +++++++++++++++++- dts/framework/utils.py | 3 + 2 files changed, 538 insertions(+), 2 deletions(-) diff --git a/dts/framework/remote_session/testpmd_shell.py b/dts/framework/remote_session/testpmd_shell.py index 77902a468d..4ca34505b7 100644 --- a/dts/framework/remote_session/testpmd_shell.py +++ b/dts/framework/remote_session/testpmd_shell.py @@ -32,7 +32,7 @@ from framework.settings import SETTINGS from framework.testbed_model.cpu import LogicalCoreCount, LogicalCoreList from framework.testbed_model.sut_node import SutNode -from framework.utils import StrEnum +from framework.utils import REGEX_FOR_MAC_ADDRESS, StrEnum P = ParamSpec("P") TestPmdShellMethod = Callable[Concatenate["TestPmdShell", P], Any] @@ -581,6 +581,506 @@ class TestPmdPortStats(TextParser): tx_bps: int = field(metadata=TextParser.find_int(r"Tx-bps:\s+(\d+)")) +class PacketOffloadFlag(Flag): + """Flag representing the Packet Offload Features Flags in DPDK. + + Values in this class are taken from the definitions in the RTE MBUF core library in DPDK + located in ``lib/mbuf/rte_mbuf_core.h``. It is expected that flag values in this class will + match the values they are set to in said DPDK library with one exception; all values must be + unique. For example, the definitions for unknown checksum flags in ``rte_mbuf_core.h`` are all + set to :data:`0`, but it is valuable to distinguish between them in this framework. For this + reason flags that are not unique in the DPDK library are set either to values within the + RTE_MBUF_F_FIRST_FREE-RTE_MBUF_F_LAST_FREE range for Rx or shifted 61+ bits for Tx. + + References: + DPDK lib: ``lib/mbuf/rte_mbuf_core.h`` + """ + + # RX flags + + #: The RX packet is a 802.1q VLAN packet, and the tci has been saved in mbuf->vlan_tci. If the + #: flag RTE_MBUF_F_RX_VLAN_STRIPPED is also present, the VLAN header has been stripped from + #: mbuf data, else it is still present. + RTE_MBUF_F_RX_VLAN = auto() + + #: RX packet with RSS hash result. + RTE_MBUF_F_RX_RSS_HASH = auto() + + #: RX packet with FDIR match indicate. + RTE_MBUF_F_RX_FDIR = auto() + + #: This flag is set when the outermost IP header checksum is detected as wrong by the hardware. + RTE_MBUF_F_RX_OUTER_IP_CKSUM_BAD = 1 << 5 + + #: A vlan has been stripped by the hardware and its tci is saved in mbuf->vlan_tci. This can + #: only happen if vlan stripping is enabled in the RX configuration of the PMD. When + #: RTE_MBUF_F_RX_VLAN_STRIPPED is set, RTE_MBUF_F_RX_VLAN must also be set. + RTE_MBUF_F_RX_VLAN_STRIPPED = auto() + + #: No information about the RX IP checksum. Value is 0 in the DPDK library. + RTE_MBUF_F_RX_IP_CKSUM_UNKNOWN = 1 << 23 + #: The IP checksum in the packet is wrong. + RTE_MBUF_F_RX_IP_CKSUM_BAD = 1 << 4 + #: The IP checksum in the packet is valid. + RTE_MBUF_F_RX_IP_CKSUM_GOOD = 1 << 7 + #: The IP checksum is not correct in the packet data, but the integrity of the IP header is + #: verified. Value is RTE_MBUF_F_RX_IP_CKSUM_BAD | RTE_MBUF_F_RX_IP_CKSUM_GOOD in the DPDK + #: library. + RTE_MBUF_F_RX_IP_CKSUM_NONE = 1 << 24 + + #: No information about the RX L4 checksum. Value is 0 in the DPDK library. + RTE_MBUF_F_RX_L4_CKSUM_UNKNOWN = 1 << 25 + #: The L4 checksum in the packet is wrong. + RTE_MBUF_F_RX_L4_CKSUM_BAD = 1 << 3 + #: The L4 checksum in the packet is valid. + RTE_MBUF_F_RX_L4_CKSUM_GOOD = 1 << 8 + #: The L4 checksum is not correct in the packet data, but the integrity of the L4 data is + #: verified. Value is RTE_MBUF_F_RX_L4_CKSUM_BAD | RTE_MBUF_F_RX_L4_CKSUM_GOOD in the DPDK + #: library. + RTE_MBUF_F_RX_L4_CKSUM_NONE = 1 << 26 + + #: RX IEEE1588 L2 Ethernet PT Packet. + RTE_MBUF_F_RX_IEEE1588_PTP = 1 << 9 + #: RX IEEE1588 L2/L4 timestamped packet. + RTE_MBUF_F_RX_IEEE1588_TMST = 1 << 10 + + #: FD id reported if FDIR match. + RTE_MBUF_F_RX_FDIR_ID = 1 << 13 + #: Flexible bytes reported if FDIR match. + RTE_MBUF_F_RX_FDIR_FLX = 1 << 14 + + #: If both RTE_MBUF_F_RX_QINQ_STRIPPED and RTE_MBUF_F_RX_VLAN_STRIPPED are set, the 2 VLANs + #: have been stripped by the hardware. If RTE_MBUF_F_RX_QINQ_STRIPPED is set and + #: RTE_MBUF_F_RX_VLAN_STRIPPED is unset, only the outer VLAN is removed from packet data. + RTE_MBUF_F_RX_QINQ_STRIPPED = auto() + + #: When packets are coalesced by a hardware or virtual driver, this flag can be set in the RX + #: mbuf, meaning that the m->tso_segsz field is valid and is set to the segment size of + #: original packets. + RTE_MBUF_F_RX_LRO = auto() + + #: Indicate that security offload processing was applied on the RX packet. + RTE_MBUF_F_RX_SEC_OFFLOAD = 1 << 18 + #: Indicate that security offload processing failed on the RX packet. + RTE_MBUF_F_RX_SEC_OFFLOAD_FAILED = auto() + + #: The RX packet is a double VLAN. If this flag is set, RTE_MBUF_F_RX_VLAN must also be set. If + #: the flag RTE_MBUF_F_RX_QINQ_STRIPPED is also present, both VLANs headers have been stripped + #: from mbuf data, else they are still present. + RTE_MBUF_F_RX_QINQ = auto() + + #: No info about the outer RX L4 checksum. Value is 0 in the DPDK library. + RTE_MBUF_F_RX_OUTER_L4_CKSUM_UNKNOWN = 1 << 27 + #: The outer L4 checksum in the packet is wrong + RTE_MBUF_F_RX_OUTER_L4_CKSUM_BAD = 1 << 21 + #: The outer L4 checksum in the packet is valid + RTE_MBUF_F_RX_OUTER_L4_CKSUM_GOOD = 1 << 22 + #: Invalid outer L4 checksum state. Value is + #: RTE_MBUF_F_RX_OUTER_L4_CKSUM_BAD | RTE_MBUF_F_RX_OUTER_L4_CKSUM_GOOD in the DPDK library. + RTE_MBUF_F_RX_OUTER_L4_CKSUM_INVALID = 1 << 28 + + # TX flags + + #: Outer UDP checksum offload flag. This flag is used for enabling outer UDP checksum in PMD. + #: To use outer UDP checksum, the user either needs to enable the following in mbuf: + #: + #: a) Fill outer_l2_len and outer_l3_len in mbuf. + #: b) Set the RTE_MBUF_F_TX_OUTER_UDP_CKSUM flag. + #: c) Set the RTE_MBUF_F_TX_OUTER_IPV4 or RTE_MBUF_F_TX_OUTER_IPV6 flag. + #: + #: Or configure RTE_ETH_TX_OFFLOAD_OUTER_UDP_CKSUM offload flag. + RTE_MBUF_F_TX_OUTER_UDP_CKSUM = 1 << 41 + + #: UDP Fragmentation Offload flag. This flag is used for enabling UDP fragmentation in SW or in + #: HW. + RTE_MBUF_F_TX_UDP_SEG = auto() + + #: Request security offload processing on the TX packet. To use Tx security offload, the user + #: needs to fill l2_len in mbuf indicating L2 header size and where L3 header starts. + #: Similarly, l3_len should also be filled along with ol_flags reflecting current L3 type. + RTE_MBUF_F_TX_SEC_OFFLOAD = auto() + + #: Offload the MACsec. This flag must be set by the application to enable this offload feature + #: for a packet to be transmitted. + RTE_MBUF_F_TX_MACSEC = auto() + + # Bits 45:48 are used for the tunnel type in ``lib/mbuf/rte_mbuf_core.h``, but some are modified + # in this Flag to maintain uniqueness. The tunnel type must be specified for TSO or checksum on + # the inner part of tunnel packets. These flags can be used with RTE_MBUF_F_TX_TCP_SEG for TSO, + # or RTE_MBUF_F_TX_xxx_CKSUM. The mbuf fields for inner and outer header lengths are required: + # outer_l2_len, outer_l3_len, l2_len, l3_len, l4_len and tso_segsz for TSO. + + #: + RTE_MBUF_F_TX_TUNNEL_VXLAN = 1 << 45 + #: + RTE_MBUF_F_TX_TUNNEL_GRE = 1 << 46 + #: Value is 3 << 45 in the DPDK library. + RTE_MBUF_F_TX_TUNNEL_IPIP = 1 << 61 + #: + RTE_MBUF_F_TX_TUNNEL_GENEVE = 1 << 47 + #: TX packet with MPLS-in-UDP RFC 7510 header. Value is 5 << 45 in the DPDK library. + RTE_MBUF_F_TX_TUNNEL_MPLSINUDP = 1 << 62 + #: Value is 6 << 45 in the DPDK library. + RTE_MBUF_F_TX_TUNNEL_VXLAN_GPE = 1 << 63 + #: Value is 7 << 45 in the DPDK library. + RTE_MBUF_F_TX_TUNNEL_GTP = 1 << 64 + #: + RTE_MBUF_F_TX_TUNNEL_ESP = 1 << 48 + #: Generic IP encapsulated tunnel type, used for TSO and checksum offload. This can be used for + #: tunnels which are not standards or listed above. It is preferred to use specific tunnel + #: flags like RTE_MBUF_F_TX_TUNNEL_GRE or RTE_MBUF_F_TX_TUNNEL_IPIP if possible. The ethdev + #: must be configured with RTE_ETH_TX_OFFLOAD_IP_TNL_TSO. Outer and inner checksums are done + #: according to the existing flags like RTE_MBUF_F_TX_xxx_CKSUM. Specific tunnel headers that + #: contain payload length, sequence id or checksum are not expected to be updated. Value is + #: 0xD << 45 in the DPDK library. + RTE_MBUF_F_TX_TUNNEL_IP = 1 << 65 + #: Generic UDP encapsulated tunnel type, used for TSO and checksum offload. UDP tunnel type + #: implies outer IP layer. It can be used for tunnels which are not standards or listed above. + #: It is preferred to use specific tunnel flags like RTE_MBUF_F_TX_TUNNEL_VXLAN if possible. + #: The ethdev must be configured with RTE_ETH_TX_OFFLOAD_UDP_TNL_TSO. Outer and inner checksums + #: are done according to the existing flags like RTE_MBUF_F_TX_xxx_CKSUM. Specific tunnel + #: headers that contain payload length, sequence id or checksum are not expected to be updated. + #: value is 0xE << 45 in the DPDK library. + RTE_MBUF_F_TX_TUNNEL_UDP = 1 << 66 + + #: Double VLAN insertion (QinQ) request to driver, driver may offload the insertion based on + #: device capability. Mbuf 'vlan_tci' & 'vlan_tci_outer' must be valid when this flag is set. + RTE_MBUF_F_TX_QINQ = 1 << 49 + + #: TCP segmentation offload. To enable this offload feature for a packet to be transmitted on + #: hardware supporting TSO: + #: + #: - set the RTE_MBUF_F_TX_TCP_SEG flag in mbuf->ol_flags (this flag implies + #: RTE_MBUF_F_TX_TCP_CKSUM) + #: - set the flag RTE_MBUF_F_TX_IPV4 or RTE_MBUF_F_TX_IPV6 + #: * if it's IPv4, set the RTE_MBUF_F_TX_IP_CKSUM flag + #: - fill the mbuf offload information: l2_len, l3_len, l4_len, tso_segsz + RTE_MBUF_F_TX_TCP_SEG = auto() + + #: TX IEEE1588 packet to timestamp. + RTE_MBUF_F_TX_IEEE1588_TMST = auto() + + # Bits 52+53 used for L4 packet type with checksum enabled in ``lib/mbuf/rte_mbuf_core.h`` but + # some values must be modified in this framework to maintain uniqueness. To use hardware + # L4 checksum offload, the user needs to: + # + # - fill l2_len and l3_len in mbuf + # - set the flags RTE_MBUF_F_TX_TCP_CKSUM, RTE_MBUF_F_TX_SCTP_CKSUM or + # RTE_MBUF_F_TX_UDP_CKSUM + # - set the flag RTE_MBUF_F_TX_IPV4 or RTE_MBUF_F_TX_IPV6 + + #: Disable L4 cksum of TX pkt. Value is 0 in the DPDK library. + RTE_MBUF_F_TX_L4_NO_CKSUM = 1 << 67 + #: TCP cksum of TX pkt. Computed by NIC. + RTE_MBUF_F_TX_TCP_CKSUM = 1 << 52 + #: SCTP cksum of TX pkt. Computed by NIC. + RTE_MBUF_F_TX_SCTP_CKSUM = 1 << 53 + #: UDP cksum of TX pkt. Computed by NIC. Value is 3 << 52 in the DPDK library. + RTE_MBUF_F_TX_UDP_CKSUM = 1 << 68 + + #: Offload the IP checksum in the hardware. The flag RTE_MBUF_F_TX_IPV4 should also be set by + #: the application, although a PMD will only check RTE_MBUF_F_TX_IP_CKSUM. + RTE_MBUF_F_TX_IP_CKSUM = 1 << 54 + + #: Packet is IPv4. This flag must be set when using any offload feature (TSO, L3 or L4 + #: checksum) to tell the NIC that the packet is an IPv4 packet. If the packet is a tunneled + #: packet, this flag is related to the inner headers. + RTE_MBUF_F_TX_IPV4 = auto() + #: Packet is IPv6. This flag must be set when using an offload feature (TSO or L4 checksum) to + #: tell the NIC that the packet is an IPv6 packet. If the packet is a tunneled packet, this + #: flag is related to the inner headers. + RTE_MBUF_F_TX_IPV6 = auto() + #: VLAN tag insertion request to driver, driver may offload the insertion based on the device + #: capability. mbuf 'vlan_tci' field must be valid when this flag is set. + RTE_MBUF_F_TX_VLAN = auto() + + #: Offload the IP checksum of an external header in the hardware. The flag + #: RTE_MBUF_F_TX_OUTER_IPV4 should also be set by the application, although a PMD will only + #: check RTE_MBUF_F_TX_OUTER_IP_CKSUM. + RTE_MBUF_F_TX_OUTER_IP_CKSUM = auto() + #: Packet outer header is IPv4. This flag must be set when using any outer offload feature (L3 + #: or L4 checksum) to tell the NIC that the outer header of the tunneled packet is an IPv4 + #: packet. + RTE_MBUF_F_TX_OUTER_IPV4 = auto() + #: Packet outer header is IPv6. This flag must be set when using any outer offload feature (L4 + #: checksum) to tell the NIC that the outer header of the tunneled packet is an IPv6 packet. + RTE_MBUF_F_TX_OUTER_IPV6 = auto() + + @classmethod + def from_str(cls, flags: str) -> Self: + """Makes an instance from a string containing whitespace-separated the flag members. + + Args: + arr: A string containing ol_flag values. + + Returns: + A new instance of the flag. + """ + flag = cls(0) + for name in flags.split(): + if hasattr(cls, name): + flag |= cls[name] + return flag + + @classmethod + def make_parser(cls) -> ParserFn: + """Makes a parser function. + + Returns: + ParserFn: A dictionary for the `dataclasses.field` metadata argument containing a + parser function that makes an instance of this flag from text. + """ + return TextParser.wrap( + TextParser.find(r"ol_flags: ([^\n]+)"), + cls.from_str, + ) + + +class RtePTypes(Flag): + """Flag representing possible packet types in DPDK verbose output. + + Values in this class are derived from definitions in the RTE MBUF ptype library in DPDK located + in lib/mbuf/rte_mbuf_ptype.h. Specifically, the names of values in this class should match the + possible return options from the methods rte_get_ptype_*_name in rte_mbuf_ptype.c. + """ + + # L2 + #: Ethernet packet type. This is used for outer packet for tunneling cases. + L2_ETHER = auto() + #: Ethernet packet type for time sync. + L2_ETHER_TIMESYNC = auto() + #: ARP (Address Resolution Protocol) packet type. + L2_ETHER_ARP = auto() + #: LLDP (Link Layer Discovery Protocol) packet type. + L2_ETHER_LLDP = auto() + #: NSH (Network Service Header) packet type. + L2_ETHER_NSH = auto() + #: VLAN packet type. + L2_ETHER_VLAN = auto() + #: QinQ packet type. + L2_ETHER_QINQ = auto() + #: PPPOE packet type. + L2_ETHER_PPPOE = auto() + #: FCoE packet type.. + L2_ETHER_FCOE = auto() + #: MPLS packet type. + L2_ETHER_MPLS = auto() + #: No L2 packet information. + L2_UNKNOWN = auto() + + # L3 + #: IP (Internet Protocol) version 4 packet type. This is used for outer packet for tunneling + #: cases, and does not contain any header option. + L3_IPV4 = auto() + #: IP (Internet Protocol) version 4 packet type. This is used for outer packet for tunneling + #: cases, and contains header options. + L3_IPV4_EXT = auto() + #: IP (Internet Protocol) version 6 packet type. This is used for outer packet for tunneling + #: cases, and does not contain any extension header. + L3_IPV6 = auto() + #: IP (Internet Protocol) version 4 packet type. This is used for outer packet for tunneling + #: cases, and may or maynot contain header options. + L3_IPV4_EXT_UNKNOWN = auto() + #: IP (Internet Protocol) version 6 packet type. This is used for outer packet for tunneling + #: cases, and contains extension headers. + L3_IPV6_EXT = auto() + #: IP (Internet Protocol) version 6 packet type. This is used for outer packet for tunneling + #: cases, and may or maynot contain extension headers. + L3_IPV6_EXT_UNKNOWN = auto() + #: No L3 packet information. + L3_UNKNOWN = auto() + + # L4 + #: TCP (Transmission Control Protocol) packet type. This is used for outer packet for tunneling + #: cases. + L4_TCP = auto() + #: UDP (User Datagram Protocol) packet type. This is used for outer packet for tunneling cases. + L4_UDP = auto() + #: Fragmented IP (Internet Protocol) packet type. This is used for outer packet for tunneling + #: cases and refers to those packets of any IP types which can be recognized as fragmented. A + #: fragmented packet cannot be recognized as any other L4 types (RTE_PTYPE_L4_TCP, + #: RTE_PTYPE_L4_UDP, RTE_PTYPE_L4_SCTP, RTE_PTYPE_L4_ICMP, RTE_PTYPE_L4_NONFRAG). + L4_FRAG = auto() + #: SCTP (Stream Control Transmission Protocol) packet type. This is used for outer packet for + #: tunneling cases. + L4_SCTP = auto() + #: ICMP (Internet Control Message Protocol) packet type. This is used for outer packet for + #: tunneling cases. + L4_ICMP = auto() + #: Non-fragmented IP (Internet Protocol) packet type. This is used for outer packet for + #: tunneling cases and refers to those packets of any IP types, that cannot be recognized as + #: any of the above L4 types (RTE_PTYPE_L4_TCP, RTE_PTYPE_L4_UDP, RTE_PTYPE_L4_FRAG, + #: RTE_PTYPE_L4_SCTP, RTE_PTYPE_L4_ICMP). + L4_NONFRAG = auto() + #: IGMP (Internet Group Management Protocol) packet type. + L4_IGMP = auto() + #: No L4 packet information. + L4_UNKNOWN = auto() + + # Tunnel + #: IP (Internet Protocol) in IP (Internet Protocol) tunneling packet type. + TUNNEL_IP = auto() + #: GRE (Generic Routing Encapsulation) tunneling packet type. + TUNNEL_GRE = auto() + #: VXLAN (Virtual eXtensible Local Area Network) tunneling packet type. + TUNNEL_VXLAN = auto() + #: NVGRE (Network Virtualization using Generic Routing Encapsulation) tunneling packet type. + TUNNEL_NVGRE = auto() + #: GENEVE (Generic Network Virtualization Encapsulation) tunneling packet type. + TUNNEL_GENEVE = auto() + #: Tunneling packet type of Teredo, VXLAN (Virtual eXtensible Local Area Network) or GRE + #: (Generic Routing Encapsulation) could be recognized as this packet type, if they can not be + #: recognized independently as of hardware capability. + TUNNEL_GRENAT = auto() + #: GTP-C (GPRS Tunnelling Protocol) control tunneling packet type. + TUNNEL_GTPC = auto() + #: GTP-U (GPRS Tunnelling Protocol) user data tunneling packet type. + TUNNEL_GTPU = auto() + #: ESP (IP Encapsulating Security Payload) tunneling packet type. + TUNNEL_ESP = auto() + #: L2TP (Layer 2 Tunneling Protocol) tunneling packet type. + TUNNEL_L2TP = auto() + #: VXLAN-GPE (VXLAN Generic Protocol Extension) tunneling packet type. + TUNNEL_VXLAN_GPE = auto() + #: MPLS-in-UDP tunneling packet type (RFC 7510). + TUNNEL_MPLS_IN_UDP = auto() + #: MPLS-in-GRE tunneling packet type (RFC 4023). + TUNNEL_MPLS_IN_GRE = auto() + #: No tunnel information found on the packet. + TUNNEL_UNKNOWN = auto() + + # Inner L2 + #: Ethernet packet type. This is used for inner packet type only. + INNER_L2_ETHER = auto() + #: Ethernet packet type with VLAN (Virtual Local Area Network) tag. + INNER_L2_ETHER_VLAN = auto() + #: QinQ packet type. + INNER_L2_ETHER_QINQ = auto() + #: No inner L2 information found on the packet. + INNER_L2_UNKNOWN = auto() + + # Inner L3 + #: IP (Internet Protocol) version 4 packet type. This is used for inner packet only, and does + #: not contain any header option. + INNER_L3_IPV4 = auto() + #: IP (Internet Protocol) version 4 packet type. This is used for inner packet only, and + #: contains header options. + INNER_L3_IPV4_EXT = auto() + #: IP (Internet Protocol) version 6 packet type. This is used for inner packet only, and does + #: not contain any extension header. + INNER_L3_IPV6 = auto() + #: IP (Internet Protocol) version 4 packet type. This is used for inner packet only, and may or + #: may not contain header options. + INNER_L3_IPV4_EXT_UNKNOWN = auto() + #: IP (Internet Protocol) version 6 packet type. This is used for inner packet only, and + #: contains extension headers. + INNER_L3_IPV6_EXT = auto() + #: IP (Internet Protocol) version 6 packet type. This is used for inner packet only, and may or + #: may not contain extension headers. + INNER_L3_IPV6_EXT_UNKNOWN = auto() + #: No inner L3 information found on the packet. + INNER_L3_UNKNOWN = auto() + + # Inner L4 + #: TCP (Transmission Control Protocol) packet type. This is used for inner packet only. + INNER_L4_TCP = auto() + #: UDP (User Datagram Protocol) packet type. This is used for inner packet only. + INNER_L4_UDP = auto() + #: Fragmented IP (Internet Protocol) packet type. This is used for inner packet only, and may + #: or maynot have a layer 4 packet. + INNER_L4_FRAG = auto() + #: SCTP (Stream Control Transmission Protocol) packet type. This is used for inner packet only. + INNER_L4_SCTP = auto() + #: ICMP (Internet Control Message Protocol) packet type. This is used for inner packet only. + INNER_L4_ICMP = auto() + #: Non-fragmented IP (Internet Protocol) packet type. It is used for inner packet only, and may + #: or may not have other unknown layer 4 packet types. + INNER_L4_NONFRAG = auto() + #: No inner L4 information found on the packet. + INNER_L4_UNKNOWN = auto() + + @classmethod + def from_str(cls, flags: str) -> Self: + """Makes an instance from a string containing whitespace-separated the flag members. + + Args: + flags: A string containing the packet types. + + Returns: + A new instance of the flag. + """ + flag = cls(0) + for name in flags.split(): + if hasattr(cls, name): + flag |= cls[name] + return flag + + @classmethod + def make_parser(cls, hw: bool) -> ParserFn: + """Makes a parser function. + + Args: + hw: Whether to make a parser for hardware ptypes or software ptypes. If :data:`True`, + hardware ptypes will be collected, otherwise software pytpes will. + + Returns: + ParserFn: A dictionary for the `dataclasses.field` metadata argument containing a + parser function that makes an instance of this flag from text. + """ + return TextParser.wrap( + TextParser.find(f"{'hw' if hw else 'sw'} ptype: ([^-]+)"), + cls.from_str, + ) + + +@dataclass +class TestPmdVerbosePacket(TextParser): + """Packet information provided by verbose output in Testpmd. + + This dataclass expects that packet information be prepended with the starting line of packet + bursts. Specifically, the line that reads "port X/queue Y: sent/received Z packets". + """ + + #: ID of the port that handled the packet. + port_id: int = field(metadata=TextParser.find_int(r"port (\d+)/queue \d+")) + #: ID of the queue that handled the packet. + queue_id: int = field(metadata=TextParser.find_int(r"port \d+/queue (\d+)")) + #: Whether the packet was received or sent by the queue/port. + was_received: bool = field(metadata=TextParser.find(r"received \d+ packets")) + #: + src_mac: str = field(metadata=TextParser.find(f"src=({REGEX_FOR_MAC_ADDRESS})")) + #: + dst_mac: str = field(metadata=TextParser.find(f"dst=({REGEX_FOR_MAC_ADDRESS})")) + #: Memory pool the packet was handled on. + pool: str = field(metadata=TextParser.find(r"pool=(\S+)")) + #: Packet type in hex. + p_type: int = field(metadata=TextParser.find_int(r"type=(0x[a-fA-F\d]+)")) + #: + length: int = field(metadata=TextParser.find_int(r"length=(\d+)")) + #: Number of segments in the packet. + nb_segs: int = field(metadata=TextParser.find_int(r"nb_segs=(\d+)")) + #: Hardware packet type. + hw_ptype: RtePTypes = field(metadata=RtePTypes.make_parser(hw=True)) + #: Software packet type. + sw_ptype: RtePTypes = field(metadata=RtePTypes.make_parser(hw=False)) + #: + l2_len: int = field(metadata=TextParser.find_int(r"l2_len=(\d+)")) + #: + ol_flags: PacketOffloadFlag = field(metadata=PacketOffloadFlag.make_parser()) + #: RSS hash of the packet in hex. + rss_hash: int | None = field( + default=None, metadata=TextParser.find_int(r"RSS hash=(0x[a-fA-F\d]+)") + ) + #: RSS queue that handled the packet in hex. + rss_queue: int | None = field( + default=None, metadata=TextParser.find_int(r"RSS queue=(0x[a-fA-F\d]+)") + ) + #: + l3_len: int | None = field(default=None, metadata=TextParser.find_int(r"l3_len=(\d+)")) + #: + l4_len: int | None = field(default=None, metadata=TextParser.find_int(r"l4_len=(\d+)")) + + def requires_stopped_ports(func: TestPmdShellMethod) -> TestPmdShellMethod: """Decorator for :class:`TestPmdShell` commands methods that require stopped ports. @@ -714,7 +1214,7 @@ def start(self, verify: bool = True) -> None: "Not all ports came up after starting packet forwarding in testpmd." ) - def stop(self, verify: bool = True) -> None: + def stop(self, verify: bool = True) -> str: """Stop packet forwarding. Args: @@ -725,6 +1225,12 @@ def stop(self, verify: bool = True) -> None: Raises: InteractiveCommandExecutionError: If `verify` is :data:`True` and the command to stop forwarding results in an error. + + Returns: + Output gathered from the stop command and all other preceding logs in the buffer. This + output is most often used to view forwarding statistics that are displayed when this + command is sent as well as any verbose packet information that hasn't been consumed + prior to calling this method. """ stop_cmd_output = self.send_command("stop") if verify: @@ -734,6 +1240,7 @@ def stop(self, verify: bool = True) -> None: ): self._logger.debug(f"Failed to stop packet forwarding: \n{stop_cmd_output}") raise InteractiveCommandExecutionError("Testpmd failed to stop packet forwarding.") + return stop_cmd_output def get_devices(self) -> list[TestPmdDevice]: """Get a list of device names that are known to testpmd. @@ -981,6 +1488,32 @@ def set_port_mtu_all(self, mtu: int, verify: bool = True) -> None: for port in self.ports: self.set_port_mtu(port.id, mtu, verify) + @staticmethod + def extract_verbose_output(output: str) -> list[TestPmdVerbosePacket]: + """Extract the verbose information present in given testpmd output. + + This method extracts sections of verbose output that begin with the line + "port X/queue Y: sent/received Z packets" and end with the ol_flags of a packet. + + Args: + output: Testpmd output that contains verbose information + + Returns: + List of parsed packet information gathered from verbose information in `output`. + """ + out: list[TestPmdVerbosePacket] = [] + prev_header: str = "" + iter = re.finditer( + r"(?P
(?:port \d+/queue \d+: (?:received|sent) \d+ packets)?)\s*" + r"(?Psrc=[\w\s=:-]+?ol_flags: [\w ]+)", + output, + ) + for match in iter: + if match.group("HEADER"): + prev_header = match.group("HEADER") + out.append(TestPmdVerbosePacket.parse(f"{prev_header}\n{match.group('PACKET')}")) + return out + def _close(self) -> None: """Overrides :meth:`~.interactive_shell.close`.""" self.stop() diff --git a/dts/framework/utils.py b/dts/framework/utils.py index c768dd0c99..d1eee55c0f 100644 --- a/dts/framework/utils.py +++ b/dts/framework/utils.py @@ -29,6 +29,9 @@ from .exception import ConfigurationError, InternalError REGEX_FOR_PCI_ADDRESS: str = "/[0-9a-fA-F]{4}:[0-9a-fA-F]{2}:[0-9a-fA-F]{2}.[0-9]{1}/" +_REGEX_FOR_COLON_OR_HYPHEN_SEP_MAC: str = r"(?:[\da-fA-F]{2}[:-]){5}[\da-fA-F]{2}" +_REGEX_FOR_DOT_SEP_MAC: str = r"(?:[\da-fA-F]{4}.){2}[\da-fA-F]{4}" +REGEX_FOR_MAC_ADDRESS: str = rf"{_REGEX_FOR_COLON_OR_HYPHEN_SEP_MAC}|{_REGEX_FOR_DOT_SEP_MAC}" def expand_range(range_str: str) -> list[int]: -- 2.46.0