DPDK patches and discussions
 help / color / mirror / Atom feed
From: jspewock@iol.unh.edu
To: thomas@monjalon.net, wathsala.vithanage@arm.com,
	paul.szczepanek@arm.com, Luca.Vizzarro@arm.com,
	probb@iol.unh.edu, npratte@iol.unh.edu,
	juraj.linkes@pantheon.tech, Honnappa.Nagarahalli@arm.com,
	yoan.picchi@foss.arm.com, alex.chapman@arm.com
Cc: dev@dpdk.org, Jeremy Spewock <jspewock@iol.unh.edu>
Subject: [PATCH v7 1/1] dts: add text parser for testpmd verbose output
Date: Thu, 26 Sep 2024 11:47:37 -0400	[thread overview]
Message-ID: <20240926154737.10743-2-jspewock@iol.unh.edu> (raw)
In-Reply-To: <20240926154737.10743-1-jspewock@iol.unh.edu>

From: Jeremy Spewock <jspewock@iol.unh.edu>

Multiple test suites from the old DTS framework rely on being able to
consume and interpret the verbose output of testpmd. The new framework
doesn't have an elegant way for handling the verbose output, but test
suites are starting to be written that rely on it. This patch creates a
TextParser class that can be used to extract the verbose information
from any testpmd output and also adjusts the `stop` method of the shell
to return all output that it collected.

Signed-off-by: Jeremy Spewock <jspewock@iol.unh.edu>
---
 dts/framework/remote_session/testpmd_shell.py | 563 +++++++++++++++++-
 dts/framework/utils.py                        |   3 +
 2 files changed, 564 insertions(+), 2 deletions(-)

diff --git a/dts/framework/remote_session/testpmd_shell.py b/dts/framework/remote_session/testpmd_shell.py
index 77902a468d..55efc9d455 100644
--- a/dts/framework/remote_session/testpmd_shell.py
+++ b/dts/framework/remote_session/testpmd_shell.py
@@ -32,7 +32,7 @@
 from framework.settings import SETTINGS
 from framework.testbed_model.cpu import LogicalCoreCount, LogicalCoreList
 from framework.testbed_model.sut_node import SutNode
-from framework.utils import StrEnum
+from framework.utils import REGEX_FOR_MAC_ADDRESS, StrEnum
 
 P = ParamSpec("P")
 TestPmdShellMethod = Callable[Concatenate["TestPmdShell", P], Any]
@@ -581,6 +581,532 @@ class TestPmdPortStats(TextParser):
     tx_bps: int = field(metadata=TextParser.find_int(r"Tx-bps:\s+(\d+)"))
 
 
+class PacketOffloadFlag(Flag):
+    """Flag representing the Packet Offload Features Flags in DPDK.
+
+    Values in this class are taken from the definitions in the RTE MBUF core library in DPDK
+    located in ``lib/mbuf/rte_mbuf_core.h``. It is expected that flag values in this class will
+    match the values they are set to in said DPDK library with one exception; all values must be
+    unique. For example, the definitions for unknown checksum flags in ``rte_mbuf_core.h`` are all
+    set to :data:`0`, but it is valuable to distinguish between them in this framework. For this
+    reason flags that are not unique in the DPDK library are set either to values within the
+    RTE_MBUF_F_FIRST_FREE-RTE_MBUF_F_LAST_FREE range for Rx or shifted 61+ bits for Tx.
+
+    References:
+        DPDK lib: ``lib/mbuf/rte_mbuf_core.h``
+    """
+
+    # RX flags
+
+    #: The RX packet is a 802.1q VLAN packet, and the tci has been saved in mbuf->vlan_tci. If the
+    #: flag RTE_MBUF_F_RX_VLAN_STRIPPED is also present, the VLAN header has been stripped from
+    #: mbuf data, else it is still present.
+    RTE_MBUF_F_RX_VLAN = auto()
+
+    #: RX packet with RSS hash result.
+    RTE_MBUF_F_RX_RSS_HASH = auto()
+
+    #: RX packet with FDIR match indicate.
+    RTE_MBUF_F_RX_FDIR = auto()
+
+    #: This flag is set when the outermost IP header checksum is detected as wrong by the hardware.
+    RTE_MBUF_F_RX_OUTER_IP_CKSUM_BAD = 1 << 5
+
+    #: A vlan has been stripped by the hardware and its tci is saved in mbuf->vlan_tci. This can
+    #: only happen if vlan stripping is enabled in the RX configuration of the PMD. When
+    #: RTE_MBUF_F_RX_VLAN_STRIPPED is set, RTE_MBUF_F_RX_VLAN must also be set.
+    RTE_MBUF_F_RX_VLAN_STRIPPED = auto()
+
+    #: No information about the RX IP checksum. Value is 0 in the DPDK library.
+    RTE_MBUF_F_RX_IP_CKSUM_UNKNOWN = 1 << 23
+    #: The IP checksum in the packet is wrong.
+    RTE_MBUF_F_RX_IP_CKSUM_BAD = 1 << 4
+    #: The IP checksum in the packet is valid.
+    RTE_MBUF_F_RX_IP_CKSUM_GOOD = 1 << 7
+    #: The IP checksum is not correct in the packet data, but the integrity of the IP header is
+    #: verified. Value is RTE_MBUF_F_RX_IP_CKSUM_BAD | RTE_MBUF_F_RX_IP_CKSUM_GOOD in the DPDK
+    #: library.
+    RTE_MBUF_F_RX_IP_CKSUM_NONE = 1 << 24
+
+    #: No information about the RX L4 checksum. Value is 0 in the DPDK library.
+    RTE_MBUF_F_RX_L4_CKSUM_UNKNOWN = 1 << 25
+    #: The L4 checksum in the packet is wrong.
+    RTE_MBUF_F_RX_L4_CKSUM_BAD = 1 << 3
+    #: The L4 checksum in the packet is valid.
+    RTE_MBUF_F_RX_L4_CKSUM_GOOD = 1 << 8
+    #: The L4 checksum is not correct in the packet data, but the integrity of the L4 data is
+    #: verified. Value is RTE_MBUF_F_RX_L4_CKSUM_BAD | RTE_MBUF_F_RX_L4_CKSUM_GOOD in the DPDK
+    #: library.
+    RTE_MBUF_F_RX_L4_CKSUM_NONE = 1 << 26
+
+    #: RX IEEE1588 L2 Ethernet PT Packet.
+    RTE_MBUF_F_RX_IEEE1588_PTP = 1 << 9
+    #: RX IEEE1588 L2/L4 timestamped packet.
+    RTE_MBUF_F_RX_IEEE1588_TMST = 1 << 10
+
+    #: FD id reported if FDIR match.
+    RTE_MBUF_F_RX_FDIR_ID = 1 << 13
+    #: Flexible bytes reported if FDIR match.
+    RTE_MBUF_F_RX_FDIR_FLX = 1 << 14
+
+    #: If both RTE_MBUF_F_RX_QINQ_STRIPPED and RTE_MBUF_F_RX_VLAN_STRIPPED are set, the 2 VLANs
+    #: have been stripped by the hardware. If RTE_MBUF_F_RX_QINQ_STRIPPED is set and
+    #: RTE_MBUF_F_RX_VLAN_STRIPPED is unset, only the outer VLAN is removed from packet data.
+    RTE_MBUF_F_RX_QINQ_STRIPPED = auto()
+
+    #: When packets are coalesced by a hardware or virtual driver, this flag can be set in the RX
+    #: mbuf, meaning that the m->tso_segsz field is valid and is set to the segment size of
+    #: original packets.
+    RTE_MBUF_F_RX_LRO = auto()
+
+    #: Indicate that security offload processing was applied on the RX packet.
+    RTE_MBUF_F_RX_SEC_OFFLOAD = 1 << 18
+    #: Indicate that security offload processing failed on the RX packet.
+    RTE_MBUF_F_RX_SEC_OFFLOAD_FAILED = auto()
+
+    #: The RX packet is a double VLAN. If this flag is set, RTE_MBUF_F_RX_VLAN must also be set. If
+    #: the flag RTE_MBUF_F_RX_QINQ_STRIPPED is also present, both VLANs headers have been stripped
+    #: from mbuf data, else they are still present.
+    RTE_MBUF_F_RX_QINQ = auto()
+
+    #: No info about the outer RX L4 checksum. Value is 0 in the DPDK library.
+    RTE_MBUF_F_RX_OUTER_L4_CKSUM_UNKNOWN = 1 << 27
+    #: The outer L4 checksum in the packet is wrong
+    RTE_MBUF_F_RX_OUTER_L4_CKSUM_BAD = 1 << 21
+    #: The outer L4 checksum in the packet is valid
+    RTE_MBUF_F_RX_OUTER_L4_CKSUM_GOOD = 1 << 22
+    #: Invalid outer L4 checksum state. Value is
+    #: RTE_MBUF_F_RX_OUTER_L4_CKSUM_BAD | RTE_MBUF_F_RX_OUTER_L4_CKSUM_GOOD in the DPDK library.
+    RTE_MBUF_F_RX_OUTER_L4_CKSUM_INVALID = 1 << 28
+
+    # TX flags
+
+    #: Outer UDP checksum offload flag. This flag is used for enabling outer UDP checksum in PMD.
+    #: To use outer UDP checksum, the user either needs to enable the following in mbuf:
+    #:
+    #:  a) Fill outer_l2_len and outer_l3_len in mbuf.
+    #:  b) Set the RTE_MBUF_F_TX_OUTER_UDP_CKSUM flag.
+    #:  c) Set the RTE_MBUF_F_TX_OUTER_IPV4 or RTE_MBUF_F_TX_OUTER_IPV6 flag.
+    #:
+    #: Or configure RTE_ETH_TX_OFFLOAD_OUTER_UDP_CKSUM offload flag.
+    RTE_MBUF_F_TX_OUTER_UDP_CKSUM = 1 << 41
+
+    #: UDP Fragmentation Offload flag. This flag is used for enabling UDP fragmentation in SW or in
+    #: HW.
+    RTE_MBUF_F_TX_UDP_SEG = auto()
+
+    #: Request security offload processing on the TX packet. To use Tx security offload, the user
+    #: needs to fill l2_len in mbuf indicating L2 header size and where L3 header starts.
+    #: Similarly, l3_len should also be filled along with ol_flags reflecting current L3 type.
+    RTE_MBUF_F_TX_SEC_OFFLOAD = auto()
+
+    #: Offload the MACsec. This flag must be set by the application to enable this offload feature
+    #: for a packet to be transmitted.
+    RTE_MBUF_F_TX_MACSEC = auto()
+
+    # Bits 45:48 are used for the tunnel type in ``lib/mbuf/rte_mbuf_core.h``, but some are modified
+    # in this Flag to maintain uniqueness. The tunnel type must be specified for TSO or checksum on
+    # the inner part of tunnel packets. These flags can be used with RTE_MBUF_F_TX_TCP_SEG for TSO,
+    # or RTE_MBUF_F_TX_xxx_CKSUM. The mbuf fields for inner and outer header lengths are required:
+    # outer_l2_len, outer_l3_len, l2_len, l3_len, l4_len and tso_segsz for TSO.
+
+    #:
+    RTE_MBUF_F_TX_TUNNEL_VXLAN = 1 << 45
+    #:
+    RTE_MBUF_F_TX_TUNNEL_GRE = 1 << 46
+    #: Value is 3 << 45 in the DPDK library.
+    RTE_MBUF_F_TX_TUNNEL_IPIP = 1 << 61
+    #:
+    RTE_MBUF_F_TX_TUNNEL_GENEVE = 1 << 47
+    #: TX packet with MPLS-in-UDP RFC 7510 header. Value is 5 << 45 in the DPDK library.
+    RTE_MBUF_F_TX_TUNNEL_MPLSINUDP = 1 << 62
+    #: Value is 6 << 45 in the DPDK library.
+    RTE_MBUF_F_TX_TUNNEL_VXLAN_GPE = 1 << 63
+    #: Value is 7 << 45 in the DPDK library.
+    RTE_MBUF_F_TX_TUNNEL_GTP = 1 << 64
+    #:
+    RTE_MBUF_F_TX_TUNNEL_ESP = 1 << 48
+    #: Generic IP encapsulated tunnel type, used for TSO and checksum offload. This can be used for
+    #: tunnels which are not standards or listed above. It is preferred to use specific tunnel
+    #: flags like RTE_MBUF_F_TX_TUNNEL_GRE or RTE_MBUF_F_TX_TUNNEL_IPIP if possible. The ethdev
+    #: must be configured with RTE_ETH_TX_OFFLOAD_IP_TNL_TSO.  Outer and inner checksums are done
+    #: according to the existing flags like RTE_MBUF_F_TX_xxx_CKSUM. Specific tunnel headers that
+    #: contain payload length, sequence id or checksum are not expected to be updated. Value is
+    #: 0xD << 45 in the DPDK library.
+    RTE_MBUF_F_TX_TUNNEL_IP = 1 << 65
+    #: Generic UDP encapsulated tunnel type, used for TSO and checksum offload. UDP tunnel type
+    #: implies outer IP layer. It can be used for tunnels which are not standards or listed above.
+    #: It is preferred to use specific tunnel flags like RTE_MBUF_F_TX_TUNNEL_VXLAN if possible.
+    #: The ethdev must be configured with RTE_ETH_TX_OFFLOAD_UDP_TNL_TSO. Outer and inner checksums
+    #: are done according to the existing flags like RTE_MBUF_F_TX_xxx_CKSUM. Specific tunnel
+    #: headers that contain payload length, sequence id or checksum are not expected to be updated.
+    #: value is 0xE << 45 in the DPDK library.
+    RTE_MBUF_F_TX_TUNNEL_UDP = 1 << 66
+
+    #: Double VLAN insertion (QinQ) request to driver, driver may offload the insertion based on
+    #: device capability. Mbuf 'vlan_tci' & 'vlan_tci_outer' must be valid when this flag is set.
+    RTE_MBUF_F_TX_QINQ = 1 << 49
+
+    #: TCP segmentation offload. To enable this offload feature for a packet to be transmitted on
+    #: hardware supporting TSO:
+    #:
+    #:  - set the RTE_MBUF_F_TX_TCP_SEG flag in mbuf->ol_flags (this flag implies
+    #:      RTE_MBUF_F_TX_TCP_CKSUM)
+    #:  - set the flag RTE_MBUF_F_TX_IPV4 or RTE_MBUF_F_TX_IPV6
+    #:      * if it's IPv4, set the RTE_MBUF_F_TX_IP_CKSUM flag
+    #:  - fill the mbuf offload information: l2_len, l3_len, l4_len, tso_segsz
+    RTE_MBUF_F_TX_TCP_SEG = auto()
+
+    #: TX IEEE1588 packet to timestamp.
+    RTE_MBUF_F_TX_IEEE1588_TMST = auto()
+
+    # Bits 52+53 used for L4 packet type with checksum enabled in ``lib/mbuf/rte_mbuf_core.h`` but
+    # some values must be modified in this framework to maintain uniqueness. To use hardware
+    # L4 checksum offload, the user needs to:
+    #
+    # - fill l2_len and l3_len in mbuf
+    # - set the flags RTE_MBUF_F_TX_TCP_CKSUM, RTE_MBUF_F_TX_SCTP_CKSUM or
+    #   RTE_MBUF_F_TX_UDP_CKSUM
+    # - set the flag RTE_MBUF_F_TX_IPV4 or RTE_MBUF_F_TX_IPV6
+
+    #: Disable L4 cksum of TX pkt. Value is 0 in the DPDK library.
+    RTE_MBUF_F_TX_L4_NO_CKSUM = 1 << 67
+    #: TCP cksum of TX pkt. Computed by NIC.
+    RTE_MBUF_F_TX_TCP_CKSUM = 1 << 52
+    #: SCTP cksum of TX pkt. Computed by NIC.
+    RTE_MBUF_F_TX_SCTP_CKSUM = 1 << 53
+    #: UDP cksum of TX pkt. Computed by NIC. Value is 3 << 52 in the DPDK library.
+    RTE_MBUF_F_TX_UDP_CKSUM = 1 << 68
+
+    #: Offload the IP checksum in the hardware. The flag RTE_MBUF_F_TX_IPV4 should also be set by
+    #: the application, although a PMD will only check RTE_MBUF_F_TX_IP_CKSUM.
+    RTE_MBUF_F_TX_IP_CKSUM = 1 << 54
+
+    #: Packet is IPv4. This flag must be set when using any offload feature (TSO, L3 or L4
+    #: checksum) to tell the NIC that the packet is an IPv4 packet. If the packet is a tunneled
+    #: packet, this flag is related to the inner headers.
+    RTE_MBUF_F_TX_IPV4 = auto()
+    #: Packet is IPv6. This flag must be set when using an offload feature (TSO or L4 checksum) to
+    #: tell the NIC that the packet is an IPv6 packet. If the packet is a tunneled packet, this
+    #: flag is related to the inner headers.
+    RTE_MBUF_F_TX_IPV6 = auto()
+    #: VLAN tag insertion request to driver, driver may offload the insertion based on the device
+    #: capability. mbuf 'vlan_tci' field must be valid when this flag is set.
+    RTE_MBUF_F_TX_VLAN = auto()
+
+    #: Offload the IP checksum of an external header in the hardware. The flag
+    #: RTE_MBUF_F_TX_OUTER_IPV4 should also be set by the application, although a PMD will only
+    #: check RTE_MBUF_F_TX_OUTER_IP_CKSUM.
+    RTE_MBUF_F_TX_OUTER_IP_CKSUM = auto()
+    #: Packet outer header is IPv4. This flag must be set when using any outer offload feature (L3
+    #: or L4 checksum) to tell the NIC that the outer header of the tunneled packet is an IPv4
+    #: packet.
+    RTE_MBUF_F_TX_OUTER_IPV4 = auto()
+    #: Packet outer header is IPv6. This flag must be set when using any outer offload feature (L4
+    #: checksum) to tell the NIC that the outer header of the tunneled packet is an IPv6 packet.
+    RTE_MBUF_F_TX_OUTER_IPV6 = auto()
+
+    @classmethod
+    def from_list_string(cls, names: str) -> Self:
+        """Makes a flag from a whitespace-separated list of names.
+
+        Args:
+            names: a whitespace-separated list containing the members of this flag.
+
+        Returns:
+            An instance of this flag.
+        """
+        flag = cls(0)
+        for name in names.split():
+            flag |= cls.from_str(name)
+        return flag
+
+    @classmethod
+    def from_str(cls, name: str) -> Self:
+        """Makes a flag matching the supplied name.
+
+        Args:
+            name: a valid member of this flag in text
+        Returns:
+            An instance of this flag.
+        """
+        member_name = name.strip().replace("-", "_")
+        return cls[member_name]
+
+    @classmethod
+    def make_parser(cls) -> ParserFn:
+        """Makes a parser function.
+
+        Returns:
+            ParserFn: A dictionary for the `dataclasses.field` metadata argument containing a
+                parser function that makes an instance of this flag from text.
+        """
+        return TextParser.wrap(
+            TextParser.find(r"ol_flags: ([^\n]+)"),
+            cls.from_list_string,
+        )
+
+
+class RtePTypes(Flag):
+    """Flag representing possible packet types in DPDK verbose output.
+
+    Values in this class are derived from definitions in the RTE MBUF ptype library in DPDK located
+    in ``lib/mbuf/rte_mbuf_ptype.h``. Specifically, the names of values in this class should match
+    the possible return options from the functions ``rte_get_ptype_*_name`` in ``rte_mbuf_ptype.c``.
+
+    References:
+        DPDK lib: ``lib/mbuf/rte_mbuf_ptype.h``
+        DPDK ptype name formatting functions: ``lib/mbuf/rte_mbuf_ptype.c:rte_get_ptype_*_name()``
+    """
+
+    # L2
+    #: Ethernet packet type. This is used for outer packet for tunneling cases.
+    L2_ETHER = auto()
+    #: Ethernet packet type for time sync.
+    L2_ETHER_TIMESYNC = auto()
+    #: ARP (Address Resolution Protocol) packet type.
+    L2_ETHER_ARP = auto()
+    #: LLDP (Link Layer Discovery Protocol) packet type.
+    L2_ETHER_LLDP = auto()
+    #: NSH (Network Service Header) packet type.
+    L2_ETHER_NSH = auto()
+    #: VLAN packet type.
+    L2_ETHER_VLAN = auto()
+    #: QinQ packet type.
+    L2_ETHER_QINQ = auto()
+    #: PPPOE packet type.
+    L2_ETHER_PPPOE = auto()
+    #: FCoE packet type..
+    L2_ETHER_FCOE = auto()
+    #: MPLS packet type.
+    L2_ETHER_MPLS = auto()
+    #: No L2 packet information.
+    L2_UNKNOWN = auto()
+
+    # L3
+    #: IP (Internet Protocol) version 4 packet type. This is used for outer packet for tunneling
+    #: cases, and does not contain any header option.
+    L3_IPV4 = auto()
+    #: IP (Internet Protocol) version 4 packet type. This is used for outer packet for tunneling
+    #: cases, and contains header options.
+    L3_IPV4_EXT = auto()
+    #: IP (Internet Protocol) version 6 packet type. This is used for outer packet for tunneling
+    #: cases, and does not contain any extension header.
+    L3_IPV6 = auto()
+    #: IP (Internet Protocol) version 4 packet type. This is used for outer packet for tunneling
+    #: cases, and may or maynot contain header options.
+    L3_IPV4_EXT_UNKNOWN = auto()
+    #: IP (Internet Protocol) version 6 packet type. This is used for outer packet for tunneling
+    #: cases, and contains extension headers.
+    L3_IPV6_EXT = auto()
+    #: IP (Internet Protocol) version 6 packet type. This is used for outer packet for tunneling
+    #: cases, and may or maynot contain extension headers.
+    L3_IPV6_EXT_UNKNOWN = auto()
+    #: No L3 packet information.
+    L3_UNKNOWN = auto()
+
+    # L4
+    #: TCP (Transmission Control Protocol) packet type. This is used for outer packet for tunneling
+    #: cases.
+    L4_TCP = auto()
+    #: UDP (User Datagram Protocol) packet type. This is used for outer packet for tunneling cases.
+    L4_UDP = auto()
+    #: Fragmented IP (Internet Protocol) packet type. This is used for outer packet for tunneling
+    #: cases and refers to those packets of any IP types which can be recognized as fragmented. A
+    #: fragmented packet cannot be recognized as any other L4 types (RTE_PTYPE_L4_TCP,
+    #: RTE_PTYPE_L4_UDP, RTE_PTYPE_L4_SCTP, RTE_PTYPE_L4_ICMP, RTE_PTYPE_L4_NONFRAG).
+    L4_FRAG = auto()
+    #: SCTP (Stream Control Transmission Protocol) packet type. This is used for outer packet for
+    #: tunneling cases.
+    L4_SCTP = auto()
+    #: ICMP (Internet Control Message Protocol) packet type. This is used for outer packet for
+    #: tunneling cases.
+    L4_ICMP = auto()
+    #: Non-fragmented IP (Internet Protocol) packet type. This is used for outer packet for
+    #: tunneling cases and refers to those packets of any IP types, that cannot be recognized as
+    #: any of the above L4 types (RTE_PTYPE_L4_TCP, RTE_PTYPE_L4_UDP, RTE_PTYPE_L4_FRAG,
+    #: RTE_PTYPE_L4_SCTP, RTE_PTYPE_L4_ICMP).
+    L4_NONFRAG = auto()
+    #: IGMP (Internet Group Management Protocol) packet type.
+    L4_IGMP = auto()
+    #: No L4 packet information.
+    L4_UNKNOWN = auto()
+
+    # Tunnel
+    #: IP (Internet Protocol) in IP (Internet Protocol) tunneling packet type.
+    TUNNEL_IP = auto()
+    #: GRE (Generic Routing Encapsulation) tunneling packet type.
+    TUNNEL_GRE = auto()
+    #: VXLAN (Virtual eXtensible Local Area Network) tunneling packet type.
+    TUNNEL_VXLAN = auto()
+    #: NVGRE (Network Virtualization using Generic Routing Encapsulation) tunneling packet type.
+    TUNNEL_NVGRE = auto()
+    #: GENEVE (Generic Network Virtualization Encapsulation) tunneling packet type.
+    TUNNEL_GENEVE = auto()
+    #: Tunneling packet type of Teredo, VXLAN (Virtual eXtensible Local Area Network) or GRE
+    #: (Generic Routing Encapsulation) could be recognized as this packet type, if they can not be
+    #: recognized independently as of hardware capability.
+    TUNNEL_GRENAT = auto()
+    #: GTP-C (GPRS Tunnelling Protocol) control tunneling packet type.
+    TUNNEL_GTPC = auto()
+    #: GTP-U (GPRS Tunnelling Protocol) user data tunneling packet type.
+    TUNNEL_GTPU = auto()
+    #: ESP (IP Encapsulating Security Payload) tunneling packet type.
+    TUNNEL_ESP = auto()
+    #: L2TP (Layer 2 Tunneling Protocol) tunneling packet type.
+    TUNNEL_L2TP = auto()
+    #: VXLAN-GPE (VXLAN Generic Protocol Extension) tunneling packet type.
+    TUNNEL_VXLAN_GPE = auto()
+    #: MPLS-in-UDP tunneling packet type (RFC 7510).
+    TUNNEL_MPLS_IN_UDP = auto()
+    #: MPLS-in-GRE tunneling packet type (RFC 4023).
+    TUNNEL_MPLS_IN_GRE = auto()
+    #: No tunnel information found on the packet.
+    TUNNEL_UNKNOWN = auto()
+
+    # Inner L2
+    #: Ethernet packet type. This is used for inner packet type only.
+    INNER_L2_ETHER = auto()
+    #: Ethernet packet type with VLAN (Virtual Local Area Network) tag.
+    INNER_L2_ETHER_VLAN = auto()
+    #: QinQ packet type.
+    INNER_L2_ETHER_QINQ = auto()
+    #: No inner L2 information found on the packet.
+    INNER_L2_UNKNOWN = auto()
+
+    # Inner L3
+    #: IP (Internet Protocol) version 4 packet type. This is used for inner packet only, and does
+    #: not contain any header option.
+    INNER_L3_IPV4 = auto()
+    #: IP (Internet Protocol) version 4 packet type. This is used for inner packet only, and
+    #: contains header options.
+    INNER_L3_IPV4_EXT = auto()
+    #: IP (Internet Protocol) version 6 packet type. This is used for inner packet only, and does
+    #: not contain any extension header.
+    INNER_L3_IPV6 = auto()
+    #: IP (Internet Protocol) version 4 packet type. This is used for inner packet only, and may or
+    #: may not contain header options.
+    INNER_L3_IPV4_EXT_UNKNOWN = auto()
+    #: IP (Internet Protocol) version 6 packet type. This is used for inner packet only, and
+    #: contains extension headers.
+    INNER_L3_IPV6_EXT = auto()
+    #: IP (Internet Protocol) version 6 packet type. This is used for inner packet only, and may or
+    #: may not contain extension headers.
+    INNER_L3_IPV6_EXT_UNKNOWN = auto()
+    #: No inner L3 information found on the packet.
+    INNER_L3_UNKNOWN = auto()
+
+    # Inner L4
+    #: TCP (Transmission Control Protocol) packet type. This is used for inner packet only.
+    INNER_L4_TCP = auto()
+    #: UDP (User Datagram Protocol) packet type. This is used for inner packet only.
+    INNER_L4_UDP = auto()
+    #: Fragmented IP (Internet Protocol) packet type. This is used for inner packet only, and may
+    #: or maynot have a layer 4 packet.
+    INNER_L4_FRAG = auto()
+    #: SCTP (Stream Control Transmission Protocol) packet type. This is used for inner packet only.
+    INNER_L4_SCTP = auto()
+    #: ICMP (Internet Control Message Protocol) packet type. This is used for inner packet only.
+    INNER_L4_ICMP = auto()
+    #: Non-fragmented IP (Internet Protocol) packet type. It is used for inner packet only, and may
+    #: or may not have other unknown layer 4 packet types.
+    INNER_L4_NONFRAG = auto()
+    #: No inner L4 information found on the packet.
+    INNER_L4_UNKNOWN = auto()
+
+    @classmethod
+    def from_list_string(cls, names: str) -> Self:
+        """Makes a flag from a whitespace-separated list of names.
+
+        Args:
+            names: a whitespace-separated list containing the members of this flag.
+
+        Returns:
+            An instance of this flag.
+        """
+        flag = cls(0)
+        for name in names.split():
+            flag |= cls.from_str(name)
+        return flag
+
+    @classmethod
+    def from_str(cls, name: str) -> Self:
+        """Makes a flag matching the supplied name.
+
+        Args:
+            name: a valid member of this flag in text
+        Returns:
+            An instance of this flag.
+        """
+        member_name = name.strip().replace("-", "_")
+        return cls[member_name]
+
+    @classmethod
+    def make_parser(cls, hw: bool) -> ParserFn:
+        """Makes a parser function.
+
+        Args:
+            hw: Whether to make a parser for hardware ptypes or software ptypes. If :data:`True`,
+                hardware ptypes will be collected, otherwise software pytpes will.
+
+        Returns:
+            ParserFn: A dictionary for the `dataclasses.field` metadata argument containing a
+                parser function that makes an instance of this flag from text.
+        """
+        return TextParser.wrap(
+            TextParser.find(f"{'hw' if hw else 'sw'} ptype: ([^-]+)"),
+            cls.from_list_string,
+        )
+
+
+@dataclass
+class TestPmdVerbosePacket(TextParser):
+    """Packet information provided by verbose output in Testpmd.
+
+    This dataclass expects that packet information be prepended with the starting line of packet
+    bursts. Specifically, the line that reads "port X/queue Y: sent/received Z packets".
+    """
+
+    #: ID of the port that handled the packet.
+    port_id: int = field(metadata=TextParser.find_int(r"port (\d+)/queue \d+"))
+    #: ID of the queue that handled the packet.
+    queue_id: int = field(metadata=TextParser.find_int(r"port \d+/queue (\d+)"))
+    #: Whether the packet was received or sent by the queue/port.
+    was_received: bool = field(metadata=TextParser.find(r"received \d+ packets"))
+    #:
+    src_mac: str = field(metadata=TextParser.find(f"src=({REGEX_FOR_MAC_ADDRESS})"))
+    #:
+    dst_mac: str = field(metadata=TextParser.find(f"dst=({REGEX_FOR_MAC_ADDRESS})"))
+    #: Memory pool the packet was handled on.
+    pool: str = field(metadata=TextParser.find(r"pool=(\S+)"))
+    #: Packet type in hex.
+    p_type: int = field(metadata=TextParser.find_int(r"type=(0x[a-fA-F\d]+)"))
+    #:
+    length: int = field(metadata=TextParser.find_int(r"length=(\d+)"))
+    #: Number of segments in the packet.
+    nb_segs: int = field(metadata=TextParser.find_int(r"nb_segs=(\d+)"))
+    #: Hardware packet type.
+    hw_ptype: RtePTypes = field(metadata=RtePTypes.make_parser(hw=True))
+    #: Software packet type.
+    sw_ptype: RtePTypes = field(metadata=RtePTypes.make_parser(hw=False))
+    #:
+    l2_len: int = field(metadata=TextParser.find_int(r"l2_len=(\d+)"))
+    #:
+    ol_flags: PacketOffloadFlag = field(metadata=PacketOffloadFlag.make_parser())
+    #: RSS hash of the packet in hex.
+    rss_hash: int | None = field(
+        default=None, metadata=TextParser.find_int(r"RSS hash=(0x[a-fA-F\d]+)")
+    )
+    #: RSS queue that handled the packet in hex.
+    rss_queue: int | None = field(
+        default=None, metadata=TextParser.find_int(r"RSS queue=(0x[a-fA-F\d]+)")
+    )
+    #:
+    l3_len: int | None = field(default=None, metadata=TextParser.find_int(r"l3_len=(\d+)"))
+    #:
+    l4_len: int | None = field(default=None, metadata=TextParser.find_int(r"l4_len=(\d+)"))
+
+
 def requires_stopped_ports(func: TestPmdShellMethod) -> TestPmdShellMethod:
     """Decorator for :class:`TestPmdShell` commands methods that require stopped ports.
 
@@ -714,7 +1240,7 @@ def start(self, verify: bool = True) -> None:
                         "Not all ports came up after starting packet forwarding in testpmd."
                     )
 
-    def stop(self, verify: bool = True) -> None:
+    def stop(self, verify: bool = True) -> str:
         """Stop packet forwarding.
 
         Args:
@@ -725,6 +1251,12 @@ def stop(self, verify: bool = True) -> None:
         Raises:
             InteractiveCommandExecutionError: If `verify` is :data:`True` and the command to stop
                 forwarding results in an error.
+
+        Returns:
+            Output gathered from the stop command and all other preceding logs in the buffer. This
+            output is most often used to view forwarding statistics that are displayed when this
+            command is sent as well as any verbose packet information that hasn't been consumed
+            prior to calling this method.
         """
         stop_cmd_output = self.send_command("stop")
         if verify:
@@ -734,6 +1266,7 @@ def stop(self, verify: bool = True) -> None:
             ):
                 self._logger.debug(f"Failed to stop packet forwarding: \n{stop_cmd_output}")
                 raise InteractiveCommandExecutionError("Testpmd failed to stop packet forwarding.")
+        return stop_cmd_output
 
     def get_devices(self) -> list[TestPmdDevice]:
         """Get a list of device names that are known to testpmd.
@@ -981,6 +1514,32 @@ def set_port_mtu_all(self, mtu: int, verify: bool = True) -> None:
         for port in self.ports:
             self.set_port_mtu(port.id, mtu, verify)
 
+    @staticmethod
+    def extract_verbose_output(output: str) -> list[TestPmdVerbosePacket]:
+        """Extract the verbose information present in given testpmd output.
+
+        This method extracts sections of verbose output that begin with the line
+        "port X/queue Y: sent/received Z packets" and end with the ol_flags of a packet.
+
+        Args:
+            output: Testpmd output that contains verbose information
+
+        Returns:
+            List of parsed packet information gathered from verbose information in `output`.
+        """
+        out: list[TestPmdVerbosePacket] = []
+        prev_header: str = ""
+        iter = re.finditer(
+            r"(?P<HEADER>(?:port \d+/queue \d+: (?:received|sent) \d+ packets)?)\s*"
+            r"(?P<PACKET>src=[\w\s=:-]+?ol_flags: [\w ]+)",
+            output,
+        )
+        for match in iter:
+            if match.group("HEADER"):
+                prev_header = match.group("HEADER")
+            out.append(TestPmdVerbosePacket.parse(f"{prev_header}\n{match.group('PACKET')}"))
+        return out
+
     def _close(self) -> None:
         """Overrides :meth:`~.interactive_shell.close`."""
         self.stop()
diff --git a/dts/framework/utils.py b/dts/framework/utils.py
index c768dd0c99..d1eee55c0f 100644
--- a/dts/framework/utils.py
+++ b/dts/framework/utils.py
@@ -29,6 +29,9 @@
 from .exception import ConfigurationError, InternalError
 
 REGEX_FOR_PCI_ADDRESS: str = "/[0-9a-fA-F]{4}:[0-9a-fA-F]{2}:[0-9a-fA-F]{2}.[0-9]{1}/"
+_REGEX_FOR_COLON_OR_HYPHEN_SEP_MAC: str = r"(?:[\da-fA-F]{2}[:-]){5}[\da-fA-F]{2}"
+_REGEX_FOR_DOT_SEP_MAC: str = r"(?:[\da-fA-F]{4}.){2}[\da-fA-F]{4}"
+REGEX_FOR_MAC_ADDRESS: str = rf"{_REGEX_FOR_COLON_OR_HYPHEN_SEP_MAC}|{_REGEX_FOR_DOT_SEP_MAC}"
 
 
 def expand_range(range_str: str) -> list[int]:
-- 
2.46.0


  reply	other threads:[~2024-09-26 15:47 UTC|newest]

Thread overview: 38+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2024-07-29 20:39 [PATCH v1 0/1] dts: testpmd verbose parser jspewock
2024-07-29 20:39 ` [PATCH v1 1/1] dts: add text parser for testpmd verbose output jspewock
2024-07-30 13:34 ` [PATCH v2 0/1] dts: testpmd verbose parser jspewock
2024-07-30 13:34   ` [PATCH v2 1/1] dts: add text parser for testpmd verbose output jspewock
2024-07-30 15:41     ` Nicholas Pratte
2024-07-30 21:30       ` Jeremy Spewock
2024-08-02 14:54         ` Nicholas Pratte
2024-08-02 17:38           ` Jeremy Spewock
2024-08-05 13:20             ` Nicholas Pratte
2024-07-30 21:33     ` Jeremy Spewock
2024-08-01  8:43       ` Luca Vizzarro
2024-08-02 13:40         ` Jeremy Spewock
2024-08-01  8:41     ` Luca Vizzarro
2024-08-02 13:35       ` Jeremy Spewock
2024-08-08 20:36 ` [PATCH v3 0/1] dts: testpmd verbose parser jspewock
2024-08-08 20:36   ` [PATCH v3 1/1] dts: add text parser for testpmd verbose output jspewock
2024-08-08 21:49     ` Jeremy Spewock
2024-08-12 17:32       ` Nicholas Pratte
2024-09-09 11:44     ` Juraj Linkeš
2024-09-17 13:40       ` Jeremy Spewock
2024-09-18  8:09         ` Juraj Linkeš
2024-09-18 16:34 ` [PATCH v4 0/1] dts: testpmd verbose parser jspewock
2024-09-18 16:34   ` [PATCH v4 1/1] dts: add text parser for testpmd verbose output jspewock
2024-09-18 17:05 ` [PATCH v5 0/1] dts: testpmd verbose parser jspewock
2024-09-18 17:05   ` [PATCH v5 1/1] dts: add text parser for testpmd verbose output jspewock
2024-09-19  9:02     ` Juraj Linkeš
2024-09-20 15:53       ` Jeremy Spewock
2024-09-23 13:30         ` Juraj Linkeš
2024-09-19 12:35     ` Juraj Linkeš
2024-09-20 15:55       ` Jeremy Spewock
2024-09-25 15:46 ` [PATCH v6 0/1] dts: testpmd verbose parser jspewock
2024-09-25 15:46   ` [PATCH v6 1/1] dts: add text parser for testpmd verbose output jspewock
2024-09-26  8:25     ` Juraj Linkeš
2024-09-26 14:43       ` Jeremy Spewock
2024-09-26 15:47 ` [PATCH v7 0/1] dts: testpmd verbose parser jspewock
2024-09-26 15:47   ` jspewock [this message]
2024-09-27  9:32     ` [PATCH v7 1/1] dts: add text parser for testpmd verbose output Juraj Linkeš
2024-09-27 11:48     ` Luca Vizzarro

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20240926154737.10743-2-jspewock@iol.unh.edu \
    --to=jspewock@iol.unh.edu \
    --cc=Honnappa.Nagarahalli@arm.com \
    --cc=Luca.Vizzarro@arm.com \
    --cc=alex.chapman@arm.com \
    --cc=dev@dpdk.org \
    --cc=juraj.linkes@pantheon.tech \
    --cc=npratte@iol.unh.edu \
    --cc=paul.szczepanek@arm.com \
    --cc=probb@iol.unh.edu \
    --cc=thomas@monjalon.net \
    --cc=wathsala.vithanage@arm.com \
    --cc=yoan.picchi@foss.arm.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).