Re: [PATCH v5 1/1] dts: add text parser for testpmd verbose output

["Juraj Linkeš" <juraj.linkes@pantheon.tech>]

> diff --git a/dts/framework/remote_session/testpmd_shell.py b/dts/framework/remote_session/testpmd_shell.py

> @@ -577,6 +577,497 @@ class TestPmdPortStats(TextParser):
>       tx_bps: int = field(metadata=TextParser.find_int(r"Tx-bps:\s+(\d+)"))
>   
>   
> +class PacketOffloadFlag(Flag):
> +    """Flag representing the Packet Offload Features Flags in DPDK.
> +
> +    Values in this class are taken from the definitions in the RTE MBUF core library in DPDK
> +    located in lib/mbuf/rte_mbuf_core.h. It is expected that flag values in this class will match
> +    the values they are set to in said DPDK library with one exception; all values must be unique.
> +    For example, the definitions for unknown checksum flags in rte_mbuf_core.h are all set to
> +    :data:`0`, but it is valuable to distinguish between them in this framework. For this reason
> +    flags that are not unique in the DPDK library are set either to values within the
> +    RTE_MBUF_F_FIRST_FREE-RTE_MBUF_F_LAST_FREE range for Rx or shifted 61+ bits for Tx.
> +    """

> +    #: No information about the RX IP checksum.
> +    RTE_MBUF_F_RX_IP_CKSUM_UNKNOWN = 1 << 23

Good idea with the UKNOWN flag values.

> +    #: The IP checksum in the packet is wrong.
> +    RTE_MBUF_F_RX_IP_CKSUM_BAD = 1 << 4
> +    #: The IP checksum in the packet is valid.
> +    RTE_MBUF_F_RX_IP_CKSUM_GOOD = 1 << 7

I see you kept the order and just used the corresponding flag values. 
Makes sense.


> +    #:
> +    RTE_MBUF_F_TX_TUNNEL_VXLAN = 1 << 45
> +    #:
> +    RTE_MBUF_F_TX_TUNNEL_GRE = 2 << 45
> +    #:
> +    RTE_MBUF_F_TX_TUNNEL_IPIP = 3 << 45
> +    #:
> +    RTE_MBUF_F_TX_TUNNEL_GENEVE = 4 << 45
> +    """ TX packet with MPLS-in-UDP RFC 7510 header. """

This should be one line below after :#

> +    #:
> +    RTE_MBUF_F_TX_TUNNEL_MPLSINUDP = 5 << 45
> +    #:
> +    RTE_MBUF_F_TX_TUNNEL_VXLAN_GPE = 6 << 45
> +    #:
> +    RTE_MBUF_F_TX_TUNNEL_GTP = 7 << 45
> +    #:
> +    RTE_MBUF_F_TX_TUNNEL_ESP = 8 << 45

So the DPDK code mixes values withing flags? Would this work? We have to 
be careful with how we use this:
PacketOffloadFlag.RTE_MBUF_F_TX_TUNNEL_VXLAN | 
PacketOffloadFlag.RTE_MBUF_F_TX_TUNNEL_GRE == 
PacketOffloadFlag.RTE_MBUF_F_TX_TUNNEL_IPIP
True

PacketOffloadFlag.RTE_MBUF_F_TX_TUNNEL_VXLAN | 
PacketOffloadFlag.RTE_MBUF_F_TX_TUNNEL_GRE is 
PacketOffloadFlag.RTE_MBUF_F_TX_TUNNEL_IPIP
True

PacketOffloadFlag.RTE_MBUF_F_TX_TUNNEL_VXLAN in 
PacketOffloadFlag.RTE_MBUF_F_TX_TUNNEL_IPIP
True

The combination of 1 | 2 == 3, even identity returns True and one flag 
is part of another. If we're looking at verbose_output.ol_flags and 
checking the RTE_MBUF_F_TX_TUNNEL_VXLAN flag, True would be returned for 
all flag that have the first bit set:
RTE_MBUF_F_TX_TUNNEL_VXLAN
RTE_MBUF_F_TX_TUNNEL_IPIP
RTE_MBUF_F_TX_TUNNEL_MPLSINUDP
RTE_MBUF_F_TX_TUNNEL_GTP

Do you know how this is handled in DPDK? Or how testpmd processes this 
to return the proper flag?

This mixing seems pretty wild to me (I guess this is to not waste space, 
since ULL is max 64 bits). We need to think this through thoroughly.


> +    #: TCP cksum of TX pkt. Computed by NIC.
> +    RTE_MBUF_F_TX_TCP_CKSUM = 1 << 52
> +    #: SCTP cksum of TX pkt. Computed by NIC.
> +    RTE_MBUF_F_TX_SCTP_CKSUM = 2 << 52
> +    #: UDP cksum of TX pkt. Computed by NIC.
> +    RTE_MBUF_F_TX_UDP_CKSUM = 3 << 52

This is the same thing as above.


> +    @classmethod
> +    def from_str_list(cls, arr: list[str]) -> Self:
> +        """Makes an instance from a list containing the flag members.
> +
> +        Args:
> +            arr: A list of strings containing ol_flag values.
> +
> +        Returns:
> +            A new instance of the flag.
> +        """
> +        flag = cls(0)
> +        for name in arr:
> +            if hasattr(cls, name):

So you used hasattr instead of cls[name] in cls. Is this to avoid the 
exception? I now realize that if we could ignore the exception then we 
won't need the condition.

The question is when the exception would be raised, or, in other words, 
what should we do when hasattr(cls, name) is False. If I understand this 
correctly, is it's False, then name is not among the flags and that 
means testpmd returned an unsupported flag, which shouldn't happen, but 
if it does in the future, we would be better off throwing an exception, 
or at very least, log a warning, so that we have an indication that we 
need to add support for a new flag.

> +                flag |= cls[name]
> +        return flag
> +
> +    @classmethod
> +    def make_parser(cls) -> ParserFn:
> +        """Makes a parser function.
> +
> +        Returns:
> +            ParserFn: A dictionary for the `dataclasses.field` metadata argument containing a
> +                parser function that makes an instance of this flag from text.
> +        """
> +        return TextParser.wrap(
> +            TextParser.wrap(TextParser.find(r"ol_flags: ([^\n]+)"), str.split),
> +            cls.from_str_list,
> +        )

The RSSOffloadTypesFlag does the split in its from_list_string method. 
Do we want to do the same here?

Maybe could create a ParsableFlag (or Creatable? Or something else) 
superclass that would implement these from_* methods (from_list_string, 
from_str) and subclass it. Flags should be subclassable if they don't 
contain members.

The superclass would be useful so that we don't redefine the same method 
over and over and so that it's clear what's already available.


> @@ -656,6 +1147,9 @@ def stop(self, verify: bool = True) -> None:
>           Raises:
>               InteractiveCommandExecutionError: If `verify` is :data:`True` and the command to stop
>                   forwarding results in an error.
> +
> +        Returns:
> +            Output gathered from sending the stop command.

This not just from sending the stop command, but everything else that 
preceded (when collecting the verbose output), right?


> diff --git a/dts/framework/utils.py b/dts/framework/utils.py

> @@ -27,6 +27,12 @@
>   from .exception import ConfigurationError
>   
>   REGEX_FOR_PCI_ADDRESS: str = "/[0-9a-fA-F]{4}:[0-9a-fA-F]{2}:[0-9a-fA-F]{2}.[0-9]{1}/"
> +_REGEX_FOR_COLON_SEP_MAC: str = r"(?:[\da-fA-F]{2}:){5}[\da-fA-F]{2}"
> +_REGEX_FOR_HYPHEN_SEP_MAC: str = r"(?:[\da-fA-F]{2}-){5,7}[\da-fA-F]{2}"

{5,7} should be just 5 repetitions. When could it be more?