From: Luca Vizzarro <luca.vizzarro@arm.com>
To: dev@dpdk.org
Cc: "Jeremy Spewock" <jspewock@iol.unh.edu>,
"Juraj Linkeš" <juraj.linkes@pantheon.tech>,
"Luca Vizzarro" <luca.vizzarro@arm.com>,
"Paul Szczepanek" <paul.szczepanek@arm.com>
Subject: [PATCH v4 3/5] dts: add parsing utility module
Date: Thu, 6 Jun 2024 10:17:16 +0100 [thread overview]
Message-ID: <20240606091718.151516-4-luca.vizzarro@arm.com> (raw)
In-Reply-To: <20240606091718.151516-1-luca.vizzarro@arm.com>
Adds parsing text into a custom dataclass. It provides a new
`TextParser` dataclass to be inherited. This implements the `parse`
method, which combined with the parser functions, it can automatically
parse the value for each field.
This new utility will facilitate and simplify the parsing of complex
command outputs, while ensuring that the codebase does not get bloated
and stays flexible.
Signed-off-by: Luca Vizzarro <luca.vizzarro@arm.com>
Reviewed-by: Paul Szczepanek <paul.szczepanek@arm.com>
---
dts/framework/exception.py | 9 ++
dts/framework/parser.py | 229 +++++++++++++++++++++++++++++++++++++
2 files changed, 238 insertions(+)
create mode 100644 dts/framework/parser.py
diff --git a/dts/framework/exception.py b/dts/framework/exception.py
index cce1e0231a..d9d690037d 100644
--- a/dts/framework/exception.py
+++ b/dts/framework/exception.py
@@ -31,6 +31,8 @@ class ErrorSeverity(IntEnum):
#:
SSH_ERR = 4
#:
+ INTERNAL_ERR = 5
+ #:
DPDK_BUILD_ERR = 10
#:
TESTCASE_VERIFY_ERR = 20
@@ -192,3 +194,10 @@ def __init__(self, suite_name: str) -> None:
def __str__(self) -> str:
"""Add some context to the string representation."""
return f"Blocking suite {self._suite_name} failed."
+
+
+class InternalError(DTSError):
+ """An internal error or bug has occurred in DTS."""
+
+ #:
+ severity: ClassVar[ErrorSeverity] = ErrorSeverity.INTERNAL_ERR
diff --git a/dts/framework/parser.py b/dts/framework/parser.py
new file mode 100644
index 0000000000..186f2a3d2c
--- /dev/null
+++ b/dts/framework/parser.py
@@ -0,0 +1,229 @@
+# SPDX-License-Identifier: BSD-3-Clause
+# Copyright(c) 2024 Arm Limited
+
+"""Parsing utility module.
+
+This module provides :class:`~TextParser` which can be used to model any dataclass to a block of
+text.
+"""
+
+import re
+from abc import ABC
+from dataclasses import MISSING, dataclass, fields
+from functools import partial
+from typing import Any, Callable, TypedDict, cast
+
+from typing_extensions import Self
+
+from framework.exception import InternalError
+
+
+class ParserFn(TypedDict):
+ """Parser function in a dict compatible with the :func:`dataclasses.field` metadata param."""
+
+ #:
+ TextParser_fn: Callable[[str], Any]
+
+
+@dataclass
+class TextParser(ABC):
+ r"""Helper abstract dataclass that parses a text according to the fields' rules.
+
+ In order to enable text parsing in a dataclass, subclass it with :class:`TextParser`.
+
+ The provided `parse` method is a factory which parses the supplied text and creates an instance
+ with populated dataclass fields. This takes text as an argument and for each field in the
+ dataclass, the field's parser function is run against the whole text. The returned value is then
+ assigned to the field of the new instance. If the field does not have a parser function its
+ default value or factory is used instead. If no default is available either, an exception is
+ raised.
+
+ This class provides a selection of parser functions and a function to wrap parser functions with
+ generic functions. Parser functions are designed to be passed to the fields' metadata param. The
+ most commonly used parser function is expected to be the `find` method, which runs a regular
+ expression against the text to find matches.
+
+ Example:
+ The following example makes use of and demonstrates every parser function available:
+
+ ..code:: python
+
+ from dataclasses import dataclass, field
+ from enum import Enum
+ from framework.parser import TextParser
+
+ class Colour(Enum):
+ BLACK = 1
+ WHITE = 2
+
+ @classmethod
+ def from_str(cls, text: str):
+ match text:
+ case "black":
+ return cls.BLACK
+ case "white":
+ return cls.WHITE
+ case _:
+ return None # unsupported colour
+
+ @classmethod
+ def make_parser(cls):
+ # make a parser function that finds a match and
+ # then makes it a Colour object through Colour.from_str
+ return TextParser.wrap(cls.from_str, TextParser.find(r"is a (\w+)"))
+
+ @dataclass
+ class Animal(TextParser):
+ kind: str = field(metadata=TextParser.find(r"is a \w+ (\w+)"))
+ name: str = field(metadata=TextParser.find(r"^(\w+)"))
+ colour: Colour = field(metadata=Colour.make_parser())
+ age: int = field(metadata=TextParser.find_int(r"aged (\d+)"))
+
+ steph = Animal.parse("Stephanie is a white cat aged 10")
+ print(steph) # Animal(kind='cat', name='Stephanie', colour=<Colour.WHITE: 2>, age=10)
+ """
+
+ """============ BEGIN PARSER FUNCTIONS ============"""
+
+ @staticmethod
+ def wrap(parser_fn: ParserFn, wrapper_fn: Callable) -> ParserFn:
+ """Makes a wrapped parser function.
+
+ `parser_fn` is called and if a non-None value is returned, `wrapper_function` is called with
+ it. Otherwise the function returns early with None. In pseudo-code:
+
+ intermediate_value := parser_fn(input)
+ if intermediary_value is None then
+ output := None
+ else
+ output := wrapper_fn(intermediate_value)
+
+ Args:
+ parser_fn: The dictionary storing the parser function to be wrapped.
+ wrapper_fn: The function that wraps `parser_fn`.
+
+ Returns:
+ ParserFn: A dictionary for the `dataclasses.field` metadata argument containing the
+ newly wrapped parser function.
+ """
+ inner_fn = parser_fn["TextParser_fn"]
+
+ def _composite_parser_fn(text: str) -> Any:
+ intermediate_value = inner_fn(text)
+ if intermediate_value is None:
+ return None
+ return wrapper_fn(intermediate_value)
+
+ return ParserFn(TextParser_fn=_composite_parser_fn)
+
+ @staticmethod
+ def find(
+ pattern: str | re.Pattern[str],
+ flags: re.RegexFlag = re.RegexFlag(0),
+ named: bool = False,
+ ) -> ParserFn:
+ """Makes a parser function that finds a regular expression match in the text.
+
+ If the pattern has any capturing groups, it returns None if no match was found, otherwise a
+ tuple containing the values per each group is returned.. If the pattern has only one
+ capturing group and a match was found, its value is returned. If the pattern has no
+ capturing groups then either True or False is returned if the pattern had a match or not.
+
+ Args:
+ pattern: The regular expression pattern.
+ flags: The regular expression flags. Ignored if the given pattern is already compiled.
+ named: If set to True only the named capturing groups will be returned, as a dictionary.
+
+ Returns:
+ ParserFn: A dictionary for the `dataclasses.field` metadata argument containing the find
+ parser function.
+ """
+ if isinstance(pattern, str):
+ pattern = re.compile(pattern, flags)
+
+ def _find(text: str) -> Any:
+ m = pattern.search(text)
+ if m is None:
+ return None if pattern.groups > 0 else False
+
+ if pattern.groups == 0:
+ return True
+
+ if named:
+ return m.groupdict()
+
+ matches = m.groups()
+ if len(matches) == 1:
+ return matches[0]
+
+ return matches
+
+ return ParserFn(TextParser_fn=_find)
+
+ @staticmethod
+ def find_int(
+ pattern: str | re.Pattern[str],
+ flags: re.RegexFlag = re.RegexFlag(0),
+ int_base: int = 0,
+ ) -> ParserFn:
+ """Makes a parser function that converts the match of :meth:`~find` to int.
+
+ This function is compatible only with a pattern containing one capturing group.
+
+ Args:
+ pattern: The regular expression pattern.
+ flags: The regular expression flags. Ignored if the given pattern is already compiled.
+ int_base: The base of the number to convert from.
+
+ Raises:
+ InternalError: If the pattern does not have exactly one capturing group.
+
+ Returns:
+ ParserFn: A dictionary for the `dataclasses.field` metadata argument containing the
+ :meth:`~find` parser function wrapped by the int built-in.
+ """
+ if isinstance(pattern, str):
+ pattern = re.compile(pattern, flags)
+
+ if pattern.groups != 1:
+ raise InternalError("only one capturing group is allowed with this parser function")
+
+ return TextParser.wrap(TextParser.find(pattern), partial(int, base=int_base))
+
+ """============ END PARSER FUNCTIONS ============"""
+
+ @classmethod
+ def parse(cls, text: str) -> Self:
+ """Creates a new instance of the class from the given text.
+
+ A new class instance is created with all the fields that have a parser function in their
+ metadata. Fields without one are ignored and are expected to have a default value, otherwise
+ the class initialization will fail.
+
+ A field is populated with the value returned by its corresponding parser function.
+
+ Args:
+ text: the text to parse
+
+ Raises:
+ InternalError: if the parser did not find a match and the field does not have a default
+ value or default factory.
+
+ Returns:
+ A new instance of the class.
+ """
+ fields_values = {}
+ for field in fields(cls):
+ parse = cast(ParserFn, field.metadata).get("TextParser_fn")
+ if parse is None:
+ continue
+
+ value = parse(text)
+ if value is not None:
+ fields_values[field.name] = value
+ elif field.default is MISSING and field.default_factory is MISSING:
+ raise InternalError(
+ f"parser for field {field.name} returned None, but the field has no default"
+ )
+
+ return cls(**fields_values)
--
2.34.1
next prev parent reply other threads:[~2024-06-06 9:17 UTC|newest]
Thread overview: 95+ messages / expand[flat|nested] mbox.gz Atom feed top
2024-04-12 11:11 [PATCH 0/5] dts: testpmd show port info/stats Luca Vizzarro
2024-04-12 11:11 ` [PATCH 1/5] dts: fix InteractiveShell command prompt filtering Luca Vizzarro
2024-04-16 8:40 ` Juraj Linkeš
2024-04-16 12:12 ` Luca Vizzarro
2024-04-17 13:06 ` Juraj Linkeš
2024-04-17 14:17 ` Luca Vizzarro
2024-04-18 6:31 ` Juraj Linkeš
2024-04-29 16:16 ` Jeremy Spewock
2024-04-12 11:11 ` [PATCH 2/5] dts: skip first line of send_command output Luca Vizzarro
2024-04-16 8:48 ` Juraj Linkeš
2024-04-16 12:15 ` Luca Vizzarro
2024-04-17 13:18 ` Juraj Linkeš
2024-04-29 15:18 ` Jeremy Spewock
2024-06-14 18:58 ` Nicholas Pratte
2024-04-12 11:11 ` [PATCH 3/5] dts: add parsing utility module Luca Vizzarro
2024-04-16 8:59 ` Juraj Linkeš
2024-04-16 12:16 ` Luca Vizzarro
2024-04-29 16:15 ` Jeremy Spewock
2024-04-30 10:49 ` Luca Vizzarro
2024-04-30 20:03 ` Jeremy Spewock
2024-04-12 11:11 ` [PATCH 4/5] dts: add `show port info` command to TestPmdShell Luca Vizzarro
2024-04-16 9:03 ` Juraj Linkeš
2024-04-16 12:24 ` Luca Vizzarro
2024-04-17 13:22 ` Juraj Linkeš
2024-04-17 14:25 ` Luca Vizzarro
2024-04-17 15:29 ` Luca Vizzarro
2024-04-18 6:41 ` Juraj Linkeš
2024-04-18 10:52 ` Luca Vizzarro
2024-04-12 11:11 ` [PATCH 5/5] dts: add `show port stats` " Luca Vizzarro
2024-04-16 9:04 ` Juraj Linkeš
2024-04-29 15:54 ` Jeremy Spewock
2024-04-30 10:51 ` Luca Vizzarro
2024-05-09 11:26 ` [PATCH v2 0/5] dts: testpmd show port info/stats Luca Vizzarro
2024-05-09 11:26 ` [PATCH v2 1/5] dts: fix InteractiveShell command prompt filtering Luca Vizzarro
2024-05-31 21:06 ` Jeremy Spewock
2024-06-04 13:57 ` Juraj Linkeš
2024-05-09 11:26 ` [PATCH v2 2/5] dts: skip first line of send command output Luca Vizzarro
2024-05-31 21:06 ` Jeremy Spewock
2024-06-04 13:58 ` Juraj Linkeš
2024-05-09 11:26 ` [PATCH v2 3/5] dts: add parsing utility module Luca Vizzarro
2024-05-31 21:06 ` Jeremy Spewock
2024-06-04 15:13 ` Juraj Linkeš
2024-06-05 10:35 ` Luca Vizzarro
2024-06-05 12:19 ` Juraj Linkeš
2024-06-05 13:00 ` Luca Vizzarro
2024-05-09 11:26 ` [PATCH v2 4/5] dts: add `show port info` command to TestPmdShell Luca Vizzarro
2024-05-31 21:06 ` Jeremy Spewock
2024-06-03 9:11 ` Luca Vizzarro
2024-06-04 15:40 ` Juraj Linkeš
2024-06-05 11:03 ` Luca Vizzarro
2024-05-09 11:26 ` [PATCH v2 5/5] dts: add `show port stats` " Luca Vizzarro
2024-05-20 14:26 ` Nicholas Pratte
2024-05-21 15:00 ` Luca Vizzarro
2024-05-31 21:07 ` Jeremy Spewock
2024-06-04 15:44 ` Juraj Linkeš
2024-06-05 11:04 ` Luca Vizzarro
2024-06-05 14:44 ` [PATCH v3 0/5] dts: testpmd show port info/stats Luca Vizzarro
2024-06-05 14:45 ` [PATCH v3 1/5] dts: fix InteractiveShell command prompt filtering Luca Vizzarro
2024-06-05 14:45 ` [PATCH v3 2/5] dts: skip first line of send command output Luca Vizzarro
2024-06-05 14:45 ` [PATCH v3 3/5] dts: add parsing utility module Luca Vizzarro
2024-06-05 14:45 ` [PATCH v3 4/5] dts: add `show port info` command to TestPmdShell Luca Vizzarro
2024-06-05 14:45 ` [PATCH v3 5/5] dts: add `show port stats` " Luca Vizzarro
2024-06-06 9:17 ` [PATCH v4 0/5] dts: testpmd show port info/stats Luca Vizzarro
2024-06-06 9:17 ` [PATCH v4 1/5] dts: fix InteractiveShell command prompt filtering Luca Vizzarro
2024-06-06 18:52 ` Jeremy Spewock
2024-06-06 9:17 ` [PATCH v4 2/5] dts: skip first line of send command output Luca Vizzarro
2024-06-06 18:52 ` Jeremy Spewock
2024-06-06 9:17 ` Luca Vizzarro [this message]
2024-06-06 18:52 ` [PATCH v4 3/5] dts: add parsing utility module Jeremy Spewock
2024-06-06 21:14 ` Luca Vizzarro
2024-06-06 9:17 ` [PATCH v4 4/5] dts: add `show port info` command to TestPmdShell Luca Vizzarro
2024-06-06 18:52 ` Jeremy Spewock
2024-06-06 9:17 ` [PATCH v4 5/5] dts: add `show port stats` " Luca Vizzarro
2024-06-06 18:53 ` Jeremy Spewock
2024-06-06 21:34 ` [PATCH v5 0/5] dts: testpmd show port info/stats Luca Vizzarro
2024-06-06 21:34 ` [PATCH v5 1/5] dts: fix InteractiveShell command prompt filtering Luca Vizzarro
2024-06-07 11:15 ` Juraj Linkeš
2024-06-07 13:10 ` Jeremy Spewock
2024-06-14 20:05 ` Nicholas Pratte
2024-06-06 21:34 ` [PATCH v5 2/5] dts: skip first line of send command output Luca Vizzarro
2024-06-07 11:16 ` Juraj Linkeš
2024-06-07 13:10 ` Jeremy Spewock
2024-06-06 21:34 ` [PATCH v5 3/5] dts: add parsing utility module Luca Vizzarro
2024-06-07 11:16 ` Juraj Linkeš
2024-06-07 13:11 ` Jeremy Spewock
2024-06-14 17:39 ` Nicholas Pratte
2024-06-06 21:34 ` [PATCH v5 4/5] dts: add `show port info` command to TestPmdShell Luca Vizzarro
2024-06-07 11:16 ` Juraj Linkeš
2024-06-07 13:11 ` Jeremy Spewock
2024-06-14 17:36 ` Nicholas Pratte
2024-06-06 21:34 ` [PATCH v5 5/5] dts: add `show port stats` " Luca Vizzarro
2024-06-07 11:16 ` Juraj Linkeš
2024-06-07 13:11 ` Jeremy Spewock
2024-06-14 17:34 ` Nicholas Pratte
2024-06-20 3:04 ` [PATCH v5 0/5] dts: testpmd show port info/stats Thomas Monjalon
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20240606091718.151516-4-luca.vizzarro@arm.com \
--to=luca.vizzarro@arm.com \
--cc=dev@dpdk.org \
--cc=jspewock@iol.unh.edu \
--cc=juraj.linkes@pantheon.tech \
--cc=paul.szczepanek@arm.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).