From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id C9A5145482; Mon, 17 Jun 2024 17:22:27 +0200 (CEST) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 8BF654028B; Mon, 17 Jun 2024 17:22:27 +0200 (CEST) Received: from mail-lj1-f169.google.com (mail-lj1-f169.google.com [209.85.208.169]) by mails.dpdk.org (Postfix) with ESMTP id DE8D74028A for ; Mon, 17 Jun 2024 17:22:25 +0200 (CEST) Received: by mail-lj1-f169.google.com with SMTP id 38308e7fff4ca-2ebd590a79cso4480311fa.1 for ; Mon, 17 Jun 2024 08:22:25 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=iol.unh.edu; s=unh-iol; t=1718637745; x=1719242545; darn=dpdk.org; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:from:to:cc:subject:date :message-id:reply-to; bh=1U4l7uDEYS12bZNB55mL3ZvTdvk/0XiMYgpruH9JblM=; b=gsMqdVVf3+ci4jxHWUZTDKoE4sE02HDnUcF/cxXi1YnHrGe0vxSY6uD/6iEdkR5YPk xqN8K7fxfEJFyNPpMMXrdQnk55II5LwhqbiWSi2gHM/QqXSj4lWoE3h+BI3Jy2+xzHVv UXrd0ayc7rm4qhA1CF1ZYSWqFsUoksGL66Gqo= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1718637745; x=1719242545; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=1U4l7uDEYS12bZNB55mL3ZvTdvk/0XiMYgpruH9JblM=; b=MXPtzMJAtT8NOzjqJMUxbaRJIZRMciP9HuMkx8dr0RBqad16NRxXTRX00tetnr0Xac ON1RvSEVWAwWENxy+7elBwyj+R5LSXlIdizWA5i3rZsW8UoMscv+yskg2LqKqL6GCnSb 0UemlXqSCTZqr8xITsGhpuNUBcqZH9FYPhUTHzhnTWazWcRimn7FL3jFKcny1ExHitKz MaDiqt/ymYyfc8+MY5DcNSzWYHELOQmRZ5jKh4fldB2cwo9ulL9KZ9J7MS1NcVK0/by+ mgWfcBcm53nhr3UEq3Zx+QEpWZEOWfm7F2/vUDfCvZcnszTi4UtdVF4gtNIMNMf8zJDp fvuQ== X-Gm-Message-State: AOJu0YwatYY9sGL4cSalbpPjM91hLKcmc1yzEMh5XEQSa5mQGBkhDPR4 y5Tg0F5Bjx+5ROCNbc/7pTuRGyR/nf+oFzxKRJJQgGyNLTeIl/XVxsmY9Wto8EgHtnWVZUkcnQY rOYaP6ZLBvqooDoMNsok0VC6ticjYpXmlcI08pQ== X-Google-Smtp-Source: AGHT+IHvQLd9aBhXcnD7Yj8GJNvFoamG7T/YPwEq38BH9BlU7Qea3fnx/yRzUKC1l+KgLWkwJZdyzsY5UFt7+Sa+gIk= X-Received: by 2002:a05:651c:547:b0:2ec:e97:3dc1 with SMTP id 38308e7fff4ca-2ec0e973e41mr61164841fa.0.1718637745180; Mon, 17 Jun 2024 08:22:25 -0700 (PDT) MIME-Version: 1.0 References: <20240326190422.577028-1-luca.vizzarro@arm.com> <20240617145409.67407-1-luca.vizzarro@arm.com> <20240617145409.67407-2-luca.vizzarro@arm.com> In-Reply-To: <20240617145409.67407-2-luca.vizzarro@arm.com> From: Nicholas Pratte Date: Mon, 17 Jun 2024 11:22:14 -0400 Message-ID: Subject: Re: [PATCH v5 1/8] dts: add params manipulation module To: Luca Vizzarro Cc: dev@dpdk.org, Jeremy Spewock , =?UTF-8?Q?Juraj_Linke=C5=A1?= , Paul Szczepanek Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Tested-by: Nicholas Pratte Reviewed-by: Nicholas Pratte On Mon, Jun 17, 2024 at 10:54=E2=80=AFAM Luca Vizzarro wrote: > > This commit introduces a new "params" module, which adds a new way > to manage command line parameters. The provided Params dataclass > is able to read the fields of its child class and produce a string > representation to supply to the command line. Any data structure > that is intended to represent command line parameters can inherit it. > > The main purpose is to make it easier to represent data structures that > map to parameters. Aiding quicker development, while minimising code > bloat. > > Signed-off-by: Luca Vizzarro > Reviewed-by: Paul Szczepanek > --- > dts/framework/params/__init__.py | 358 +++++++++++++++++++++++++++++++ > 1 file changed, 358 insertions(+) > create mode 100644 dts/framework/params/__init__.py > > diff --git a/dts/framework/params/__init__.py b/dts/framework/params/__in= it__.py > new file mode 100644 > index 0000000000..107b070ed2 > --- /dev/null > +++ b/dts/framework/params/__init__.py > @@ -0,0 +1,358 @@ > +# SPDX-License-Identifier: BSD-3-Clause > +# Copyright(c) 2024 Arm Limited > + > +"""Parameter manipulation module. > + > +This module provides :class:`Params` which can be used to model any data= structure > +that is meant to represent any command line parameters. > +""" > + > +from dataclasses import dataclass, fields > +from enum import Flag > +from typing import ( > + Any, > + Callable, > + Iterable, > + Literal, > + Reversible, > + TypedDict, > + TypeVar, > + cast, > +) > + > +from typing_extensions import Self > + > +T =3D TypeVar("T") > + > +#: Type for a function taking one argument. > +FnPtr =3D Callable[[Any], Any] > +#: Type for a switch parameter. > +Switch =3D Literal[True, None] > +#: Type for a yes/no switch parameter. > +YesNoSwitch =3D Literal[True, False, None] > + > + > +def _reduce_functions(funcs: Iterable[FnPtr]) -> FnPtr: > + """Reduces an iterable of :attr:`FnPtr` from left to right to a sing= le function. > + > + If the iterable is empty, the created function just returns its fed = value back. > + > + Args: > + funcs: An iterable containing the functions to be chained from l= eft to right. > + > + Returns: > + FnPtr: A function that calls the given functions from left to ri= ght. > + """ > + > + def reduced_fn(value): > + for fn in funcs: > + value =3D fn(value) > + return value > + > + return reduced_fn > + > + > +def modify_str(*funcs: FnPtr) -> Callable[[T], T]: > + """Class decorator modifying the ``__str__`` method with a function = created from its arguments. > + > + The :attr:`FnPtr`s fed to the decorator are executed from left to ri= ght in the arguments list > + order. > + > + Args: > + *funcs: The functions to chain from left to right. > + > + Returns: > + The decorator. > + > + Example: > + .. code:: python > + > + @convert_str(hex_from_flag_value) > + class BitMask(enum.Flag): > + A =3D auto() > + B =3D auto() > + > + will allow ``BitMask`` to render as a hexadecimal value. > + """ > + > + def _class_decorator(original_class): > + original_class.__str__ =3D _reduce_functions(funcs) > + return original_class > + > + return _class_decorator > + > + > +def comma_separated(values: Iterable[Any]) -> str: > + """Converts an iterable into a comma-separated string. > + > + Args: > + values: An iterable of objects. > + > + Returns: > + A comma-separated list of stringified values. > + """ > + return ",".join([str(value).strip() for value in values if value is = not None]) > + > + > +def bracketed(value: str) -> str: > + """Adds round brackets to the input. > + > + Args: > + value: Any string. > + > + Returns: > + A string surrounded by round brackets. > + """ > + return f"({value})" > + > + > +def str_from_flag_value(flag: Flag) -> str: > + """Returns the value from a :class:`enum.Flag` as a string. > + > + Args: > + flag: An instance of :class:`Flag`. > + > + Returns: > + The stringified value of the given flag. > + """ > + return str(flag.value) > + > + > +def hex_from_flag_value(flag: Flag) -> str: > + """Returns the value from a :class:`enum.Flag` converted to hexadeci= mal. > + > + Args: > + flag: An instance of :class:`Flag`. > + > + Returns: > + The value of the given flag in hexadecimal representation. > + """ > + return hex(flag.value) > + > + > +class ParamsModifier(TypedDict, total=3DFalse): > + """Params modifiers dict compatible with the :func:`dataclasses.fiel= d` metadata parameter.""" > + > + #: > + Params_short: str > + #: > + Params_long: str > + #: > + Params_multiple: bool > + #: > + Params_convert_value: Reversible[FnPtr] > + > + > +@dataclass > +class Params: > + """Dataclass that renders its fields into command line arguments. > + > + The parameter name is taken from the field name by default. The foll= owing: > + > + .. code:: python > + > + name: str | None =3D "value" > + > + is rendered as ``--name=3Dvalue``. > + Through :func:`dataclasses.field` the resulting parameter can be man= ipulated by applying > + this class' metadata modifier functions. These return regular dictio= naries which can be combined > + together using the pipe (OR) operator. > + > + To use fields as switches, set the value to ``True`` to render them.= If you > + use a yes/no switch you can also set ``False`` which would render a = switch > + prefixed with ``--no-``. Examples: > + > + .. code:: python > + > + interactive: Switch =3D True # renders --interactive > + numa: YesNoSwitch =3D False # renders --no-numa > + > + Setting ``None`` will prevent it from being rendered. The :attr:`~Sw= itch` type alias is provided > + for regular switches, whereas :attr:`~YesNoSwitch` is offered for ye= s/no ones. > + > + An instance of a dataclass inheriting ``Params`` can also be assigne= d to an attribute, > + this helps with grouping parameters together. > + The attribute holding the dataclass will be ignored and the latter w= ill just be rendered as > + expected. > + """ > + > + _suffix =3D "" > + """Holder of the plain text value of Params when called directly. A = suffix for child classes.""" > + > + """=3D=3D=3D=3D=3D=3D=3D=3D=3D BEGIN FIELD METADATA MODIFIER FUNCTIO= NS =3D=3D=3D=3D=3D=3D=3D=3D""" > + > + @staticmethod > + def short(name: str) -> ParamsModifier: > + """Overrides any parameter name with the given short option. > + > + Args: > + name: The short parameter name. > + > + Returns: > + ParamsModifier: A dictionary for the `dataclasses.field` met= adata argument containing > + the parameter short name modifier. > + > + Example: > + .. code:: python > + > + logical_cores: str | None =3D field(default=3D"1-4", met= adata=3DParams.short("l")) > + > + will render as ``-l=3D1-4`` instead of ``--logical-cores=3D1= -4``. > + """ > + return ParamsModifier(Params_short=3Dname) > + > + @staticmethod > + def long(name: str) -> ParamsModifier: > + """Overrides the inferred parameter name to the specified one. > + > + Args: > + name: The long parameter name. > + > + Returns: > + ParamsModifier: A dictionary for the `dataclasses.field` met= adata argument containing > + the parameter long name modifier. > + > + Example: > + .. code:: python > + > + x_name: str | None =3D field(default=3D"y", metadata=3DP= arams.long("x")) > + > + will render as ``--x=3Dy``, but the field is accessed and mo= dified through ``x_name``. > + """ > + return ParamsModifier(Params_long=3Dname) > + > + @staticmethod > + def multiple() -> ParamsModifier: > + """Specifies that this parameter is set multiple times. The para= meter type must be a list. > + > + Returns: > + ParamsModifier: A dictionary for the `dataclasses.field` met= adata argument containing > + the multiple parameters modifier. > + > + Example: > + .. code:: python > + > + ports: list[int] | None =3D field( > + default_factory=3Dlambda: [0, 1, 2], > + metadata=3DParams.multiple() | Params.long("port") > + ) > + > + will render as ``--port=3D0 --port=3D1 --port=3D2``. > + """ > + return ParamsModifier(Params_multiple=3DTrue) > + > + @staticmethod > + def convert_value(*funcs: FnPtr) -> ParamsModifier: > + """Takes in a variable number of functions to convert the value = text representation. > + > + Functions can be chained together, executed from left to right i= n the arguments list order. > + > + Args: > + *funcs: The functions to chain from left to right. > + > + Returns: > + ParamsModifier: A dictionary for the `dataclasses.field` met= adata argument containing > + the convert value modifier. > + > + Example: > + .. code:: python > + > + hex_bitmask: int | None =3D field( > + default=3D0b1101, > + metadata=3DParams.convert_value(hex) | Params.long("= mask") > + ) > + > + will render as ``--mask=3D0xd``. > + """ > + return ParamsModifier(Params_convert_value=3Dfuncs) > + > + """=3D=3D=3D=3D=3D=3D=3D=3D=3D END FIELD METADATA MODIFIER FUNCTIONS= =3D=3D=3D=3D=3D=3D=3D=3D""" > + > + def append_str(self, text: str) -> None: > + """Appends a string at the end of the string representation. > + > + Args: > + text: Any text to append at the end of the parameters string= representation. > + """ > + self._suffix +=3D text > + > + def __iadd__(self, text: str) -> Self: > + """Appends a string at the end of the string representation. > + > + Args: > + text: Any text to append at the end of the parameters string= representation. > + > + Returns: > + The given instance back. > + """ > + self.append_str(text) > + return self > + > + @classmethod > + def from_str(cls, text: str) -> Self: > + """Creates a plain Params object from a string. > + > + Args: > + text: The string parameters. > + > + Returns: > + A new plain instance of :class:`Params`. > + """ > + obj =3D cls() > + obj.append_str(text) > + return obj > + > + @staticmethod > + def _make_switch( > + name: str, is_short: bool =3D False, is_no: bool =3D False, valu= e: str | None =3D None > + ) -> str: > + """Make the string representation of the parameter. > + > + Args: > + name: The name of the parameters. > + is_short: If the parameters is short or not. > + is_no: If the parameter is negated or not. > + value: The value of the parameter. > + > + Returns: > + The complete command line parameter. > + """ > + prefix =3D f"{'-' if is_short else '--'}{'no-' if is_no else ''}= " > + name =3D name.replace("_", "-") > + value =3D f"{' ' if is_short else '=3D'}{value}" if value else "= " > + return f"{prefix}{name}{value}" > + > + def __str__(self) -> str: > + """Returns a string of command-line-ready arguments from the cla= ss fields.""" > + arguments: list[str] =3D [] > + > + for field in fields(self): > + value =3D getattr(self, field.name) > + modifiers =3D cast(ParamsModifier, field.metadata) > + > + if value is None: > + continue > + > + if isinstance(value, Params): > + arguments.append(str(value)) > + continue > + > + # take the short modifier, or the long modifier, or infer fr= om field name > + switch_name =3D modifiers.get("Params_short", modifiers.get(= "Params_long", field.name)) > + is_short =3D "Params_short" in modifiers > + > + if isinstance(value, bool): > + arguments.append(self._make_switch(switch_name, is_short= , is_no=3D(not value))) > + continue > + > + convert =3D _reduce_functions(modifiers.get("Params_convert_= value", [])) > + multiple =3D modifiers.get("Params_multiple", False) > + > + values =3D value if multiple else [value] > + for value in values: > + arguments.append(self._make_switch(switch_name, is_short= , value=3Dconvert(value))) > + > + if self._suffix: > + arguments.append(self._suffix) > + > + return " ".join(arguments) > -- > 2.34.1 >