From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 4FC0A43380; Mon, 20 Nov 2023 17:36:47 +0100 (CET) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 4071142DDE; Mon, 20 Nov 2023 17:36:47 +0100 (CET) Received: from mail-ej1-f41.google.com (mail-ej1-f41.google.com [209.85.218.41]) by mails.dpdk.org (Postfix) with ESMTP id CCBBD42DD2 for ; Mon, 20 Nov 2023 17:36:45 +0100 (CET) Received: by mail-ej1-f41.google.com with SMTP id a640c23a62f3a-9fd0059a967so266727066b.1 for ; Mon, 20 Nov 2023 08:36:45 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=pantheon.tech; s=google; t=1700498205; x=1701103005; darn=dpdk.org; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:from:to:cc:subject:date :message-id:reply-to; bh=80h9wLZI7rcgarlYP6KTc15JuwEl75JXrLX1vNqDqzU=; b=p+kJBBoE8e9FEehsCRUgYR/yGNt+zVVyYIrpGTVPXvmWNs5qVKnPl48JOef88o9DYv I6vikHwp2f8UEGCSZOfTeLUWQzyTSF15XCq3awBlwEOajeH7HOthzaWEBnPvZMfvBKeb Khy/sXOiyTGFU1CQxkJS3MlSr+QIK5vfH37IEA9r3JS95WzDHPWDnHe0p1PNSoFch50g GpY77Ub5kTHiHS7lMKj7CpgMJGU84QTRBYvAL+L7kblPAKOh6OBKoODBO0cWqFPav61f UZRD54Yms7mimipYrHf+ahyqlUAP4J54IWC/gX6VQ8fVp3ctjNcwE1qm2WcOIgXio4Mc CtsQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1700498205; x=1701103005; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=80h9wLZI7rcgarlYP6KTc15JuwEl75JXrLX1vNqDqzU=; b=GWCZu4Wd7NCBU+Hjk44raRw8f1KgxJypWocL1WXZQQbimef5SwX2V5TvISyQSAqQxc o0oGK1X9QX3yZwikSqRsPW+LSkX55tbOdEQUocqy8feHUIlLHbJCsRj3VXqCUfq5S/Fv TICc3LhGMN639IfoHeXCGrd9+xcLAjClh+rguOWRv4/c+l4M0SwhKzwUCSTW7wBi4NLJ fIIs1SOaYxTRPDLv35fNlr+M2PBuAzmkyzbBP6edPppDv6s+rNOYU79TMqTNPHVbff7N 1spUJgSSzYsmuOlFbPDY2l7gqB2KirC9iYtwOaS310xaNMuwyCP608FTD45vUp6iFltj DGWA== X-Gm-Message-State: AOJu0YzChdBg9LDDzstI0AQBiEDKXOH2SEAWTNnE9kdrP1kF2MeVZKsX 4/03BEN5h7HxrgdAWdZ7cweldvJ/MYa6BMbbYW6XOwSUV3fhQoBSGk8= X-Google-Smtp-Source: AGHT+IFfUFkRphhkHGJcps+MY6CE1yrtWMazESv/unlwvhDYXOuYNQLXiuMs3Rd+L/wBPDRlllgQUepIg05WkLqvgwA= X-Received: by 2002:a17:906:51dc:b0:9a2:295a:9bbc with SMTP id v28-20020a17090651dc00b009a2295a9bbcmr1887553ejk.37.1700498205416; Mon, 20 Nov 2023 08:36:45 -0800 (PST) MIME-Version: 1.0 References: <20231108125324.191005-23-juraj.linkes@pantheon.tech> <20231115130959.39420-1-juraj.linkes@pantheon.tech> <20231115130959.39420-7-juraj.linkes@pantheon.tech> <5e7ae58b-ebbe-4c83-bbe7-71451a350ed2@foss.arm.com> In-Reply-To: <5e7ae58b-ebbe-4c83-bbe7-71451a350ed2@foss.arm.com> From: =?UTF-8?Q?Juraj_Linke=C5=A1?= Date: Mon, 20 Nov 2023 17:36:34 +0100 Message-ID: Subject: Re: [PATCH v7 06/21] dts: logger and utils docstring update To: Yoan Picchi Cc: thomas@monjalon.net, Honnappa.Nagarahalli@arm.com, jspewock@iol.unh.edu, probb@iol.unh.edu, paul.szczepanek@arm.com, dev@dpdk.org Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org On Mon, Nov 20, 2023 at 5:23=E2=80=AFPM Yoan Picchi wrote: > > On 11/15/23 13:09, Juraj Linke=C5=A1 wrote: > > Format according to the Google format and PEP257, with slight > > deviations. > > > > Signed-off-by: Juraj Linke=C5=A1 > > --- > > dts/framework/logger.py | 72 ++++++++++++++++++++++----------- > > dts/framework/utils.py | 88 +++++++++++++++++++++++++++++-----------= - > > 2 files changed, 113 insertions(+), 47 deletions(-) > > > > diff --git a/dts/framework/logger.py b/dts/framework/logger.py > > index bb2991e994..d3eb75a4e4 100644 > > --- a/dts/framework/logger.py > > +++ b/dts/framework/logger.py > > @@ -3,9 +3,9 @@ > > # Copyright(c) 2022-2023 PANTHEON.tech s.r.o. > > # Copyright(c) 2022-2023 University of New Hampshire > > > > -""" > > -DTS logger module with several log level. DTS framework and TestSuite = logs > > -are saved in different log files. > > +"""DTS logger module. > > + > > +DTS framework and TestSuite logs are saved in different log files. > > """ > > > > import logging > > @@ -18,19 +18,21 @@ > > stream_fmt =3D "%(asctime)s - %(name)s - %(levelname)s - %(message)s" > > > > > > -class LoggerDictType(TypedDict): > > - logger: "DTSLOG" > > - name: str > > - node: str > > - > > +class DTSLOG(logging.LoggerAdapter): > > + """DTS logger adapter class for framework and testsuites. > > > > -# List for saving all using loggers > > -Loggers: list[LoggerDictType] =3D [] > > + The :option:`--verbose` command line argument and the :envvar:`DTS= _VERBOSE` environment > > + variable control the verbosity of output. If enabled, all messages= will be emitted to the > > + console. > > > > + The :option:`--output` command line argument and the :envvar:`DTS_= OUTPUT_DIR` environment > > + variable modify the directory where the logs will be stored. > > > > -class DTSLOG(logging.LoggerAdapter): > > - """ > > - DTS log class for framework and testsuite. > > + Attributes: > > + node: The additional identifier. Currently unused. > > + sh: The handler which emits logs to console. > > + fh: The handler which emits logs to a file. > > + verbose_fh: Just as fh, but logs with a different, more verbos= e, format. > > """ > > > > _logger: logging.Logger > > @@ -40,6 +42,15 @@ class DTSLOG(logging.LoggerAdapter): > > verbose_fh: logging.FileHandler > > > > def __init__(self, logger: logging.Logger, node: str =3D "suite")= : > > + """Extend the constructor with additional handlers. > > + > > + One handler logs to the console, the other one to a file, with= either a regular or verbose > > + format. > > + > > + Args: > > + logger: The logger from which to create the logger adapter= . > > + node: An additional identifier. Currently unused. > > + """ > > self._logger =3D logger > > # 1 means log everything, this will be used by file handlers = if their level > > # is not set > > @@ -92,26 +103,43 @@ def __init__(self, logger: logging.Logger, node: s= tr =3D "suite"): > > super(DTSLOG, self).__init__(self._logger, dict(node=3Dself.n= ode)) > > > > def logger_exit(self) -> None: > > - """ > > - Remove stream handler and logfile handler. > > - """ > > + """Remove the stream handler and the logfile handler.""" > > for handler in (self.sh, self.fh, self.verbose_fh): > > handler.flush() > > self._logger.removeHandler(handler) > > > > > > +class _LoggerDictType(TypedDict): > > + logger: DTSLOG > > + name: str > > + node: str > > + > > + > > +# List for saving all loggers in use > > +_Loggers: list[_LoggerDictType] =3D [] > > + > > + > > def getLogger(name: str, node: str =3D "suite") -> DTSLOG: > > + """Get DTS logger adapter identified by name and node. > > + > > + An existing logger will be return if one with the exact name and n= ode already exists. > > An existing logger will be return*ed* > Ack, will fix. > > + A new one will be created and stored otherwise. > > + > > + Args: > > + name: The name of the logger. > > + node: An additional identifier for the logger. > > + > > + Returns: > > + A logger uniquely identified by both name and node. > > """ > > - Get logger handler and if there's no handler for specified Node wi= ll create one. > > - """ > > - global Loggers > > + global _Loggers > > # return saved logger > > - logger: LoggerDictType > > - for logger in Loggers: > > + logger: _LoggerDictType > > + for logger in _Loggers: > > if logger["name"] =3D=3D name and logger["node"] =3D=3D node: > > return logger["logger"] > > > > # return new logger > > dts_logger: DTSLOG =3D DTSLOG(logging.getLogger(name), node) > > - Loggers.append({"logger": dts_logger, "name": name, "node": node}) > > + _Loggers.append({"logger": dts_logger, "name": name, "node": node}= ) > > return dts_logger > > diff --git a/dts/framework/utils.py b/dts/framework/utils.py > > index f0c916471c..5016e3be10 100644 > > --- a/dts/framework/utils.py > > +++ b/dts/framework/utils.py > > @@ -3,6 +3,16 @@ > > # Copyright(c) 2022-2023 PANTHEON.tech s.r.o. > > # Copyright(c) 2022-2023 University of New Hampshire > > > > +"""Various utility classes and functions. > > + > > +These are used in multiple modules across the framework. They're here = because > > +they provide some non-specific functionality, greatly simplify imports= or just don't > > +fit elsewhere. > > + > > +Attributes: > > + REGEX_FOR_PCI_ADDRESS: The regex representing a PCI address, e.g. = ``0000:00:08.0``. > > +""" > > + > > import atexit > > import json > > import os > > @@ -19,12 +29,20 @@ > > > > > > def expand_range(range_str: str) -> list[int]: > > - """ > > - Process range string into a list of integers. There are two possib= le formats: > > - n - a single integer > > - n-m - a range of integers > > + """Process `range_str` into a list of integers. > > + > > + There are two possible formats of `range_str`: > > + > > + * ``n`` - a single integer, > > + * ``n-m`` - a range of integers. > > > > - The returned range includes both n and m. Empty string returns an = empty list. > > + The returned range includes both ``n`` and ``m``. Empty string ret= urns an empty list. > > + > > + Args: > > + range_str: The range to expand. > > + > > + Returns: > > + All the numbers from the range. > > """ > > expanded_range: list[int] =3D [] > > if range_str: > > @@ -39,6 +57,14 @@ def expand_range(range_str: str) -> list[int]: > > > > > > def get_packet_summaries(packets: list[Packet]) -> str: > > + """Format a string summary from `packets`. > > + > > + Args: > > + packets: The packets to format. > > + > > + Returns: > > + The summary of `packets`. > > + """ > > if len(packets) =3D=3D 1: > > packet_summaries =3D packets[0].summary() > > else: > > @@ -49,6 +75,8 @@ def get_packet_summaries(packets: list[Packet]) -> st= r: > > > > > > class StrEnum(Enum): > > + """Enum with members stored as strings.""" > > + > > @staticmethod > > def _generate_next_value_( > > name: str, start: int, count: int, last_values: object > > @@ -56,22 +84,29 @@ def _generate_next_value_( > > return name > > > > def __str__(self) -> str: > > + """The string representation is the name of the member.""" > > return self.name > > > > > > class MesonArgs(object): > > - """ > > - Aggregate the arguments needed to build DPDK: > > - default_library: Default library type, Meson allows "shared", "sta= tic" and "both". > > - Defaults to None, in which case the argument won't be u= sed. > > - Keyword arguments: The arguments found in meson_options.txt in roo= t DPDK directory. > > - Do not use -D with them, for example: > > - meson_args =3D MesonArgs(enable_kmods=3DTrue). > > - """ > > + """Aggregate the arguments needed to build DPDK.""" > > > > _default_library: str > > > > def __init__(self, default_library: str | None =3D None, **dpdk_a= rgs: str | bool): > > + """Initialize the meson arguments. > > + > > + Args: > > + default_library: The default library type, Meson supports = ``shared``, ``static`` and > > + ``both``. Defaults to :data:`None`, in which case the = argument won't be used. > > + dpdk_args: The arguments found in ``meson_options.txt`` in= root DPDK directory. > > + Do not use ``-D`` with them. > > + > > + Example: > > + :: > > + > > + meson_args =3D MesonArgs(enable_kmods=3DTrue). > > + """ > > self._default_library =3D ( > > f"--default-library=3D{default_library}" if default_libra= ry else "" > > ) > > @@ -83,6 +118,7 @@ def __init__(self, default_library: str | None =3D N= one, **dpdk_args: str | bool): > > ) > > > > def __str__(self) -> str: > > + """The actual args.""" > > return " ".join(f"{self._default_library} {self._dpdk_args}".= split()) > > > > > > @@ -104,24 +140,14 @@ class _TarCompressionFormat(StrEnum): > > > > > > class DPDKGitTarball(object): > > - """Create a compressed tarball of DPDK from the repository. > > - > > - The DPDK version is specified with git object git_ref. > > - The tarball will be compressed with _TarCompressionFormat, > > - which must be supported by the DTS execution environment. > > - The resulting tarball will be put into output_dir. > > + """Compressed tarball of DPDK from the repository. > > > > - The class supports the os.PathLike protocol, > > + The class supports the :class:`os.PathLike` protocol, > > which is used to get the Path of the tarball:: > > > > from pathlib import Path > > tarball =3D DPDKGitTarball("HEAD", "output") > > tarball_path =3D Path(tarball) > > - > > - Arguments: > > - git_ref: A git commit ID, tag ID or tree ID. > > - output_dir: The directory where to put the resulting tarball. > > - tar_compression_format: The compression format to use. > > """ > > > > _git_ref: str > > @@ -136,6 +162,17 @@ def __init__( > > output_dir: str, > > tar_compression_format: _TarCompressionFormat =3D _TarCompres= sionFormat.xz, > > ): > > + """Create the tarball during initialization. > > + > > + The DPDK version is specified with `git_ref`. The tarball will= be compressed with > > + `tar_compression_format`, which must be supported by the DTS e= xecution environment. > > + The resulting tarball will be put into `output_dir`. > > + > > + Args: > > + git_ref: A git commit ID, tag ID or tree ID. > > + output_dir: The directory where to put the resulting tarba= ll. > > + tar_compression_format: The compression format to use. > > + """ > > self._git_ref =3D git_ref > > self._tar_compression_format =3D tar_compression_format > > > > @@ -204,4 +241,5 @@ def _delete_tarball(self) -> None: > > os.remove(self._tarball_path) > > > > def __fspath__(self) -> str: > > + """The os.PathLike protocol implementation.""" > > return str(self._tarball_path) >