From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 6C58BA0548; Thu, 8 Sep 2022 10:31:26 +0200 (CEST) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 0E51D40143; Thu, 8 Sep 2022 10:31:26 +0200 (CEST) Received: from mga14.intel.com (mga14.intel.com [192.55.52.115]) by mails.dpdk.org (Postfix) with ESMTP id EDB44400D6 for ; Thu, 8 Sep 2022 10:31:23 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1662625884; x=1694161884; h=date:from:to:cc:subject:message-id:references: content-transfer-encoding:in-reply-to:mime-version; bh=mv0WSTZkHaPN75atolZO1HTKohNmmpGNk/Bi3128mfM=; b=dcHis2xYNj5N8dGArbpgW2ca6kLG9I1YfH/QrBTefQeArS+KDiKpxHkc e3NdAy1KVbeVfU/8Zf8ZjQp6L4YXHvrVjsUPFoVmLcGQB2nkMtkDQlEBO IM7gnhVhS4q4UzU2xHZsMIY8L5boE0Mpg9l0SAciHnUJ1IvBYz0zSNPjY INRgMt8rWnch6Co4pzhjN/M5yAqT3mh/dlEH21qLIQ9dPhLGJXVpVPSZm z1koTMrPKGlgTvdvdszFP0KH1YEkl58+uhAXDsaynag/9oNhPdszLhVZz /N4AkHFJGgEILLgR1v/7nOqfQ2ViSNO49ivKQXm5wdz7bB87MSoK4ndls w==; X-IronPort-AV: E=McAfee;i="6500,9779,10463"; a="297109623" X-IronPort-AV: E=Sophos;i="5.93,299,1654585200"; d="scan'208";a="297109623" Received: from orsmga001.jf.intel.com ([10.7.209.18]) by fmsmga103.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 08 Sep 2022 01:31:22 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.93,299,1654585200"; d="scan'208";a="647959973" Received: from orsmsx601.amr.corp.intel.com ([10.22.229.14]) by orsmga001.jf.intel.com with ESMTP; 08 Sep 2022 01:31:22 -0700 Received: from orsmsx607.amr.corp.intel.com (10.22.229.20) by ORSMSX601.amr.corp.intel.com (10.22.229.14) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2375.31; Thu, 8 Sep 2022 01:31:22 -0700 Received: from orsmsx607.amr.corp.intel.com (10.22.229.20) by ORSMSX607.amr.corp.intel.com (10.22.229.20) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2375.31; Thu, 8 Sep 2022 01:31:22 -0700 Received: from orsedg603.ED.cps.intel.com (10.7.248.4) by orsmsx607.amr.corp.intel.com (10.22.229.20) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2375.31 via Frontend Transport; Thu, 8 Sep 2022 01:31:22 -0700 Received: from NAM11-BN8-obe.outbound.protection.outlook.com (104.47.58.169) by edgegateway.intel.com (134.134.137.100) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.1.2375.31; Thu, 8 Sep 2022 01:31:21 -0700 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=WnL7bDB4hEQX/rBMQ4WL4Jco4gBmBiV9ZRXn5hGGg5ftsTOSxjtoOUh/g5wGbHKqfWG5v88VRDbMDiOZvbRHwYbzhkiUJSdcpiJCqvFKSijZFBIv7uep8q3vAXbzPL2IewiKf8/oGZNksWjIwARQ8EwNIqU2vJAm7zzKWDlLbwqAmS1xeDF1NIxnN3hhb5vgJ8YlJnOC+4VuwHKbnzPvkYqxEEeaj2zlMBqh1CscpHvLaOmnR7Jpci7178R9aFbMQpRG6H2faaD53q1guZrwTraWiVGh8LlEavhBwYwPRZyz+wILKTyh3ScAwFj8BW8bO2lb7E4yHZG5poDG42O1LA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=Bv7E9GIGlPYEkjRGh1sVVm+tceGTJsSQ0ElyFW9V7MA=; b=iMPjXBTi4OnewwV7Po1YZWSLsu4VjfzaPLyhqvH6cjw1UsXj3UWPHaF+0SgwvrPxg8jWcGLfyaXlz+heH/QKTWgZ42EDK4b+N8VcraetGzf0y+iXVa62Qw96U+Zg8BLx4oHG43xbbq1ZIRxCx4tm7HCgU/zkZ8qxyFx10qseLyO9HbQj++7hQN451BYKHFIOxtBSLoMNFyLGGJl0mvM8UnDFBN1XGZHuS1lzsyj2BiVnx4KYR3YU3Sca0yNFpTCjZEesmhSpHac/yKQYsqtHtpZk9NIZ03/2/ex4B/i2JJJf4D3PgwaZGtLOJOsxDcRPTqdCPwcPNNlUZe5TuqOqtg== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=intel.com; dmarc=pass action=none header.from=intel.com; dkim=pass header.d=intel.com; arc=none Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=intel.com; Received: from MWHPR11MB1629.namprd11.prod.outlook.com (2603:10b6:301:d::21) by CY4PR11MB1895.namprd11.prod.outlook.com (2603:10b6:903:126::7) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5588.17; Thu, 8 Sep 2022 08:31:14 +0000 Received: from MWHPR11MB1629.namprd11.prod.outlook.com ([fe80::13c:8120:d994:16d2]) by MWHPR11MB1629.namprd11.prod.outlook.com ([fe80::13c:8120:d994:16d2%6]) with mapi id 15.20.5612.014; Thu, 8 Sep 2022 08:31:14 +0000 Date: Thu, 8 Sep 2022 09:31:07 +0100 From: Bruce Richardson To: Juraj =?utf-8?Q?Linke=C5=A1?= CC: , , , , , , Subject: Re: [PATCH v4 3/9] dts: add basic logging facility Message-ID: References: <20220728100044.1318484-1-juraj.linkes@pantheon.tech> <20220729105550.1382664-1-juraj.linkes@pantheon.tech> <20220729105550.1382664-4-juraj.linkes@pantheon.tech> Content-Type: text/plain; charset="utf-8" Content-Disposition: inline Content-Transfer-Encoding: 8bit In-Reply-To: <20220729105550.1382664-4-juraj.linkes@pantheon.tech> X-ClientProxiedBy: PA7P264CA0034.FRAP264.PROD.OUTLOOK.COM (2603:10a6:102:34b::15) To MWHPR11MB1629.namprd11.prod.outlook.com (2603:10b6:301:d::21) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: MWHPR11MB1629:EE_|CY4PR11MB1895:EE_ X-MS-Office365-Filtering-Correlation-Id: c89e956b-ae14-4d19-e7ea-08da91747dfb X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: pxTD115EfiBR1hkQw7Fas7bktXRzQPQdvNwVCaTLBCpUxpj87pX0Le3om7wX/gczPQJUhwtXsPOqpAzpYv3lIQSzAVHdaXMLqgj+uCbwC7sKBnZiIZjSMtjRBnx1OBY8T4KmeEyaoHXOW7qhDAXdpO6COiSR9rZjX7PwoQboQ5qagleqgqruOGvhSbOkBYThy7Ya255y2cXp3jB9wY9ylO3gv2Rf0/QUl/XZGVWOieAZD3Vh0Y8qh6zcyOPeXBSwO5E6A5EgBMtP35WACi1oV7xdnIZjLt2NAHgNPduUq1IDZVLzjNQpkcCDrThrm+D305vL4xC7y9nb0gk3xEikXIAr7YsFwX7tQo3LR8a1wLCDsSwXqVyXMebIlO9AIDXmmt9F4GOJ6sKVY6jvz6YUU0Jd0CCO2uojVwyX6ulVdLc3CJzLE24rCaZe2aSz41WA9uGV2qX6qKoiayTpdSmbdpzN8AxZYZhOty9lTEDQjL12THSBVFgBRLnGqNM/hgaMbJEf2jugyFMcPEdURcxpONF8CvZtHaLjIFhmm53XQ1U7cS7vjC1c68onO2BBQab8PHCwPRTYj31Qni+13NIpOAaKi9CNtQfDR4sKdAv6bWZReIb4PwAUILJb5zcdnInWzttrF0j/gRfQdO1n7HKdbaBMO1/gP9fjwzPBpd65X9P+AhF5bFrEK61Dgz0vuuHh7Y2aG4dw4W4LrJPnYCF1iA== X-Forefront-Antispam-Report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:MWHPR11MB1629.namprd11.prod.outlook.com; PTR:; CAT:NONE; SFS:(13230016)(39860400002)(366004)(396003)(376002)(136003)(346002)(82960400001)(186003)(6666004)(44832011)(6512007)(26005)(6506007)(5660300002)(6486002)(41300700001)(478600001)(316002)(38100700002)(83380400001)(6916009)(2906002)(66476007)(86362001)(8676002)(66556008)(8936002)(66946007)(4326008); DIR:OUT; SFP:1102; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?utf-8?B?OUxvY0lSZ3VHdGVQajBtUGtESm42NERjRHRwNk5oTE1GcHFlK3MxZTRGaHhh?= =?utf-8?B?NE1NdHdsekw2cCsxa1l2YVp2YllpZzlFd29pelZER1ZlUTQyNi9TdVVFbGhE?= =?utf-8?B?RkhUa3R2THdLQ090ZHpMMlBxV0txVHA5KzdPbzBiOWROK0l5SVBkMGQxTlFj?= =?utf-8?B?dG5UL1hsQmFlT2w4aFpzR0hVSmJXRnIvczk2SmViMjkyTkhDNnJNQk5QRWpX?= =?utf-8?B?YnNDTHJpZ3A0U3BqYlRnZ1daWEd0MlpzaGZHblJONkhIR2UzMEpGTVl1ODU2?= =?utf-8?B?eHhmc2NPL3JDUDZSS0dEcFFmOEdGTWhpS1gzQTAwcndJYTRKM2RiL2I5M2o3?= =?utf-8?B?UXJNSlhZbmJHN1hxODg3R0phYUw1K0FWYkpibERSVFJSSllyRzJOSXdTaGhS?= =?utf-8?B?MExjWWJ3M01VcFFQcFNVVVJ0ZXlPeXU4SUtQaWxocmVyU2ZlakVET2ZmUVZQ?= =?utf-8?B?Qzh2enJMM0FCaXk5ODhjbUlrYXB2T2t3MjgyNFNpem91UXNLc3FHL3FoNVBy?= =?utf-8?B?RXArWEsxRi8vVjR5MWxZSkcxaExFVEJ4RVI1S0I5VWk2dHU1Z1hycW93VEhL?= =?utf-8?B?RHlKMWZVbnE5V0R1TE92cUZsM00rYk9oV0tiRXBYUnJ0UktqM09BSEN2UGdI?= =?utf-8?B?eTJHbjhvQWhOZU1UY0NuNS9VNUd0YUVKblhIRUJPZk9zZW1kdVNNVWdpSW9k?= =?utf-8?B?bmFTM2Jpd2J5Z1dxRjhqVk5UWmpaNXZUKzJSYUFMTnEyQ2Y3WWRmZ2dSQkJu?= =?utf-8?B?di9FSVd2QlZlWlN6dWozblJCNU0rWk5oZHBHN2xWZVYxQUZPaSt5akNZQ1g3?= =?utf-8?B?OEZuaE90d1AyQWpNeGoxZlo4UENLZWpveFg3YUhBVmt4S2pLRVBsRENieGFH?= =?utf-8?B?c2dhWjlGbC9XV0dnaEFjQWg0K3RmSnFURnhsVXdzbVZOL2Vab0QrOFBCWUtz?= =?utf-8?B?Wmk0NEhWVXVoYnZXb0F4MmRJRkgxOGFTbm5PYS9SM0NwWC90cWlabW1QNjlZ?= =?utf-8?B?ZlM3R3hEb3kycGxIb1Zqb0hRR3RMdzl3Wk5TdGFWTVREOFZyK2xhYzNXMjRU?= =?utf-8?B?YTRBcG9GNkVrYWRhckJBc29hOG5WWTA4UGJoejZHemhXRWl1THlMSFRzelBn?= =?utf-8?B?UE1WMm55MDBiSXdLWXk2NVpIeDY3WFduaWY2Y1d2eGhFSUFWT3NIZTdQUUU1?= =?utf-8?B?VXpCTDFzQTMvZWtPN1JkQ2dySDFYR3pNQnB0SGlvR0VrVjNCbWRqN3FpVldl?= =?utf-8?B?WEJJamF5NDlvQkhzN0dteVhwcGpwdHVXb3BUZGFGb2JRamRkRC9xRVM5aE0r?= =?utf-8?B?eHk3ZGZFcE1YenJDc29GaEx6SUtoSUcwUU5iVlZNZWZvQnpramtWaWo3a3ly?= =?utf-8?B?MG95MmRUUnNJRlp2QjZqNEdJSksrTXA0Z0l4clNtVFJXbXV3QjRKUmozLzc0?= =?utf-8?B?QWtQZmRSMU9TZzFOYWd6V29hY3hHenlmWEkyUzFzbGNXUVhORUNqamtvNFBn?= =?utf-8?B?S1cwaFdLUjVHNGp0LzFlaWRmZ3F3ZnNwWlhnUlh1TExraUsxMkVTSnlFME1L?= =?utf-8?B?QkJENGdlc3lrN2lmRm5xcHBIL3Y3WlN6L0s4SXNwaSt3Sm5MUWwyY0ZiWDV5?= =?utf-8?B?UVIvL1RkcTYwUUJGb1lyTDBGdmhVSFArekJyOXlBM0xKUFBhUGZQUVQzS0E0?= =?utf-8?B?eHdlVkUyVjRzS2x4RklleTZQc2VndFlnK3dxNFBqejJpVGQwMEFIakhnMmlI?= =?utf-8?B?VmxiK2U0OVBuTjBkY05wU1BVREhXSTZjSHpCVW1Jd2NaSTBrUXE4YmlCWEE5?= =?utf-8?B?SEhJalhqYjJpOEdWWldBODZWc2lyMzZ5ekZ6NlV3dWZ6Z0RKUDR1eHM3YWhi?= =?utf-8?B?aDlXQmNVUlVDOFF1aXpmOEJnaldRTitHUnpTMUZjeE43ZFRZQm9EMGV0eXpX?= =?utf-8?B?L2tZaHpwcXNLU05IRGV6cGJCajRuYm1oTHpXeHBkZG5XeS9ZVUF5aWRHTlNS?= =?utf-8?B?Z1lwcCtpS0wyVFMxYlBTYktiSFNkT0t6U2RZZVpZZ0h1SitZS0hTeVJjNFUw?= =?utf-8?B?Rk9EV1EreDFNQzkzWjAwb2VtSEhuZmdTRlQrQWVzeFIvT0tSWlhycnYyb3dF?= =?utf-8?B?VUZGa3BBck5yRGFFUkJHc1JGa3hCMjZrcHI0K0RUTndOVHYxTFA4TGlId2Jk?= =?utf-8?B?NGc9PQ==?= X-MS-Exchange-CrossTenant-Network-Message-Id: c89e956b-ae14-4d19-e7ea-08da91747dfb X-MS-Exchange-CrossTenant-AuthSource: MWHPR11MB1629.namprd11.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 08 Sep 2022 08:31:14.3647 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 46c98d88-e344-4ed4-8496-4ed7712e255d X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: jGP8e/tU2drpa5xnzp4EqNlizQYtHmtouFFZVWYgeVEEFchGq0NPbxlYVNmyaFbGtsS94IiIO/LspOzWrE/FioFOmc0p1C6rNpJPrcCFR/E= X-MS-Exchange-Transport-CrossTenantHeadersStamped: CY4PR11MB1895 X-OriginatorOrg: intel.com X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org On Fri, Jul 29, 2022 at 10:55:44AM +0000, Juraj Linkeš wrote: > The logging module provides loggers distinguished by two attributes, > a custom format and a verbosity switch. The loggers log to both console > and more verbosely to files. > > Signed-off-by: Owen Hilyard > Signed-off-by: Juraj Linkeš Few small comments inline below. Thanks, /Bruce > --- > dts/framework/__init__.py | 3 + > dts/framework/logger.py | 124 ++++++++++++++++++++++++++++++++++++++ > 2 files changed, 127 insertions(+) > create mode 100644 dts/framework/__init__.py > create mode 100644 dts/framework/logger.py > > diff --git a/dts/framework/__init__.py b/dts/framework/__init__.py > new file mode 100644 > index 0000000000..3c30bccf43 > --- /dev/null > +++ b/dts/framework/__init__.py > @@ -0,0 +1,3 @@ > +# SPDX-License-Identifier: BSD-3-Clause > +# Copyright(c) 2022 PANTHEON.tech s.r.o. > +# > diff --git a/dts/framework/logger.py b/dts/framework/logger.py > new file mode 100644 > index 0000000000..920ce0fb15 > --- /dev/null > +++ b/dts/framework/logger.py > @@ -0,0 +1,124 @@ > +# SPDX-License-Identifier: BSD-3-Clause > +# Copyright(c) 2010-2014 Intel Corporation > +# Copyright(c) 2022 PANTHEON.tech s.r.o. > +# Copyright(c) 2022 University of New Hampshire > +# > + > +import logging > +import os.path > +from typing import TypedDict > + > +""" > +DTS logger module with several log level. DTS framework and TestSuite log > +will saved into different log files. > +""" > +verbose = False > +date_fmt = "%d/%m/%Y %H:%M:%S" Please use Year-month-day ordering for dates, since it's unambiguous - as well as being an ISO standard date format. (ISO 8601) > +stream_fmt = "%(asctime)s - %(name)s - %(levelname)s - %(message)s" > + > + > +class LoggerDictType(TypedDict): > + logger: "DTSLOG" > + name: str > + node: str > + > + > +# List for saving all using loggers > +global Loggers > +Loggers: list[LoggerDictType] = [] > + > + > +def set_verbose() -> None: > + global verbose > + verbose = True > + Is there a need for a clear_verbose() or "set_not_verbose()" API? > + > +class DTSLOG(logging.LoggerAdapter): > + """ > + DTS log class for framework and testsuite. > + """ > + > + node: str > + logger: logging.Logger > + sh: logging.StreamHandler > + fh: logging.FileHandler > + verbose_handler: logging.FileHandler > + > + def __init__(self, logger: logging.Logger, node: str = "suite"): > + global log_dir > + > + self.logger = logger > + self.logger.setLevel(1) # 1 means log everything > + > + self.node = node > + > + # add handler to emit to stdout > + sh = logging.StreamHandler() > + sh.setFormatter(logging.Formatter(stream_fmt, date_fmt)) > + > + sh.setLevel(logging.DEBUG) # file handler default level > + global verbose > + if verbose is True: > + sh.setLevel(logging.DEBUG) > + else: > + sh.setLevel(logging.INFO) # console handler defaultlevel The global should be defined at the top of the function. Looks like some of the setlevel calls are unnecessary; two should be enough rather than three. For example: sh.setLevel(logging.INFO) if verbose: sh.setLevel(logging.DEGUG) > + > + self.logger.addHandler(sh) > + self.sh = sh > + > + if not os.path.exists("output"): > + os.mkdir("output") > + > + fh = logging.FileHandler(f"output/{node}.log") > + fh.setFormatter( > + logging.Formatter( > + fmt="%(asctime)s - %(name)s - %(levelname)s - %(message)s", > + datefmt=date_fmt, > + ) > + ) > + > + fh.setLevel(1) # We want all the logs we can get in the file > + self.logger.addHandler(fh) > + self.fh = fh > + > + # This outputs EVERYTHING, intended for post-mortem debugging > + # Also optimized for processing via AWK (awk -F '|' ...) > + verbose_handler = logging.FileHandler(f"output/{node}.verbose.log") > + verbose_handler.setFormatter( > + logging.Formatter( > + fmt="%(asctime)s|%(name)s|%(levelname)s|%(pathname)s|%(lineno)d|%(funcName)s|" > + "%(process)d|%(thread)d|%(threadName)s|%(message)s", > + datefmt=date_fmt, > + ) > + ) > + > + verbose_handler.setLevel(1) # We want all the logs we can get in the file > + self.logger.addHandler(verbose_handler) > + self.verbose_handler = verbose_handler > + > + super(DTSLOG, self).__init__(self.logger, dict(node=self.node)) > + > + def logger_exit(self) -> None: > + """ > + Remove stream handler and logfile handler. > + """ > + for handler in (self.sh, self.fh, self.verbose_handler): > + handler.flush() > + self.logger.removeHandler(handler) > + > + > +def getLogger(name: str, node: str = "suite") -> DTSLOG: > + """ > + Get logger handler and if there's no handler for specified Node will create one. > + """ > + global Loggers > + # return saved logger > + logger: LoggerDictType > + for logger in Loggers: > + if logger["name"] == name and logger["node"] == node: > + return logger["logger"] > + > + # return new logger > + dts_logger: DTSLOG = DTSLOG(logging.getLogger(name), node) > + Loggers.append({"logger": dts_logger, "name": name, "node": node}) > + return dts_logger Looking through this patch alone, I see the "verbose" global only seems to be used to set the log-level in the logger init function. If this is the only use of it across all the patches in the set, it may be more readable to change the variable from a "verbose" flag, to instead being a log-level one. That way your global define is: log_level = logging.INFO and set_verbose() becomes: global log_level log_level = logging.DEBUG thereby removing the branch from you logging init fn. NOTE: I have not yet had the chance to review the rest of the series, so if verbose is used elsewhere, please ignore this comment. > -- > 2.30.2 >