From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 5E9A2A0C43; Wed, 1 Sep 2021 14:46:27 +0200 (CEST) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id D056340140; Wed, 1 Sep 2021 14:46:26 +0200 (CEST) Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [216.205.24.124]) by mails.dpdk.org (Postfix) with ESMTP id 041834013F for ; Wed, 1 Sep 2021 14:46:24 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1630500384; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=if1eaZRR06DafFOr5ML7p0ATfV84fU0oRfuq8HPIGwo=; b=W0j+arYTi7KprOORhFA814uCi8wDEXeR8FzHoYIcr+YIQWzIbHx4l8DPjpZy2Etf0p6XLk J+AAVitltQqruKqQW8SCQIGtXa+3Rce1L5+uc2EC1PNFwC2hMbbxk6i/gzbcapvkun06Z4 8xYZad/qIrOwZ7mHIjVJ1zNtILrnsQE= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-155-avSp1KCrOie5qmaGnm8Vkw-1; Wed, 01 Sep 2021 08:46:19 -0400 X-MC-Unique: avSp1KCrOie5qmaGnm8Vkw-1 Received: from smtp.corp.redhat.com (int-mx07.intmail.prod.int.phx2.redhat.com [10.5.11.22]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 4798092B; Wed, 1 Sep 2021 12:46:18 +0000 (UTC) Received: from RHTPC1VM0NT (unknown [10.22.17.187]) by smtp.corp.redhat.com (Postfix) with ESMTPS id 1E3A910114AE; Wed, 1 Sep 2021 12:46:17 +0000 (UTC) From: Aaron Conole To: Ray Kinsella Cc: dev@dpdk.org, bruce.richardson@intel.com, stephen@networkplumber.org, ferruh.yigit@intel.com, thomas@monjalon.net, ktraynor@redhat.com References: <20210618163659.85933-1-mdr@ashroe.eu> <20210831145017.856776-1-mdr@ashroe.eu> <20210831145017.856776-3-mdr@ashroe.eu> Date: Wed, 01 Sep 2021 08:46:16 -0400 In-Reply-To: <20210831145017.856776-3-mdr@ashroe.eu> (Ray Kinsella's message of "Tue, 31 Aug 2021 15:50:16 +0100") Message-ID: User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/27.2 (gnu/linux) MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.84 on 10.5.11.22 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=aconole@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Type: text/plain Subject: Re: [dpdk-dev] [PATCH v10 2/3] devtools: script to send notifications of expired symbols X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" Ray Kinsella writes: > Use this script with the output of the DPDK symbol tool, to notify > maintainers of expired symbols by email. You need to define the environment > variable DPDK_GETMAINTAINER_PATH for this tool to work. > > Use terminal output to review the emails before sending. > e.g. > $ devtools/symbol-tool.py list-expired --format-output csv \ > | DPDK_GETMAINTAINER_PATH=/get_maintainer.pl \ > devtools/notify_expired_symbols.py --format-output terminal > > Then use email output to send the emails to the maintainers. > e.g. > $ devtools/symbol-tool.py list-expired --format-output csv \ > | DPDK_GETMAINTAINER_PATH=/get_maintainer.pl \ > devtools/notify_expired_symbols.py --format-output email \ > --smtp-server --sender \ > --password --cc > > Signed-off-by: Ray Kinsella > --- > devtools/notify-symbol-maintainers.py | 256 ++++++++++++++++++++++++++ > 1 file changed, 256 insertions(+) > create mode 100755 devtools/notify-symbol-maintainers.py > > diff --git a/devtools/notify-symbol-maintainers.py b/devtools/notify-symbol-maintainers.py > new file mode 100755 > index 0000000000..ee554687ff > --- /dev/null > +++ b/devtools/notify-symbol-maintainers.py > @@ -0,0 +1,256 @@ > +#!/usr/bin/env python3 > +# SPDX-License-Identifier: BSD-3-Clause > +# Copyright(c) 2021 Intel Corporation > +# pylint: disable=invalid-name > +'''Tool to notify maintainers of expired symbols''' > +import smtplib > +import ssl > +import sys > +import subprocess > +import argparse > +from argparse import RawTextHelpFormatter > +import time > +from email.message import EmailMessage > + > +DESCRIPTION = ''' > +Use this script with the output of the DPDK symbol tool, to notify maintainers > +and contributors of expired symbols by email. You need to define the environment > +variable DPDK_GETMAINTAINER_PATH for this tool to work. > + > +Use terminal output to review the emails before sending. > +e.g. > +$ devtools/symbol-tool.py list-expired --format-output csv \\ > +| DPDK_GETMAINTAINER_PATH=/get_maintainer.pl \\ > +{s} --format-output terminal > + > +Then use email output to send the emails to the maintainers. > +e.g. > +$ devtools/symbol-tool.py list-expired --format-output csv \\ > +| DPDK_GETMAINTAINER_PATH=/get_maintainer.pl \\ > +{s} --format-output email \\ > +--smtp-server --sender --password \\ > +--cc > +''' > + > +EMAIL_TEMPLATE = '''Hi there, > + > +Please note the symbols listed below have expired. In line with the DPDK ABI > +policy, they should be scheduled for removal, in the next DPDK release. > + > +For more information, please see the DPDK ABI Policy, section 3.5.3. > +https://doc.dpdk.org/guides/contributing/abi_policy.html > + > +Thanks, > + > +The DPDK Symbol Bot > + > +''' > + > +ABI_POLICY = 'doc/guides/contributing/abi_policy.rst' > +MAINTAINERS = 'MAINTAINERS' > +get_maintainer = ['devtools/get-maintainer.sh', \ > + '--email', '-f'] Maybe it's best to make this something that can be overridden. There's a series to change the .sh files to .py files. Perhaps an environment variable or argument? > +def _get_maintainers(libpath): > + '''Get the maintainers for given library''' > + try: > + cmd = get_maintainer + [libpath] > + result = subprocess.run(cmd, \ > + stdout=subprocess.PIPE, \ > + stderr=subprocess.PIPE, > + check=True) > + except subprocess.CalledProcessError: > + return None You might consider handling except FileNotFoundError: .... With a graceful exit and error message. In case the get_maintainers path changes. > + if result is None: > + return None > + > + email = result.stdout.decode('utf-8') > + if email == '': > + return None > + > + email = list(filter(None,email.split('\n'))) > + return email > + > +default_maintainers = _get_maintainers(ABI_POLICY) + \ > + _get_maintainers(MAINTAINERS) > + > +def get_maintainers(libpath): > + '''Get the maintainers for given library''' > + maintainers=_get_maintainers(libpath) > + > + if maintainers is None: > + maintainers = default_maintainers > + > + return maintainers > + > +def get_message(library, symbols, config): > + '''Build email message from symbols, config and maintainers''' > + contributors = {} > + message = {} > + maintainers = get_maintainers(library) > + > + if maintainers != default_maintainers: > + message['CC'] = default_maintainers.copy() > + > + if 'CC' in config: > + message.setdefault('CC',[]).append(config['CC']) > + > + message['Subject'] = 'Expired symbols in {}\n'.format(library) > + > + body = EMAIL_TEMPLATE > + body += '{:<50}{:<25}{:<25}\n'.format('Symbol','Contributor','Email') > + for sym in symbols: > + body += ('{:<50}{:<25}{:<25}\n'.format(sym,\ > + symbols[sym]['name'], > + symbols[sym]['email'], > + )) > + email = symbols[sym]['email'] > + contributors[email] = '' > + > + contributors = list(contributors.keys()) > + > + message['To'] = maintainers + contributors > + message['Body'] = body > + > + return message > + > +class OutputEmail(): > + '''Format the output for email''' > + def __init__(self, config): > + self.config = config > + > + self.terminal = OutputTerminal(config) > + context = ssl.create_default_context() > + > + # Try to log in to server and send email > + try: > + self.server = smtplib.SMTP(config['smtp_server'], 587) > + self.server.starttls(context=context) # Secure the connection > + self.server.login(config['sender'], config['password']) > + except Exception as exception: > + print(exception) > + raise exception > + > + def message(self,message): > + '''send email''' > + self.terminal.message(message) > + > + msg = EmailMessage() > + msg.set_content(message.pop('Body')) > + > + for key in message.keys(): > + msg[key] = message[key] > + > + msg['From'] = self.config['sender'] > + msg['Reply-To'] = 'no-reply@dpdk.org' > + > + self.server.send_message(msg) > + > + time.sleep(1) Why this sleep is needed? > + > + def __del__(self): > + self.server.quit() > + > +class OutputTerminal(): # pylint: disable=too-few-public-methods > + '''Format the output for the terminal''' > + def __init__(self, config): > + self.config = config > + > + def message(self,message): > + '''Print email to terminal''' > + > + terminal = 'To:' + ', '.join(message['To']) + '\n' > + if 'sender' in self.config.keys(): > + terminal += 'From:' + self.config['sender'] + '\n' > + > + terminal += 'Reply-To:' + 'no-reply@dpdk.org' + '\n' > + > + if 'CC' in message: > + terminal += 'CC:' + ', '.join(message['CC']) + '\n' > + > + terminal += 'Subject:' + message['Subject'] + '\n' > + terminal += 'Body:' + message['Body'] + '\n' > + > + print(terminal) > + print('-' * 80) > + > +def parse_config(args): > + '''put the command line args in the right places''' > + config = {} > + error_msg = None > + > + outputs = { > + None : OutputTerminal, > + 'terminal' : OutputTerminal, > + 'email' : OutputEmail > + } > + > + if args.format_output == 'email': > + if args.smtp_server is None: > + error_msg = 'SMTP server' > + else: > + config['smtp_server'] = args.smtp_server > + > + if args.sender is None: > + error_msg = 'sender' > + else: > + config['sender'] = args.sender > + > + if args.password is None: > + error_msg = 'password' > + else: > + config['password'] = args.password > + > + if args.cc is not None: > + config['CC'] = args.cc > + > + if error_msg is not None: > + print('Please specify a {} for email output'.format(error_msg)) > + return None > + > + config['output'] = outputs[args.format_output] > + return config > + > +def main(): > + '''Main entry point''' > + parser = argparse.ArgumentParser(description=DESCRIPTION.format(s=__file__), \ > + formatter_class=RawTextHelpFormatter) > + parser.add_argument('--format-output', choices=['terminal','email'], \ > + default='terminal') > + parser.add_argument('--smtp-server') > + parser.add_argument('--password') > + parser.add_argument('--sender') > + parser.add_argument('--cc') > + > + args = parser.parse_args() > + config = parse_config(args) > + if config is None: > + return > + > + symbols = {} > + lastlib = library = '' > + > + output = config['output'](config) > + > + for line in sys.stdin: > + line = line.rstrip('\n') > + > + if line.find('mapfile') >= 0: > + continue > + library, symbol, name, email = line.split(',') > + > + if library != lastlib: > + message = get_message(lastlib, symbols, config) > + output.message(message) > + symbols = {} > + > + lastlib = library > + symbols[symbol] = {'name' : name, 'email' : email} > + > + #print the last library > + message = get_message(lastlib, symbols, config) > + output.message(message) > + > +if __name__ == '__main__': > + main()