From: Aaron Conole <aconole@redhat.com>
To: Ray Kinsella <mdr@ashroe.eu>
Cc: dev@dpdk.org, bruce.richardson@intel.com,
stephen@networkplumber.org, ferruh.yigit@intel.com,
thomas@monjalon.net, ktraynor@redhat.com
Subject: Re: [dpdk-dev] [PATCH v10 2/3] devtools: script to send notifications of expired symbols
Date: Wed, 01 Sep 2021 08:46:16 -0400 [thread overview]
Message-ID: <f7tzgsw79bb.fsf@redhat.com> (raw)
In-Reply-To: <20210831145017.856776-3-mdr@ashroe.eu> (Ray Kinsella's message of "Tue, 31 Aug 2021 15:50:16 +0100")
Ray Kinsella <mdr@ashroe.eu> writes:
> Use this script with the output of the DPDK symbol tool, to notify
> maintainers of expired symbols by email. You need to define the environment
> variable DPDK_GETMAINTAINER_PATH for this tool to work.
>
> Use terminal output to review the emails before sending.
> e.g.
> $ devtools/symbol-tool.py list-expired --format-output csv \
> | DPDK_GETMAINTAINER_PATH=<somewhere>/get_maintainer.pl \
> devtools/notify_expired_symbols.py --format-output terminal
>
> Then use email output to send the emails to the maintainers.
> e.g.
> $ devtools/symbol-tool.py list-expired --format-output csv \
> | DPDK_GETMAINTAINER_PATH=<somewhere>/get_maintainer.pl \
> devtools/notify_expired_symbols.py --format-output email \
> --smtp-server <server> --sender <someone@somewhere.com> \
> --password <password> --cc <someone@somewhere.com>
>
> Signed-off-by: Ray Kinsella <mdr@ashroe.eu>
> ---
> devtools/notify-symbol-maintainers.py | 256 ++++++++++++++++++++++++++
> 1 file changed, 256 insertions(+)
> create mode 100755 devtools/notify-symbol-maintainers.py
>
> diff --git a/devtools/notify-symbol-maintainers.py b/devtools/notify-symbol-maintainers.py
> new file mode 100755
> index 0000000000..ee554687ff
> --- /dev/null
> +++ b/devtools/notify-symbol-maintainers.py
> @@ -0,0 +1,256 @@
> +#!/usr/bin/env python3
> +# SPDX-License-Identifier: BSD-3-Clause
> +# Copyright(c) 2021 Intel Corporation
> +# pylint: disable=invalid-name
> +'''Tool to notify maintainers of expired symbols'''
> +import smtplib
> +import ssl
> +import sys
> +import subprocess
> +import argparse
> +from argparse import RawTextHelpFormatter
> +import time
> +from email.message import EmailMessage
> +
> +DESCRIPTION = '''
> +Use this script with the output of the DPDK symbol tool, to notify maintainers
> +and contributors of expired symbols by email. You need to define the environment
> +variable DPDK_GETMAINTAINER_PATH for this tool to work.
> +
> +Use terminal output to review the emails before sending.
> +e.g.
> +$ devtools/symbol-tool.py list-expired --format-output csv \\
> +| DPDK_GETMAINTAINER_PATH=<somewhere>/get_maintainer.pl \\
> +{s} --format-output terminal
> +
> +Then use email output to send the emails to the maintainers.
> +e.g.
> +$ devtools/symbol-tool.py list-expired --format-output csv \\
> +| DPDK_GETMAINTAINER_PATH=<somewhere>/get_maintainer.pl \\
> +{s} --format-output email \\
> +--smtp-server <server> --sender <someone@somewhere.com> --password <password> \\
> +--cc <someone@somewhere.com>
> +'''
> +
> +EMAIL_TEMPLATE = '''Hi there,
> +
> +Please note the symbols listed below have expired. In line with the DPDK ABI
> +policy, they should be scheduled for removal, in the next DPDK release.
> +
> +For more information, please see the DPDK ABI Policy, section 3.5.3.
> +https://doc.dpdk.org/guides/contributing/abi_policy.html
> +
> +Thanks,
> +
> +The DPDK Symbol Bot
> +
> +'''
> +
> +ABI_POLICY = 'doc/guides/contributing/abi_policy.rst'
> +MAINTAINERS = 'MAINTAINERS'
> +get_maintainer = ['devtools/get-maintainer.sh', \
> + '--email', '-f']
Maybe it's best to make this something that can be overridden. There's
a series to change the .sh files to .py files. Perhaps an environment
variable or argument?
> +def _get_maintainers(libpath):
> + '''Get the maintainers for given library'''
> + try:
> + cmd = get_maintainer + [libpath]
> + result = subprocess.run(cmd, \
> + stdout=subprocess.PIPE, \
> + stderr=subprocess.PIPE,
> + check=True)
> + except subprocess.CalledProcessError:
> + return None
You might consider handling
except FileNotFoundError:
....
With a graceful exit and error message. In case the get_maintainers
path changes.
> + if result is None:
> + return None
> +
> + email = result.stdout.decode('utf-8')
> + if email == '':
> + return None
> +
> + email = list(filter(None,email.split('\n')))
> + return email
> +
> +default_maintainers = _get_maintainers(ABI_POLICY) + \
> + _get_maintainers(MAINTAINERS)
> +
> +def get_maintainers(libpath):
> + '''Get the maintainers for given library'''
> + maintainers=_get_maintainers(libpath)
> +
> + if maintainers is None:
> + maintainers = default_maintainers
> +
> + return maintainers
> +
> +def get_message(library, symbols, config):
> + '''Build email message from symbols, config and maintainers'''
> + contributors = {}
> + message = {}
> + maintainers = get_maintainers(library)
> +
> + if maintainers != default_maintainers:
> + message['CC'] = default_maintainers.copy()
> +
> + if 'CC' in config:
> + message.setdefault('CC',[]).append(config['CC'])
> +
> + message['Subject'] = 'Expired symbols in {}\n'.format(library)
> +
> + body = EMAIL_TEMPLATE
> + body += '{:<50}{:<25}{:<25}\n'.format('Symbol','Contributor','Email')
> + for sym in symbols:
> + body += ('{:<50}{:<25}{:<25}\n'.format(sym,\
> + symbols[sym]['name'],
> + symbols[sym]['email'],
> + ))
> + email = symbols[sym]['email']
> + contributors[email] = ''
> +
> + contributors = list(contributors.keys())
> +
> + message['To'] = maintainers + contributors
> + message['Body'] = body
> +
> + return message
> +
> +class OutputEmail():
> + '''Format the output for email'''
> + def __init__(self, config):
> + self.config = config
> +
> + self.terminal = OutputTerminal(config)
> + context = ssl.create_default_context()
> +
> + # Try to log in to server and send email
> + try:
> + self.server = smtplib.SMTP(config['smtp_server'], 587)
> + self.server.starttls(context=context) # Secure the connection
> + self.server.login(config['sender'], config['password'])
> + except Exception as exception:
> + print(exception)
> + raise exception
> +
> + def message(self,message):
> + '''send email'''
> + self.terminal.message(message)
> +
> + msg = EmailMessage()
> + msg.set_content(message.pop('Body'))
> +
> + for key in message.keys():
> + msg[key] = message[key]
> +
> + msg['From'] = self.config['sender']
> + msg['Reply-To'] = 'no-reply@dpdk.org'
> +
> + self.server.send_message(msg)
> +
> + time.sleep(1)
Why this sleep is needed?
> +
> + def __del__(self):
> + self.server.quit()
> +
> +class OutputTerminal(): # pylint: disable=too-few-public-methods
> + '''Format the output for the terminal'''
> + def __init__(self, config):
> + self.config = config
> +
> + def message(self,message):
> + '''Print email to terminal'''
> +
> + terminal = 'To:' + ', '.join(message['To']) + '\n'
> + if 'sender' in self.config.keys():
> + terminal += 'From:' + self.config['sender'] + '\n'
> +
> + terminal += 'Reply-To:' + 'no-reply@dpdk.org' + '\n'
> +
> + if 'CC' in message:
> + terminal += 'CC:' + ', '.join(message['CC']) + '\n'
> +
> + terminal += 'Subject:' + message['Subject'] + '\n'
> + terminal += 'Body:' + message['Body'] + '\n'
> +
> + print(terminal)
> + print('-' * 80)
> +
> +def parse_config(args):
> + '''put the command line args in the right places'''
> + config = {}
> + error_msg = None
> +
> + outputs = {
> + None : OutputTerminal,
> + 'terminal' : OutputTerminal,
> + 'email' : OutputEmail
> + }
> +
> + if args.format_output == 'email':
> + if args.smtp_server is None:
> + error_msg = 'SMTP server'
> + else:
> + config['smtp_server'] = args.smtp_server
> +
> + if args.sender is None:
> + error_msg = 'sender'
> + else:
> + config['sender'] = args.sender
> +
> + if args.password is None:
> + error_msg = 'password'
> + else:
> + config['password'] = args.password
> +
> + if args.cc is not None:
> + config['CC'] = args.cc
> +
> + if error_msg is not None:
> + print('Please specify a {} for email output'.format(error_msg))
> + return None
> +
> + config['output'] = outputs[args.format_output]
> + return config
> +
> +def main():
> + '''Main entry point'''
> + parser = argparse.ArgumentParser(description=DESCRIPTION.format(s=__file__), \
> + formatter_class=RawTextHelpFormatter)
> + parser.add_argument('--format-output', choices=['terminal','email'], \
> + default='terminal')
> + parser.add_argument('--smtp-server')
> + parser.add_argument('--password')
> + parser.add_argument('--sender')
> + parser.add_argument('--cc')
> +
> + args = parser.parse_args()
> + config = parse_config(args)
> + if config is None:
> + return
> +
> + symbols = {}
> + lastlib = library = ''
> +
> + output = config['output'](config)
> +
> + for line in sys.stdin:
> + line = line.rstrip('\n')
> +
> + if line.find('mapfile') >= 0:
> + continue
> + library, symbol, name, email = line.split(',')
> +
> + if library != lastlib:
> + message = get_message(lastlib, symbols, config)
> + output.message(message)
> + symbols = {}
> +
> + lastlib = library
> + symbols[symbol] = {'name' : name, 'email' : email}
> +
> + #print the last library
> + message = get_message(lastlib, symbols, config)
> + output.message(message)
> +
> +if __name__ == '__main__':
> + main()
next prev parent reply other threads:[~2021-09-01 12:46 UTC|newest]
Thread overview: 50+ messages / expand[flat|nested] mbox.gz Atom feed top
2021-06-18 16:36 [dpdk-dev] [PATCH] devtools: script to track map symbols Ray Kinsella
2021-06-18 19:40 ` Stephen Hemminger
2021-06-21 9:18 ` Kinsella, Ray
2021-06-21 15:11 ` Ray Kinsella
2021-06-21 15:25 ` [dpdk-dev] [PATCH v3] " Ray Kinsella
2021-06-21 15:35 ` [dpdk-dev] [PATCH v4] " Ray Kinsella
2021-06-21 18:14 ` Stephen Hemminger
2021-06-22 10:19 ` [dpdk-dev] [PATCH v5] " Ray Kinsella
2021-08-04 16:23 ` [dpdk-dev] [PATCH v6] " Ray Kinsella
2021-08-04 16:27 ` [dpdk-dev] [PATCH v7] " Ray Kinsella
2021-08-06 17:54 ` [dpdk-dev] [PATCH v8 0/2] devtools: scripts to count and track symbols Ray Kinsella
2021-08-06 17:54 ` [dpdk-dev] [PATCH v8 1/2] devtools: script to track map symbols Ray Kinsella
2021-08-06 17:54 ` [dpdk-dev] [PATCH v8 2/2] devtools: script to send notifications of expired symbols Ray Kinsella
2021-08-09 12:53 ` [dpdk-dev] [PATCH v9 0/2] devtools: scripts to count and track symbols Ray Kinsella
2021-08-09 12:53 ` [dpdk-dev] [PATCH v9 1/2] devtools: script to track symbols over releases Ray Kinsella
2021-08-09 12:53 ` [dpdk-dev] [PATCH v9 2/2] devtools: script to send notifications of expired symbols Ray Kinsella
2021-08-31 14:50 ` [dpdk-dev] [PATCH v10 0/3] devtools: scripts to count and track symbols Ray Kinsella
2021-08-31 14:50 ` [dpdk-dev] [PATCH v10 1/3] devtools: script to track symbols over releases Ray Kinsella
2021-08-31 14:50 ` [dpdk-dev] [PATCH v10 2/3] devtools: script to send notifications of expired symbols Ray Kinsella
2021-09-01 12:46 ` Aaron Conole [this message]
2021-09-03 11:15 ` Kinsella, Ray
2021-09-03 13:32 ` Kinsella, Ray
2021-09-01 13:01 ` David Marchand
2021-09-03 13:28 ` Kinsella, Ray
2021-09-03 13:34 ` Kinsella, Ray
2021-08-31 14:50 ` [dpdk-dev] [PATCH v10 3/3] maintainers: add new abi scripts Ray Kinsella
2021-09-01 12:31 ` [dpdk-dev] [PATCH v10 0/3] devtools: scripts to count and track symbols Aaron Conole
2021-09-01 17:17 ` Stephen Hemminger
2021-09-01 19:04 ` Aaron Conole
2021-09-03 11:17 ` Kinsella, Ray
2021-09-03 13:23 ` [dpdk-dev] [PATCH v11 " Ray Kinsella
2021-09-03 13:23 ` [dpdk-dev] [PATCH v11 1/3] devtools: script to track symbols over releases Ray Kinsella
2021-09-03 13:23 ` [dpdk-dev] [PATCH v11 2/3] devtools: script to send notifications of expired symbols Ray Kinsella
2021-09-03 13:23 ` [dpdk-dev] [PATCH v11 3/3] maintainers: add new abi scripts Ray Kinsella
2021-09-08 15:12 ` [dpdk-dev] [PATCH v11 0/3] devtools: scripts to count and track symbols Ray Kinsella
2021-09-08 15:12 ` [dpdk-dev] [PATCH v11 1/3] devtools: script to track symbols over releases Ray Kinsella
2021-09-08 15:12 ` [dpdk-dev] [PATCH v11 2/3] devtools: script to send notifications of expired symbols Ray Kinsella
2021-09-08 15:12 ` [dpdk-dev] [PATCH v11 3/3] maintainers: add new abi scripts Ray Kinsella
2021-09-08 15:13 ` [dpdk-dev] [PATCH v12 0/4] devtools: scripts to count and track symbols Ray Kinsella
2021-09-08 15:13 ` [dpdk-dev] [PATCH v12 1/4] devtools: script to track symbols over releases Ray Kinsella
2021-09-08 15:13 ` [dpdk-dev] [PATCH v12 2/4] devtools: script to send notifications of expired symbols Ray Kinsella
2021-09-08 15:13 ` [dpdk-dev] [PATCH v12 3/4] maintainers: add new abi scripts Ray Kinsella
2021-09-08 15:13 ` [dpdk-dev] [PATCH v12 4/4] devtools: add asym crypto to symbol-tool ignore Ray Kinsella
2021-09-08 15:23 ` [dpdk-dev] [EXT] " Akhil Goyal
2021-09-09 13:48 ` [dpdk-dev] [PATCH v13 0/4] devtools: scripts to count and track symbols Ray Kinsella
2021-09-09 13:48 ` [dpdk-dev] [PATCH v13 1/4] devtools: script to track symbols over releases Ray Kinsella
2021-09-09 13:48 ` [dpdk-dev] [PATCH v13 2/4] devtools: script to send notifications of expired symbols Ray Kinsella
2021-09-09 13:48 ` [dpdk-dev] [PATCH v13 3/4] maintainers: add new abi scripts Ray Kinsella
2021-09-09 13:48 ` [dpdk-dev] [PATCH v13 4/4] devtools: add asym crypto to symbol-tool ignore Ray Kinsella
2023-07-06 19:13 ` [dpdk-dev] [PATCH v13 0/4] devtools: scripts to count and track symbols Stephen Hemminger
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=f7tzgsw79bb.fsf@redhat.com \
--to=aconole@redhat.com \
--cc=bruce.richardson@intel.com \
--cc=dev@dpdk.org \
--cc=ferruh.yigit@intel.com \
--cc=ktraynor@redhat.com \
--cc=mdr@ashroe.eu \
--cc=stephen@networkplumber.org \
--cc=thomas@monjalon.net \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).