From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id D0C19A0C4E; Thu, 12 Aug 2021 17:36:51 +0200 (CEST) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 9FCAB4014D; Thu, 12 Aug 2021 17:36:51 +0200 (CEST) Received: from mail-vs1-f98.google.com (mail-vs1-f98.google.com [209.85.217.98]) by mails.dpdk.org (Postfix) with ESMTP id 714FF40042 for ; Thu, 12 Aug 2021 17:36:50 +0200 (CEST) Received: by mail-vs1-f98.google.com with SMTP id b10so4133881vsp.2 for ; Thu, 12 Aug 2021 08:36:50 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=iol.unh.edu; s=unh-iol; h=from:to:cc:subject:date:message-id:mime-version :content-transfer-encoding; bh=2P/jIT83aMDGiQMQhV/n3EkqquXiXd52MXjkzzArq4k=; b=S15vpx5FkKEIsJikzZud/JZZ8t7iDGHz9Fb+D3YRrPKqF6vqXc7Qn8X9S8uW1rtVSl vcfbYc8w6REmzUb0+vgTkLwQTFoAVspHXuIXFq7Yk33dMnuOi1OkinU/4xdRYePQGtQy D1guIUcmFlOasiy+L1tG9/jwPv9K/bHBRUnMc= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:mime-version :content-transfer-encoding; bh=2P/jIT83aMDGiQMQhV/n3EkqquXiXd52MXjkzzArq4k=; b=kX/BXrS2CL3EZuoEg8HzdqFZrKGIJsGq6s/KAw/XjYNKh9WnW0rf+tnW7DmFNeDKww kxc85Lyfm+RgCIvaxQpnk+Iyl+ecufwO8sfWueqLDa0s7pZBALZe8JmrZ5t9JhKEkd+8 ltFu1hx5lSQTHKQQzPfHJC+lsw2IcTI5V9Q58N54AQyYVt5zKFFQAta+kBV6AX/pUFxr DIc8sK34OcqUqzO8OnAd7gfVqxT2S2zavcpKP/GxQQb4zz3vZSmpw/oJpUn4YT1wcTc7 Vgp0q44V4gr6qfXxA9FVn/olVYQoGnZm5Qo635z7rFabn30Zgd2zlMnA6GbZ/S4hEFwy Bk+Q== X-Gm-Message-State: AOAM530fQJKC6JUnFR1Fvo+uC4P+TRnTAJHf2RIhmmGwYJgR0XVtjn1Q A3S8B+RC1gm1jW7dR9KhTsReWPfhjE+ieKNLwx2Pk14ezSNcQ21r2w1nH1RQNgtzFp72ueX7F0/ hkEeQk+lpkenJbdhv6wNtp2p/4BE1qw7ap5w1KudOClqTdou2C5S0L8RG6rc7huJdAb+fBS8SUQ == X-Google-Smtp-Source: ABdhPJw2igQlgiIxfVtowU+ARMJlMzMcDBGPRgk6YKcb2L9vASQH+fTD8mc4fe6xZPby2KfVTqm40lNMfQr9 X-Received: by 2002:a67:2e43:: with SMTP id u64mr4035652vsu.30.1628782609696; Thu, 12 Aug 2021 08:36:49 -0700 (PDT) Received: from postal.iol.unh.edu (postal.iol.unh.edu. [132.177.123.84]) by smtp-relay.gmail.com with ESMTPS id y8sm668062vkd.2.2021.08.12.08.36.49 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Thu, 12 Aug 2021 08:36:49 -0700 (PDT) X-Relaying-Domain: iol.unh.edu Received: from iol.unh.edu (unknown [IPv6:2606:4100:3880:1257::105d]) by postal.iol.unh.edu (Postfix) with ESMTP id 12150605248E; Thu, 12 Aug 2021 11:36:49 -0400 (EDT) From: ohilyard@iol.unh.edu To: dts@dpdk.org Cc: lijuan.tu@intel.com, Owen Hilyard Date: Thu, 12 Aug 2021 11:36:47 -0400 Message-Id: <20210812153647.74262-1-ohilyard@iol.unh.edu> X-Mailer: git-send-email 2.30.2 MIME-Version: 1.0 Content-Transfer-Encoding: 8bit Subject: [dts] [PATCH v2] ci/initial: Added script to get the tests for a patchset X-BeenThere: dts@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: test suite reviews and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dts-bounces@dpdk.org Sender: "dts" From: Owen Hilyard This script should be run after the patchset has been applied. It will check all files that have a diff to the git ref in DTS_MAIN_BRANCH_REF (currently origin/master). It will also issue warnings to standard error if a "protected path" is changed. This is currently configured to only by the ci scripts folder, since under most circumstances a patch should not need to change anything in there. This warning will be in the format: "WARNING: {file_name} is protected" The script will also issue a warning if a config file is changed. This warning is also sent to standard error and takes the form of: "WARNING: {file_name} is a config file and was changed" The script will output a list of the test suites to run to standard out, with each entry having one line. Signed-off-by: Owen Hilyard --- ci/Dockerfile | 18 ++++ ci/README.txt | 37 +++++++ ci/build_image.sh | 0 ci/get_tests_for_patchset.py | 197 +++++++++++++++++++++++++++++++++++ ci/requirements.txt | 32 ++++++ 5 files changed, 284 insertions(+) create mode 100644 ci/Dockerfile create mode 100644 ci/README.txt create mode 100644 ci/build_image.sh create mode 100644 ci/get_tests_for_patchset.py create mode 100644 ci/requirements.txt diff --git a/ci/Dockerfile b/ci/Dockerfile new file mode 100644 index 00000000..89645b36 --- /dev/null +++ b/ci/Dockerfile @@ -0,0 +1,18 @@ +# This container should be build in the ci directory, and then the +# DTS directory should be mounted as a volume at /dts/ +FROM python:3.9-slim-buster + +ENV DEBIAN_FRONTEND=noninteractive + +COPY requirements.txt . +COPY dts_requirements.txt dts_requirements.txt + +RUN apt-get update && apt-get install --no-install-recommends -y \ + # Add a C compiler for all of the c modules in DTS + build-essential make gcc git libpcap-dev\ + python3-pip + +RUN pip3 install -r requirements.txt +RUN pip3 install -r dts_requirements.txt +# install formatter +RUN pip3 install black \ No newline at end of file diff --git a/ci/README.txt b/ci/README.txt new file mode 100644 index 00000000..281329f7 --- /dev/null +++ b/ci/README.txt @@ -0,0 +1,37 @@ +# BSD LICENSE +# +# Copyright(c) 2021 University of New Hampshire Interoperability Laboratory. All rights reserved. +# All rights reserved. +# +# Redistribution and use in source and binary forms, with or without +# modification, are permitted provided that the following conditions +# are met: +# +# * Redistributions of source code must retain the above copyright +# notice, this list of conditions and the following disclaimer. +# * Redistributions in binary form must reproduce the above copyright +# notice, this list of conditions and the following disclaimer in +# the documentation and/or other materials provided with the +# distribution. +# * Neither the name of Intel Corporation nor the names of its +# contributors may be used to endorse or promote products derived +# from this software without specific prior written permission. +# +# THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS +# "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT +# LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR +# A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT +# OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, +# SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT +# LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, +# DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY +# THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT +# (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE +# OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. + +This directory contains scripts used for ci in DTS. Nothing in this directory +should be modified during the course of submitting a patch to DPDK. + +Additional python requirements only needed for running DTS in ci are present in the +requirements.txt file in this directory. + diff --git a/ci/build_image.sh b/ci/build_image.sh new file mode 100644 index 00000000..e69de29b diff --git a/ci/get_tests_for_patchset.py b/ci/get_tests_for_patchset.py new file mode 100644 index 00000000..40e37bce --- /dev/null +++ b/ci/get_tests_for_patchset.py @@ -0,0 +1,197 @@ +# BSD LICENSE +# +# Copyright(c) 2021 University of New Hampshire Interoperability Laboratory. All rights reserved. +# All rights reserved. +# +# Redistribution and use in source and binary forms, with or without +# modification, are permitted provided that the following conditions +# are met: +# +# * Redistributions of source code must retain the above copyright +# notice, this list of conditions and the following disclaimer. +# * Redistributions in binary form must reproduce the above copyright +# notice, this list of conditions and the following disclaimer in +# the documentation and/or other materials provided with the +# distribution. +# * Neither the name of Intel Corporation nor the names of its +# contributors may be used to endorse or promote products derived +# from this software without specific prior written permission. +# +# THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS +# "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT +# LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR +# A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT +# OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, +# SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT +# LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, +# DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY +# THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT +# (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE +# OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. +import argparse +import os +import pkgutil +import re +import subprocess +import sys +from pkgutil import walk_packages +from types import ModuleType +from typing import List, Iterable + +DTS_MAIN_BRANCH_REF: str = "origin/master" + +DTS_TEST_MODULE_PATH: str = "tests" +# Will be unnecessary once DTS moves to a normal module structure +DTS_MODULE_PATHS: List[str] = ["framework", "nic", "dep", DTS_TEST_MODULE_PATH] + +# This is primarily intended for folders which contain CI scripts, as these +# should not be modified in the course of normal CI. A file in any of these +# folders being modified will cause a warning to be emitted to standard error +DTS_PROTECTED_PATHS: List[str] = [ + "ci", +] + +# This is intended to help detect when a config files have been changed. +# The intention behind this detection is to enable additional regression +# tests related to ensuring stable config file formats +DTS_CONFIG_PATHS: List[str] = [ + "conf", + "execution.cfg", +] + + +def get_args() -> str: + parser: argparse.ArgumentParser = argparse.ArgumentParser( + description="After a patchset is applied, run this script" + "It will then output a list " + "of applicable test suites to standard output, with 1 " + "test suite per line. All other output, such as warnings," + " errors or informational messages will be sent to " + "standard error. This script may produce no output at all," + "in which case it should be assumed that there are no " + "applicable test suites.\n\n " + "Exit Codes:\n" + "value | meaning\n" + "----- | -------\n" + " 0 | OK \n" + " 1 | Error, check standard error", + ) + + dts_directory: str = os.path.dirname(os.path.dirname(os.path.join(os.getcwd(), __file__))) + parser.add_argument("-d", "--dts-directory", type=str, default=dts_directory, required=False) + args = parser.parse_args() + if not os.path.isdir(args.dts_directory): + print(f"{args.dts_directory} is not a directory.", file=sys.stderr) + exit(1) + + return args.dts_directory + + +def get_modified_files() -> List[str]: + cmd = ['git', 'diff', '--name-only', DTS_MAIN_BRANCH_REF, 'HEAD'] + process: subprocess.CompletedProcess = subprocess.run(cmd, capture_output=True) + if process.returncode != 0: + print(f"{' '.join(cmd)} returned {process.returncode}") + exit(1) + + # This explicit conversion to utf8 will catch encoding errors + stdout: bytes = process.stdout + stdout_str: str = stdout.decode("utf-8") + + return stdout_str.splitlines() + + +def get_names_of_modified_python_files(files: List[str]) -> List[str]: + return list( + map( + lambda f: str(re.sub("\\.py", "", f)), + map( + lambda f: os.path.basename(f), + filter( + lambda f: f.endswith(".py"), files + ) + ) + ) + ) + + +def get_modules() -> List[ModuleType]: + return list(map(lambda m: pkgutil.resolve_name(m.name), walk_packages(DTS_MODULE_PATHS))) + + +def get_module_imports(mod: ModuleType) -> Iterable[ModuleType]: + imports = [] + for attribute_label in dir(mod): + try: + attribute = getattr(mod, attribute_label) + except ModuleNotFoundError as _: # some standard library modules don't like being directly imported + continue + if isinstance(attribute, type(mod)): + imports.append(attribute) + return imports + + +def get_modules_in_tree(mod: ModuleType) -> Iterable[ModuleType]: + yield from get_module_imports(mod) + + +def get_only_test_suites(modules: Iterable[ModuleType]) -> Iterable[ModuleType]: + test_package_path = os.path.join(os.getcwd(), DTS_TEST_MODULE_PATH) + mod: ModuleType + return filter(lambda mod: mod.__name__.startswith("TestSuite_"), + filter(lambda mod: mod.__file__.startswith(test_package_path), + filter(lambda mod: "__file__" in dir(mod), modules))) + + +def get_test_suite_names(modules: Iterable[ModuleType]) -> Iterable[str]: + # Moving this into a set is there to ensure that there are no duplicates + return set(list(map(lambda mod: re.sub("TestSuite_", "", mod.__name__), modules))) + + +def get_tests_to_run() -> List[str]: + dts_directory: str + dts_directory = get_args() + + # This all needs to be done at the top level, so I have to do it here. + + # chdir to the DTS directory, since we want that to be + # the context for any commands that are run. + os.chdir(dts_directory) + + for path in DTS_MODULE_PATHS: + sys.path.append(os.path.join(dts_directory, path)) + + files: List[str] = get_modified_files() + changed_module_name = get_names_of_modified_python_files(files) + + for protected_folder in DTS_PROTECTED_PATHS: + for file_name in files: + if file_name.startswith(protected_folder): + print(f"WARNING: {file_name} is protected", file=sys.stderr) + + for config_file_path in DTS_CONFIG_PATHS: + for file_name in files: + if file_name.startswith(config_file_path): + print(f"WARNING: {file_name} is a config file and was changed", file=sys.stderr) + + # Each index is 1 level of the tree + module_list: List[ModuleType] = [pkgutil.resolve_name(name) for name in changed_module_name] + current_index: int = 0 + while current_index < len(module_list) and len(module_list) > 0: + mod = module_list[current_index] + if module_list.count(mod) == 1: + module_list = module_list + list(get_modules_in_tree(mod)) + current_index += 1 + + test_suites_to_run: List[str] = list(get_test_suite_names(get_only_test_suites(module_list))) + test_suites_to_run.sort() + return test_suites_to_run + + +def main(): + test_suites_to_run = get_tests_to_run() + print("\n".join(test_suites_to_run)) + + +if __name__ == "__main__": + main() diff --git a/ci/requirements.txt b/ci/requirements.txt new file mode 100644 index 00000000..0abd6716 --- /dev/null +++ b/ci/requirements.txt @@ -0,0 +1,32 @@ +# BSD LICENSE +# +# Copyright(c) 2021 University of New Hampshire Interoperability Laboratory. All rights reserved. +# All rights reserved. +# +# Redistribution and use in source and binary forms, with or without +# modification, are permitted provided that the following conditions +# are met: +# +# * Redistributions of source code must retain the above copyright +# notice, this list of conditions and the following disclaimer. +# * Redistributions in binary form must reproduce the above copyright +# notice, this list of conditions and the following disclaimer in +# the documentation and/or other materials provided with the +# distribution. +# * Neither the name of Intel Corporation nor the names of its +# contributors may be used to endorse or promote products derived +# from this software without specific prior written permission. +# +# THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS +# "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT +# LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR +# A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT +# OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, +# SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT +# LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, +# DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY +# THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT +# (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE +# OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. + +argparse==1.4.0 \ No newline at end of file -- 2.30.2