DPDK CI discussions
 help / color / mirror / Atom feed
* [dpdk-ci] [PATCH v2 1/4] create_new_execution_file_from_tags: change tag argument
@ 2021-10-13 14:00 ohilyard
  2021-10-13 14:00 ` [dpdk-ci] [PATCH v2 2/4] create_new_execution_file_from_tags: add test argument ohilyard
                   ` (2 more replies)
  0 siblings, 3 replies; 4+ messages in thread
From: ohilyard @ 2021-10-13 14:00 UTC (permalink / raw)
  To: ci; +Cc: aconole, alialnu, Owen Hilyard

From: Owen Hilyard <ohilyard@iol.unh.edu>

BREAKING CHANGE: --tags syntax changed

Due to limitations of the argparse library, you can't have 2 varidic
arguments. Since upcoming requirements for the CI require the ability to
specify individual tests to be run (ex: weekly rte_flow testing), this
script must have the ability to pass in those tests as well. This is a
change made to support that.

Old syntax:
"--tags core documentation"

New syntax:
"--tag core --tag documentation"

Signed-off-by: Owen Hilyard <ohilyard@iol.unh.edu>
---
 tools/create_new_execution_file_from_tags.py | 13 ++++++++++---
 1 file changed, 10 insertions(+), 3 deletions(-)

diff --git a/tools/create_new_execution_file_from_tags.py b/tools/create_new_execution_file_from_tags.py
index d1d4447..b88fa93 100755
--- a/tools/create_new_execution_file_from_tags.py
+++ b/tools/create_new_execution_file_from_tags.py
@@ -73,7 +73,8 @@ def __str__(self):
     parser.add_argument('output_path', help='The path to the output execution file')
     parser.add_argument('testing_type', type=TestingType, choices=list(TestingType),
                         help='What type of testing to create an execution file for')
-    parser.add_argument('tags', metavar='tag', type=str, nargs='*', help='The tags to create an execution file for.')
+    parser.add_argument('--tag', type=str, action='append',
+                        help='The tags to create an execution file for.')
 
     args = parser.parse_args()
 
@@ -86,7 +87,9 @@ def __str__(self):
     test_map = {key: parse_comma_delimited_list_from_string(value.strip()) for key, value in
                 tag_to_test_map_parser[str(args.testing_type)].items()}
 
-    tests = map_tags_to_tests(args.tags, test_map)
+    tests = set()
+    if args.tag is not None:
+        tests = map_tags_to_tests(args.tag, test_map)
 
     try:
         output_file = open(args.output_path, 'x')
@@ -98,7 +101,11 @@ def __str__(self):
         if execution_plan != 'DEFAULT':
             test_allowlist = parse_comma_delimited_list_from_string(
                 template_execution_file_parser[execution_plan]['test_suites'])
-            tests_to_run = list(set(test_allowlist).intersection(tests))
+            if len(tests) != 0:
+                tests_to_run = list(set(test_allowlist).intersection(tests))
+            else:  # no tags given
+                tests_to_run = [entry for entry in test_allowlist if entry != '']
+
             tests_to_run.sort()
             template_execution_file_parser[execution_plan]['test_suites'] = ", ".join(tests_to_run)
 
-- 
2.30.2


^ permalink raw reply	[flat|nested] 4+ messages in thread

* [dpdk-ci] [PATCH v2 2/4] create_new_execution_file_from_tags: add test argument
  2021-10-13 14:00 [dpdk-ci] [PATCH v2 1/4] create_new_execution_file_from_tags: change tag argument ohilyard
@ 2021-10-13 14:00 ` ohilyard
  2021-10-13 14:00 ` [dpdk-ci] [PATCH v2 3/4] guess_git_tree: module doc comment moved ohilyard
  2021-10-13 14:00 ` [dpdk-ci] [PATCH v2 4/4] guess_git_tree: fix crash caused by empty diff ohilyard
  2 siblings, 0 replies; 4+ messages in thread
From: ohilyard @ 2021-10-13 14:00 UTC (permalink / raw)
  To: ci; +Cc: aconole, alialnu, Owen Hilyard

From: Owen Hilyard <ohilyard@iol.unh.edu>

Adds an argument to allow passing in individual test cases to be added
to the resulting DTS execution file. This change was made to facilitate
periodic testing of testcases not currently in CI, such as rte_flow. A
test passed in using this argument will bypass the normal behavior of
only allowing test cases specified in the template execution file. This
behavior because the mapping of DPDK files to tags is not complete, and
to avoid not testing patches, the decision was made to run all tests if
the tests for a patch could not be determined.

Signed-off-by: Owen Hilyard <ohilyard@iol.unh.edu>
---
 tools/create_new_execution_file_from_tags.py | 6 ++++++
 1 file changed, 6 insertions(+)

diff --git a/tools/create_new_execution_file_from_tags.py b/tools/create_new_execution_file_from_tags.py
index b88fa93..61d801d 100755
--- a/tools/create_new_execution_file_from_tags.py
+++ b/tools/create_new_execution_file_from_tags.py
@@ -75,6 +75,8 @@ def __str__(self):
                         help='What type of testing to create an execution file for')
     parser.add_argument('--tag', type=str, action='append',
                         help='The tags to create an execution file for.')
+    parser.add_argument('--test', type=str, action='append',
+                        help='The tests to run along with the tests required by the provided tags')
 
     args = parser.parse_args()
 
@@ -106,6 +108,10 @@ def __str__(self):
             else:  # no tags given
                 tests_to_run = [entry for entry in test_allowlist if entry != '']
 
+            if args.test is not None:
+                for test in args.test:
+                    tests_to_run.append(test)
+
             tests_to_run.sort()
             template_execution_file_parser[execution_plan]['test_suites'] = ", ".join(tests_to_run)
 
-- 
2.30.2


^ permalink raw reply	[flat|nested] 4+ messages in thread

* [dpdk-ci] [PATCH v2 3/4] guess_git_tree: module doc comment moved
  2021-10-13 14:00 [dpdk-ci] [PATCH v2 1/4] create_new_execution_file_from_tags: change tag argument ohilyard
  2021-10-13 14:00 ` [dpdk-ci] [PATCH v2 2/4] create_new_execution_file_from_tags: add test argument ohilyard
@ 2021-10-13 14:00 ` ohilyard
  2021-10-13 14:00 ` [dpdk-ci] [PATCH v2 4/4] guess_git_tree: fix crash caused by empty diff ohilyard
  2 siblings, 0 replies; 4+ messages in thread
From: ohilyard @ 2021-10-13 14:00 UTC (permalink / raw)
  To: ci; +Cc: aconole, alialnu, Owen Hilyard

From: Owen Hilyard <ohilyard@iol.unh.edu>

The import statements in the file have been moved below the module
doc comment per PEP 257.

Signed-off-by: Owen Hilyard <ohilyard@iol.unh.edu>
---
 tools/guess_git_tree.py | 25 ++++++++++++-------------
 1 file changed, 12 insertions(+), 13 deletions(-)

diff --git a/tools/guess_git_tree.py b/tools/guess_git_tree.py
index c9eef39..e186f5e 100755
--- a/tools/guess_git_tree.py
+++ b/tools/guess_git_tree.py
@@ -1,20 +1,8 @@
-#!/usr/bin/env python
+#!/usr/bin/env python3
 
 # SPDX-License-Identifier: (BSD-3-Clause AND GPL-2.0-or-later AND MIT)
 # Copyright 2019 Mellanox Technologies, Ltd
 
-import os
-import sys
-import re
-import argparse
-import fnmatch
-
-from requests.exceptions import HTTPError
-
-from git_pw import config
-from git_pw import api
-from git_pw import utils
-
 """
 Description:
 This script uses the git-pw API to retrieve Patchwork's
@@ -49,6 +37,17 @@
     tree = maintainers.get_tree(files)
 """
 
+import os
+import sys
+import re
+import argparse
+import fnmatch
+
+from requests.exceptions import HTTPError
+
+from git_pw import config
+from git_pw import api
+from git_pw import utils
 
 MAINTAINERS_FILE_PATH = os.environ.get('MAINTAINERS_FILE_PATH')
 if not MAINTAINERS_FILE_PATH:
-- 
2.30.2


^ permalink raw reply	[flat|nested] 4+ messages in thread

* [dpdk-ci] [PATCH v2 4/4] guess_git_tree: fix crash caused by empty diff
  2021-10-13 14:00 [dpdk-ci] [PATCH v2 1/4] create_new_execution_file_from_tags: change tag argument ohilyard
  2021-10-13 14:00 ` [dpdk-ci] [PATCH v2 2/4] create_new_execution_file_from_tags: add test argument ohilyard
  2021-10-13 14:00 ` [dpdk-ci] [PATCH v2 3/4] guess_git_tree: module doc comment moved ohilyard
@ 2021-10-13 14:00 ` ohilyard
  2 siblings, 0 replies; 4+ messages in thread
From: ohilyard @ 2021-10-13 14:00 UTC (permalink / raw)
  To: ci; +Cc: aconole, alialnu, Owen Hilyard

From: Owen Hilyard <ohilyard@iol.unh.edu>

A sanity check has been added to find_filenames. Occasionally, due to
how the community lab internally handles getting patches from
patchworks, a patch will result in no diff. This patch adds handling for
this case.

Signed-off-by: Owen Hilyard <ohilyard@iol.unh.edu>
---
 tools/guess_git_tree.py | 5 +++++
 1 file changed, 5 insertions(+)

diff --git a/tools/guess_git_tree.py b/tools/guess_git_tree.py
index e186f5e..63892e1 100755
--- a/tools/guess_git_tree.py
+++ b/tools/guess_git_tree.py
@@ -92,6 +92,11 @@ def find_filenames(diff):
             - Moved _filename_re into the method.
             - Reduced newlines.
         """
+        # sanity check diff
+        # for patches without any diff, it will try to run diff.replace
+        # while diff is None. just return an empty list
+        if diff is None:
+            return []
         _filename_re = re.compile(r'^(---|\+\+\+) (\S+)')
         # normalise spaces
         diff = diff.replace('\r', '')
-- 
2.30.2


^ permalink raw reply	[flat|nested] 4+ messages in thread

end of thread, other threads:[~2021-10-13 14:00 UTC | newest]

Thread overview: 4+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2021-10-13 14:00 [dpdk-ci] [PATCH v2 1/4] create_new_execution_file_from_tags: change tag argument ohilyard
2021-10-13 14:00 ` [dpdk-ci] [PATCH v2 2/4] create_new_execution_file_from_tags: add test argument ohilyard
2021-10-13 14:00 ` [dpdk-ci] [PATCH v2 3/4] guess_git_tree: module doc comment moved ohilyard
2021-10-13 14:00 ` [dpdk-ci] [PATCH v2 4/4] guess_git_tree: fix crash caused by empty diff ohilyard

DPDK CI discussions

This inbox may be cloned and mirrored by anyone:

	git clone --mirror http://inbox.dpdk.org/ci/0 ci/git/0.git

	# If you have public-inbox 1.1+ installed, you may
	# initialize and index your mirror using the following commands:
	public-inbox-init -V2 ci ci/ http://inbox.dpdk.org/ci \
		ci@dpdk.org
	public-inbox-index ci

Example config snippet for mirrors.
Newsgroup available over NNTP:
	nntp://inbox.dpdk.org/inbox.dpdk.ci


AGPL code for this site: git clone https://public-inbox.org/public-inbox.git