DPDK CI discussions
 help / color / mirror / Atom feed
* [dpdk-ci] [PATCH] ci: added patch parser for patch files
@ 2020-12-04 19:45 Owen Hilyard
  2021-01-14 16:53 ` Owen Hilyard
  2021-04-15 20:38 ` [dpdk-ci] [PATCH] ci: added patch parser for patch files Aaron Conole
  0 siblings, 2 replies; 9+ messages in thread
From: Owen Hilyard @ 2020-12-04 19:45 UTC (permalink / raw)
  To: ci; +Cc: ohilyard

This commit contains a script, patch_parser.py, and a config file,
patch_parser.cfg. These are tooling that the UNH CI team has been
testing in order to reduce the number of tests that need to be run
per patch. This resulted from our push to increase the number of
functional tests running in the CI. While working on expanding test
coverage, we found that DTS could easily take over 6 hours to run, so
we decided to begin work on tagging patches and then only running the
required tests.

The script works by taking in an address for the config file and then
a list of patch files, which it will parse and then produce a list of
tags for that list of patches based on the config file. The config file
is designed to work as a mapping for a base path to a set of tags. It
also contains an ordered list of priorities for tags so that this may
also be used by hierarchical tools rather than modular ones.

The intention of the UNH team with giving this tooling to the wider
DPDK community is to have people more familiar with the internal
functionality of DPDK provide most of the tagging. This would allow
UNH to have a better turn around time for testing by eliminating
unnecessary tests, while still increasing the number of tests in the
CI.

The different patch tags are currently defined as such:

core:
    Core DPDK functionality. Examples include kernel modules and
    librte_eal. This tag should be used sparingly as it is intended
     to signal to automated test suites that it is necessary to
     run most of the tests for DPDK and as such will consume CI
     resources for a long period of time.

driver:
    For NIC drivers and other hardware interface code. This should be
    used as a generic tag with each driver getting it's own tag.

application:
    Used in a similar manner to "driver". This tag is intended for
    code used in only in applications that DPDK provides, such as
    testpmd or helloworld. This tag should be accompanied by a tag
    which denotes which application specifically has been changed.

documentation:
    This is intended to be used as a tag for paths which only contain
    documentation, such as "doc/". It's intended use is as a way to
    trigger the automatic re-building of the documentation website.

Signed-off-by: Owen Hilyard <ohilyard@iol.unh.edu>
---
 config/patch_parser.cfg | 25 ++++++++++++++++
 tools/patch_parser.py   | 64 +++++++++++++++++++++++++++++++++++++++++
 2 files changed, 89 insertions(+)
 create mode 100644 config/patch_parser.cfg
 create mode 100755 tools/patch_parser.py

diff --git a/config/patch_parser.cfg b/config/patch_parser.cfg
new file mode 100644
index 0000000..5757f9a
--- /dev/null
+++ b/config/patch_parser.cfg
@@ -0,0 +1,25 @@
+# Description of the categories as initially designed
+
+[Paths]
+drivers =
+    driver,
+    core
+kernel = core
+doc = documentation
+lib = core
+meson_options.txt = core
+examples = application
+app = application
+license = documentation
+VERSION = documentation
+build = core
+
+# This is an ordered list of the importance of each patch classification.
+# It should be used to determine which classification to use on tools which
+# do not support multiple patch classifications.
+[Priority]
+priority_list =
+    core,
+    driver,
+    application,
+    documentation
diff --git a/tools/patch_parser.py b/tools/patch_parser.py
new file mode 100755
index 0000000..01fc55d
--- /dev/null
+++ b/tools/patch_parser.py
@@ -0,0 +1,64 @@
+#!/usr/bin/env python3
+
+import itertools
+import sys
+from configparser import ConfigParser
+from typing import List, Dict, Set
+
+
+def get_patch_files(patch_file: str) -> List[str]:
+    with open(patch_file, 'r') as f:
+        lines = list(itertools.takewhile(
+            lambda line: line.strip().endswith('+') or line.strip().endswith('-'),
+            itertools.dropwhile(
+                lambda line: not line.strip().startswith("---"),
+                f.readlines()
+            )
+        ))
+        filenames = map(lambda line: line.strip().split(' ')[0], lines)
+        # takewhile includes the --- which starts the filenames
+        return list(filenames)[1:]
+
+
+def get_all_files_from_patches(patch_files: List[str]) -> Set[str]:
+    return set(itertools.chain.from_iterable(map(get_patch_files, patch_files)))
+
+
+def parse_comma_delimited_list_from_string(mod_str: str) -> List[str]:
+    return list(map(str.strip, mod_str.split(',')))
+
+
+def get_dictionary_attributes_from_config_file(conf_obj: ConfigParser) -> Dict[str, Set[str]]:
+    return {
+        directory: parse_comma_delimited_list_from_string(module_string) for directory, module_string in
+        conf_obj['Paths'].items()
+    }
+
+
+def get_tags_for_patch_file(patch_file: str, dir_attrs: Dict[str, Set[str]]) -> Set[str]:
+    return set(itertools.chain.from_iterable(
+        tags for directory, tags in dir_attrs.items() if patch_file.startswith(directory)
+    ))
+
+
+def get_tags_for_patches(patch_files: Set[str], dir_attrs: Dict[str, Set[str]]) -> Set[str]:
+    return set(itertools.chain.from_iterable(
+        map(lambda patch_file: get_tags_for_patch_file(patch_file, dir_attrs), patch_files)
+    ))
+
+
+if len(sys.argv) < 3:
+    print("usage: patch_parser.py <path to patch_parser.cfg> <patch file>...")
+    exit(1)
+
+conf_obj = ConfigParser()
+conf_obj.read(sys.argv[1])
+
+patch_files = get_all_files_from_patches(sys.argv[2:])
+dir_attrs = get_dictionary_attributes_from_config_file(conf_obj)
+priority_list = parse_comma_delimited_list_from_string(conf_obj['Priority']['priority_list'])
+
+unordered_tags: Set[str] = get_tags_for_patches(patch_files, dir_attrs)
+ordered_tags: List[str] = [tag for tag in priority_list if tag in unordered_tags]
+
+print("\n".join(ordered_tags))
-- 
2.27.0


^ permalink raw reply	[flat|nested] 9+ messages in thread

* Re: [dpdk-ci] [PATCH] ci: added patch parser for patch files
  2020-12-04 19:45 [dpdk-ci] [PATCH] ci: added patch parser for patch files Owen Hilyard
@ 2021-01-14 16:53 ` Owen Hilyard
  2021-01-14 17:59   ` [dpdk-ci] [PATCH V2] patch-tagging: Added tool to convert tags into dts execution file ohilyard
  2021-04-15 20:38 ` [dpdk-ci] [PATCH] ci: added patch parser for patch files Aaron Conole
  1 sibling, 1 reply; 9+ messages in thread
From: Owen Hilyard @ 2021-01-14 16:53 UTC (permalink / raw)
  To: ci; +Cc: Thomas Monjalon

[-- Attachment #1: Type: text/plain, Size: 6395 bytes --]

A bit of fuzz testing found some edge cases where this script crashes or
fails to properly parse the patch file. I am currently working on a
rewrite using a dedicated library to avoid these and similar issues.

On Fri, Dec 4, 2020 at 2:45 PM Owen Hilyard <ohilyard@iol.unh.edu> wrote:

> This commit contains a script, patch_parser.py, and a config file,
> patch_parser.cfg. These are tooling that the UNH CI team has been
> testing in order to reduce the number of tests that need to be run
> per patch. This resulted from our push to increase the number of
> functional tests running in the CI. While working on expanding test
> coverage, we found that DTS could easily take over 6 hours to run, so
> we decided to begin work on tagging patches and then only running the
> required tests.
>
> The script works by taking in an address for the config file and then
> a list of patch files, which it will parse and then produce a list of
> tags for that list of patches based on the config file. The config file
> is designed to work as a mapping for a base path to a set of tags. It
> also contains an ordered list of priorities for tags so that this may
> also be used by hierarchical tools rather than modular ones.
>
> The intention of the UNH team with giving this tooling to the wider
> DPDK community is to have people more familiar with the internal
> functionality of DPDK provide most of the tagging. This would allow
> UNH to have a better turn around time for testing by eliminating
> unnecessary tests, while still increasing the number of tests in the
> CI.
>
> The different patch tags are currently defined as such:
>
> core:
>     Core DPDK functionality. Examples include kernel modules and
>     librte_eal. This tag should be used sparingly as it is intended
>      to signal to automated test suites that it is necessary to
>      run most of the tests for DPDK and as such will consume CI
>      resources for a long period of time.
>
> driver:
>     For NIC drivers and other hardware interface code. This should be
>     used as a generic tag with each driver getting it's own tag.
>
> application:
>     Used in a similar manner to "driver". This tag is intended for
>     code used in only in applications that DPDK provides, such as
>     testpmd or helloworld. This tag should be accompanied by a tag
>     which denotes which application specifically has been changed.
>
> documentation:
>     This is intended to be used as a tag for paths which only contain
>     documentation, such as "doc/". It's intended use is as a way to
>     trigger the automatic re-building of the documentation website.
>
> Signed-off-by: Owen Hilyard <ohilyard@iol.unh.edu>
> ---
>  config/patch_parser.cfg | 25 ++++++++++++++++
>  tools/patch_parser.py   | 64 +++++++++++++++++++++++++++++++++++++++++
>  2 files changed, 89 insertions(+)
>  create mode 100644 config/patch_parser.cfg
>  create mode 100755 tools/patch_parser.py
>
> diff --git a/config/patch_parser.cfg b/config/patch_parser.cfg
> new file mode 100644
> index 0000000..5757f9a
> --- /dev/null
> +++ b/config/patch_parser.cfg
> @@ -0,0 +1,25 @@
> +# Description of the categories as initially designed
> +
> +[Paths]
> +drivers =
> +    driver,
> +    core
> +kernel = core
> +doc = documentation
> +lib = core
> +meson_options.txt = core
> +examples = application
> +app = application
> +license = documentation
> +VERSION = documentation
> +build = core
> +
> +# This is an ordered list of the importance of each patch classification.
> +# It should be used to determine which classification to use on tools
> which
> +# do not support multiple patch classifications.
> +[Priority]
> +priority_list =
> +    core,
> +    driver,
> +    application,
> +    documentation
> diff --git a/tools/patch_parser.py b/tools/patch_parser.py
> new file mode 100755
> index 0000000..01fc55d
> --- /dev/null
> +++ b/tools/patch_parser.py
> @@ -0,0 +1,64 @@
> +#!/usr/bin/env python3
> +
> +import itertools
> +import sys
> +from configparser import ConfigParser
> +from typing import List, Dict, Set
> +
> +
> +def get_patch_files(patch_file: str) -> List[str]:
> +    with open(patch_file, 'r') as f:
> +        lines = list(itertools.takewhile(
> +            lambda line: line.strip().endswith('+') or
> line.strip().endswith('-'),
> +            itertools.dropwhile(
> +                lambda line: not line.strip().startswith("---"),
> +                f.readlines()
> +            )
> +        ))
> +        filenames = map(lambda line: line.strip().split(' ')[0], lines)
> +        # takewhile includes the --- which starts the filenames
> +        return list(filenames)[1:]
> +
> +
> +def get_all_files_from_patches(patch_files: List[str]) -> Set[str]:
> +    return set(itertools.chain.from_iterable(map(get_patch_files,
> patch_files)))
> +
> +
> +def parse_comma_delimited_list_from_string(mod_str: str) -> List[str]:
> +    return list(map(str.strip, mod_str.split(',')))
> +
> +
> +def get_dictionary_attributes_from_config_file(conf_obj: ConfigParser) ->
> Dict[str, Set[str]]:
> +    return {
> +        directory: parse_comma_delimited_list_from_string(module_string)
> for directory, module_string in
> +        conf_obj['Paths'].items()
> +    }
> +
> +
> +def get_tags_for_patch_file(patch_file: str, dir_attrs: Dict[str,
> Set[str]]) -> Set[str]:
> +    return set(itertools.chain.from_iterable(
> +        tags for directory, tags in dir_attrs.items() if
> patch_file.startswith(directory)
> +    ))
> +
> +
> +def get_tags_for_patches(patch_files: Set[str], dir_attrs: Dict[str,
> Set[str]]) -> Set[str]:
> +    return set(itertools.chain.from_iterable(
> +        map(lambda patch_file: get_tags_for_patch_file(patch_file,
> dir_attrs), patch_files)
> +    ))
> +
> +
> +if len(sys.argv) < 3:
> +    print("usage: patch_parser.py <path to patch_parser.cfg> <patch
> file>...")
> +    exit(1)
> +
> +conf_obj = ConfigParser()
> +conf_obj.read(sys.argv[1])
> +
> +patch_files = get_all_files_from_patches(sys.argv[2:])
> +dir_attrs = get_dictionary_attributes_from_config_file(conf_obj)
> +priority_list =
> parse_comma_delimited_list_from_string(conf_obj['Priority']['priority_list'])
> +
> +unordered_tags: Set[str] = get_tags_for_patches(patch_files, dir_attrs)
> +ordered_tags: List[str] = [tag for tag in priority_list if tag in
> unordered_tags]
> +
> +print("\n".join(ordered_tags))
> --
> 2.27.0
>
>

[-- Attachment #2: Type: text/html, Size: 7501 bytes --]

^ permalink raw reply	[flat|nested] 9+ messages in thread

* [dpdk-ci] [PATCH V2] patch-tagging: Added tool to convert tags into dts execution file
  2021-01-14 16:53 ` Owen Hilyard
@ 2021-01-14 17:59   ` ohilyard
  2021-01-14 18:19     ` Owen Hilyard
  0 siblings, 1 reply; 9+ messages in thread
From: ohilyard @ 2021-01-14 17:59 UTC (permalink / raw)
  To: ci; +Cc: thomas, Owen Hilyard

Fixed parsing bugs with the tag parsing script
	Parsing the patch file directly proved to be too unreliabler, so the approach to patch parsing was changed so that it relies instead on using a directory with those patches already applied in them and then use a git diff command to get the changed files. This avoids needing to parse the patch files directly and should be more reliable.

Added a tool to create an execution file from tags using another execution file as a template
	This tool is intended to allow a list of patch tags to be used in combination with an execution file containing information on crbs to only change which test suites are run. This tool enables using the mappings in the tests_for_tag.cfg file to run only necessary tests. At present, the mappings represent how tests are currently run at the UNH-IOL Community Lab, with the same tests being run for every patch, with the requested change from bug 511 where documentation changes do not cause any testing.

Signed-off-by: Owen Hilyard <ohilyard@iol.unh.edu>
---
 config/tests_for_tag.cfg | 19 ++++++++++++--
 tools/patch_parser.py    | 54 ++++++++++++++++++++++++----------------
 2 files changed, 49 insertions(+), 24 deletions(-)

diff --git a/config/tests_for_tag.cfg b/config/tests_for_tag.cfg
index 77f797f..7d95c4a 100644
--- a/config/tests_for_tag.cfg
+++ b/config/tests_for_tag.cfg
@@ -5,11 +5,26 @@ core = dynamic_config,
        mtu_update,
        scatter,
        stats_checks,
+       unit_tests_mbuf
+driver = dynamic_config,
+       link_status_interrupt,
+       mac_filter,
+       mtu_update,
+       scatter,
+       stats_checks,
+       unit_tests_mbuf
+application = dynamic_config,
+       link_status_interrupt,
+       mac_filter,
+       mtu_update,
+       scatter,
        stats_checks,
        unit_tests_mbuf
-
 ; Nothing in documentation
 documentation =
 
 [performance]
-core = nic_single_core_perf,
+core = nic_single_core_perf
+driver = nic_single_core_perf
+application = nic_single_core_perf
+documentation =
diff --git a/tools/patch_parser.py b/tools/patch_parser.py
index 80b8194..73df91f 100755
--- a/tools/patch_parser.py
+++ b/tools/patch_parser.py
@@ -31,30 +31,39 @@
 # THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
 # (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
 # OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
-
+import subprocess
 
 import itertools
 from configparser import ConfigParser
 from typing import List, Dict, Set
 import argparse
+import re
 
 
 def get_patch_files(patch_file: str) -> List[str]:
+    file_match_pattern = re.compile(r"[\w\-\/.]+ +\| +\d+ [+\-]+")
     with open(patch_file, 'r') as f:
-        lines = list(itertools.takewhile(
-            lambda line: line.strip().endswith('+') or line.strip().endswith('-'),
-            itertools.dropwhile(
-                lambda line: not line.strip().startswith("---"),
-                f.readlines()
+        file_lines = f.readlines()
+        lines = list(filter(
+            lambda line: file_match_pattern.match(line.strip()),
+            itertools.takewhile(
+                lambda line: not line.startswith('---'),
+                list(
+                    itertools.dropwhile(
+                        lambda line: not line.strip().startswith("---"),
+                        file_lines
+                    )
+                )[1:]
             )
         ))
         filenames = map(lambda line: line.strip().split(' ')[0], lines)
-        # takewhile includes the --- which starts the filenames
-        return list(filenames)[1:]
+        return list(filenames)
 
 
 def get_all_files_from_patches(patch_files: List[str]) -> Set[str]:
-    return set(itertools.chain.from_iterable(map(get_patch_files, patch_files)))
+    num_patch_files: int = len(patch_files)
+    files = subprocess.check_output(f"cd dpdk && git diff --name-only HEAD~{num_patch_files}", shell=True).decode('utf-8').splitlines()
+    return set(files)
 
 
 def parse_comma_delimited_list_from_string(mod_str: str) -> List[str]:
@@ -80,21 +89,22 @@ def get_tags_for_patches(patch_files: Set[str], dir_attrs: Dict[str, Set[str]])
     ))
 
 
-parser = argparse.ArgumentParser(
-    description='Takes a patch file and a config file and creates a list of tags for that patch')
-parser.add_argument('config_file_path', help='The path to patch_parser.cfg', default='config/patch_parser.cfg')
-parser.add_argument('patch_file_paths', help='A list of patch files', type=str, metavar='patch file', nargs='+')
+if __name__ == '__main__':
+    parser = argparse.ArgumentParser(
+        description='Takes a patch file and a config file and creates a list of tags for that patch')
+    parser.add_argument('config_file_path', help='The path to patch_parser.cfg', default='config/patch_parser.cfg')
+    parser.add_argument('patch_file_paths', help='A list of patch files', type=str, metavar='patch file', nargs='+')
 
-args = parser.parse_args()
+    args = parser.parse_args()
 
-conf_obj = ConfigParser()
-conf_obj.read(args.config_file_path)
+    conf_obj = ConfigParser()
+    conf_obj.read(args.config_file_path)
 
-patch_files = get_all_files_from_patches(args.patch_file_paths)
-dir_attrs = get_dictionary_attributes_from_config_file(conf_obj)
-priority_list = parse_comma_delimited_list_from_string(conf_obj['Priority']['priority_list'])
+    patch_files = get_all_files_from_patches(args.patch_file_paths)
+    dir_attrs = get_dictionary_attributes_from_config_file(conf_obj)
+    priority_list = parse_comma_delimited_list_from_string(conf_obj['Priority']['priority_list'])
 
-unordered_tags: Set[str] = get_tags_for_patches(patch_files, dir_attrs)
-ordered_tags: List[str] = [tag for tag in priority_list if tag in unordered_tags]
+    unordered_tags: Set[str] = get_tags_for_patches(patch_files, dir_attrs)
+    ordered_tags: List[str] = [tag for tag in priority_list if tag in unordered_tags]
 
-print("\n".join(ordered_tags))
+    print("\n".join(ordered_tags))
-- 
2.27.0


^ permalink raw reply	[flat|nested] 9+ messages in thread

* Re: [dpdk-ci] [PATCH V2] patch-tagging: Added tool to convert tags into dts execution file
  2021-01-14 17:59   ` [dpdk-ci] [PATCH V2] patch-tagging: Added tool to convert tags into dts execution file ohilyard
@ 2021-01-14 18:19     ` Owen Hilyard
  2021-04-15 20:42       ` Aaron Conole
  0 siblings, 1 reply; 9+ messages in thread
From: Owen Hilyard @ 2021-01-14 18:19 UTC (permalink / raw)
  To: ci; +Cc: Thomas Monjalon

[-- Attachment #1: Type: text/plain, Size: 6510 bytes --]

Please ignore this patch. I messed up directories and this patch was not
created from the correct file.

On Thu, Jan 14, 2021 at 12:59 PM <ohilyard@iol.unh.edu> wrote:

> Fixed parsing bugs with the tag parsing script
>         Parsing the patch file directly proved to be too unreliabler, so
> the approach to patch parsing was changed so that it relies instead on
> using a directory with those patches already applied in them and then use a
> git diff command to get the changed files. This avoids needing to parse the
> patch files directly and should be more reliable.
>
> Added a tool to create an execution file from tags using another execution
> file as a template
>         This tool is intended to allow a list of patch tags to be used in
> combination with an execution file containing information on crbs to only
> change which test suites are run. This tool enables using the mappings in
> the tests_for_tag.cfg file to run only necessary tests. At present, the
> mappings represent how tests are currently run at the UNH-IOL Community
> Lab, with the same tests being run for every patch, with the requested
> change from bug 511 where documentation changes do not cause any testing.
>
> Signed-off-by: Owen Hilyard <ohilyard@iol.unh.edu>
> ---
>  config/tests_for_tag.cfg | 19 ++++++++++++--
>  tools/patch_parser.py    | 54 ++++++++++++++++++++++++----------------
>  2 files changed, 49 insertions(+), 24 deletions(-)
>
> diff --git a/config/tests_for_tag.cfg b/config/tests_for_tag.cfg
> index 77f797f..7d95c4a 100644
> --- a/config/tests_for_tag.cfg
> +++ b/config/tests_for_tag.cfg
> @@ -5,11 +5,26 @@ core = dynamic_config,
>         mtu_update,
>         scatter,
>         stats_checks,
> +       unit_tests_mbuf
> +driver = dynamic_config,
> +       link_status_interrupt,
> +       mac_filter,
> +       mtu_update,
> +       scatter,
> +       stats_checks,
> +       unit_tests_mbuf
> +application = dynamic_config,
> +       link_status_interrupt,
> +       mac_filter,
> +       mtu_update,
> +       scatter,
>         stats_checks,
>         unit_tests_mbuf
> -
>  ; Nothing in documentation
>  documentation =
>
>  [performance]
> -core = nic_single_core_perf,
> +core = nic_single_core_perf
> +driver = nic_single_core_perf
> +application = nic_single_core_perf
> +documentation =
> diff --git a/tools/patch_parser.py b/tools/patch_parser.py
> index 80b8194..73df91f 100755
> --- a/tools/patch_parser.py
> +++ b/tools/patch_parser.py
> @@ -31,30 +31,39 @@
>  # THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
>  # (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
>  # OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
> -
> +import subprocess
>
>  import itertools
>  from configparser import ConfigParser
>  from typing import List, Dict, Set
>  import argparse
> +import re
>
>
>  def get_patch_files(patch_file: str) -> List[str]:
> +    file_match_pattern = re.compile(r"[\w\-\/.]+ +\| +\d+ [+\-]+")
>      with open(patch_file, 'r') as f:
> -        lines = list(itertools.takewhile(
> -            lambda line: line.strip().endswith('+') or
> line.strip().endswith('-'),
> -            itertools.dropwhile(
> -                lambda line: not line.strip().startswith("---"),
> -                f.readlines()
> +        file_lines = f.readlines()
> +        lines = list(filter(
> +            lambda line: file_match_pattern.match(line.strip()),
> +            itertools.takewhile(
> +                lambda line: not line.startswith('---'),
> +                list(
> +                    itertools.dropwhile(
> +                        lambda line: not line.strip().startswith("---"),
> +                        file_lines
> +                    )
> +                )[1:]
>              )
>          ))
>          filenames = map(lambda line: line.strip().split(' ')[0], lines)
> -        # takewhile includes the --- which starts the filenames
> -        return list(filenames)[1:]
> +        return list(filenames)
>
>
>  def get_all_files_from_patches(patch_files: List[str]) -> Set[str]:
> -    return set(itertools.chain.from_iterable(map(get_patch_files,
> patch_files)))
> +    num_patch_files: int = len(patch_files)
> +    files = subprocess.check_output(f"cd dpdk && git diff --name-only
> HEAD~{num_patch_files}", shell=True).decode('utf-8').splitlines()
> +    return set(files)
>
>
>  def parse_comma_delimited_list_from_string(mod_str: str) -> List[str]:
> @@ -80,21 +89,22 @@ def get_tags_for_patches(patch_files: Set[str],
> dir_attrs: Dict[str, Set[str]])
>      ))
>
>
> -parser = argparse.ArgumentParser(
> -    description='Takes a patch file and a config file and creates a list
> of tags for that patch')
> -parser.add_argument('config_file_path', help='The path to
> patch_parser.cfg', default='config/patch_parser.cfg')
> -parser.add_argument('patch_file_paths', help='A list of patch files',
> type=str, metavar='patch file', nargs='+')
> +if __name__ == '__main__':
> +    parser = argparse.ArgumentParser(
> +        description='Takes a patch file and a config file and creates a
> list of tags for that patch')
> +    parser.add_argument('config_file_path', help='The path to
> patch_parser.cfg', default='config/patch_parser.cfg')
> +    parser.add_argument('patch_file_paths', help='A list of patch files',
> type=str, metavar='patch file', nargs='+')
>
> -args = parser.parse_args()
> +    args = parser.parse_args()
>
> -conf_obj = ConfigParser()
> -conf_obj.read(args.config_file_path)
> +    conf_obj = ConfigParser()
> +    conf_obj.read(args.config_file_path)
>
> -patch_files = get_all_files_from_patches(args.patch_file_paths)
> -dir_attrs = get_dictionary_attributes_from_config_file(conf_obj)
> -priority_list =
> parse_comma_delimited_list_from_string(conf_obj['Priority']['priority_list'])
> +    patch_files = get_all_files_from_patches(args.patch_file_paths)
> +    dir_attrs = get_dictionary_attributes_from_config_file(conf_obj)
> +    priority_list =
> parse_comma_delimited_list_from_string(conf_obj['Priority']['priority_list'])
>
> -unordered_tags: Set[str] = get_tags_for_patches(patch_files, dir_attrs)
> -ordered_tags: List[str] = [tag for tag in priority_list if tag in
> unordered_tags]
> +    unordered_tags: Set[str] = get_tags_for_patches(patch_files,
> dir_attrs)
> +    ordered_tags: List[str] = [tag for tag in priority_list if tag in
> unordered_tags]
>
> -print("\n".join(ordered_tags))
> +    print("\n".join(ordered_tags))
> --
> 2.27.0
>
>

[-- Attachment #2: Type: text/html, Size: 7811 bytes --]

^ permalink raw reply	[flat|nested] 9+ messages in thread

* Re: [dpdk-ci] [PATCH] ci: added patch parser for patch files
  2020-12-04 19:45 [dpdk-ci] [PATCH] ci: added patch parser for patch files Owen Hilyard
  2021-01-14 16:53 ` Owen Hilyard
@ 2021-04-15 20:38 ` Aaron Conole
  2021-04-15 21:11   ` [dpdk-ci] [PATCH] patch parsing: Added library for parsing " ohilyard
  1 sibling, 1 reply; 9+ messages in thread
From: Aaron Conole @ 2021-04-15 20:38 UTC (permalink / raw)
  To: Owen Hilyard; +Cc: ci

Owen Hilyard writes:

> This commit contains a script, patch_parser.py, and a config file,
> patch_parser.cfg. These are tooling that the UNH CI team has been
> testing in order to reduce the number of tests that need to be run
> per patch. This resulted from our push to increase the number of
> functional tests running in the CI. While working on expanding test
> coverage, we found that DTS could easily take over 6 hours to run, so
> we decided to begin work on tagging patches and then only running the
> required tests.
>
> The script works by taking in an address for the config file and then
> a list of patch files, which it will parse and then produce a list of
> tags for that list of patches based on the config file. The config file
> is designed to work as a mapping for a base path to a set of tags. It
> also contains an ordered list of priorities for tags so that this may
> also be used by hierarchical tools rather than modular ones.
>
> The intention of the UNH team with giving this tooling to the wider
> DPDK community is to have people more familiar with the internal
> functionality of DPDK provide most of the tagging. This would allow
> UNH to have a better turn around time for testing by eliminating
> unnecessary tests, while still increasing the number of tests in the
> CI.
>
> The different patch tags are currently defined as such:
>
> core:
>     Core DPDK functionality. Examples include kernel modules and
>     librte_eal. This tag should be used sparingly as it is intended
>      to signal to automated test suites that it is necessary to
>      run most of the tests for DPDK and as such will consume CI
>      resources for a long period of time.
>
> driver:
>     For NIC drivers and other hardware interface code. This should be
>     used as a generic tag with each driver getting it's own tag.
>
> application:
>     Used in a similar manner to "driver". This tag is intended for
>     code used in only in applications that DPDK provides, such as
>     testpmd or helloworld. This tag should be accompanied by a tag
>     which denotes which application specifically has been changed.
>
> documentation:
>     This is intended to be used as a tag for paths which only contain
>     documentation, such as "doc/". It's intended use is as a way to
>     trigger the automatic re-building of the documentation website.
>
> Signed-off-by: Owen Hilyard <ohilyard at iol.unh.edu>
> ---

Applied to dpdk-ci.git

Thanks!


^ permalink raw reply	[flat|nested] 9+ messages in thread

* Re: [dpdk-ci] [PATCH V2] patch-tagging: Added tool to convert tags into dts execution file
  2021-01-14 18:19     ` Owen Hilyard
@ 2021-04-15 20:42       ` Aaron Conole
  2021-04-15 21:14         ` Owen Hilyard
  0 siblings, 1 reply; 9+ messages in thread
From: Aaron Conole @ 2021-04-15 20:42 UTC (permalink / raw)
  To: Owen Hilyard; +Cc: ci, Thomas Monjalon

Owen Hilyard <ohilyard@iol.unh.edu> writes:

> Please ignore this patch. I messed up directories and this patch was not created from the correct file. 

Is there an update for this that I missed?

> On Thu, Jan 14, 2021 at 12:59 PM <ohilyard@iol.unh.edu> wrote:
>
>  Fixed parsing bugs with the tag parsing script
>          Parsing the patch file directly proved to be too unreliabler, so the approach to patch parsing was changed so that
>  it relies instead on using a directory with those patches already applied in them and then use a git diff command to get
>  the changed files. This avoids needing to parse the patch files directly and should be more reliable.
>
>  Added a tool to create an execution file from tags using another execution file as a template
>          This tool is intended to allow a list of patch tags to be used in combination with an execution file containing
>  information on crbs to only change which test suites are run. This tool enables using the mappings in the
>  tests_for_tag.cfg file to run only necessary tests. At present, the mappings represent how tests are currently run at the
>  UNH-IOL Community Lab, with the same tests being run for every patch, with the requested change from bug 511
>  where documentation changes do not cause any testing.
>
>  Signed-off-by: Owen Hilyard <ohilyard@iol.unh.edu>
>  ---
>   config/tests_for_tag.cfg | 19 ++++++++++++--
>   tools/patch_parser.py    | 54 ++++++++++++++++++++++++----------------
>   2 files changed, 49 insertions(+), 24 deletions(-)
>
>  diff --git a/config/tests_for_tag.cfg b/config/tests_for_tag.cfg
>  index 77f797f..7d95c4a 100644
>  --- a/config/tests_for_tag.cfg
>  +++ b/config/tests_for_tag.cfg
>  @@ -5,11 +5,26 @@ core = dynamic_config,
>          mtu_update,
>          scatter,
>          stats_checks,
>  +       unit_tests_mbuf
>  +driver = dynamic_config,
>  +       link_status_interrupt,
>  +       mac_filter,
>  +       mtu_update,
>  +       scatter,
>  +       stats_checks,
>  +       unit_tests_mbuf
>  +application = dynamic_config,
>  +       link_status_interrupt,
>  +       mac_filter,
>  +       mtu_update,
>  +       scatter,
>          stats_checks,
>          unit_tests_mbuf
>  -
>   ; Nothing in documentation
>   documentation =
>
>   [performance]
>  -core = nic_single_core_perf,
>  +core = nic_single_core_perf
>  +driver = nic_single_core_perf
>  +application = nic_single_core_perf
>  +documentation =
>  diff --git a/tools/patch_parser.py b/tools/patch_parser.py
>  index 80b8194..73df91f 100755
>  --- a/tools/patch_parser.py
>  +++ b/tools/patch_parser.py
>  @@ -31,30 +31,39 @@
>   # THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
>   # (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
>   # OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
>  -
>  +import subprocess
>
>   import itertools
>   from configparser import ConfigParser
>   from typing import List, Dict, Set
>   import argparse
>  +import re
>
>   def get_patch_files(patch_file: str) -> List[str]:
>  +    file_match_pattern = re.compile(r"[\w\-\/.]+ +\| +\d+ [+\-]+")
>       with open(patch_file, 'r') as f:
>  -        lines = list(itertools.takewhile(
>  -            lambda line: line.strip().endswith('+') or line.strip().endswith('-'),
>  -            itertools.dropwhile(
>  -                lambda line: not line.strip().startswith("---"),
>  -                f.readlines()
>  +        file_lines = f.readlines()
>  +        lines = list(filter(
>  +            lambda line: file_match_pattern.match(line.strip()),
>  +            itertools.takewhile(
>  +                lambda line: not line.startswith('---'),
>  +                list(
>  +                    itertools.dropwhile(
>  +                        lambda line: not line.strip().startswith("---"),
>  +                        file_lines
>  +                    )
>  +                )[1:]
>               )
>           ))
>           filenames = map(lambda line: line.strip().split(' ')[0], lines)
>  -        # takewhile includes the --- which starts the filenames
>  -        return list(filenames)[1:]
>  +        return list(filenames)
>
>   def get_all_files_from_patches(patch_files: List[str]) -> Set[str]:
>  -    return set(itertools.chain.from_iterable(map(get_patch_files, patch_files)))
>  +    num_patch_files: int = len(patch_files)
>  +    files = subprocess.check_output(f"cd dpdk && git diff --name-only HEAD~{num_patch_files}", shell=True).decode
>  ('utf-8').splitlines()
>  +    return set(files)
>
>   def parse_comma_delimited_list_from_string(mod_str: str) -> List[str]:
>  @@ -80,21 +89,22 @@ def get_tags_for_patches(patch_files: Set[str], dir_attrs: Dict[str, Set[str]])
>       ))
>
>  -parser = argparse.ArgumentParser(
>  -    description='Takes a patch file and a config file and creates a list of tags for that patch')
>  -parser.add_argument('config_file_path', help='The path to patch_parser.cfg', default='config/patch_parser.cfg')
>  -parser.add_argument('patch_file_paths', help='A list of patch files', type=str, metavar='patch file', nargs='+')
>  +if __name__ == '__main__':
>  +    parser = argparse.ArgumentParser(
>  +        description='Takes a patch file and a config file and creates a list of tags for that patch')
>  +    parser.add_argument('config_file_path', help='The path to patch_parser.cfg', default='config/patch_parser.cfg')
>  +    parser.add_argument('patch_file_paths', help='A list of patch files', type=str, metavar='patch file', nargs='+')
>
>  -args = parser.parse_args()
>  +    args = parser.parse_args()
>
>  -conf_obj = ConfigParser()
>  -conf_obj.read(args.config_file_path)
>  +    conf_obj = ConfigParser()
>  +    conf_obj.read(args.config_file_path)
>
>  -patch_files = get_all_files_from_patches(args.patch_file_paths)
>  -dir_attrs = get_dictionary_attributes_from_config_file(conf_obj)
>  -priority_list = parse_comma_delimited_list_from_string(conf_obj['Priority']['priority_list'])
>  +    patch_files = get_all_files_from_patches(args.patch_file_paths)
>  +    dir_attrs = get_dictionary_attributes_from_config_file(conf_obj)
>  +    priority_list = parse_comma_delimited_list_from_string(conf_obj['Priority']['priority_list'])
>
>  -unordered_tags: Set[str] = get_tags_for_patches(patch_files, dir_attrs)
>  -ordered_tags: List[str] = [tag for tag in priority_list if tag in unordered_tags]
>  +    unordered_tags: Set[str] = get_tags_for_patches(patch_files, dir_attrs)
>  +    ordered_tags: List[str] = [tag for tag in priority_list if tag in unordered_tags]
>
>  -print("\n".join(ordered_tags))
>  +    print("\n".join(ordered_tags))
>  -- 
>  2.27.0


^ permalink raw reply	[flat|nested] 9+ messages in thread

* [dpdk-ci] [PATCH] patch parsing: Added library for parsing patch files
  2021-04-15 20:38 ` [dpdk-ci] [PATCH] ci: added patch parser for patch files Aaron Conole
@ 2021-04-15 21:11   ` ohilyard
  2021-04-15 21:37     ` Aaron Conole
  0 siblings, 1 reply; 9+ messages in thread
From: ohilyard @ 2021-04-15 21:11 UTC (permalink / raw)
  To: aconole; +Cc: ci, Owen Hilyard

From: Owen Hilyard <ohilyard@iol.unh.edu>

Due to the number of edge cases, there were difficulties doing the parsing using pattern matching, so a dedicated library, whatthepatch, is now used for parsing the patches to ensure correctness.

Also added the default behavior that if no tags are given, create_new_execution_file_from_tags.py will act as though all possible tags were given. This is to allow patches where no tags could be found to still have test coverage.
---
 config/tests_for_tag.cfg                     |  30 +++++
 requirements.txt                             |   1 +
 tools/create_new_execution_file_from_tags.py | 114 +++++++++++++++++++
 tools/patch_parser.py                        |  85 ++++++++++----
 4 files changed, 205 insertions(+), 25 deletions(-)
 create mode 100644 config/tests_for_tag.cfg
 create mode 100644 requirements.txt
 create mode 100755 tools/create_new_execution_file_from_tags.py

diff --git a/config/tests_for_tag.cfg b/config/tests_for_tag.cfg
new file mode 100644
index 0000000..7d95c4a
--- /dev/null
+++ b/config/tests_for_tag.cfg
@@ -0,0 +1,30 @@
+[functional]
+core = dynamic_config,
+       link_status_interrupt,
+       mac_filter,
+       mtu_update,
+       scatter,
+       stats_checks,
+       unit_tests_mbuf
+driver = dynamic_config,
+       link_status_interrupt,
+       mac_filter,
+       mtu_update,
+       scatter,
+       stats_checks,
+       unit_tests_mbuf
+application = dynamic_config,
+       link_status_interrupt,
+       mac_filter,
+       mtu_update,
+       scatter,
+       stats_checks,
+       unit_tests_mbuf
+; Nothing in documentation
+documentation =
+
+[performance]
+core = nic_single_core_perf
+driver = nic_single_core_perf
+application = nic_single_core_perf
+documentation =
diff --git a/requirements.txt b/requirements.txt
new file mode 100644
index 0000000..f20067d
--- /dev/null
+++ b/requirements.txt
@@ -0,0 +1 @@
+whatthepatch==1.0.2
\ No newline at end of file
diff --git a/tools/create_new_execution_file_from_tags.py b/tools/create_new_execution_file_from_tags.py
new file mode 100755
index 0000000..d1d4447
--- /dev/null
+++ b/tools/create_new_execution_file_from_tags.py
@@ -0,0 +1,114 @@
+#!/usr/bin/env python3
+
+# BSD LICENSE
+#
+# Copyright(c) 2020 Intel Corporation. All rights reserved.
+# Copyright © 2018[, 2019] The University of New Hampshire. All rights reserved.
+# All rights reserved.
+#
+# Redistribution and use in source and binary forms, with or without
+# modification, are permitted provided that the following conditions
+# are met:
+#
+#   * Redistributions of source code must retain the above copyright
+#     notice, this list of conditions and the following disclaimer.
+#   * Redistributions in binary form must reproduce the above copyright
+#     notice, this list of conditions and the following disclaimer in
+#     the documentation and/or other materials provided with the
+#     distribution.
+#   * Neither the name of Intel Corporation nor the names of its
+#     contributors may be used to endorse or promote products derived
+#     from this software without specific prior written permission.
+#
+# THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
+# "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
+# LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
+# A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
+# OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
+# SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
+# LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
+# DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
+# THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+# (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
+# OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+import sys
+from enum import Enum
+
+import itertools
+from configparser import ConfigParser
+from typing import List, Dict, Set
+import argparse
+
+
+def parse_comma_delimited_list_from_string(mod_str: str) -> List[str]:
+    return list(map(str.strip, mod_str.split(',')))
+
+
+def map_tags_to_tests(tags: List[str], test_map: Dict[str, List[str]]) -> Set[str]:
+    """
+    Returns a list that is the union of all of the map lookups.
+    """
+    try:
+        return set(
+            filter(lambda test: test != '', set(itertools.chain.from_iterable(map(lambda tag: test_map[tag], tags)))))
+    except KeyError as e:
+        print(f'Tag {e} is not present in tests_for_tag.cfg')
+        exit(1)
+
+
+class TestingType(Enum):
+    functional = 'functional'
+    performance = 'performance'
+
+    def __str__(self):
+        return self.value
+
+
+if __name__ == '__main__':
+    parser = argparse.ArgumentParser(
+        description='Take a template execution file and add the relevant tests'
+                    ' for the given tags to it, creating a new file.')
+    parser.add_argument('config_file_path', help='The path to tests_for_tag.cfg', default='config/tests_for_tag.cfg')
+    parser.add_argument('template_execution_file', help='The path to the execution file to use as a template')
+    parser.add_argument('output_path', help='The path to the output execution file')
+    parser.add_argument('testing_type', type=TestingType, choices=list(TestingType),
+                        help='What type of testing to create an execution file for')
+    parser.add_argument('tags', metavar='tag', type=str, nargs='*', help='The tags to create an execution file for.')
+
+    args = parser.parse_args()
+
+    tag_to_test_map_parser = ConfigParser()
+    tag_to_test_map_parser.read(args.config_file_path)
+
+    template_execution_file_parser = ConfigParser()
+    template_execution_file_parser.read(args.template_execution_file)
+
+    test_map = {key: parse_comma_delimited_list_from_string(value.strip()) for key, value in
+                tag_to_test_map_parser[str(args.testing_type)].items()}
+
+    tests = map_tags_to_tests(args.tags, test_map)
+
+    try:
+        output_file = open(args.output_path, 'x')
+    except FileExistsError:
+        output_file = open(args.output_path, 'w')
+
+    for execution_plan in template_execution_file_parser:
+        # The DEFAULT section is always present and contains top-level items, so it needs to be ignored
+        if execution_plan != 'DEFAULT':
+            test_allowlist = parse_comma_delimited_list_from_string(
+                template_execution_file_parser[execution_plan]['test_suites'])
+            tests_to_run = list(set(test_allowlist).intersection(tests))
+            tests_to_run.sort()
+            template_execution_file_parser[execution_plan]['test_suites'] = ", ".join(tests_to_run)
+
+            if args.testing_type == TestingType.functional:
+                template_execution_file_parser[execution_plan]['parameters'] += ':func=true:perf=false'
+            elif args.testing_type == TestingType.performance:
+                template_execution_file_parser[execution_plan]['parameters'] += ':func=false:perf=true'
+            else:
+                # This code should be unreachable, since this is checked at the top of the file
+                print("Fatal error: testing type is neither performance nor functional", file=sys.stderr)
+                exit(1)
+
+    template_execution_file_parser.write(output_file)
diff --git a/tools/patch_parser.py b/tools/patch_parser.py
index 01fc55d..cc91cd2 100755
--- a/tools/patch_parser.py
+++ b/tools/patch_parser.py
@@ -1,27 +1,58 @@
 #!/usr/bin/env python3
 
-import itertools
-import sys
+import argparse
 from configparser import ConfigParser
 from typing import List, Dict, Set
 
+import itertools
+# BSD LICENSE
+#
+# Copyright(c) 2020 Intel Corporation. All rights reserved.
+# Copyright © 2018[, 2019] The University of New Hampshire. All rights reserved.
+# All rights reserved.
+#
+# Redistribution and use in source and binary forms, with or without
+# modification, are permitted provided that the following conditions
+# are met:
+#
+#   * Redistributions of source code must retain the above copyright
+#     notice, this list of conditions and the following disclaimer.
+#   * Redistributions in binary form must reproduce the above copyright
+#     notice, this list of conditions and the following disclaimer in
+#     the documentation and/or other materials provided with the
+#     distribution.
+#   * Neither the name of Intel Corporation nor the names of its
+#     contributors may be used to endorse or promote products derived
+#     from this software without specific prior written permission.
+#
+# THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
+# "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
+# LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
+# A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
+# OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
+# SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
+# LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
+# DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
+# THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+# (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
+# OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+import sys
 
-def get_patch_files(patch_file: str) -> List[str]:
+try:
+    import whatthepatch
+except ImportError:
+    print("Please install whatthepatch, a patch parsing library", file=sys.stderr)
+    exit(1)
+
+
+def get_changed_files_in_patch(patch_file: str) -> List[str]:
     with open(patch_file, 'r') as f:
-        lines = list(itertools.takewhile(
-            lambda line: line.strip().endswith('+') or line.strip().endswith('-'),
-            itertools.dropwhile(
-                lambda line: not line.strip().startswith("---"),
-                f.readlines()
-            )
-        ))
-        filenames = map(lambda line: line.strip().split(' ')[0], lines)
-        # takewhile includes the --- which starts the filenames
-        return list(filenames)[1:]
+        filenames = map(lambda diff: diff.header.new_path, whatthepatch.parse_patch(f.read()))
+        return list(filenames)
 
 
 def get_all_files_from_patches(patch_files: List[str]) -> Set[str]:
-    return set(itertools.chain.from_iterable(map(get_patch_files, patch_files)))
+    return set(itertools.chain.from_iterable(map(get_changed_files_in_patch, patch_files)))
 
 
 def parse_comma_delimited_list_from_string(mod_str: str) -> List[str]:
@@ -47,18 +78,22 @@ def get_tags_for_patches(patch_files: Set[str], dir_attrs: Dict[str, Set[str]])
     ))
 
 
-if len(sys.argv) < 3:
-    print("usage: patch_parser.py <path to patch_parser.cfg> <patch file>...")
-    exit(1)
+if __name__ == '__main__':
+    parser = argparse.ArgumentParser(
+        description='Takes a patch file and a config file and creates a list of tags for that patch')
+    parser.add_argument('config_file_path', help='The path to patch_parser.cfg', default='config/patch_parser.cfg')
+    parser.add_argument('patch_file_paths', help='A list of patch files', type=str, metavar='patch file', nargs='+')
+
+    args = parser.parse_args()
 
-conf_obj = ConfigParser()
-conf_obj.read(sys.argv[1])
+    conf_obj = ConfigParser()
+    conf_obj.read(args.config_file_path)
 
-patch_files = get_all_files_from_patches(sys.argv[2:])
-dir_attrs = get_dictionary_attributes_from_config_file(conf_obj)
-priority_list = parse_comma_delimited_list_from_string(conf_obj['Priority']['priority_list'])
+    patch_files = get_all_files_from_patches(args.patch_file_paths)
+    dir_attrs = get_dictionary_attributes_from_config_file(conf_obj)
+    priority_list = parse_comma_delimited_list_from_string(conf_obj['Priority']['priority_list'])
 
-unordered_tags: Set[str] = get_tags_for_patches(patch_files, dir_attrs)
-ordered_tags: List[str] = [tag for tag in priority_list if tag in unordered_tags]
+    unordered_tags: Set[str] = get_tags_for_patches(patch_files, dir_attrs)
+    ordered_tags: List[str] = [tag for tag in priority_list if tag in unordered_tags]
 
-print("\n".join(ordered_tags))
+    print("\n".join(ordered_tags))
-- 
2.27.0


^ permalink raw reply	[flat|nested] 9+ messages in thread

* Re: [dpdk-ci] [PATCH V2] patch-tagging: Added tool to convert tags into dts execution file
  2021-04-15 20:42       ` Aaron Conole
@ 2021-04-15 21:14         ` Owen Hilyard
  0 siblings, 0 replies; 9+ messages in thread
From: Owen Hilyard @ 2021-04-15 21:14 UTC (permalink / raw)
  To: Aaron Conole; +Cc: ci, Thomas Monjalon

[-- Attachment #1: Type: text/plain, Size: 7409 bytes --]

It seems like the v2 never got properly sent. It's in my outbox but the
mailing list archive doesn't have a record of it. I've sent in another
patch with a couple of stability changes and minor feature tweaks that were
made. That should complete the list of features in the original commit.

Sorry about that

On Thu, Apr 15, 2021 at 4:42 PM Aaron Conole <aconole@redhat.com> wrote:

> Owen Hilyard <ohilyard@iol.unh.edu> writes:
>
> > Please ignore this patch. I messed up directories and this patch was not
> created from the correct file.
>
> Is there an update for this that I missed?
>
> > On Thu, Jan 14, 2021 at 12:59 PM <ohilyard@iol.unh.edu> wrote:
> >
> >  Fixed parsing bugs with the tag parsing script
> >          Parsing the patch file directly proved to be too unreliabler,
> so the approach to patch parsing was changed so that
> >  it relies instead on using a directory with those patches already
> applied in them and then use a git diff command to get
> >  the changed files. This avoids needing to parse the patch files
> directly and should be more reliable.
> >
> >  Added a tool to create an execution file from tags using another
> execution file as a template
> >          This tool is intended to allow a list of patch tags to be used
> in combination with an execution file containing
> >  information on crbs to only change which test suites are run. This tool
> enables using the mappings in the
> >  tests_for_tag.cfg file to run only necessary tests. At present, the
> mappings represent how tests are currently run at the
> >  UNH-IOL Community Lab, with the same tests being run for every patch,
> with the requested change from bug 511
> >  where documentation changes do not cause any testing.
> >
> >  Signed-off-by: Owen Hilyard <ohilyard@iol.unh.edu>
> >  ---
> >   config/tests_for_tag.cfg | 19 ++++++++++++--
> >   tools/patch_parser.py    | 54 ++++++++++++++++++++++++----------------
> >   2 files changed, 49 insertions(+), 24 deletions(-)
> >
> >  diff --git a/config/tests_for_tag.cfg b/config/tests_for_tag.cfg
> >  index 77f797f..7d95c4a 100644
> >  --- a/config/tests_for_tag.cfg
> >  +++ b/config/tests_for_tag.cfg
> >  @@ -5,11 +5,26 @@ core = dynamic_config,
> >          mtu_update,
> >          scatter,
> >          stats_checks,
> >  +       unit_tests_mbuf
> >  +driver = dynamic_config,
> >  +       link_status_interrupt,
> >  +       mac_filter,
> >  +       mtu_update,
> >  +       scatter,
> >  +       stats_checks,
> >  +       unit_tests_mbuf
> >  +application = dynamic_config,
> >  +       link_status_interrupt,
> >  +       mac_filter,
> >  +       mtu_update,
> >  +       scatter,
> >          stats_checks,
> >          unit_tests_mbuf
> >  -
> >   ; Nothing in documentation
> >   documentation =
> >
> >   [performance]
> >  -core = nic_single_core_perf,
> >  +core = nic_single_core_perf
> >  +driver = nic_single_core_perf
> >  +application = nic_single_core_perf
> >  +documentation =
> >  diff --git a/tools/patch_parser.py b/tools/patch_parser.py
> >  index 80b8194..73df91f 100755
> >  --- a/tools/patch_parser.py
> >  +++ b/tools/patch_parser.py
> >  @@ -31,30 +31,39 @@
> >   # THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
> >   # (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
> >   # OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
> >  -
> >  +import subprocess
> >
> >   import itertools
> >   from configparser import ConfigParser
> >   from typing import List, Dict, Set
> >   import argparse
> >  +import re
> >
> >   def get_patch_files(patch_file: str) -> List[str]:
> >  +    file_match_pattern = re.compile(r"[\w\-\/.]+ +\| +\d+ [+\-]+")
> >       with open(patch_file, 'r') as f:
> >  -        lines = list(itertools.takewhile(
> >  -            lambda line: line.strip().endswith('+') or
> line.strip().endswith('-'),
> >  -            itertools.dropwhile(
> >  -                lambda line: not line.strip().startswith("---"),
> >  -                f.readlines()
> >  +        file_lines = f.readlines()
> >  +        lines = list(filter(
> >  +            lambda line: file_match_pattern.match(line.strip()),
> >  +            itertools.takewhile(
> >  +                lambda line: not line.startswith('---'),
> >  +                list(
> >  +                    itertools.dropwhile(
> >  +                        lambda line: not
> line.strip().startswith("---"),
> >  +                        file_lines
> >  +                    )
> >  +                )[1:]
> >               )
> >           ))
> >           filenames = map(lambda line: line.strip().split(' ')[0], lines)
> >  -        # takewhile includes the --- which starts the filenames
> >  -        return list(filenames)[1:]
> >  +        return list(filenames)
> >
> >   def get_all_files_from_patches(patch_files: List[str]) -> Set[str]:
> >  -    return set(itertools.chain.from_iterable(map(get_patch_files,
> patch_files)))
> >  +    num_patch_files: int = len(patch_files)
> >  +    files = subprocess.check_output(f"cd dpdk && git diff --name-only
> HEAD~{num_patch_files}", shell=True).decode
> >  ('utf-8').splitlines()
> >  +    return set(files)
> >
> >   def parse_comma_delimited_list_from_string(mod_str: str) -> List[str]:
> >  @@ -80,21 +89,22 @@ def get_tags_for_patches(patch_files: Set[str],
> dir_attrs: Dict[str, Set[str]])
> >       ))
> >
> >  -parser = argparse.ArgumentParser(
> >  -    description='Takes a patch file and a config file and creates a
> list of tags for that patch')
> >  -parser.add_argument('config_file_path', help='The path to
> patch_parser.cfg', default='config/patch_parser.cfg')
> >  -parser.add_argument('patch_file_paths', help='A list of patch files',
> type=str, metavar='patch file', nargs='+')
> >  +if __name__ == '__main__':
> >  +    parser = argparse.ArgumentParser(
> >  +        description='Takes a patch file and a config file and creates
> a list of tags for that patch')
> >  +    parser.add_argument('config_file_path', help='The path to
> patch_parser.cfg', default='config/patch_parser.cfg')
> >  +    parser.add_argument('patch_file_paths', help='A list of patch
> files', type=str, metavar='patch file', nargs='+')
> >
> >  -args = parser.parse_args()
> >  +    args = parser.parse_args()
> >
> >  -conf_obj = ConfigParser()
> >  -conf_obj.read(args.config_file_path)
> >  +    conf_obj = ConfigParser()
> >  +    conf_obj.read(args.config_file_path)
> >
> >  -patch_files = get_all_files_from_patches(args.patch_file_paths)
> >  -dir_attrs = get_dictionary_attributes_from_config_file(conf_obj)
> >  -priority_list =
> parse_comma_delimited_list_from_string(conf_obj['Priority']['priority_list'])
> >  +    patch_files = get_all_files_from_patches(args.patch_file_paths)
> >  +    dir_attrs = get_dictionary_attributes_from_config_file(conf_obj)
> >  +    priority_list =
> parse_comma_delimited_list_from_string(conf_obj['Priority']['priority_list'])
> >
> >  -unordered_tags: Set[str] = get_tags_for_patches(patch_files, dir_attrs)
> >  -ordered_tags: List[str] = [tag for tag in priority_list if tag in
> unordered_tags]
> >  +    unordered_tags: Set[str] = get_tags_for_patches(patch_files,
> dir_attrs)
> >  +    ordered_tags: List[str] = [tag for tag in priority_list if tag in
> unordered_tags]
> >
> >  -print("\n".join(ordered_tags))
> >  +    print("\n".join(ordered_tags))
> >  --
> >  2.27.0
>
>

[-- Attachment #2: Type: text/html, Size: 9444 bytes --]

^ permalink raw reply	[flat|nested] 9+ messages in thread

* Re: [dpdk-ci] [PATCH] patch parsing: Added library for parsing patch files
  2021-04-15 21:11   ` [dpdk-ci] [PATCH] patch parsing: Added library for parsing " ohilyard
@ 2021-04-15 21:37     ` Aaron Conole
  0 siblings, 0 replies; 9+ messages in thread
From: Aaron Conole @ 2021-04-15 21:37 UTC (permalink / raw)
  To: ohilyard; +Cc: ci

ohilyard@iol.unh.edu writes:

> From: Owen Hilyard <ohilyard@iol.unh.edu>
>
> Due to the number of edge cases, there were difficulties doing the parsing using pattern matching, so a dedicated library, whatthepatch, is now used for parsing the patches to ensure correctness.
>
> Also added the default behavior that if no tags are given, create_new_execution_file_from_tags.py will act as though all possible tags were given. This is to allow patches where no tags could be found to still have test coverage.
> ---

Missing sign-off.

Also, please reformat the commit message, preferring 75 column width.
Unlike code, commit logs are very sensitive when viewing history, etc.

>  config/tests_for_tag.cfg                     |  30 +++++
>  requirements.txt                             |   1 +
>  tools/create_new_execution_file_from_tags.py | 114 +++++++++++++++++++
>  tools/patch_parser.py                        |  85 ++++++++++----
>  4 files changed, 205 insertions(+), 25 deletions(-)
>  create mode 100644 config/tests_for_tag.cfg
>  create mode 100644 requirements.txt
>  create mode 100755 tools/create_new_execution_file_from_tags.py
>
> diff --git a/config/tests_for_tag.cfg b/config/tests_for_tag.cfg
> new file mode 100644
> index 0000000..7d95c4a
> --- /dev/null
> +++ b/config/tests_for_tag.cfg
> @@ -0,0 +1,30 @@
> +[functional]
> +core = dynamic_config,
> +       link_status_interrupt,
> +       mac_filter,
> +       mtu_update,
> +       scatter,
> +       stats_checks,
> +       unit_tests_mbuf
> +driver = dynamic_config,
> +       link_status_interrupt,
> +       mac_filter,
> +       mtu_update,
> +       scatter,
> +       stats_checks,
> +       unit_tests_mbuf
> +application = dynamic_config,
> +       link_status_interrupt,
> +       mac_filter,
> +       mtu_update,
> +       scatter,
> +       stats_checks,
> +       unit_tests_mbuf
> +; Nothing in documentation
> +documentation =
> +
> +[performance]
> +core = nic_single_core_perf
> +driver = nic_single_core_perf
> +application = nic_single_core_perf
> +documentation =
> diff --git a/requirements.txt b/requirements.txt
> new file mode 100644
> index 0000000..f20067d
> --- /dev/null
> +++ b/requirements.txt
> @@ -0,0 +1 @@
> +whatthepatch==1.0.2
> \ No newline at end of file
> diff --git a/tools/create_new_execution_file_from_tags.py b/tools/create_new_execution_file_from_tags.py
> new file mode 100755
> index 0000000..d1d4447
> --- /dev/null
> +++ b/tools/create_new_execution_file_from_tags.py
> @@ -0,0 +1,114 @@
> +#!/usr/bin/env python3
> +
> +# BSD LICENSE
> +#
> +# Copyright(c) 2020 Intel Corporation. All rights reserved.
> +# Copyright © 2018[, 2019] The University of New Hampshire. All rights reserved.
> +# All rights reserved.
> +#
> +# Redistribution and use in source and binary forms, with or without
> +# modification, are permitted provided that the following conditions
> +# are met:
> +#
> +#   * Redistributions of source code must retain the above copyright
> +#     notice, this list of conditions and the following disclaimer.
> +#   * Redistributions in binary form must reproduce the above copyright
> +#     notice, this list of conditions and the following disclaimer in
> +#     the documentation and/or other materials provided with the
> +#     distribution.
> +#   * Neither the name of Intel Corporation nor the names of its
> +#     contributors may be used to endorse or promote products derived
> +#     from this software without specific prior written permission.
> +#
> +# THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
> +# "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
> +# LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
> +# A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
> +# OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
> +# SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
> +# LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
> +# DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
> +# THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
> +# (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
> +# OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
> +import sys
> +from enum import Enum
> +
> +import itertools
> +from configparser import ConfigParser
> +from typing import List, Dict, Set
> +import argparse
> +
> +
> +def parse_comma_delimited_list_from_string(mod_str: str) -> List[str]:
> +    return list(map(str.strip, mod_str.split(',')))
> +
> +
> +def map_tags_to_tests(tags: List[str], test_map: Dict[str, List[str]]) -> Set[str]:
> +    """
> +    Returns a list that is the union of all of the map lookups.
> +    """
> +    try:
> +        return set(
> +            filter(lambda test: test != '', set(itertools.chain.from_iterable(map(lambda tag: test_map[tag], tags)))))
> +    except KeyError as e:
> +        print(f'Tag {e} is not present in tests_for_tag.cfg')
> +        exit(1)
> +
> +
> +class TestingType(Enum):
> +    functional = 'functional'
> +    performance = 'performance'
> +
> +    def __str__(self):
> +        return self.value
> +
> +
> +if __name__ == '__main__':
> +    parser = argparse.ArgumentParser(
> +        description='Take a template execution file and add the relevant tests'
> +                    ' for the given tags to it, creating a new file.')
> +    parser.add_argument('config_file_path', help='The path to tests_for_tag.cfg', default='config/tests_for_tag.cfg')
> +    parser.add_argument('template_execution_file', help='The path to the execution file to use as a template')
> +    parser.add_argument('output_path', help='The path to the output execution file')
> +    parser.add_argument('testing_type', type=TestingType, choices=list(TestingType),
> +                        help='What type of testing to create an execution file for')
> +    parser.add_argument('tags', metavar='tag', type=str, nargs='*', help='The tags to create an execution file for.')
> +
> +    args = parser.parse_args()
> +
> +    tag_to_test_map_parser = ConfigParser()
> +    tag_to_test_map_parser.read(args.config_file_path)
> +
> +    template_execution_file_parser = ConfigParser()
> +    template_execution_file_parser.read(args.template_execution_file)
> +
> +    test_map = {key: parse_comma_delimited_list_from_string(value.strip()) for key, value in
> +                tag_to_test_map_parser[str(args.testing_type)].items()}
> +
> +    tests = map_tags_to_tests(args.tags, test_map)
> +
> +    try:
> +        output_file = open(args.output_path, 'x')
> +    except FileExistsError:
> +        output_file = open(args.output_path, 'w')
> +
> +    for execution_plan in template_execution_file_parser:
> +        # The DEFAULT section is always present and contains top-level items, so it needs to be ignored
> +        if execution_plan != 'DEFAULT':
> +            test_allowlist = parse_comma_delimited_list_from_string(
> +                template_execution_file_parser[execution_plan]['test_suites'])
> +            tests_to_run = list(set(test_allowlist).intersection(tests))
> +            tests_to_run.sort()
> +            template_execution_file_parser[execution_plan]['test_suites'] = ", ".join(tests_to_run)
> +
> +            if args.testing_type == TestingType.functional:
> +                template_execution_file_parser[execution_plan]['parameters'] += ':func=true:perf=false'
> +            elif args.testing_type == TestingType.performance:
> +                template_execution_file_parser[execution_plan]['parameters'] += ':func=false:perf=true'
> +            else:
> +                # This code should be unreachable, since this is checked at the top of the file
> +                print("Fatal error: testing type is neither performance nor functional", file=sys.stderr)
> +                exit(1)
> +
> +    template_execution_file_parser.write(output_file)
> diff --git a/tools/patch_parser.py b/tools/patch_parser.py
> index 01fc55d..cc91cd2 100755
> --- a/tools/patch_parser.py
> +++ b/tools/patch_parser.py
> @@ -1,27 +1,58 @@
>  #!/usr/bin/env python3
>  
> -import itertools
> -import sys
> +import argparse
>  from configparser import ConfigParser
>  from typing import List, Dict, Set
>  
> +import itertools
> +# BSD LICENSE
> +#
> +# Copyright(c) 2020 Intel Corporation. All rights reserved.
> +# Copyright © 2018[, 2019] The University of New Hampshire. All rights reserved.
> +# All rights reserved.
> +#
> +# Redistribution and use in source and binary forms, with or without
> +# modification, are permitted provided that the following conditions
> +# are met:
> +#
> +#   * Redistributions of source code must retain the above copyright
> +#     notice, this list of conditions and the following disclaimer.
> +#   * Redistributions in binary form must reproduce the above copyright
> +#     notice, this list of conditions and the following disclaimer in
> +#     the documentation and/or other materials provided with the
> +#     distribution.
> +#   * Neither the name of Intel Corporation nor the names of its
> +#     contributors may be used to endorse or promote products derived
> +#     from this software without specific prior written permission.
> +#
> +# THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
> +# "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
> +# LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
> +# A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
> +# OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
> +# SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
> +# LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
> +# DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
> +# THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
> +# (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
> +# OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
> +import sys
>  
> -def get_patch_files(patch_file: str) -> List[str]:
> +try:
> +    import whatthepatch
> +except ImportError:
> +    print("Please install whatthepatch, a patch parsing library", file=sys.stderr)
> +    exit(1)
> +
> +
> +def get_changed_files_in_patch(patch_file: str) -> List[str]:
>      with open(patch_file, 'r') as f:
> -        lines = list(itertools.takewhile(
> -            lambda line: line.strip().endswith('+') or line.strip().endswith('-'),
> -            itertools.dropwhile(
> -                lambda line: not line.strip().startswith("---"),
> -                f.readlines()
> -            )
> -        ))
> -        filenames = map(lambda line: line.strip().split(' ')[0], lines)
> -        # takewhile includes the --- which starts the filenames
> -        return list(filenames)[1:]
> +        filenames = map(lambda diff: diff.header.new_path, whatthepatch.parse_patch(f.read()))
> +        return list(filenames)
>  
>  
>  def get_all_files_from_patches(patch_files: List[str]) -> Set[str]:
> -    return set(itertools.chain.from_iterable(map(get_patch_files, patch_files)))
> +    return set(itertools.chain.from_iterable(map(get_changed_files_in_patch, patch_files)))
>  
>  
>  def parse_comma_delimited_list_from_string(mod_str: str) -> List[str]:
> @@ -47,18 +78,22 @@ def get_tags_for_patches(patch_files: Set[str], dir_attrs: Dict[str, Set[str]])
>      ))
>  
>  
> -if len(sys.argv) < 3:
> -    print("usage: patch_parser.py <path to patch_parser.cfg> <patch file>...")
> -    exit(1)
> +if __name__ == '__main__':
> +    parser = argparse.ArgumentParser(
> +        description='Takes a patch file and a config file and creates a list of tags for that patch')
> +    parser.add_argument('config_file_path', help='The path to patch_parser.cfg', default='config/patch_parser.cfg')
> +    parser.add_argument('patch_file_paths', help='A list of patch files', type=str, metavar='patch file', nargs='+')
> +
> +    args = parser.parse_args()
>  
> -conf_obj = ConfigParser()
> -conf_obj.read(sys.argv[1])
> +    conf_obj = ConfigParser()
> +    conf_obj.read(args.config_file_path)
>  
> -patch_files = get_all_files_from_patches(sys.argv[2:])
> -dir_attrs = get_dictionary_attributes_from_config_file(conf_obj)
> -priority_list = parse_comma_delimited_list_from_string(conf_obj['Priority']['priority_list'])
> +    patch_files = get_all_files_from_patches(args.patch_file_paths)
> +    dir_attrs = get_dictionary_attributes_from_config_file(conf_obj)
> +    priority_list = parse_comma_delimited_list_from_string(conf_obj['Priority']['priority_list'])
>  
> -unordered_tags: Set[str] = get_tags_for_patches(patch_files, dir_attrs)
> -ordered_tags: List[str] = [tag for tag in priority_list if tag in unordered_tags]
> +    unordered_tags: Set[str] = get_tags_for_patches(patch_files, dir_attrs)
> +    ordered_tags: List[str] = [tag for tag in priority_list if tag in unordered_tags]
>  
> -print("\n".join(ordered_tags))
> +    print("\n".join(ordered_tags))


^ permalink raw reply	[flat|nested] 9+ messages in thread

end of thread, other threads:[~2021-04-15 21:37 UTC | newest]

Thread overview: 9+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2020-12-04 19:45 [dpdk-ci] [PATCH] ci: added patch parser for patch files Owen Hilyard
2021-01-14 16:53 ` Owen Hilyard
2021-01-14 17:59   ` [dpdk-ci] [PATCH V2] patch-tagging: Added tool to convert tags into dts execution file ohilyard
2021-01-14 18:19     ` Owen Hilyard
2021-04-15 20:42       ` Aaron Conole
2021-04-15 21:14         ` Owen Hilyard
2021-04-15 20:38 ` [dpdk-ci] [PATCH] ci: added patch parser for patch files Aaron Conole
2021-04-15 21:11   ` [dpdk-ci] [PATCH] patch parsing: Added library for parsing " ohilyard
2021-04-15 21:37     ` Aaron Conole

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).