DPDK patches and discussions
 help / color / mirror / Atom feed
From: Anatoly Burakov <anatoly.burakov@intel.com>
To: dev@dpdk.org
Cc: Pawel Modrak <pawelx.modrak@intel.com>,
	john.mcnamara@intel.com, ray.kinsella@intel.com,
	bruce.richardson@intel.com, thomas@monjalon.net,
	david.marchand@redhat.com
Subject: [dpdk-dev] [PATCH v5 02/10] buildtools: add script for updating symbols abi version
Date: Thu, 24 Oct 2019 10:46:39 +0100	[thread overview]
Message-ID: <0bdec3d7e6659e3a41b9888a8b3a13eb428426f6.1571910363.git.anatoly.burakov@intel.com> (raw)
In-Reply-To: <cover.1571910363.git.anatoly.burakov@intel.com>
In-Reply-To: <cover.1571910363.git.anatoly.burakov@intel.com>

From: Pawel Modrak <pawelx.modrak@intel.com>

Add a script that automatically merges all stable ABI's under one
ABI section with the new version, while leaving experimental
section exactly as it is.

Signed-off-by: Pawel Modrak <pawelx.modrak@intel.com>
Signed-off-by: Anatoly Burakov <anatoly.burakov@intel.com>
Acked-by: Bruce Richardson <bruce.richardson@intel.com>
---

Notes:
    v3:
    - Add comments to regex patterns
    
    v2:
    - Reworked script to be pep8-compliant and more reliable

 buildtools/update_version_map_abi.py | 170 +++++++++++++++++++++++++++
 1 file changed, 170 insertions(+)
 create mode 100755 buildtools/update_version_map_abi.py

diff --git a/buildtools/update_version_map_abi.py b/buildtools/update_version_map_abi.py
new file mode 100755
index 0000000000..50283e6a3d
--- /dev/null
+++ b/buildtools/update_version_map_abi.py
@@ -0,0 +1,170 @@
+#!/usr/bin/env python
+# SPDX-License-Identifier: BSD-3-Clause
+# Copyright(c) 2019 Intel Corporation
+
+"""
+A Python program to update the ABI version and function names in a DPDK
+lib_*_version.map file. Called from the buildtools/update_abi.sh utility.
+"""
+
+from __future__ import print_function
+import argparse
+import sys
+import re
+
+
+def __parse_map_file(f_in):
+    # match function name, followed by semicolon, followed by EOL, optionally
+    # with whitespace inbetween each item
+    func_line_regex = re.compile(r"\s*"
+                                 r"(?P<func>[a-zA-Z_0-9]+)"
+                                 r"\s*"
+                                 r";"
+                                 r"\s*"
+                                 r"$")
+    # match section name, followed by opening bracked, followed by EOL,
+    # optionally with whitespace inbetween each item
+    section_begin_regex = re.compile(r"\s*"
+                                     r"(?P<version>[a-zA-Z0-9_\.]+)"
+                                     r"\s*"
+                                     r"{"
+                                     r"\s*"
+                                     r"$")
+    # match closing bracket, optionally followed by section name (for when we
+    # inherit from another ABI version), followed by semicolon, followed by
+    # EOL, optionally with whitespace inbetween each item
+    section_end_regex = re.compile(r"\s*"
+                                   r"}"
+                                   r"\s*"
+                                   r"(?P<parent>[a-zA-Z0-9_\.]+)?"
+                                   r"\s*"
+                                   r";"
+                                   r"\s*"
+                                   r"$")
+
+    # for stable ABI, we don't care about which version introduced which
+    # function, we just flatten the list. there are dupes in certain files, so
+    # use a set instead of a list
+    stable_lines = set()
+    # copy experimental section as is
+    experimental_lines = []
+    is_experimental = False
+
+    # gather all functions
+    for line in f_in:
+        # clean up the line
+        line = line.strip('\n').strip()
+
+        # is this an end of section?
+        match = section_end_regex.match(line)
+        if match:
+            # whatever section this was, it's not active any more
+            is_experimental = False
+            continue
+
+        # if we're in the middle of experimental section, we need to copy
+        # the section verbatim, so just add the line
+        if is_experimental:
+            experimental_lines += [line]
+            continue
+
+        # skip empty lines
+        if not line:
+            continue
+
+        # is this a beginning of a new section?
+        match = section_begin_regex.match(line)
+        if match:
+            cur_section = match.group("version")
+            # is it experimental?
+            is_experimental = cur_section == "EXPERIMENTAL"
+            continue
+
+        # is this a function?
+        match = func_line_regex.match(line)
+        if match:
+            stable_lines.add(match.group("func"))
+
+    return stable_lines, experimental_lines
+
+
+def __regenerate_map_file(f_out, abi_version, stable_lines,
+                          experimental_lines):
+    # print ABI version header
+    print("DPDK_{} {{".format(abi_version), file=f_out)
+
+    if stable_lines:
+        # print global section
+        print("\tglobal:", file=f_out)
+        # blank line
+        print(file=f_out)
+
+        # print all stable lines, alphabetically sorted
+        for line in sorted(stable_lines):
+            print("\t{};".format(line), file=f_out)
+
+        # another blank line
+        print(file=f_out)
+
+    # print local section
+    print("\tlocal: *;", file=f_out)
+
+    # end stable version
+    print("};", file=f_out)
+
+    # do we have experimental lines?
+    if not experimental_lines:
+        return
+
+    # another blank line
+    print(file=f_out)
+
+    # start experimental section
+    print("EXPERIMENTAL {", file=f_out)
+
+    # print all experimental lines as they were
+    for line in experimental_lines:
+        # don't print empty whitespace
+        if not line:
+            print("", file=f_out)
+        else:
+            print("\t{}".format(line), file=f_out)
+
+    # end section
+    print("};", file=f_out)
+
+
+def __main():
+    arg_parser = argparse.ArgumentParser(
+        description='Merge versions in linker version script.')
+
+    arg_parser.add_argument("map_file", type=str,
+                            help='path to linker version script file '
+                                 '(pattern: *version.map)')
+    arg_parser.add_argument("abi_version", type=str,
+                            help='target ABI version (pattern: MAJOR.MINOR)')
+
+    parsed = arg_parser.parse_args()
+
+    if not parsed.map_file.endswith('version.map'):
+        print("Invalid input file: {}".format(parsed.map_file),
+              file=sys.stderr)
+        arg_parser.print_help()
+        sys.exit(1)
+
+    if not re.match(r"\d{1,2}\.\d{1,2}", parsed.abi_version):
+        print("Invalid ABI version: {}".format(parsed.abi_version),
+              file=sys.stderr)
+        arg_parser.print_help()
+        sys.exit(1)
+
+    with open(parsed.map_file) as f_in:
+        stable_lines, experimental_lines = __parse_map_file(f_in)
+
+    with open(parsed.map_file, 'w') as f_out:
+        __regenerate_map_file(f_out, parsed.abi_version, stable_lines,
+                              experimental_lines)
+
+
+if __name__ == "__main__":
+    __main()
-- 
2.17.1

  parent reply	other threads:[~2019-10-24  9:47 UTC|newest]

Thread overview: 131+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2019-09-30  9:21 [dpdk-dev] [PATCH 1/8] config: change ABI versioning for global Marcin Baran
2019-09-30  9:21 ` [dpdk-dev] [PATCH 2/8] buildtools: scripts for updating symbols abi version Marcin Baran
2019-09-30  9:21 ` [dpdk-dev] [PATCH 3/8] buildtools: add ABI versioning check script Marcin Baran
2019-09-30 10:27   ` Bruce Richardson
2019-09-30  9:21 ` [dpdk-dev] [PATCH 4/8] build: change ABI version to 20.0 Marcin Baran
2019-09-30  9:21 ` [dpdk-dev] [PATCH 5/8] lib: remove dead code from timer Marcin Baran
2019-09-30  9:21 ` [dpdk-dev] [PATCH 6/8] lib: remove dead code from lpm Marcin Baran
2019-09-30  9:21 ` [dpdk-dev] [PATCH 7/8] lib: remove dead code from distributor Marcin Baran
2019-09-30  9:21 ` [dpdk-dev] [PATCH 8/8] lib: change function suffix in distributor Marcin Baran
2019-10-16 12:43 ` [dpdk-dev] [PATCH v2 00/10] Implement the new ABI policy and add helper scripts Anatoly Burakov
2019-10-16 17:03   ` [dpdk-dev] [PATCH v3 0/9] " Anatoly Burakov
2019-10-17  8:50     ` Bruce Richardson
2019-10-17 14:31     ` [dpdk-dev] [PATCH v4 00/10] " Anatoly Burakov
2019-10-24  9:46       ` [dpdk-dev] [PATCH v5 " Anatoly Burakov
2019-11-06 16:54         ` [dpdk-dev] [PATCH v6 " Anatoly Burakov
2019-11-08 16:25           ` [dpdk-dev] [PATCH v7 " Anatoly Burakov
2019-11-20 17:23             ` [dpdk-dev] [PATCH v8 00/12] " Anatoly Burakov
2019-11-20 20:17               ` Thomas Monjalon
2019-11-20 22:13                 ` David Marchand
2019-11-21 10:22                   ` Burakov, Anatoly
2019-11-21 13:24                 ` Kinsella, Ray
2019-11-20 17:23             ` [dpdk-dev] [PATCH v8 01/12] config: change ABI versioning to global Anatoly Burakov
2019-11-20 19:51               ` David Marchand
2019-11-20 22:01                 ` David Marchand
2019-11-20 17:23             ` [dpdk-dev] [PATCH v8 02/12] config: remove CONFIG_RTE_MAJOR_ABI option Anatoly Burakov
2019-11-20 17:23             ` [dpdk-dev] [PATCH v8 03/12] build: remove individual library versions Anatoly Burakov
2019-11-20 19:35               ` David Marchand
2019-11-20 17:23             ` [dpdk-dev] [PATCH v8 04/12] buildtools: add script for updating symbols abi version Anatoly Burakov
2019-11-20 20:05               ` David Marchand
2019-11-20 17:23             ` [dpdk-dev] [PATCH v8 05/12] buildtools: add ABI update shell script Anatoly Burakov
2019-11-20 17:23             ` [dpdk-dev] [PATCH v8 06/12] timer: remove deprecated code Anatoly Burakov
2019-11-20 17:23             ` [dpdk-dev] [PATCH v8 07/12] lpm: " Anatoly Burakov
2019-11-20 17:23             ` [dpdk-dev] [PATCH v8 08/12] distributor: " Anatoly Burakov
2019-11-20 17:23             ` [dpdk-dev] [PATCH v8 09/12] distributor: rename v2.0 ABI to _single suffix Anatoly Burakov
2019-11-20 17:23             ` [dpdk-dev] [PATCH v8 10/12] drivers/octeontx: add missing public symbol Anatoly Burakov
2019-11-20 17:23             ` [dpdk-dev] [PATCH v8 11/12] build: change ABI version to 20.0 Anatoly Burakov
2019-11-20 20:47               ` David Marchand
2019-11-20 17:23             ` [dpdk-dev] [PATCH v8 12/12] buildtools: add ABI versioning check script Anatoly Burakov
2019-11-08 16:25           ` [dpdk-dev] [PATCH v7 01/10] config: change ABI versioning to global Anatoly Burakov
2019-11-19 13:53             ` Thomas Monjalon
2019-11-19 15:48               ` Burakov, Anatoly
2019-11-20 12:10             ` Kinsella, Ray
2019-11-20 13:31               ` Thomas Monjalon
2019-11-20 14:10                 ` Kinsella, Ray
2019-11-08 16:25           ` [dpdk-dev] [PATCH v7 02/10] buildtools: add script for updating symbols abi version Anatoly Burakov
2019-11-19 14:01             ` Thomas Monjalon
2019-11-19 15:38               ` Burakov, Anatoly
2019-11-19 16:05                 ` Thomas Monjalon
2019-11-08 16:25           ` [dpdk-dev] [PATCH v7 03/10] buildtools: add ABI update shell script Anatoly Burakov
2019-11-19 17:38             ` Thomas Monjalon
2019-11-20 11:50               ` Burakov, Anatoly
2019-11-08 16:25           ` [dpdk-dev] [PATCH v7 04/10] timer: remove deprecated code Anatoly Burakov
2019-11-19 21:42             ` David Marchand
2019-11-08 16:25           ` [dpdk-dev] [PATCH v7 05/10] lpm: " Anatoly Burakov
2019-11-19 21:43             ` David Marchand
2019-11-08 16:25           ` [dpdk-dev] [PATCH v7 06/10] distributor: " Anatoly Burakov
2019-11-08 16:25           ` [dpdk-dev] [PATCH v7 07/10] distributor: rename v2.0 ABI to _single suffix Anatoly Burakov
2019-11-08 16:25           ` [dpdk-dev] [PATCH v7 08/10] drivers/octeontx: add missing public symbol Anatoly Burakov
2019-11-08 16:25           ` [dpdk-dev] [PATCH v7 09/10] build: change ABI version to 20.0 Anatoly Burakov
2019-11-19 17:46             ` Thomas Monjalon
2019-11-19 21:50             ` David Marchand
2019-11-22  7:07               ` Sachin Saxena
2019-11-08 16:25           ` [dpdk-dev] [PATCH v7 10/10] buildtools: add ABI versioning check script Anatoly Burakov
2019-11-19 18:16             ` Thomas Monjalon
2019-11-06 16:54         ` [dpdk-dev] [PATCH v6 01/10] config: change ABI versioning to global Anatoly Burakov
2019-11-06 16:54         ` [dpdk-dev] [PATCH v6 02/10] buildtools: add script for updating symbols abi version Anatoly Burakov
2019-11-06 16:54         ` [dpdk-dev] [PATCH v6 03/10] buildtools: add ABI update shell script Anatoly Burakov
2019-11-06 16:54         ` [dpdk-dev] [PATCH v6 04/10] timer: remove deprecated code Anatoly Burakov
2019-11-06 16:54         ` [dpdk-dev] [PATCH v6 05/10] lpm: " Anatoly Burakov
2019-11-06 16:54         ` [dpdk-dev] [PATCH v6 06/10] distributor: " Anatoly Burakov
2019-11-06 16:54         ` [dpdk-dev] [PATCH v6 07/10] distributor: rename v2.0 ABI to _single suffix Anatoly Burakov
2019-11-06 16:54         ` [dpdk-dev] [PATCH v6 08/10] drivers/octeontx: add missing public symbol Anatoly Burakov
2019-11-06 16:54         ` [dpdk-dev] [PATCH v6 09/10] build: change ABI version to 20.0 Anatoly Burakov
2019-11-06 16:54         ` [dpdk-dev] [PATCH v6 10/10] buildtools: add ABI versioning check script Anatoly Burakov
2019-10-24  9:46       ` [dpdk-dev] [PATCH v5 01/10] config: change ABI versioning to global Anatoly Burakov
2019-11-05 11:05         ` David Marchand
2019-11-05 13:50           ` Bruce Richardson
2019-10-24  9:46       ` Anatoly Burakov [this message]
2019-11-06 15:38         ` [dpdk-dev] [PATCH v5 02/10] buildtools: add script for updating symbols abi version David Marchand
2019-10-24  9:46       ` [dpdk-dev] [PATCH v5 03/10] buildtools: add ABI update shell script Anatoly Burakov
2019-10-24  9:46       ` [dpdk-dev] [PATCH v5 04/10] timer: remove deprecated code Anatoly Burakov
2019-10-24  9:46       ` [dpdk-dev] [PATCH v5 05/10] lpm: " Anatoly Burakov
2019-10-24  9:46       ` [dpdk-dev] [PATCH v5 06/10] distributor: " Anatoly Burakov
2019-10-24  9:46       ` [dpdk-dev] [PATCH v5 07/10] distributor: rename v2.0 ABI to _single suffix Anatoly Burakov
2019-10-24  9:46       ` [dpdk-dev] [PATCH v5 08/10] drivers/octeontx: add missing public symbol Anatoly Burakov
2019-10-24  9:46       ` [dpdk-dev] [PATCH v5 09/10] build: change ABI version to 20.0 Anatoly Burakov
2019-10-24  9:46       ` [dpdk-dev] [PATCH v5 10/10] buildtools: add ABI versioning check script Anatoly Burakov
2019-10-17 14:31     ` [dpdk-dev] [PATCH v4 01/10] config: change ABI versioning to global Anatoly Burakov
2019-10-17 14:31     ` [dpdk-dev] [PATCH v4 02/10] buildtools: add script for updating symbols abi version Anatoly Burakov
2019-10-17 14:31     ` [dpdk-dev] [PATCH v4 03/10] buildtools: add ABI update shell script Anatoly Burakov
2019-10-17 14:31     ` [dpdk-dev] [PATCH v4 04/10] timer: remove deprecated code Anatoly Burakov
2019-10-17 21:04       ` Carrillo, Erik G
2019-10-21 13:24       ` Kevin Traynor
2019-10-24  9:07         ` Burakov, Anatoly
2019-10-17 14:31     ` [dpdk-dev] [PATCH v4 05/10] lpm: " Anatoly Burakov
2019-10-17 14:31     ` [dpdk-dev] [PATCH v4 06/10] distributor: " Anatoly Burakov
2019-10-17 15:59       ` Hunt, David
2019-10-17 14:31     ` [dpdk-dev] [PATCH v4 07/10] distributor: rename v2.0 ABI to _single suffix Anatoly Burakov
2019-10-17 16:00       ` Hunt, David
2019-10-17 14:31     ` [dpdk-dev] [PATCH v4 08/10] drivers/octeontx: add missing public symbol Anatoly Burakov
2019-10-17 14:31     ` [dpdk-dev] [PATCH v4 09/10] build: change ABI version to 20.0 Anatoly Burakov
2019-10-17 14:32     ` [dpdk-dev] [PATCH v4 10/10] buildtools: add ABI versioning check script Anatoly Burakov
2019-10-16 17:03   ` [dpdk-dev] [PATCH v3 1/9] config: change ABI versioning to global Anatoly Burakov
2019-10-17  8:44     ` Bruce Richardson
2019-10-17 10:25       ` Burakov, Anatoly
2019-10-17 14:09       ` Luca Boccassi
2019-10-17 14:12         ` Bruce Richardson
2019-10-18 10:07         ` Kevin Traynor
2019-10-16 17:03   ` [dpdk-dev] [PATCH v3 2/9] buildtools: add script for updating symbols abi version Anatoly Burakov
2019-10-16 17:03   ` [dpdk-dev] [PATCH v3 3/9] buildtools: add ABI update shell script Anatoly Burakov
2019-10-16 17:03   ` [dpdk-dev] [PATCH v3 4/9] timer: remove deprecated code Anatoly Burakov
2019-10-16 17:03   ` [dpdk-dev] [PATCH v3 5/9] lpm: " Anatoly Burakov
2019-10-16 17:03   ` [dpdk-dev] [PATCH v3 6/9] distributor: " Anatoly Burakov
2019-10-17 10:53     ` Hunt, David
2019-10-16 17:03   ` [dpdk-dev] [PATCH v3 7/9] drivers/octeontx: add missing public symbol Anatoly Burakov
2019-10-16 17:03   ` [dpdk-dev] [PATCH v3 8/9] build: change ABI version to 20.0 Anatoly Burakov
2019-10-16 17:03   ` [dpdk-dev] [PATCH v3 9/9] buildtools: add ABI versioning check script Anatoly Burakov
2019-10-16 12:43 ` [dpdk-dev] [PATCH v2 01/10] config: change ABI versioning for global Anatoly Burakov
2019-10-16 13:22   ` Bruce Richardson
2019-10-16 12:43 ` [dpdk-dev] [PATCH v2 02/10] buildtools: add script for updating symbols abi version Anatoly Burakov
2019-10-16 13:25   ` Bruce Richardson
2019-10-16 12:43 ` [dpdk-dev] [PATCH v2 03/10] buildtools: add ABI update shell script Anatoly Burakov
2019-10-16 13:33   ` Bruce Richardson
2019-10-16 12:43 ` [dpdk-dev] [PATCH v2 04/10] timer: remove deprecated code Anatoly Burakov
2019-10-16 12:43 ` [dpdk-dev] [PATCH v2 05/10] lpm: " Anatoly Burakov
2019-10-16 12:43 ` [dpdk-dev] [PATCH v2 06/10] distributor: " Anatoly Burakov
2019-10-16 12:43 ` [dpdk-dev] [PATCH v2 07/10] lib: change function suffix in distributor Anatoly Burakov
2019-10-16 13:18   ` Hunt, David
2019-10-16 12:43 ` [dpdk-dev] [PATCH v2 08/10] drivers/octeontx: add missing public symbol Anatoly Burakov
2019-10-16 12:43 ` [dpdk-dev] [PATCH v2 09/10] build: change ABI version to 20.0 Anatoly Burakov
2019-10-16 12:43 ` [dpdk-dev] [PATCH v2 10/10] buildtools: add ABI versioning check script Anatoly Burakov

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=0bdec3d7e6659e3a41b9888a8b3a13eb428426f6.1571910363.git.anatoly.burakov@intel.com \
    --to=anatoly.burakov@intel.com \
    --cc=bruce.richardson@intel.com \
    --cc=david.marchand@redhat.com \
    --cc=dev@dpdk.org \
    --cc=john.mcnamara@intel.com \
    --cc=pawelx.modrak@intel.com \
    --cc=ray.kinsella@intel.com \
    --cc=thomas@monjalon.net \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).