[dpdk-dev] [PATCH 0/3] pmdinfogen fixes

[dpdk-dev] [PATCH 0/3] pmdinfogen fixes

[Dmitry Kozlyuk]

Recent pmdinfogen patches introduced regressions with older systems.
This is a follow-up cleaning the mess.

[1]: http://patchwork.dpdk.org/project/dpdk/list/?series=13153
[2]: http://patchwork.dpdk.org/project/dpdk/list/?series=14274

Dmitry Kozlyuk (3):
  pmdinfogen: fix build with pyelftools < 0.24
  buildtools: fix archive extraction for Python 3.5
  buildtools: use build subdirectory for temporary files

 buildtools/gen-pmdinfo-cfile.py |  6 +++---
 buildtools/meson.build          |  2 +-
 buildtools/pmdinfogen.py        | 20 ++++++++++++++++++--
 3 files changed, 22 insertions(+), 6 deletions(-)

-- 
2.29.2

[dpdk-dev] [PATCH 1/3] pmdinfogen: fix build with pyelftools < 0.24

[Dmitry Kozlyuk]

pyelftools had some breaking changes [1] and API enhancements [2]
between 0.23 (used in Ubuntu 16.04) and 0.24. Ensure compatibility with
both legacy and modern versions.

[1]: https://github.com/eliben/pyelftools/pull/76
[2]: https://github.com/eliben/pyelftools/pull/56

Signed-off-by: Dmitry Kozlyuk <dmitry.kozliuk@gmail.com>
---
To be squashed with 043e0ba6c ("buildtools: add Python pmdinfogen").

 buildtools/pmdinfogen.py | 20 ++++++++++++++++++--
 1 file changed, 18 insertions(+), 2 deletions(-)

diff --git a/buildtools/pmdinfogen.py b/buildtools/pmdinfogen.py
index 893a6c571..7a739ec7d 100755
--- a/buildtools/pmdinfogen.py
+++ b/buildtools/pmdinfogen.py
@@ -10,6 +10,7 @@
 import tempfile
 
 try:
+    import elftools
     from elftools.elf.elffile import ELFFile
     from elftools.elf.sections import SymbolTableSection
 except ImportError:
@@ -38,8 +39,13 @@ def get_value(self, offset, size):
 
 class ELFImage:
     def __init__(self, data):
+        version = tuple(int(c) for c in elftools.__version__.split("."))
+        self._legacy_elftools = version < (0, 24)
+
         self._image = ELFFile(data)
-        self._symtab = self._image.get_section_by_name(".symtab")
+
+        section = b".symtab" if self._legacy_elftools else ".symtab"
+        self._symtab = self._image.get_section_by_name(section)
         if not isinstance(self._symtab, SymbolTableSection):
             raise Exception(".symtab section is not a symbol table")
 
@@ -48,10 +54,20 @@ def is_big_endian(self):
         return not self._image.little_endian
 
     def find_by_name(self, name):
-        symbol = self._symtab.get_symbol_by_name(name)
+        symbol = self._get_symbol_by_name(name)
         return ELFSymbol(self._image, symbol[0]) if symbol else None
 
+    def _get_symbol_by_name(self, name):
+        if not self._legacy_elftools:
+            return self._symtab.get_symbol_by_name(name)
+        name = name.encode("utf-8")
+        for symbol in self._symtab.iter_symbols():
+            if symbol.name == name:
+                return [symbol]
+        return None
+
     def find_by_prefix(self, prefix):
+        prefix = prefix.encode("utf-8") if self._legacy_elftools else prefix
         for i in range(self._symtab.num_symbols()):
             symbol = self._symtab.get_symbol(i)
             if symbol.name.startswith(prefix):
-- 
2.29.2

[dpdk-dev] [PATCH 2/3] buildtools: fix archive extraction for Python 3.5

[Dmitry Kozlyuk]

Python 3.5 subprocess.run() has no capture_output parameter.
Use subprocess.PIPE available in all versions.

Signed-off-by: Dmitry Kozlyuk <dmitry.kozliuk@gmail.com>
---
To be squashed with caaca1ec6 ("buildtools: support object file
extraction for Windows").

 buildtools/gen-pmdinfo-cfile.py | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/buildtools/gen-pmdinfo-cfile.py b/buildtools/gen-pmdinfo-cfile.py
index f1f289ffe..c60ecd7f6 100644
--- a/buildtools/gen-pmdinfo-cfile.py
+++ b/buildtools/gen-pmdinfo-cfile.py
@@ -11,7 +11,7 @@
 with tempfile.TemporaryDirectory() as temp:
     proc = subprocess.run(
         # Don't use "ar p", because its output is corrupted on Windows.
-        [ar, "xv", os.path.abspath(archive)], capture_output=True, check=True, cwd=temp
+        [ar, "xv", os.path.abspath(archive)], stdout=subprocess.PIPE, check=True, cwd=temp
     )
     lines = proc.stdout.decode().splitlines()
     names = [line[len("x - ") :] for line in lines]
-- 
2.29.2

[dpdk-dev] [PATCH 3/3] buildtools: use build subdirectory for temporary files

[Dmitry Kozlyuk]

Use current build directory as base for temporary directories,
so that all build files are isolated there.

Fixes: caaca1ec6a8e ("buildtools: support object file extraction for Windows")

Signed-off-by: Dmitry Kozlyuk <dmitry.kozliuk@gmail.com>
---
Thomas asked to send this as a follow-up before regression has been
noticed, may be squashed with commit mentioned above or applied on its own.

 buildtools/gen-pmdinfo-cfile.py | 4 ++--
 buildtools/meson.build          | 2 +-
 2 files changed, 3 insertions(+), 3 deletions(-)

diff --git a/buildtools/gen-pmdinfo-cfile.py b/buildtools/gen-pmdinfo-cfile.py
index c60ecd7f6..a4e080199 100644
--- a/buildtools/gen-pmdinfo-cfile.py
+++ b/buildtools/gen-pmdinfo-cfile.py
@@ -7,8 +7,8 @@
 import sys
 import tempfile
 
-_, ar, archive, output, *pmdinfogen = sys.argv
-with tempfile.TemporaryDirectory() as temp:
+_, tmp_root, ar, archive, output, *pmdinfogen = sys.argv
+with tempfile.TemporaryDirectory(dir=tmp_root) as temp:
     proc = subprocess.run(
         # Don't use "ar p", because its output is corrupted on Windows.
         [ar, "xv", os.path.abspath(archive)], stdout=subprocess.PIPE, check=True, cwd=temp
diff --git a/buildtools/meson.build b/buildtools/meson.build
index 0a2e91a7b..9c9347457 100644
--- a/buildtools/meson.build
+++ b/buildtools/meson.build
@@ -18,7 +18,7 @@ map_to_win_cmd = py3 + files('map_to_win.py')
 sphinx_wrapper = py3 + files('call-sphinx-build.py')
 
 # select library and object file format
-pmdinfo = py3 + files('gen-pmdinfo-cfile.py')
+pmdinfo = py3 + files('gen-pmdinfo-cfile.py') + [meson.current_build_dir()]
 pmdinfogen = py3 + files('pmdinfogen.py')
 if host_machine.system() == 'windows'
 	if cc.get_id() == 'gcc'
-- 
2.29.2

Re: [dpdk-dev] [PATCH 0/3] pmdinfogen fixes

[Thomas Monjalon]

25/01/2021 23:12, Dmitry Kozlyuk:
> Recent pmdinfogen patches introduced regressions with older systems.
> This is a follow-up cleaning the mess.
> 
> [1]: http://patchwork.dpdk.org/project/dpdk/list/?series=13153
> [2]: http://patchwork.dpdk.org/project/dpdk/list/?series=14274
> 
> Dmitry Kozlyuk (3):
>   pmdinfogen: fix build with pyelftools < 0.24
>   buildtools: fix archive extraction for Python 3.5
>   buildtools: use build subdirectory for temporary files

Applied, thanks for the quick fixes.