[dpdk-dev] [PATCH] build: automatically create windows exports file

[dpdk-dev] [PATCH] build: automatically create windows exports file

[Bruce Richardson]

Rather than having a separate version.map file for linux/BSD and an
exports definition file for windows for each library, generate the
latter from the former automatically at build time.

Signed-off-by: Bruce Richardson <bruce.richardson@intel.com>
---
 buildtools/map_to_def.py                 | 40 ++++++++++++++++++++++++
 buildtools/meson.build                   | 12 ++++---
 buildtools/pmdinfogen/meson.build        |  7 ++++-
 lib/librte_kvargs/rte_kvargs_exports.def |  7 -----
 lib/meson.build                          | 14 ++++++---
 meson.build                              |  2 +-
 6 files changed, 64 insertions(+), 18 deletions(-)
 create mode 100644 buildtools/map_to_def.py
 delete mode 100644 lib/librte_kvargs/rte_kvargs_exports.def

diff --git a/buildtools/map_to_def.py b/buildtools/map_to_def.py
new file mode 100644
index 000000000..6775b54a9
--- /dev/null
+++ b/buildtools/map_to_def.py
@@ -0,0 +1,40 @@
+#!/usr/bin/env python
+# SPDX-License-Identifier: BSD-3-Clause
+# Copyright(c) 2019 Intel Corporation
+
+from __future__ import print_function
+import sys
+from os.path import dirname, basename, join, exists
+
+
+def is_function_line(ln):
+    return ln.startswith('\t') and ln.endswith(';\n') and ":" not in ln
+
+
+def main(args):
+    if not args[1].endswith('version.map') or \
+            not args[2].endswith('exports.def'):
+        return 1
+
+# special case, allow override if an def file already exists alongside map file
+    override_file = join(dirname(args[1]), basename(args[2]))
+    if exists(override_file):
+        with open(override_file) as f_in:
+            functions = f_in.readlines()
+
+# generate def file from map file.
+# This works taking indented lines only which end with a ";" and which don't
+# have a colon in them, i.e. the lines defining functions only.
+    else:
+        with open(args[1]) as f_in:
+            functions = [ln[:-2] + '\n' for ln in sorted(f_in.readlines())
+                         if is_function_line(ln)]
+            functions = ["EXPORTS\n"] + functions
+
+    with open(args[2], 'w') as f_out:
+        f_out.writelines(functions)
+    return 0
+
+
+if __name__ == "__main__":
+    sys.exit(main(sys.argv))
diff --git a/buildtools/meson.build b/buildtools/meson.build
index 0209bec8f..32c79c130 100644
--- a/buildtools/meson.build
+++ b/buildtools/meson.build
@@ -1,10 +1,14 @@
 # SPDX-License-Identifier: BSD-3-Clause
 # Copyright(c) 2017-2019 Intel Corporation
 
-if host_machine.system() == 'windows'
-	subdir_done()
-endif
-
 subdir('pmdinfogen')
 
 pmdinfo = find_program('gen-pmdinfo-cfile.sh')
+
+# set up map-to-def script using python, either built-in or external
+python3 = import('python').find_installation(required: false)
+if python3.found()
+	map_to_def_cmd = [python3, files('map_to_def.py')]
+else
+	map_to_def_cmd = ['meson', 'runpython', files('map_to_def.py')]
+endif
diff --git a/buildtools/pmdinfogen/meson.build b/buildtools/pmdinfogen/meson.build
index a219a8e96..899ba112c 100644
--- a/buildtools/pmdinfogen/meson.build
+++ b/buildtools/pmdinfogen/meson.build
@@ -1,7 +1,12 @@
 # SPDX-License-Identifier: BSD-3-Clause
 # Copyright(c) 2017 Intel Corporation
 
-pmdinfogen_inc = eal_inc
+if host_machine.system() == 'windows'
+	subdir_done()
+endif
+
+pmdinfogen_inc = [global_inc]
+pmdinfogen_inc += include_directories('../../lib/librte_eal/common/include')
 pmdinfogen_inc += include_directories('../../lib/librte_pci')
 pmdinfogen = executable('pmdinfogen',
 	'pmdinfogen.c',
diff --git a/lib/librte_kvargs/rte_kvargs_exports.def b/lib/librte_kvargs/rte_kvargs_exports.def
deleted file mode 100644
index 10e839e00..000000000
--- a/lib/librte_kvargs/rte_kvargs_exports.def
+++ /dev/null
@@ -1,7 +0,0 @@
-EXPORTS
-	rte_kvargs_count
-	rte_kvargs_free
-	rte_kvargs_parse
-	rte_kvargs_process
-	rte_kvargs_parse_delim
-	rte_kvargs_strcmp
diff --git a/lib/meson.build b/lib/meson.build
index 595314d7d..ba29b1244 100644
--- a/lib/meson.build
+++ b/lib/meson.build
@@ -121,15 +121,19 @@ foreach l:libraries
 			objs += static_lib.extract_all_objects(recursive: false)
 			version_map = '@0@/@1@/rte_@2@_version.map'.format(
 					meson.current_source_dir(), dir_name, name)
-			exports = []
 			implib = dir_name + '.dll.a'
+
+			def_file = custom_target(name + '_def',
+				command: [map_to_def_cmd, '@INPUT@', '@OUTPUT@'],
+				input: version_map,
+				output: 'rte_@0@_exports.def'.format(name))
 			if host_machine.system() == 'windows'
-				exports = '@0@/@1@/rte_@2@_exports.def'.format(
-					meson.current_source_dir(), dir_name, name)
-				lk_args = ['-Wl,/def:' + exports, '-Wl,/implib:lib\\' + implib]
+				lk_args = ['-Wl,/def:' + def_file.full_path(),
+					'-Wl,/implib:lib\\' + implib]
 			else
 				lk_args = ['-Wl,--version-script=' + version_map]
 			endif
+
 			shared_lib = shared_library(libname,
 					sources,
 					objects: objs,
@@ -137,7 +141,7 @@ foreach l:libraries
 					dependencies: shared_deps,
 					include_directories: includes,
 					link_args: lk_args,
-					link_depends: [version_map, exports],
+					link_depends: [version_map, def_file],
 					version: lib_version,
 					soversion: so_version,
 					install: true)
diff --git a/meson.build b/meson.build
index a96486597..02f4dd73b 100644
--- a/meson.build
+++ b/meson.build
@@ -31,8 +31,8 @@ global_inc = include_directories('.', 'config',
 subdir('config')
 
 # build libs and drivers
-subdir('lib')
 subdir('buildtools')
+subdir('lib')
 subdir('drivers')
 
 # build binaries and installable tools
-- 
2.17.0

[dpdk-dev] [PATCH] build: automatically create windows exports file

[Bruce Richardson]

Rather than having a separate version.map file for linux/BSD and an
exports definition file for windows for each library, generate the
latter from the former automatically at build time.

Signed-off-by: Bruce Richardson <bruce.richardson@intel.com>
---
 buildtools/map_to_def.py                 | 40 ++++++++++++++++++++++++
 buildtools/meson.build                   | 12 ++++---
 buildtools/pmdinfogen/meson.build        |  7 ++++-
 lib/librte_kvargs/rte_kvargs_exports.def |  7 -----
 lib/meson.build                          | 14 ++++++---
 meson.build                              |  2 +-
 6 files changed, 64 insertions(+), 18 deletions(-)
 create mode 100644 buildtools/map_to_def.py
 delete mode 100644 lib/librte_kvargs/rte_kvargs_exports.def

diff --git a/buildtools/map_to_def.py b/buildtools/map_to_def.py
new file mode 100644
index 000000000..6775b54a9
--- /dev/null
+++ b/buildtools/map_to_def.py
@@ -0,0 +1,40 @@
+#!/usr/bin/env python
+# SPDX-License-Identifier: BSD-3-Clause
+# Copyright(c) 2019 Intel Corporation
+
+from __future__ import print_function
+import sys
+from os.path import dirname, basename, join, exists
+
+
+def is_function_line(ln):
+    return ln.startswith('\t') and ln.endswith(';\n') and ":" not in ln
+
+
+def main(args):
+    if not args[1].endswith('version.map') or \
+            not args[2].endswith('exports.def'):
+        return 1
+
+# special case, allow override if an def file already exists alongside map file
+    override_file = join(dirname(args[1]), basename(args[2]))
+    if exists(override_file):
+        with open(override_file) as f_in:
+            functions = f_in.readlines()
+
+# generate def file from map file.
+# This works taking indented lines only which end with a ";" and which don't
+# have a colon in them, i.e. the lines defining functions only.
+    else:
+        with open(args[1]) as f_in:
+            functions = [ln[:-2] + '\n' for ln in sorted(f_in.readlines())
+                         if is_function_line(ln)]
+            functions = ["EXPORTS\n"] + functions
+
+    with open(args[2], 'w') as f_out:
+        f_out.writelines(functions)
+    return 0
+
+
+if __name__ == "__main__":
+    sys.exit(main(sys.argv))
diff --git a/buildtools/meson.build b/buildtools/meson.build
index 0209bec8f..32c79c130 100644
--- a/buildtools/meson.build
+++ b/buildtools/meson.build
@@ -1,10 +1,14 @@
 # SPDX-License-Identifier: BSD-3-Clause
 # Copyright(c) 2017-2019 Intel Corporation
 
-if host_machine.system() == 'windows'
-	subdir_done()
-endif
-
 subdir('pmdinfogen')
 
 pmdinfo = find_program('gen-pmdinfo-cfile.sh')
+
+# set up map-to-def script using python, either built-in or external
+python3 = import('python').find_installation(required: false)
+if python3.found()
+	map_to_def_cmd = [python3, files('map_to_def.py')]
+else
+	map_to_def_cmd = ['meson', 'runpython', files('map_to_def.py')]
+endif
diff --git a/buildtools/pmdinfogen/meson.build b/buildtools/pmdinfogen/meson.build
index a219a8e96..899ba112c 100644
--- a/buildtools/pmdinfogen/meson.build
+++ b/buildtools/pmdinfogen/meson.build
@@ -1,7 +1,12 @@
 # SPDX-License-Identifier: BSD-3-Clause
 # Copyright(c) 2017 Intel Corporation
 
-pmdinfogen_inc = eal_inc
+if host_machine.system() == 'windows'
+	subdir_done()
+endif
+
+pmdinfogen_inc = [global_inc]
+pmdinfogen_inc += include_directories('../../lib/librte_eal/common/include')
 pmdinfogen_inc += include_directories('../../lib/librte_pci')
 pmdinfogen = executable('pmdinfogen',
 	'pmdinfogen.c',
diff --git a/lib/librte_kvargs/rte_kvargs_exports.def b/lib/librte_kvargs/rte_kvargs_exports.def
deleted file mode 100644
index 10e839e00..000000000
--- a/lib/librte_kvargs/rte_kvargs_exports.def
+++ /dev/null
@@ -1,7 +0,0 @@
-EXPORTS
-	rte_kvargs_count
-	rte_kvargs_free
-	rte_kvargs_parse
-	rte_kvargs_process
-	rte_kvargs_parse_delim
-	rte_kvargs_strcmp
diff --git a/lib/meson.build b/lib/meson.build
index 595314d7d..ba29b1244 100644
--- a/lib/meson.build
+++ b/lib/meson.build
@@ -121,15 +121,19 @@ foreach l:libraries
 			objs += static_lib.extract_all_objects(recursive: false)
 			version_map = '@0@/@1@/rte_@2@_version.map'.format(
 					meson.current_source_dir(), dir_name, name)
-			exports = []
 			implib = dir_name + '.dll.a'
+
+			def_file = custom_target(name + '_def',
+				command: [map_to_def_cmd, '@INPUT@', '@OUTPUT@'],
+				input: version_map,
+				output: 'rte_@0@_exports.def'.format(name))
 			if host_machine.system() == 'windows'
-				exports = '@0@/@1@/rte_@2@_exports.def'.format(
-					meson.current_source_dir(), dir_name, name)
-				lk_args = ['-Wl,/def:' + exports, '-Wl,/implib:lib\\' + implib]
+				lk_args = ['-Wl,/def:' + def_file.full_path(),
+					'-Wl,/implib:lib\\' + implib]
 			else
 				lk_args = ['-Wl,--version-script=' + version_map]
 			endif
+
 			shared_lib = shared_library(libname,
 					sources,
 					objects: objs,
@@ -137,7 +141,7 @@ foreach l:libraries
 					dependencies: shared_deps,
 					include_directories: includes,
 					link_args: lk_args,
-					link_depends: [version_map, exports],
+					link_depends: [version_map, def_file],
 					version: lib_version,
 					soversion: so_version,
 					install: true)
diff --git a/meson.build b/meson.build
index a96486597..02f4dd73b 100644
--- a/meson.build
+++ b/meson.build
@@ -31,8 +31,8 @@ global_inc = include_directories('.', 'config',
 subdir('config')
 
 # build libs and drivers
-subdir('lib')
 subdir('buildtools')
+subdir('lib')
 subdir('drivers')
 
 # build binaries and installable tools
-- 
2.17.0

Re: [dpdk-dev] [PATCH] build: automatically create windows exports file

[Anand Rawat]

On 4/12/2019 1:29 AM, Bruce Richardson wrote:
> Rather than having a separate version.map file for linux/BSD and an
> exports definition file for windows for each library, generate the
> latter from the former automatically at build time.
> 
> Signed-off-by: Bruce Richardson <bruce.richardson@intel.com>
> 

Acked-by: Anand Rawat <anand.rawat@intel.com>
-- 
Anand Rawat

Re: [dpdk-dev] [PATCH] build: automatically create windows exports file

[Anand Rawat]

On 4/12/2019 1:29 AM, Bruce Richardson wrote:
> Rather than having a separate version.map file for linux/BSD and an
> exports definition file for windows for each library, generate the
> latter from the former automatically at build time.
> 
> Signed-off-by: Bruce Richardson <bruce.richardson@intel.com>
> 

Acked-by: Anand Rawat <anand.rawat@intel.com>
-- 
Anand Rawat

Re: [dpdk-dev] [PATCH] build: automatically create windows exports file

[Thomas Monjalon]

13/04/2019 00:56, Anand Rawat:
> On 4/12/2019 1:29 AM, Bruce Richardson wrote:
> > Rather than having a separate version.map file for linux/BSD and an
> > exports definition file for windows for each library, generate the
> > latter from the former automatically at build time.
> > 
> > Signed-off-by: Bruce Richardson <bruce.richardson@intel.com>
> 
> Acked-by: Anand Rawat <anand.rawat@intel.com>

Please apply it in the draft repo for Windows so we can have more feedbacks.

Re: [dpdk-dev] [PATCH] build: automatically create windows exports file

[Thomas Monjalon]

13/04/2019 00:56, Anand Rawat:
> On 4/12/2019 1:29 AM, Bruce Richardson wrote:
> > Rather than having a separate version.map file for linux/BSD and an
> > exports definition file for windows for each library, generate the
> > latter from the former automatically at build time.
> > 
> > Signed-off-by: Bruce Richardson <bruce.richardson@intel.com>
> 
> Acked-by: Anand Rawat <anand.rawat@intel.com>

Please apply it in the draft repo for Windows so we can have more feedbacks.

Re: [dpdk-dev] [PATCH] build: automatically create windows exports file

[Bruce Richardson]

On Wed, Apr 17, 2019 at 06:21:29PM +0200, Thomas Monjalon wrote:
> 13/04/2019 00:56, Anand Rawat:
> > On 4/12/2019 1:29 AM, Bruce Richardson wrote:
> > > Rather than having a separate version.map file for linux/BSD and an
> > > exports definition file for windows for each library, generate the
> > > latter from the former automatically at build time.
> > > 
> > > Signed-off-by: Bruce Richardson <bruce.richardson@intel.com>
> > 
> > Acked-by: Anand Rawat <anand.rawat@intel.com>
> 
> Please apply it in the draft repo for Windows so we can have more feedbacks.
>

Ping on this. Has it already been applied to the draft repo, or is
something blocking it?

/Bruce

Re: [dpdk-dev] [PATCH] build: automatically create windows exports file

[Thomas Monjalon]

12/04/2019 10:29, Bruce Richardson:
> Rather than having a separate version.map file for linux/BSD and an
> exports definition file for windows for each library, generate the
> latter from the former automatically at build time.
> 
> Signed-off-by: Bruce Richardson <bruce.richardson@intel.com>

For an unknown reason, this patch was not tested
in the Windows draft repo, but I choose to merge in master.

Applied, thanks