Re: [dpdk-dev] [EXT] Re: [PATCH v5 3/8] kvargs: adding a module definition file

[Bruce Richardson <bruce.richardson@intel.com>]

On Tue, Mar 26, 2019 at 03:35:36PM +0000, Jerin Jacob Kollanukkaran wrote:
> On Tue, 2019-03-26 at 14:40 +0000, Bruce Richardson wrote:
> > > It is painful due to the fact that, If it is windows ONLY file then
> > > developer need to test on Windows as well as it may break Windows.
> > > If it is a common file, at least, it will be tested on one
> > > platform.
> > > So responsibly wise it is a clean partition between windows eal
> > > maintainers vs generic library maintainers.
> > > 
> > Yes, good point. However, once we get some windows support into the
> > main
> > repo then there is the requirement not to break that, so some testing
> > on
> > windows before merge will prove necessary. Hopefully that can be done
> > just
> > via CI, rather than having maintainers/committers do so manually.
> > 
> > > > have been submitted over the years which failed shared library
> > > > build
> > > > because map file updates were forgotten.
> > > > 
> > > > However, my hope is that down the road we can have the def file
> > > > generated from the map file (or potentially vice versa). Perhaps
> > > > the
> > > > meson python module could be used to allow us to script it a bit.
> > > 
> > > Make sense. Do we want to support shared lib for Windows for the
> > > first
> > > version? or Can we live with static lib till we find a proper
> > > solution.
> > > I do believe the base Windows Helloworld support needs to added
> > > this
> > > release in main repo and add the subsequent features step by
> > > step.  I
> > > would treat, shared lib as subsequent feature if it is not clean.
> > > 
> > Yes, I did consider that possibility. However, turning off shared
> > builds
> > for windows is more of a hack than adding a definition file, since it
> > involves more (temporary) changes to the meson.build for both lib and
> > driver.  If I get the chance, I'll see how complicated it might be to
> > autogenerate them at build. Otherwise, I'd suggest keeping the .def
> > files
> > for now, since only 2 libraries are involved, but then we need to
> > come up
> > with a proper solution before the number of libraries compiled on
> > windows
> > goes above that initial 2.
> 
> I am OK with a short term hack to get Window support for DPDK, Provided
> it will be revisited before adding the next .def file.
> 
Ok, some hacking has led to this as a possible approach to solve this. I've
only tested this on linux to verify it creates something approximating a
module definition file but not actually tried using it on windows. The nice
thing about meson being based on python is that we are guaranteed to have a
python3 interpreter available on whatever os we are running on. [Yes, I
made the script python2 compatible too, though I probably didn't need to!]

I've also included in this version an (untested) override option for EAL,
to allow us to keep the .def file for EAL until we can export all the
functions listed in the map file for it. Other libraries shouldn't need
this, since they aren't as insanely big as EAL.

/Bruce

diff --git a/buildtools/map_to_def.py b/buildtools/map_to_def.py
new file mode 100644
index 000000000..6c2f56c5d
--- /dev/null
+++ b/buildtools/map_to_def.py
@@ -0,0 +1,30 @@
+#!/usr/bin/env python
+# SPDX-License-Identifier: BSD-3-Clause
+# Copyright(c) 2019 Intel Corporation
+
+from __future__ import print_function
+import sys
+from os.path import dirname, basename, join, exists
+
+def main(args):
+  if not args[1].endswith('version.map') or not args[2].endswith('exports.def'):
+    return 1
+
+# special case, allow override if an def file already exists alongside map file
+  override_file = join(dirname(args[1]), basename(args[2]))
+  if exists(override_file):
+    with open(override_file) as f_in:
+      functions = f_in.readlines()
+  else:
+# generate def file from map file
+    with open(args[1]) as f_in:
+      functions = ["EXPORTS\n"] + \
+           [ln[:-2] + '\n' for ln in sorted(f_in.readlines()) \
+              if ln.startswith('\t') and ln.endswith(';\n') and ":" not in ln]
+
+  with open(args[2], 'w') as f_out:
+    f_out.writelines(functions)
+  return 0;
+
+if __name__ == "__main__":
+  sys.exit(main(sys.argv))
diff --git a/buildtools/meson.build b/buildtools/meson.build
index cdd38ed24..9ae2bc7c8 100644
--- a/buildtools/meson.build
+++ b/buildtools/meson.build
@@ -4,3 +4,4 @@
 subdir('pmdinfogen')
 
 pmdinfo = find_program('gen-pmdinfo-cfile.sh')
+map_to_def = files('map_to_def.py')
diff --git a/buildtools/pmdinfogen/meson.build b/buildtools/pmdinfogen/meson.build
index a219a8e96..938137300 100644
--- a/buildtools/pmdinfogen/meson.build
+++ b/buildtools/pmdinfogen/meson.build
@@ -1,7 +1,8 @@
 # SPDX-License-Identifier: BSD-3-Clause
 # Copyright(c) 2017 Intel Corporation
 
-pmdinfogen_inc = eal_inc
+pmdinfogen_inc = [global_inc]
+pmdinfogen_inc += include_directories('../../lib/librte_eal/common/include')
 pmdinfogen_inc += include_directories('../../lib/librte_pci')
 pmdinfogen = executable('pmdinfogen',
 	'pmdinfogen.c',
diff --git a/lib/meson.build b/lib/meson.build
index 99957ba7d..c4606e7f2 100644
--- a/lib/meson.build
+++ b/lib/meson.build
@@ -117,6 +117,12 @@ foreach l:libraries
 			objs += static_lib.extract_all_objects(recursive: false)
 			version_map = '@0@/@1@/rte_@2@_version.map'.format(
 					meson.current_source_dir(), dir_name, name)
+			exports_def = 'rte_@0@_exports.def'.format(name)
+			def_file = custom_target(exports_def,
+				command: [python3, map_to_def, '@INPUT@', '@OUTPUT@'],
+				input: version_map,
+				output: exports_def,
+				build_by_default: true)
 			shared_lib = shared_library(libname,
 					sources,
 					objects: objs,
diff --git a/meson.build b/meson.build
index 69833de82..81a9e93f7 100644
--- a/meson.build
+++ b/meson.build
@@ -8,6 +8,10 @@ project('DPDK', 'C',
 	meson_version: '>= 0.47.1'
 )
 
+# get python available, using the python interpreter for running meson
+pymod = import('python')
+python3 = pymod.find_installation()
+
 # set up some global vars for compiler, platform, configuration, etc.
 cc = meson.get_compiler('c')
 dpdk_conf = configuration_data()
@@ -36,8 +40,8 @@ global_inc = include_directories('.', 'config', 'lib/librte_eal/common/include')
 subdir('config')
 
 # build libs and drivers
-subdir('lib')
 subdir('buildtools')
+subdir('lib')
 subdir('drivers')
 
 # build binaries and installable tools