[dpdk-dev] [PATCH] drivers: improve pmdinfo generation when using meson

[dpdk-dev] [PATCH] drivers: improve pmdinfo generation when using meson

[Bruce Richardson]

Since meson 0.44, changing any file inside a PMD directory (quite
correctly) triggers a full re-run of meson on build, rather than an
incremental build as with earlier versions. This rerun is needed because
we use "grep" in meson to search for files on which to run pmdinfogen, and
changing any of those files means that grep and, therefore meson, needs to
be rerun. [Previous versions of meson did not track this dependency on the
grep command, and so did incremental builds only.]

If, however, we take advantage of pmdinfogen's ability to use stdin and
stdout instead of files, we can instead use a shell script to process an
entire static archive and generate a single .c file from it. This
eliminates the need for grep, and means that changes to a PMD file only
need an incremental build - a significant time saving.

Signed-off-by: Bruce Richardson <bruce.richardson@intel.com>
---

NOTE: this patch depends upon http://dpdk.org/dev/patchwork/patch/34469/

 buildtools/gen-pmdinfo-cfile.sh |  9 +++++----
 drivers/meson.build             | 31 +++++++++++++++----------------
 2 files changed, 20 insertions(+), 20 deletions(-)

diff --git a/buildtools/gen-pmdinfo-cfile.sh b/buildtools/gen-pmdinfo-cfile.sh
index 565e08913..c721bd1c4 100755
--- a/buildtools/gen-pmdinfo-cfile.sh
+++ b/buildtools/gen-pmdinfo-cfile.sh
@@ -6,7 +6,8 @@ arfile=$1
 output=$2
 pmdinfogen=$3
 
-tmp_o=${output%.c.pmd.c}.tmp.o
-
-ar p $arfile > $tmp_o && \
-		$pmdinfogen $tmp_o $output
+echo > $output
+for ofile in `ar t $arfile` ; do
+	ar p $arfile $ofile | $pmdinfogen - - >> $output 2> /dev/null
+done
+exit 0
diff --git a/drivers/meson.build b/drivers/meson.build
index 604384f07..234be62d6 100644
--- a/drivers/meson.build
+++ b/drivers/meson.build
@@ -44,6 +44,7 @@ foreach class:driver_classes
 
 		if build
 			dpdk_conf.set(config_flag_fmt.format(name.to_upper()),1)
+			lib_name = driver_name_fmt.format(name)
 
 			# get dependency objs from strings
 			shared_objs = []
@@ -56,21 +57,22 @@ foreach class:driver_classes
 			static_objs += ext_deps
 			dpdk_extra_ldflags += pkgconfig_extra_libs
 
-			# generate pmdinfo sources
-			pmdinfogen_srcs = run_command('grep', '--files-with-matches',
-				'RTE_PMD_REGISTER_.*(.*)', sources).stdout().strip().split()
-			foreach src: pmdinfogen_srcs
-				out_filename = '@0@.pmd.c'.format(src.split('/')[-1])
-				tmp_lib = static_library('tmp_@0@'.format(src.underscorify()),
-					src, include_directories: includes,
+			# generate pmdinfo sources by building a temporary
+			# lib and then running pmdinfogen on the contents of
+			# that lib. The final lib reuses the object files and
+			# adds in the new source file.
+			out_filename = lib_name + '.pmd.c'
+			tmp_lib = static_library('tmp_' + lib_name,
+					sources,
+					include_directories: includes,
 					dependencies: static_objs,
 					c_args: cflags)
-				sources += custom_target(out_filename,
-						command: [pmdinfo, tmp_lib.full_path(),
-							'@OUTPUT@', pmdinfogen],
-						output: out_filename,
-						depends: [pmdinfogen, tmp_lib])
-			endforeach
+			objs += tmp_lib.extract_all_objects()
+			sources = custom_target(out_filename,
+					command: [pmdinfo, tmp_lib.full_path(),
+						'@OUTPUT@', pmdinfogen],
+					output: out_filename,
+					depends: [pmdinfogen, tmp_lib])
 
 			if get_option('per_library_versions')
 				lib_version = '@0@.1'.format(version)
@@ -83,7 +85,6 @@ foreach class:driver_classes
 			endif
 
 			# now build the static driver
-			lib_name = driver_name_fmt.format(name)
 			static_lib = static_library(lib_name,
 				sources,
 				objects: objs,
@@ -93,8 +94,6 @@ foreach class:driver_classes
 				install: true)
 
 			# now build the shared driver
-			sources = []
-			objs += static_lib.extract_all_objects()
 			version_map = '@0@/@1@/@2@_version.map'.format(
 					meson.current_source_dir(),
 					drv_path, lib_name)
-- 
2.14.3

Re: [dpdk-dev] [PATCH] drivers: improve pmdinfo generation when using meson

[Bruce Richardson]

On Thu, Jan 25, 2018 at 11:14:43AM +0000, Bruce Richardson wrote:
> Since meson 0.44, changing any file inside a PMD directory (quite
> correctly) triggers a full re-run of meson on build, rather than an
> incremental build as with earlier versions. This rerun is needed because
> we use "grep" in meson to search for files on which to run pmdinfogen, and
> changing any of those files means that grep and, therefore meson, needs to
> be rerun. [Previous versions of meson did not track this dependency on the
> grep command, and so did incremental builds only.]
> 
> If, however, we take advantage of pmdinfogen's ability to use stdin and
> stdout instead of files, we can instead use a shell script to process an
> entire static archive and generate a single .c file from it. This
> eliminates the need for grep, and means that changes to a PMD file only
> need an incremental build - a significant time saving.
> 
> Signed-off-by: Bruce Richardson <bruce.richardson@intel.com>
> ---
Applied to dpdk-next-build

/Bruce