DPDK patches and discussions
 help / color / mirror / Atom feed
* [dpdk-dev] [PATCH] build: fix symlink of drivers for Windows
@ 2021-04-01 12:27 Nick Connolly
  2021-04-04  0:00 ` Dmitry Kozlyuk
                   ` (2 more replies)
  0 siblings, 3 replies; 11+ messages in thread
From: Nick Connolly @ 2021-04-01 12:27 UTC (permalink / raw)
  To: Bruce Richardson, Ranjit Menon, Nick Connolly
  Cc: dev, dmitry.kozliuk, navasile, dmitrym, pallavi.kadam, stable

The symlink-drivers-solibs.sh script was disabled as part of 'install'
for Windows because there is no support for shell scripts. However,
this means that driver related DLLs are not present in the installed
'libdir' directory.

Replace the shell script with a python script to perform the symbolic
linking to improve cross platform support. On Linux the functionality
is unchanged.

On Windows, symbolic links are somewhat problematic since the
SeCreateSymbolicLinkPrivilege is required to be able to create them.
In addition, different cross-compilation environments handle symbolic
links differently, e.g. WSL, Msys2, Cygwin. Rather than trying to
distinguish these scenarios, the python script will perform a file copy
for any Windows specific names.

On Windows, the shared library outputs have different names depending
upon which toolset has been used to build them. The script currently
handles Clang and GCC.

Fixes: 5c7d86948764 ("build: fix install on Windows")
Cc: stable@dpdk.org
Signed-off-by: Nick Connolly <nick.connolly@mayadata.io>
---
 buildtools/symlink-drivers-solibs.py | 49 ++++++++++++++++++++++++++++
 buildtools/symlink-drivers-solibs.sh | 13 --------
 config/meson.build                   |  7 ++--
 3 files changed, 51 insertions(+), 18 deletions(-)
 create mode 100644 buildtools/symlink-drivers-solibs.py
 delete mode 100644 buildtools/symlink-drivers-solibs.sh

diff --git a/buildtools/symlink-drivers-solibs.py b/buildtools/symlink-drivers-solibs.py
new file mode 100644
index 000000000..5627ddd9d
--- /dev/null
+++ b/buildtools/symlink-drivers-solibs.py
@@ -0,0 +1,49 @@
+#!/usr/bin/env python3
+# SPDX-License-Identifier: BSD-3-Clause
+# Copyright(c) 2021 Intel Corporation
+
+import os
+import sys
+import glob
+import shutil
+
+# post-install script for meson/ninja builds to symlink the PMDs stored in
+# $libdir/dpdk/pmds-*/ to $libdir. This is needed as some PMDs depend on
+# others, e.g. PCI device PMDs depending on the PCI bus driver.
+
+# parameters to script are paths relative to install prefix:
+# 1. directory for installed regular libs e.g. lib64
+# 2. subdirectory of libdir where the pmds are
+# 3. directory for installed regular binaries e.g. bin
+
+os.chdir(os.environ['MESON_INSTALL_DESTDIR_PREFIX'])
+
+lib_dir = sys.argv[1]
+pmd_subdir = sys.argv[2]
+bin_dir = sys.argv[3]
+pmd_dir = os.path.join(lib_dir, pmd_subdir)
+
+# copy Windows PMDs to avoid any issues with symlinks since the
+# build could be a cross-compilation under WSL, Msys or Cygnus.
+# the filenames are dependent upon the specific toolchain in use.
+
+def copy_pmd_files(pattern, to_dir):
+	for file in glob.glob(os.path.join(pmd_dir, pattern)):
+		to = os.path.join(to_dir, os.path.basename(file))
+		shutil.copy2(file, to)
+		print(to + ' -> ' + file)
+
+copy_pmd_files('*rte_*.dll', bin_dir)
+copy_pmd_files('*rte_*.pdb', bin_dir)
+copy_pmd_files('*rte_*.lib', lib_dir)
+copy_pmd_files('*rte_*.dll.a', lib_dir)
+
+# symlink shared objects
+
+os.chdir(lib_dir)
+for file in glob.glob(os.path.join(pmd_subdir, 'librte_*.so*')):
+	to = os.path.basename(file)
+	if os.path.exists(to):
+		os.remove(to)
+	os.symlink(file, to)
+	print(to + ' -> ' + file)
diff --git a/buildtools/symlink-drivers-solibs.sh b/buildtools/symlink-drivers-solibs.sh
deleted file mode 100644
index 42985e855..000000000
--- a/buildtools/symlink-drivers-solibs.sh
+++ /dev/null
@@ -1,13 +0,0 @@
-#! /bin/sh
-# SPDX-License-Identifier: BSD-3-Clause
-# Copyright(c) 2017 Intel Corporation
-
-# post-install script for meson/ninja builds to symlink the PMDs stored in
-# $libdir/dpdk/drivers/ to $libdir. This is needed as some PMDs depend on
-# others, e.g. PCI device PMDs depending on the PCI bus driver.
-
-# parameters to script are paths relative to install prefix:
-# 1. directory for installed regular libs e.g. lib64
-# 2. subdirectory of libdir where the pmds are
-
-cd ${MESON_INSTALL_DESTDIR_PREFIX}/$1 && ln -sfv $2/librte_*.so* .
diff --git a/config/meson.build b/config/meson.build
index 66a2edcc4..c51669b7d 100644
--- a/config/meson.build
+++ b/config/meson.build
@@ -57,11 +57,8 @@ eal_pmd_path = join_paths(get_option('prefix'), driver_install_path)
 # driver .so files often depend upon the bus drivers for their connect bus,
 # e.g. ixgbe depends on librte_bus_pci. This means that the bus drivers need
 # to be in the library path, so symlink the drivers from the main lib directory.
-if not is_windows
-	meson.add_install_script('../buildtools/symlink-drivers-solibs.sh',
-			get_option('libdir'),
-			pmd_subdir_opt)
-endif
+meson.add_install_script(py3, '../buildtools/symlink-drivers-solibs.py',
+	get_option('libdir'), pmd_subdir_opt, get_option('bindir'))
 
 # set the machine type and cflags for it
 if meson.is_cross_build()
-- 
2.25.1


^ permalink raw reply	[flat|nested] 11+ messages in thread

* Re: [dpdk-dev] [PATCH] build: fix symlink of drivers for Windows
  2021-04-01 12:27 [dpdk-dev] [PATCH] build: fix symlink of drivers for Windows Nick Connolly
@ 2021-04-04  0:00 ` Dmitry Kozlyuk
  2021-04-06 10:24   ` Nick Connolly
  2021-04-10  8:01 ` [dpdk-dev] [PATCH v2] " Nick Connolly
  2021-04-26 10:07 ` [dpdk-dev] [PATCH v3] " Nick Connolly
  2 siblings, 1 reply; 11+ messages in thread
From: Dmitry Kozlyuk @ 2021-04-04  0:00 UTC (permalink / raw)
  To: Nick Connolly
  Cc: Bruce Richardson, Ranjit Menon, dev, navasile, dmitrym,
	pallavi.kadam, stable

2021-04-01 13:27 (UTC+0100), Nick Connolly:
[...]
> +def copy_pmd_files(pattern, to_dir):
> +	for file in glob.glob(os.path.join(pmd_dir, pattern)):
> +		to = os.path.join(to_dir, os.path.basename(file))
> +		shutil.copy2(file, to)
> +		print(to + ' -> ' + file)
> +
> +copy_pmd_files('*rte_*.dll', bin_dir)
> +copy_pmd_files('*rte_*.pdb', bin_dir)

PDB (debuginfo) files can be quite large, do we want to install them?

[...]
> diff --git a/config/meson.build b/config/meson.build
> index 66a2edcc4..c51669b7d 100644
> --- a/config/meson.build
> +++ b/config/meson.build
> @@ -57,11 +57,8 @@ eal_pmd_path = join_paths(get_option('prefix'), driver_install_path)
>  # driver .so files often depend upon the bus drivers for their connect bus,
>  # e.g. ixgbe depends on librte_bus_pci. This means that the bus drivers need
>  # to be in the library path, so symlink the drivers from the main lib directory.
> -if not is_windows
> -	meson.add_install_script('../buildtools/symlink-drivers-solibs.sh',
> -			get_option('libdir'),
> -			pmd_subdir_opt)
> -endif
> +meson.add_install_script(py3, '../buildtools/symlink-drivers-solibs.py',
> +	get_option('libdir'), pmd_subdir_opt, get_option('bindir'))
>  
>  # set the machine type and cflags for it
>  if meson.is_cross_build()

As you may have seen, build fails because find_program() result cannot be
used in meson.add_install_script() until 0.55. Since your script has
Unix-specific part anyway and Windows recommends Meson 0.56, maybe Unices
should continue using shell variant and Python script can be Windows-only.

^ permalink raw reply	[flat|nested] 11+ messages in thread

* Re: [dpdk-dev] [PATCH] build: fix symlink of drivers for Windows
  2021-04-04  0:00 ` Dmitry Kozlyuk
@ 2021-04-06 10:24   ` Nick Connolly
  0 siblings, 0 replies; 11+ messages in thread
From: Nick Connolly @ 2021-04-06 10:24 UTC (permalink / raw)
  To: Dmitry Kozlyuk
  Cc: Bruce Richardson, Ranjit Menon, dev, navasile, dmitrym,
	pallavi.kadam, stable


> PDB (debuginfo) files can be quite large, do we want to install them?
The script copies will only copy the .pdb files if they have already 
been installed into
lib/dpdk/pmds-*/. If they haven't, we'll just skip the step.

Whether they should be installed is a different question :-) Installing 
them seems to
be a deliberate choice by meson 
(https://github.com/mesonbuild/meson/issues/1442).
In my opinion, it's probably best to have them there.

I looked at options apart from copying (including hard links) but in the 
end decided to
keep it as simple as possible so it will 'just work' for cross builds 
from Linux, WSL-1,
WSL-2, Msys, Cygnus etc.

> +meson.add_install_script(py3, '../buildtools/symlink-drivers-solibs.py',
> +	get_option('libdir'), pmd_subdir_opt, get_option('bindir'))
>   
>   # set the machine type and cflags for it
>   if meson.is_cross_build()
> As you may have seen, build fails because find_program() result cannot be
> used in meson.add_install_script() until 0.55. Since your script has
> Unix-specific part anyway and Windows recommends Meson 0.56, maybe Unices
> should continue using shell variant and Python script can be Windows-only.

Yes, I 'd been thinking about the best way to handle this. Your 
suggestion of having both
scripts may well be the best option in the short term.

Thanks,
Nick


^ permalink raw reply	[flat|nested] 11+ messages in thread

* [dpdk-dev] [PATCH v2] build: fix symlink of drivers for Windows
  2021-04-01 12:27 [dpdk-dev] [PATCH] build: fix symlink of drivers for Windows Nick Connolly
  2021-04-04  0:00 ` Dmitry Kozlyuk
@ 2021-04-10  8:01 ` Nick Connolly
  2021-04-24  0:53   ` Narcisa Ana Maria Vasile
  2021-04-26 10:07 ` [dpdk-dev] [PATCH v3] " Nick Connolly
  2 siblings, 1 reply; 11+ messages in thread
From: Nick Connolly @ 2021-04-10  8:01 UTC (permalink / raw)
  To: Bruce Richardson, Nick Connolly, Ranjit Menon
  Cc: dev, dmitry.kozliuk, navasile, dmitrym, pallavi.kadam, stable

The symlink-drivers-solibs.sh script was disabled as part of 'install'
for Windows because there is no support for shell scripts. However,
this means that driver related DLLs are not present in the installed
'libdir' directory. Add a python script to perform the install and use
it for Windows if the version of meson supports using an external
program with add_install_script (>= 0.55.0).

On Windows, symbolic links are somewhat problematic since the
SeCreateSymbolicLinkPrivilege is required to be able to create them.
In addition, different cross-compilation environments handle symbolic
links differently, e.g. WSL, Msys2, Cygwin. Rather than trying to
distinguish these scenarios, the python script will perform a file copy
for any Windows specific names.

On Windows, the shared library outputs have different names depending
upon which toolset has been used to build them. The script currently
handles Clang and GCC.

On Linux the functionality is unchanged, but could be replaced with the
python script once the required minimum version of meson is >= 0.55.0.

Fixes: 5c7d86948764 ("build: fix install on Windows")
Cc: stable@dpdk.org

Signed-off-by: Nick Connolly <nick.connolly@mayadata.io>
---
 buildtools/symlink-drivers-solibs.py | 49 ++++++++++++++++++++++++++++
 config/meson.build                   |  4 +++
 2 files changed, 53 insertions(+)
 create mode 100644 buildtools/symlink-drivers-solibs.py

diff --git a/buildtools/symlink-drivers-solibs.py b/buildtools/symlink-drivers-solibs.py
new file mode 100644
index 000000000..5627ddd9d
--- /dev/null
+++ b/buildtools/symlink-drivers-solibs.py
@@ -0,0 +1,49 @@
+#!/usr/bin/env python3
+# SPDX-License-Identifier: BSD-3-Clause
+# Copyright(c) 2021 Intel Corporation
+
+import os
+import sys
+import glob
+import shutil
+
+# post-install script for meson/ninja builds to symlink the PMDs stored in
+# $libdir/dpdk/pmds-*/ to $libdir. This is needed as some PMDs depend on
+# others, e.g. PCI device PMDs depending on the PCI bus driver.
+
+# parameters to script are paths relative to install prefix:
+# 1. directory for installed regular libs e.g. lib64
+# 2. subdirectory of libdir where the pmds are
+# 3. directory for installed regular binaries e.g. bin
+
+os.chdir(os.environ['MESON_INSTALL_DESTDIR_PREFIX'])
+
+lib_dir = sys.argv[1]
+pmd_subdir = sys.argv[2]
+bin_dir = sys.argv[3]
+pmd_dir = os.path.join(lib_dir, pmd_subdir)
+
+# copy Windows PMDs to avoid any issues with symlinks since the
+# build could be a cross-compilation under WSL, Msys or Cygnus.
+# the filenames are dependent upon the specific toolchain in use.
+
+def copy_pmd_files(pattern, to_dir):
+	for file in glob.glob(os.path.join(pmd_dir, pattern)):
+		to = os.path.join(to_dir, os.path.basename(file))
+		shutil.copy2(file, to)
+		print(to + ' -> ' + file)
+
+copy_pmd_files('*rte_*.dll', bin_dir)
+copy_pmd_files('*rte_*.pdb', bin_dir)
+copy_pmd_files('*rte_*.lib', lib_dir)
+copy_pmd_files('*rte_*.dll.a', lib_dir)
+
+# symlink shared objects
+
+os.chdir(lib_dir)
+for file in glob.glob(os.path.join(pmd_subdir, 'librte_*.so*')):
+	to = os.path.basename(file)
+	if os.path.exists(to):
+		os.remove(to)
+	os.symlink(file, to)
+	print(to + ' -> ' + file)
diff --git a/config/meson.build b/config/meson.build
index 66a2edcc4..d0a54b02a 100644
--- a/config/meson.build
+++ b/config/meson.build
@@ -61,6 +61,10 @@ if not is_windows
 	meson.add_install_script('../buildtools/symlink-drivers-solibs.sh',
 			get_option('libdir'),
 			pmd_subdir_opt)
+elif meson.version().version_compare('>=0.55.0')
+	# 0.55.0 is required to use external program with add_install_script
+	meson.add_install_script(py3, '../buildtools/symlink-drivers-solibs.py',
+		get_option('libdir'), pmd_subdir_opt, get_option('bindir'))
 endif
 
 # set the machine type and cflags for it
-- 
2.25.1


^ permalink raw reply	[flat|nested] 11+ messages in thread

* Re: [dpdk-dev] [PATCH v2] build: fix symlink of drivers for Windows
  2021-04-10  8:01 ` [dpdk-dev] [PATCH v2] " Nick Connolly
@ 2021-04-24  0:53   ` Narcisa Ana Maria Vasile
  0 siblings, 0 replies; 11+ messages in thread
From: Narcisa Ana Maria Vasile @ 2021-04-24  0:53 UTC (permalink / raw)
  To: Nick Connolly
  Cc: Bruce Richardson, Ranjit Menon, dev, dmitry.kozliuk, dmitrym,
	pallavi.kadam, stable

On Sat, Apr 10, 2021 at 09:01:43AM +0100, Nick Connolly wrote:
> The symlink-drivers-solibs.sh script was disabled as part of 'install'
> for Windows because there is no support for shell scripts. However,
> this means that driver related DLLs are not present in the installed
> 'libdir' directory. Add a python script to perform the install and use
> it for Windows if the version of meson supports using an external
> program with add_install_script (>= 0.55.0).
> 
> On Windows, symbolic links are somewhat problematic since the
> SeCreateSymbolicLinkPrivilege is required to be able to create them.
> In addition, different cross-compilation environments handle symbolic
> links differently, e.g. WSL, Msys2, Cygwin. Rather than trying to
> distinguish these scenarios, the python script will perform a file copy
> for any Windows specific names.
> 
> On Windows, the shared library outputs have different names depending
> upon which toolset has been used to build them. The script currently
> handles Clang and GCC.
> 
> On Linux the functionality is unchanged, but could be replaced with the
> python script once the required minimum version of meson is >= 0.55.0.
> 
> Fixes: 5c7d86948764 ("build: fix install on Windows")
> Cc: stable@dpdk.org
> 
> Signed-off-by: Nick Connolly <nick.connolly@mayadata.io>
> ---
>  buildtools/symlink-drivers-solibs.py | 49 ++++++++++++++++++++++++++++
>  config/meson.build                   |  4 +++
>  2 files changed, 53 insertions(+)
>  create mode 100644 buildtools/symlink-drivers-solibs.py
> 
> diff --git a/buildtools/symlink-drivers-solibs.py b/buildtools/symlink-drivers-solibs.py

Tested-by: Narcisa Vasile <navasile@linux.microsoft.com>
Acked-by: Narcisa Vasile <navasile@linux.microsoft.com>

Note that it needs rebasing.

^ permalink raw reply	[flat|nested] 11+ messages in thread

* [dpdk-dev] [PATCH v3] build: fix symlink of drivers for Windows
  2021-04-01 12:27 [dpdk-dev] [PATCH] build: fix symlink of drivers for Windows Nick Connolly
  2021-04-04  0:00 ` Dmitry Kozlyuk
  2021-04-10  8:01 ` [dpdk-dev] [PATCH v2] " Nick Connolly
@ 2021-04-26 10:07 ` Nick Connolly
  2021-05-27 17:37   ` Nick Connolly
  2021-05-28  8:19   ` Bruce Richardson
  2 siblings, 2 replies; 11+ messages in thread
From: Nick Connolly @ 2021-04-26 10:07 UTC (permalink / raw)
  To: Bruce Richardson, Nick Connolly, Ranjit Menon
  Cc: dev, dmitry.kozliuk, navasile, dmitrym, pallavi.kadam, stable

The symlink-drivers-solibs.sh script was disabled as part of 'install'
for Windows because there is no support for shell scripts. However,
this means that driver related DLLs are not present in the installed
'libdir' directory. Add a python script to perform the install and use
it for Windows if the version of meson supports using an external
program with add_install_script (>= 0.55.0).

On Windows, symbolic links are somewhat problematic since the
SeCreateSymbolicLinkPrivilege is required to be able to create them.
In addition, different cross-compilation environments handle symbolic
links differently, e.g. WSL, Msys2, Cygwin. Rather than trying to
distinguish these scenarios, the python script will perform a file copy
for any Windows specific names.

On Windows, the shared library outputs have different names depending
upon which toolset has been used to build them. The script currently
handles Clang and GCC.

On Linux the functionality is unchanged, but could be replaced with the
python script once the required minimum version of meson is >= 0.55.0.

Fixes: 5c7d86948764 ("build: fix install on Windows")
Cc: stable@dpdk.org

Signed-off-by: Nick Connolly <nick.connolly@mayadata.io>
Tested-by: Narcisa Vasile <navasile@linux.microsoft.com>
Acked-by: Narcisa Vasile <navasile@linux.microsoft.com>
---
v3:
* rebase

v2:
* 0.55.0 is required to use external program with add_install_script

 buildtools/symlink-drivers-solibs.py | 49 ++++++++++++++++++++++++++++
 config/meson.build                   |  4 +++
 2 files changed, 53 insertions(+)
 create mode 100644 buildtools/symlink-drivers-solibs.py

diff --git a/buildtools/symlink-drivers-solibs.py b/buildtools/symlink-drivers-solibs.py
new file mode 100644
index 000000000..5627ddd9d
--- /dev/null
+++ b/buildtools/symlink-drivers-solibs.py
@@ -0,0 +1,49 @@
+#!/usr/bin/env python3
+# SPDX-License-Identifier: BSD-3-Clause
+# Copyright(c) 2021 Intel Corporation
+
+import os
+import sys
+import glob
+import shutil
+
+# post-install script for meson/ninja builds to symlink the PMDs stored in
+# $libdir/dpdk/pmds-*/ to $libdir. This is needed as some PMDs depend on
+# others, e.g. PCI device PMDs depending on the PCI bus driver.
+
+# parameters to script are paths relative to install prefix:
+# 1. directory for installed regular libs e.g. lib64
+# 2. subdirectory of libdir where the pmds are
+# 3. directory for installed regular binaries e.g. bin
+
+os.chdir(os.environ['MESON_INSTALL_DESTDIR_PREFIX'])
+
+lib_dir = sys.argv[1]
+pmd_subdir = sys.argv[2]
+bin_dir = sys.argv[3]
+pmd_dir = os.path.join(lib_dir, pmd_subdir)
+
+# copy Windows PMDs to avoid any issues with symlinks since the
+# build could be a cross-compilation under WSL, Msys or Cygnus.
+# the filenames are dependent upon the specific toolchain in use.
+
+def copy_pmd_files(pattern, to_dir):
+	for file in glob.glob(os.path.join(pmd_dir, pattern)):
+		to = os.path.join(to_dir, os.path.basename(file))
+		shutil.copy2(file, to)
+		print(to + ' -> ' + file)
+
+copy_pmd_files('*rte_*.dll', bin_dir)
+copy_pmd_files('*rte_*.pdb', bin_dir)
+copy_pmd_files('*rte_*.lib', lib_dir)
+copy_pmd_files('*rte_*.dll.a', lib_dir)
+
+# symlink shared objects
+
+os.chdir(lib_dir)
+for file in glob.glob(os.path.join(pmd_subdir, 'librte_*.so*')):
+	to = os.path.basename(file)
+	if os.path.exists(to):
+		os.remove(to)
+	os.symlink(file, to)
+	print(to + ' -> ' + file)
diff --git a/config/meson.build b/config/meson.build
index 017bb2efb..9e04c4c76 100644
--- a/config/meson.build
+++ b/config/meson.build
@@ -59,6 +59,10 @@ eal_pmd_path = join_paths(get_option('prefix'), driver_install_path)
 if not is_windows
     meson.add_install_script('../buildtools/symlink-drivers-solibs.sh',
             get_option('libdir'), pmd_subdir_opt)
+elif meson.version().version_compare('>=0.55.0')
+    # 0.55.0 is required to use external program with add_install_script
+    meson.add_install_script(py3, '../buildtools/symlink-drivers-solibs.py',
+            get_option('libdir'), pmd_subdir_opt, get_option('bindir'))
 endif
 
 # init disable/enable driver lists that will be populated in different places
-- 
2.25.1


^ permalink raw reply	[flat|nested] 11+ messages in thread

* Re: [dpdk-dev] [PATCH v3] build: fix symlink of drivers for Windows
  2021-04-26 10:07 ` [dpdk-dev] [PATCH v3] " Nick Connolly
@ 2021-05-27 17:37   ` Nick Connolly
  2021-05-28  8:19     ` Bruce Richardson
  2021-05-28  8:19   ` Bruce Richardson
  1 sibling, 1 reply; 11+ messages in thread
From: Nick Connolly @ 2021-05-27 17:37 UTC (permalink / raw)
  To: Bruce Richardson, Ranjit Menon
  Cc: dev, dmitry.kozliuk, navasile, dmitrym, pallavi.kadam, stable

Hi Bruce,

Would you have some time to take a look at this?  It's a replacement for the
symlink-drivers-solibs.sh script for windows builds.

Thanks,
Nick

On 26/04/2021 11:07, Nick Connolly wrote:
> The symlink-drivers-solibs.sh script was disabled as part of 'install'
> for Windows because there is no support for shell scripts. However,
> this means that driver related DLLs are not present in the installed
> 'libdir' directory. Add a python script to perform the install and use
> it for Windows if the version of meson supports using an external
> program with add_install_script (>= 0.55.0).
>
> On Windows, symbolic links are somewhat problematic since the
> SeCreateSymbolicLinkPrivilege is required to be able to create them.
> In addition, different cross-compilation environments handle symbolic
> links differently, e.g. WSL, Msys2, Cygwin. Rather than trying to
> distinguish these scenarios, the python script will perform a file copy
> for any Windows specific names.
>
> On Windows, the shared library outputs have different names depending
> upon which toolset has been used to build them. The script currently
> handles Clang and GCC.
>
> On Linux the functionality is unchanged, but could be replaced with the
> python script once the required minimum version of meson is >= 0.55.0.
>
> Fixes: 5c7d86948764 ("build: fix install on Windows")
> Cc: stable@dpdk.org
>
> Signed-off-by: Nick Connolly <nick.connolly@mayadata.io>
> Tested-by: Narcisa Vasile <navasile@linux.microsoft.com>
> Acked-by: Narcisa Vasile <navasile@linux.microsoft.com>
> ---
> v3:
> * rebase
>
> v2:
> * 0.55.0 is required to use external program with add_install_script
>
>   buildtools/symlink-drivers-solibs.py | 49 ++++++++++++++++++++++++++++
>   config/meson.build                   |  4 +++
>   2 files changed, 53 insertions(+)
>   create mode 100644 buildtools/symlink-drivers-solibs.py
>
> diff --git a/buildtools/symlink-drivers-solibs.py b/buildtools/symlink-drivers-solibs.py
> new file mode 100644
> index 000000000..5627ddd9d
> --- /dev/null
> +++ b/buildtools/symlink-drivers-solibs.py
> @@ -0,0 +1,49 @@
> +#!/usr/bin/env python3
> +# SPDX-License-Identifier: BSD-3-Clause
> +# Copyright(c) 2021 Intel Corporation
> +
> +import os
> +import sys
> +import glob
> +import shutil
> +
> +# post-install script for meson/ninja builds to symlink the PMDs stored in
> +# $libdir/dpdk/pmds-*/ to $libdir. This is needed as some PMDs depend on
> +# others, e.g. PCI device PMDs depending on the PCI bus driver.
> +
> +# parameters to script are paths relative to install prefix:
> +# 1. directory for installed regular libs e.g. lib64
> +# 2. subdirectory of libdir where the pmds are
> +# 3. directory for installed regular binaries e.g. bin
> +
> +os.chdir(os.environ['MESON_INSTALL_DESTDIR_PREFIX'])
> +
> +lib_dir = sys.argv[1]
> +pmd_subdir = sys.argv[2]
> +bin_dir = sys.argv[3]
> +pmd_dir = os.path.join(lib_dir, pmd_subdir)
> +
> +# copy Windows PMDs to avoid any issues with symlinks since the
> +# build could be a cross-compilation under WSL, Msys or Cygnus.
> +# the filenames are dependent upon the specific toolchain in use.
> +
> +def copy_pmd_files(pattern, to_dir):
> +	for file in glob.glob(os.path.join(pmd_dir, pattern)):
> +		to = os.path.join(to_dir, os.path.basename(file))
> +		shutil.copy2(file, to)
> +		print(to + ' -> ' + file)
> +
> +copy_pmd_files('*rte_*.dll', bin_dir)
> +copy_pmd_files('*rte_*.pdb', bin_dir)
> +copy_pmd_files('*rte_*.lib', lib_dir)
> +copy_pmd_files('*rte_*.dll.a', lib_dir)
> +
> +# symlink shared objects
> +
> +os.chdir(lib_dir)
> +for file in glob.glob(os.path.join(pmd_subdir, 'librte_*.so*')):
> +	to = os.path.basename(file)
> +	if os.path.exists(to):
> +		os.remove(to)
> +	os.symlink(file, to)
> +	print(to + ' -> ' + file)
> diff --git a/config/meson.build b/config/meson.build
> index 017bb2efb..9e04c4c76 100644
> --- a/config/meson.build
> +++ b/config/meson.build
> @@ -59,6 +59,10 @@ eal_pmd_path = join_paths(get_option('prefix'), driver_install_path)
>   if not is_windows
>       meson.add_install_script('../buildtools/symlink-drivers-solibs.sh',
>               get_option('libdir'), pmd_subdir_opt)
> +elif meson.version().version_compare('>=0.55.0')
> +    # 0.55.0 is required to use external program with add_install_script
> +    meson.add_install_script(py3, '../buildtools/symlink-drivers-solibs.py',
> +            get_option('libdir'), pmd_subdir_opt, get_option('bindir'))
>   endif
>   
>   # init disable/enable driver lists that will be populated in different places


^ permalink raw reply	[flat|nested] 11+ messages in thread

* Re: [dpdk-dev] [PATCH v3] build: fix symlink of drivers for Windows
  2021-04-26 10:07 ` [dpdk-dev] [PATCH v3] " Nick Connolly
  2021-05-27 17:37   ` Nick Connolly
@ 2021-05-28  8:19   ` Bruce Richardson
  2021-07-09 15:05     ` [dpdk-dev] [dpdk-stable] " Thomas Monjalon
  1 sibling, 1 reply; 11+ messages in thread
From: Bruce Richardson @ 2021-05-28  8:19 UTC (permalink / raw)
  To: Nick Connolly
  Cc: Ranjit Menon, dev, dmitry.kozliuk, navasile, dmitrym,
	pallavi.kadam, stable

On Mon, Apr 26, 2021 at 11:07:32AM +0100, Nick Connolly wrote:
> The symlink-drivers-solibs.sh script was disabled as part of 'install'
> for Windows because there is no support for shell scripts. However,
> this means that driver related DLLs are not present in the installed
> 'libdir' directory. Add a python script to perform the install and use
> it for Windows if the version of meson supports using an external
> program with add_install_script (>= 0.55.0).
> 
> On Windows, symbolic links are somewhat problematic since the
> SeCreateSymbolicLinkPrivilege is required to be able to create them.
> In addition, different cross-compilation environments handle symbolic
> links differently, e.g. WSL, Msys2, Cygwin. Rather than trying to
> distinguish these scenarios, the python script will perform a file copy
> for any Windows specific names.
> 
> On Windows, the shared library outputs have different names depending
> upon which toolset has been used to build them. The script currently
> handles Clang and GCC.
> 
> On Linux the functionality is unchanged, but could be replaced with the
> python script once the required minimum version of meson is >= 0.55.0.
> 
> Fixes: 5c7d86948764 ("build: fix install on Windows")
> Cc: stable@dpdk.org
> 
> Signed-off-by: Nick Connolly <nick.connolly@mayadata.io>
> Tested-by: Narcisa Vasile <navasile@linux.microsoft.com>
> Acked-by: Narcisa Vasile <navasile@linux.microsoft.com>
> ---
Reviewed-by: Bruce Richardson <bruce.richardson@intel.com>


^ permalink raw reply	[flat|nested] 11+ messages in thread

* Re: [dpdk-dev] [PATCH v3] build: fix symlink of drivers for Windows
  2021-05-27 17:37   ` Nick Connolly
@ 2021-05-28  8:19     ` Bruce Richardson
  2021-05-28 10:50       ` Nick Connolly
  0 siblings, 1 reply; 11+ messages in thread
From: Bruce Richardson @ 2021-05-28  8:19 UTC (permalink / raw)
  To: Nick Connolly
  Cc: Ranjit Menon, dev, dmitry.kozliuk, navasile, dmitrym,
	pallavi.kadam, stable

On Thu, May 27, 2021 at 06:37:57PM +0100, Nick Connolly wrote:
> Hi Bruce,
> 
> Would you have some time to take a look at this?  It's a replacement for the
> symlink-drivers-solibs.sh script for windows builds.
> 
> Thanks,
> Nick
> 
I've reviewed it now and it looks good. Couldn't even find anything to
nit-pick in it! :-)

^ permalink raw reply	[flat|nested] 11+ messages in thread

* Re: [dpdk-dev] [PATCH v3] build: fix symlink of drivers for Windows
  2021-05-28  8:19     ` Bruce Richardson
@ 2021-05-28 10:50       ` Nick Connolly
  0 siblings, 0 replies; 11+ messages in thread
From: Nick Connolly @ 2021-05-28 10:50 UTC (permalink / raw)
  To: Bruce Richardson
  Cc: Ranjit Menon, dev, dmitry.kozliuk, navasile, dmitrym,
	pallavi.kadam, stable

> I've reviewed it now and it looks good. Couldn't even find anything to
> nit-pick in it! :-)

Thanks Bruce - appreciated  :-)
Nick

^ permalink raw reply	[flat|nested] 11+ messages in thread

* Re: [dpdk-dev] [dpdk-stable] [PATCH v3] build: fix symlink of drivers for Windows
  2021-05-28  8:19   ` Bruce Richardson
@ 2021-07-09 15:05     ` Thomas Monjalon
  0 siblings, 0 replies; 11+ messages in thread
From: Thomas Monjalon @ 2021-07-09 15:05 UTC (permalink / raw)
  To: Nick Connolly
  Cc: stable, Ranjit Menon, dev, dmitry.kozliuk, navasile, dmitrym,
	pallavi.kadam, Bruce Richardson

28/05/2021 10:19, Bruce Richardson:
> On Mon, Apr 26, 2021 at 11:07:32AM +0100, Nick Connolly wrote:
> > The symlink-drivers-solibs.sh script was disabled as part of 'install'
> > for Windows because there is no support for shell scripts. However,
> > this means that driver related DLLs are not present in the installed
> > 'libdir' directory. Add a python script to perform the install and use
> > it for Windows if the version of meson supports using an external
> > program with add_install_script (>= 0.55.0).
> > 
> > On Windows, symbolic links are somewhat problematic since the
> > SeCreateSymbolicLinkPrivilege is required to be able to create them.
> > In addition, different cross-compilation environments handle symbolic
> > links differently, e.g. WSL, Msys2, Cygwin. Rather than trying to
> > distinguish these scenarios, the python script will perform a file copy
> > for any Windows specific names.
> > 
> > On Windows, the shared library outputs have different names depending
> > upon which toolset has been used to build them. The script currently
> > handles Clang and GCC.
> > 
> > On Linux the functionality is unchanged, but could be replaced with the
> > python script once the required minimum version of meson is >= 0.55.0.
> > 
> > Fixes: 5c7d86948764 ("build: fix install on Windows")
> > Cc: stable@dpdk.org
> > 
> > Signed-off-by: Nick Connolly <nick.connolly@mayadata.io>
> > Tested-by: Narcisa Vasile <navasile@linux.microsoft.com>
> > Acked-by: Narcisa Vasile <navasile@linux.microsoft.com>
> > ---
> Reviewed-by: Bruce Richardson <bruce.richardson@intel.com>

Added the new file in MAINTAINERS.

Applied, thanks.



^ permalink raw reply	[flat|nested] 11+ messages in thread

end of thread, other threads:[~2021-07-09 15:05 UTC | newest]

Thread overview: 11+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2021-04-01 12:27 [dpdk-dev] [PATCH] build: fix symlink of drivers for Windows Nick Connolly
2021-04-04  0:00 ` Dmitry Kozlyuk
2021-04-06 10:24   ` Nick Connolly
2021-04-10  8:01 ` [dpdk-dev] [PATCH v2] " Nick Connolly
2021-04-24  0:53   ` Narcisa Ana Maria Vasile
2021-04-26 10:07 ` [dpdk-dev] [PATCH v3] " Nick Connolly
2021-05-27 17:37   ` Nick Connolly
2021-05-28  8:19     ` Bruce Richardson
2021-05-28 10:50       ` Nick Connolly
2021-05-28  8:19   ` Bruce Richardson
2021-07-09 15:05     ` [dpdk-dev] [dpdk-stable] " Thomas Monjalon

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).