[dpdk-dev] [PATCH 0/6] MinGW-w64 support

[dpdk-dev] [PATCH 0/6] MinGW-w64 support

[Dmitry Kozlyuk]

This patch series add support for building DPDK using MinGW-w64.

MinGW-w64 provides GNU toolchain and independent platform SDK on
Windows. It also supports cross-compilation to Windows from POSIX
systems by providing cross tollchains and libraries [0]. It does NOT
emulate a full POSIX environment, like Cygwin or MSYS do.

There are advantages in using MinGW-w64 in addition to Clang:

1. Cross-compilation out-of-the-box. MinGW-w64 is provides a pthread
   implementation, GNU getopt, and Windows platform SDK.

2. Easier porting of POSIX applications using DPDK to Windows, because
   application code can use the same benefits as mentioned above.

3. Having both primary compilers enabled on Windows provides more
   diagnostics and generally prevents non-portable code.

[0]: http://mingw-w64.org

Dmitry Kozlyuk (6):
  eal: introduce portable format attribute
  eal: use portable format attribute
  cmdline: use portable format attribute
  build: MinGW-w64 support for Meson
  build: add cross-file for MinGW-w64
  doc: guide for Windows build using MinGW-w64

 config/meson.build                          | 14 +++++
 doc/guides/windows_gsg/build_dpdk.rst       | 57 +++++++++++++++++----
 lib/librte_cmdline/cmdline.h                |  4 +-
 lib/librte_eal/common/include/rte_common.h  | 17 +++++-
 lib/librte_eal/common/include/rte_debug.h   |  2 +-
 lib/librte_eal/common/include/rte_devargs.h |  2 +-
 lib/librte_eal/common/include/rte_log.h     |  4 +-
 lib/librte_eal/meson.build                  |  3 ++
 lib/meson.build                             |  8 ++-
 meson_mingw.txt                             | 14 +++++
 10 files changed, 108 insertions(+), 17 deletions(-)
 create mode 100644 meson_mingw.txt

-- 
2.25.0

[dpdk-dev] [PATCH 1/6] eal: introduce portable format attribute

[Dmitry Kozlyuk]

When using __attribute__((format(...)) on functions, GCC on Windows
assumes MS-specific format string by default, even if the underlying
stdio implementation is ANSI-compliant (either MS Unicersal CRT
or MinGW implementation). Wrap attribute into a macro that forces
GNU-specific format string when using GCC.

Signed-off-by: Dmitry Kozlyuk <dmitry.kozliuk@gmail.com>

diff --git a/lib/librte_eal/common/include/rte_common.h b/lib/librte_eal/common/include/rte_common.h
index 4b5f3a31f..2f086bb9c 100644
--- a/lib/librte_eal/common/include/rte_common.h
+++ b/lib/librte_eal/common/include/rte_common.h
@@ -89,6 +89,21 @@ typedef uint16_t unaligned_uint16_t;
  */
 #define RTE_SET_USED(x) (void)(x)
 
+/**
+ * Check format string and its arguments at compile-time.
+ *
+ * GCC on Windows assumes MS-specific format string by default,
+ * even if the underlying stdio implementation is ANSI-compliant,
+ * so this must be overridden.
+ */
+#if defined(RTE_TOOLCHAIN_GCC)
+#define __rte_format(archetype, format_index, first_arg) \
+	__attribute__((format(gnu_##archetype, format_index, first_arg)))
+#else
+#define __rte_format(archetype, format_index, first_arg) \
+	__attribute__((format(archetype, format_index, first_arg)))
+#endif
+
 #define RTE_PRIORITY_LOG 101
 #define RTE_PRIORITY_BUS 110
 #define RTE_PRIORITY_CLASS 120
-- 
2.25.0

[dpdk-dev] [PATCH 2/6] eal: use portable format attribute

[Dmitry Kozlyuk]

Use portable format attribute for logging and panic messages.

Signed-off-by: Dmitry Kozlyuk <dmitry.kozliuk@gmail.com>

diff --git a/lib/librte_eal/common/include/rte_common.h b/lib/librte_eal/common/include/rte_common.h
index 2f086bb9c..dc406ce27 100644
--- a/lib/librte_eal/common/include/rte_common.h
+++ b/lib/librte_eal/common/include/rte_common.h
@@ -799,7 +799,7 @@ rte_str_to_size(const char *str)
 void
 rte_exit(int exit_code, const char *format, ...)
 	__attribute__((noreturn))
-	__attribute__((format(printf, 2, 3)));
+	__rte_format(printf, 2, 3);
 
 #ifdef __cplusplus
 }
diff --git a/lib/librte_eal/common/include/rte_debug.h b/lib/librte_eal/common/include/rte_debug.h
index 748d32c80..c47c6bcdc 100644
--- a/lib/librte_eal/common/include/rte_debug.h
+++ b/lib/librte_eal/common/include/rte_debug.h
@@ -73,7 +73,7 @@ void __rte_panic(const char *funcname , const char *format, ...)
 #endif
 #endif
 	__attribute__((noreturn))
-	__attribute__((format(printf, 2, 3)));
+	__rte_format(printf, 2, 3);
 
 #ifdef __cplusplus
 }
diff --git a/lib/librte_eal/common/include/rte_devargs.h b/lib/librte_eal/common/include/rte_devargs.h
index 882dfa0ab..4418e02f2 100644
--- a/lib/librte_eal/common/include/rte_devargs.h
+++ b/lib/librte_eal/common/include/rte_devargs.h
@@ -137,7 +137,7 @@ rte_devargs_parse(struct rte_devargs *da, const char *dev);
 int
 rte_devargs_parsef(struct rte_devargs *da,
 		   const char *format, ...)
-__attribute__((format(printf, 2, 0)));
+__rte_format(printf, 2, 0);
 
 /**
  * Insert an rte_devargs in the global list.
diff --git a/lib/librte_eal/common/include/rte_log.h b/lib/librte_eal/common/include/rte_log.h
index 1bb0e6694..823efea4e 100644
--- a/lib/librte_eal/common/include/rte_log.h
+++ b/lib/librte_eal/common/include/rte_log.h
@@ -282,7 +282,7 @@ int rte_log(uint32_t level, uint32_t logtype, const char *format, ...)
 	__attribute__((cold))
 #endif
 #endif
-	__attribute__((format(printf, 3, 4)));
+	__rte_format(printf, 3, 4);
 
 /**
  * Generates a log message.
@@ -311,7 +311,7 @@ int rte_log(uint32_t level, uint32_t logtype, const char *format, ...)
  *   - Negative on error.
  */
 int rte_vlog(uint32_t level, uint32_t logtype, const char *format, va_list ap)
-	__attribute__((format(printf,3,0)));
+	__rte_format(printf, 3, 0);
 
 /**
  * Generates a log message.
-- 
2.25.0

[dpdk-dev] [PATCH 3/6] cmdline: use portable format attribute

[Dmitry Kozlyuk]

Use portable format attribute for output strings.

Signed-off-by: Dmitry Kozlyuk <dmitry.kozliuk@gmail.com>

diff --git a/lib/librte_cmdline/cmdline.h b/lib/librte_cmdline/cmdline.h
index 27d2effdf..952a50829 100644
--- a/lib/librte_cmdline/cmdline.h
+++ b/lib/librte_cmdline/cmdline.h
@@ -7,6 +7,8 @@
 #ifndef _CMDLINE_H_
 #define _CMDLINE_H_
 
+#include <rte_common.h>
+
 #include <termios.h>
 #include <cmdline_rdline.h>
 #include <cmdline_parse.h>
@@ -34,7 +36,7 @@ struct cmdline *cmdline_new(cmdline_parse_ctx_t *ctx, const char *prompt, int s_
 void cmdline_set_prompt(struct cmdline *cl, const char *prompt);
 void cmdline_free(struct cmdline *cl);
 void cmdline_printf(const struct cmdline *cl, const char *fmt, ...)
-	__attribute__((format(printf,2,3)));
+	__rte_format(printf, 2, 3);
 int cmdline_in(struct cmdline *cl, const char *buf, int size);
 int cmdline_write_char(struct rdline *rdl, char c);
 
-- 
2.25.0

[dpdk-dev] [PATCH 4/6] build: MinGW-w64 support for Meson

[Dmitry Kozlyuk]

MinGW-w64 linker does not mimic MS linker options, so the build system
must differentiate between linkers on Windows. Use GNU linker options
with GCC and MS linker options with Clang.

MinGW-w64 by default uses MSVCRT stdio, which does not comply to ANSI,
most notably its formatting and string handling functions. MinGW-w64
support for the Universal CRT (UCRT) is ongoing, but the toolchain
provides its own standard-complying implementation of stdio. The latter
is used in the patch to support formatting in DPDK.

Signed-off-by: Dmitry Kozlyuk <dmitry.kozliuk@gmail.com>

diff --git a/config/meson.build b/config/meson.build
index 28a57f56f..a2f6f5ab1 100644
--- a/config/meson.build
+++ b/config/meson.build
@@ -14,6 +14,10 @@ foreach env:supported_exec_envs
 	set_variable('is_' + env, exec_env == env)
 endforeach
 
+# MS linker requires special treatment.
+# FIXME: use cc.get_linker_id() after upgrading to Meson >=0.53.
+is_ms_linker = is_windows and (cc.get_id() == 'clang')
+
 # set the major version, which might be used by drivers and libraries
 # depending on the configuration options
 pver = meson.project_version().split('.')
@@ -241,6 +245,16 @@ if is_freebsd
 	add_project_arguments('-D__BSD_VISIBLE', language: 'c')
 endif
 
+if is_windows
+	# Require platform SDK for Windows 7 and above.
+	add_project_arguments('-D_WIN32_WINNT=0x0601', language: 'c')
+
+	# Use MinGW-w64 stdio, because DPDK assumes ANSI-compliant formatting.
+	if cc.get_id() == 'gcc'
+		add_project_arguments('-D__USE_MINGW_ANSI_STDIO', language: 'c')
+	endif
+endif
+
 if get_option('b_lto')
 	if cc.has_argument('-ffat-lto-objects')
 		add_project_arguments('-ffat-lto-objects', language: 'c')
diff --git a/lib/librte_eal/meson.build b/lib/librte_eal/meson.build
index 4be5118ce..eae8c2ba8 100644
--- a/lib/librte_eal/meson.build
+++ b/lib/librte_eal/meson.build
@@ -20,6 +20,9 @@ endif
 if cc.has_function('getentropy', prefix : '#include <unistd.h>')
 	cflags += '-DRTE_LIBEAL_USE_GETENTROPY'
 endif
+if cc.get_id() == 'gcc'
+	cflags += '-D__USE_MINGW_ANSI_STDIO'
+endif
 sources = common_sources + env_sources
 objs = common_objs + env_objs
 headers = common_headers + env_headers
diff --git a/lib/meson.build b/lib/meson.build
index 0af3efab2..9c3cc55d5 100644
--- a/lib/meson.build
+++ b/lib/meson.build
@@ -148,12 +148,16 @@ foreach l:libraries
 				command: [map_to_def_cmd, '@INPUT@', '@OUTPUT@'],
 				input: version_map,
 				output: 'rte_@0@_exports.def'.format(name))
-			lk_deps = [version_map, def_file]
-			if is_windows
+
+			if is_ms_linker
 				lk_args = ['-Wl,/def:' + def_file.full_path(),
 					'-Wl,/implib:lib\\' + implib]
 			else
 				lk_args = ['-Wl,--version-script=' + version_map]
+			endif
+
+			lk_deps = [version_map, def_file]
+			if not is_windows
 				# on unix systems check the output of the
 				# experimental syms script, using it as a
 				# dependency of the .so build
-- 
2.25.0

[dpdk-dev] [PATCH 5/6] build: add cross-file for MinGW-w64

[Dmitry Kozlyuk]

Add Meson configuration to cross-compile for Windows using MinGW-w64.
It may require adjustments in some cases, but at least it provides
the foundation.

Signed-off-by: Dmitry Kozlyuk <dmitry.kozliuk@gmail.com>

 create mode 100644 meson_mingw.txt

diff --git a/meson_mingw.txt b/meson_mingw.txt
new file mode 100644
index 000000000..80f04343a
--- /dev/null
+++ b/meson_mingw.txt
@@ -0,0 +1,14 @@
+[binaries]
+c = '/usr/bin/x86_64-w64-mingw32-gcc'
+cpp = '/usr/bin/x86_64-w64-mingw32-g++'
+ld = '/usr/bin/x86_64-w64-mingw32-ld' # Meson 0.53.0
+c_ld = '/usr/bin/x86_64-w64-mingw32-ld'
+ar = '/usr/bin/x86_64-w64-mingw32-ar'
+strip = '/usr/bin/x86_64-w64-mingw32-strip'
+pkgconfig = '/usr/bin/x86_64-w64-mingw32-pkg-config'
+
+[host_machine]
+system = 'windows'
+cpu_family = 'x86_64'
+cpu = 'native'
+endian = 'little'
-- 
2.25.0

[dpdk-dev] [PATCH 6/6] doc: guide for Windows build using MinGW-w64

[Dmitry Kozlyuk]

Instructions for different toolchains presented as options on the
corresponging steps of the guide, so that common parts may be reused.

Signed-off-by: Dmitry Kozlyuk <dmitry.kozliuk@gmail.com>

diff --git a/doc/guides/windows_gsg/build_dpdk.rst b/doc/guides/windows_gsg/build_dpdk.rst
index 6711e07e2..eabc459fb 100644
--- a/doc/guides/windows_gsg/build_dpdk.rst
+++ b/doc/guides/windows_gsg/build_dpdk.rst
@@ -7,15 +7,22 @@ Compiling the DPDK Target from Source
 System Requirements
 -------------------
 
-The DPDK and its applications require the Clang-LLVM C compiler
-and Microsoft MSVC linker.
+Building the DPDK and its applications requires one of the following
+environments:
+
+* The Clang-LLVM C compiler and Microsoft MSVC linker.
+* The MinGW-w64 toolchain (either native or cross).
+
 The Meson Build system is used to prepare the sources for compilation
 with the Ninja backend.
 The installation of these tools is covered in this section.
 
 
+Option 1. Clang-LLVM C Compiler and Microsoft MSVC Linker
+---------------------------------------------------------
+
 Install the Compiler
---------------------
+~~~~~~~~~~~~~~~~~~~~
 
 Download and install the clang compiler from
 `LLVM website <http://releases.llvm.org/download.html>`_.
@@ -25,7 +32,7 @@ For example, Clang-LLVM direct download link::
 
 
 Install the Linker
-------------------
+~~~~~~~~~~~~~~~~~~
 
 Download and install the Build Tools for Visual Studio to link and build the
 files on windows,
@@ -34,6 +41,15 @@ When installing build tools, select the "Visual C++ build tools" option
 and ensure the Windows SDK is selected.
 
 
+Option 2. MinGW-w64 Toolchain
+-----------------------------
+
+Obtain the latest version from
+`MinGW-w64 website <http://mingw-w64.org/doku.php/download>`_.
+On Windows, install to a folder without spaces in its name, like ``C:\MinGW``.
+This path is assumed for the rest of this guide.
+
+
 Install the Build System
 ------------------------
 
@@ -56,23 +72,41 @@ Build the code
 The build environment is setup to build the EAL and the helloworld example by
 default.
 
-Using the ninja backend
-~~~~~~~~~~~~~~~~~~~~~~~~
+Option 1. Native Build on Windows
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
 
-Specifying the compiler might be required to complete the meson command.
+When using Clang-LLVM, specifying the compiler might be required to complete
+the meson command:
 
 .. code-block:: console
 
     set CC=clang
 
+When using MinGW-w64, it is sufficient to have toolchain executables in PATH:
+
+.. code-block:: console
+
+    set PATH=C:\MinGW\mingw64\bin;%PATH%
+
 To compile the examples, the flag ``-Dexamples`` is required.
 
 .. code-block:: console
 
     cd C:\Users\me\dpdk
     meson -Dexamples=helloworld build
-    cd build
-    ninja
+    ninja -C build
+
+Option 2. Cross-Compile with MinGW-w64
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+The cross-file option must be specified for Meson.
+Depending on the distribution, paths in this file may need adjustments.
+
+.. code-block:: console
+
+    meson --cross-file meson_mingw.txt -Dexamples=helloworld build
+    ninja -C build
+
 
 Run the helloworld example
 ==========================
@@ -87,3 +121,8 @@ Navigate to the examples in the build directory and run `dpdk-helloworld.exe`.
     hello from core 3
     hello from core 0
     hello from core 2
+
+Note for MinGW-w64: applications are linked to ``libwinpthread-1.dll``
+by default. To run the example, either add toolchain executables directory
+to the PATH or copy the library to the working directory.
+Alternatively, static linking may be used (mind the LGPLv2.1 license).
-- 
2.25.0

Re: [dpdk-dev] [PATCH 4/6] build: MinGW-w64 support for Meson

[Thomas Monjalon]

31/01/2020 04:07, Dmitry Kozlyuk:
> MinGW-w64 linker does not mimic MS linker options, so the build system
> must differentiate between linkers on Windows. Use GNU linker options
> with GCC and MS linker options with Clang.
> 
> MinGW-w64 by default uses MSVCRT stdio, which does not comply to ANSI,
> most notably its formatting and string handling functions. MinGW-w64
> support for the Universal CRT (UCRT) is ongoing, but the toolchain
> provides its own standard-complying implementation of stdio. The latter
> is used in the patch to support formatting in DPDK.
> 
> Signed-off-by: Dmitry Kozlyuk <dmitry.kozliuk@gmail.com>

I really like this patch.
So both GCC (with MinGW) and native clang are supported?

[...]
> +# MS linker requires special treatment.
> +# FIXME: use cc.get_linker_id() after upgrading to Meson >=0.53.

What does it mean? It won't work with meson 0.53?

> +is_ms_linker = is_windows and (cc.get_id() == 'clang')
[...]
> +if is_windows
> +	# Require platform SDK for Windows 7 and above.
> +	add_project_arguments('-D_WIN32_WINNT=0x0601', language: 'c')

Please explain. Why Windows 7 is needed? What this define is doing?

[...]
> -			if is_windows
> +
> +			if is_ms_linker
>  				lk_args = ['-Wl,/def:' + def_file.full_path(),
>  					'-Wl,/implib:lib\\' + implib]
>  			else
>  				lk_args = ['-Wl,--version-script=' + version_map]
> +			endif

Looks good.

Re: [dpdk-dev] [PATCH 5/6] build: add cross-file for MinGW-w64

[Thomas Monjalon]

31/01/2020 04:07, Dmitry Kozlyuk:
> Add Meson configuration to cross-compile for Windows using MinGW-w64.
> It may require adjustments in some cases, but at least it provides
> the foundation.
> 
> Signed-off-by: Dmitry Kozlyuk <dmitry.kozliuk@gmail.com>
> 
>  create mode 100644 meson_mingw.txt
> 
> --- /dev/null
> +++ b/meson_mingw.txt

I think this file should be in config/x86/

Re: [dpdk-dev] [PATCH 6/6] doc: guide for Windows build using MinGW-w64

[Thomas Monjalon]

31/01/2020 04:07, Dmitry Kozlyuk:
> Instructions for different toolchains presented as options on the
> corresponging steps of the guide, so that common parts may be reused.
> 
> Signed-off-by: Dmitry Kozlyuk <dmitry.kozliuk@gmail.com>
[...]
> --- a/doc/guides/windows_gsg/build_dpdk.rst
> +++ b/doc/guides/windows_gsg/build_dpdk.rst
> -The DPDK and its applications require the Clang-LLVM C compiler
> -and Microsoft MSVC linker.
> +Building the DPDK and its applications requires one of the following
> +environments:
> +
> +* The Clang-LLVM C compiler and Microsoft MSVC linker.
> +* The MinGW-w64 toolchain (either native or cross).
[...]
> +Option 1. Native Build on Windows
> +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
>  
> -Specifying the compiler might be required to complete the meson command.
> +When using Clang-LLVM, specifying the compiler might be required to complete
> +the meson command:
[...]
> +When using MinGW-w64, it is sufficient to have toolchain executables in PATH:
[...]
> +Option 2. Cross-Compile with MinGW-w64
> +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

I think having the choice between GCC and Clang for Windows is very cool.

When starting the Windows port, I asked the fundamental question of the
supported compilers. The answer was MinGW adds "non-standard" DLLs:
	https://mails.dpdk.org/archives/dev/2019-January/124236.html

Now the question is to know which one is the easiest to use?
If I understand well, MinGW brings the missing parts we are trying
to add in the DPDK repository for compliance with POSIX libraries.

Re: [dpdk-dev] [PATCH 4/6] build: MinGW-w64 support for Meson

[Dmitry Kozlyuk]

> I really like this patch.
> So both GCC (with MinGW) and native clang are supported?

Thanks. Yes, I tested both toolchains.

> [...]
> > +# MS linker requires special treatment.
> > +# FIXME: use cc.get_linker_id() after upgrading to Meson >=0.53.  
> 
> What does it mean? It won't work with meson 0.53?

It will work for any Meson version, I meant that Meson 0.54 will introduce a
better way to determine linker ID. Can you suggest a better wording?

> 
> > +is_ms_linker = is_windows and (cc.get_id() == 'clang')  
> [...]
> > +if is_windows
> > +	# Require platform SDK for Windows 7 and above.
> > +	add_project_arguments('-D_WIN32_WINNT=0x0601', language: 'c')  
> 
> Please explain. Why Windows 7 is needed? What this define is doing?

Yes, Windows 7 and above is need for already existing code in eal_lcore.c,
specifically for GetLogicalProcessorInformation() call.

When including <windows.h>, one must define minimum API version the
application is compiled against [0]. MSVC and Clang default to the version of
platform SDK (that is, maximum supported). MinGW defaults to Windows XP, so
this definition must be either in <rte_os.h> before #include <windows.h> or
here. Because other files may include <windows.h>, I'd prefer to have a
global definition via compiler command-line.

[0]:
https://docs.microsoft.com/en-us/windows/win32/WinProg/using-the-windows-headers

-- 
Dmitry Kozlyuk

Re: [dpdk-dev] [PATCH 5/6] build: add cross-file for MinGW-w64

[Dmitry Kozlyuk]

> > --- /dev/null
> > +++ b/meson_mingw.txt  
> 
> I think this file should be in config/x86/

OK, will fix in v2.

-- 
Dmitry Kozlyuk

Re: [dpdk-dev] [PATCH 6/6] doc: guide for Windows build using MinGW-w64

[Dmitry Kozlyuk]

> I think having the choice between GCC and Clang for Windows is very cool.
> 
> When starting the Windows port, I asked the fundamental question of the
> supported compilers. The answer was MinGW adds "non-standard" DLLs:
> 	https://mails.dpdk.org/archives/dev/2019-January/124236.html

Indeed, it does add a pthread implementation DLL, I even documented it. It
presents no technical or legal trouble though.

> Now the question is to know which one is the easiest to use?

The issues are a few (almost all in this message), after they are resolved,
there is no significant difference. With MinGW, more things work out of the
box. It also allows for immediate cross-compilation from Linux, which is
important while Windows drivers are unstable and can crash the system hard.

> If I understand well, MinGW brings the missing parts we are trying
> to add in the DPDK repository for compliance with POSIX libraries.

Yes, having pthread implementation out-of-the-box helps with porting EAL.
Clang can use external pthread, e. g. winpthreads [0] (not to be confused with
winpthread without S). MinGW doesn't aim to provide POSIX support, it only
brings the parts required for GCC toolchain.

Clang has structure alignment issue, which is a show-stopper for rte_mbuf,
because its carefully crafted layout is broken and becomes 3 cache-lines
instead of two:

    https://bugs.llvm.org/show_bug.cgi?id=24383

For GCC, there is a way force rte_mbuf layout:

    https://github.com/PlushBeaver/dpdk/commit/37f052cb18d1d5d425818196d5e1d15a7ada0de0

GCC, in its turn, has an AVX bug, although it can be worked around by
force-inlining functions in rte_acl (as I did):

    https://gcc.gnu.org/bugzilla/show_bug.cgi?id=54412

[0]:
https://sourceforge.net/p/mingw-w64/mingw-w64/ci/master/tree/mingw-w64-libraries/winpthreads/

-- 
Dmitry Kozlyuk

Re: [dpdk-dev] [PATCH 4/6] build: MinGW-w64 support for Meson

[Thomas Monjalon]

05/02/2020 00:21, Dmitry Kozlyuk:
> > I really like this patch.
> > So both GCC (with MinGW) and native clang are supported?
> 
> Thanks. Yes, I tested both toolchains.
> 
> > [...]
> > > +# MS linker requires special treatment.
> > > +# FIXME: use cc.get_linker_id() after upgrading to Meson >=0.53.  
> > 
> > What does it mean? It won't work with meson 0.53?
> 
> It will work for any Meson version, I meant that Meson 0.54 will introduce a
> better way to determine linker ID. Can you suggest a better wording?

FIXME: use cc.get_linker_id() with Meson >= 0.54


> > > +is_ms_linker = is_windows and (cc.get_id() == 'clang')  
> > [...]
> > > +if is_windows
> > > +	# Require platform SDK for Windows 7 and above.
> > > +	add_project_arguments('-D_WIN32_WINNT=0x0601', language: 'c')  
> > 
> > Please explain. Why Windows 7 is needed? What this define is doing?
> 
> Yes, Windows 7 and above is need for already existing code in eal_lcore.c,
> specifically for GetLogicalProcessorInformation() call.
> 
> When including <windows.h>, one must define minimum API version the
> application is compiled against [0]. MSVC and Clang default to the version of
> platform SDK (that is, maximum supported). MinGW defaults to Windows XP, so
> this definition must be either in <rte_os.h> before #include <windows.h> or
> here. Because other files may include <windows.h>, I'd prefer to have a
> global definition via compiler command-line.
> 
> [0]:
> https://docs.microsoft.com/en-us/windows/win32/WinProg/using-the-windows-headers

OK, thanks.
Please reword the comment with something like
	"Minimum supported API is Windows 7."

Re: [dpdk-dev] [EXTERNAL] [PATCH 0/6] MinGW-w64 support

[Narcisa Ana Maria Vasile]

Hi Dmitry, 

Thanks for the patches! I was able to compile and run natively, after applying the Meson patch that you mentioned.

I'm having some trouble with cross-compilation:

python3 meson.py -Dexamples=helloworld ../../dpdk/build ../../dpdk --cross-file ../../dpdk/meson_mingw.txt
The Meson build system
Version: 0.53.1
Source dir: /mnt/d/dpdk
Build dir: /mnt/d/dpdk/build
Build type: cross build
Program cat found: YES (/bin/cat)
Project name: DPDK
Project version: 20.02.0-rc1
C compiler for the build machine: cc (gcc 7.4.0 "cc (Ubuntu 7.4.0-1ubuntu1~18.04.1) 7.4.0")
C linker for the build machine: cc GNU ld.bfd 2.30

meson.build:4:0: ERROR: Unable to determine dynamic linker

Any ideas on how to fix this issue?

Thanks,
Narcisa Vasile

-----Original Message-----
From: dev <dev-bounces@dpdk.org> On Behalf Of Dmitry Kozlyuk
Sent: Thursday, January 30, 2020 7:08 PM
To: dev@dpdk.org
Cc: Dmitry Kozlyuk <dmitry.kozliuk@gmail.com>; Bruce Richardson <bruce.richardson@intel.com>; Thomas Monjalon <thomas@monjalon.net>; Olivier Matz <olivier.matz@6wind.com>; Harini Ramakrishnan <Harini.Ramakrishnan@microsoft.com>; Omar Cardona <ocardona@microsoft.com>; Pallavi Kadam <pallavi.kadam@intel.com>; Ranjit Menon <ranjit.menon@intel.com>; John McNamara <john.mcnamara@intel.com>; Marko Kovacevic <marko.kovacevic@intel.com>
Subject: [EXTERNAL] [dpdk-dev] [PATCH 0/6] MinGW-w64 support

This patch series add support for building DPDK using MinGW-w64.

MinGW-w64 provides GNU toolchain and independent platform SDK on Windows. It also supports cross-compilation to Windows from POSIX systems by providing cross tollchains and libraries [0]. It does NOT emulate a full POSIX environment, like Cygwin or MSYS do.

There are advantages in using MinGW-w64 in addition to Clang:

1. Cross-compilation out-of-the-box. MinGW-w64 is provides a pthread
   implementation, GNU getopt, and Windows platform SDK.

2. Easier porting of POSIX applications using DPDK to Windows, because
   application code can use the same benefits as mentioned above.

3. Having both primary compilers enabled on Windows provides more
   diagnostics and generally prevents non-portable code.

[0]: https://nam06.safelinks.protection.outlook.com/?url=http%3A%2F%2Fmingw-w64.org&amp;data=02%7C01%7CNarcisa.Vasile%40microsoft.com%7C9ecc756fe41941539c3208d7a5fac686%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637160368803628802&amp;sdata=GcrGNIMkp0PjfmOXoKkbdA4ttMYQLv3cB24riDfPx%2Fk%3D&amp;reserved=0

Dmitry Kozlyuk (6):
  eal: introduce portable format attribute
  eal: use portable format attribute
  cmdline: use portable format attribute
  build: MinGW-w64 support for Meson
  build: add cross-file for MinGW-w64
  doc: guide for Windows build using MinGW-w64

 config/meson.build                          | 14 +++++
 doc/guides/windows_gsg/build_dpdk.rst       | 57 +++++++++++++++++----
 lib/librte_cmdline/cmdline.h                |  4 +-
 lib/librte_eal/common/include/rte_common.h  | 17 +++++-
 lib/librte_eal/common/include/rte_debug.h   |  2 +-
 lib/librte_eal/common/include/rte_devargs.h |  2 +-
 lib/librte_eal/common/include/rte_log.h     |  4 +-
 lib/librte_eal/meson.build                  |  3 ++
 lib/meson.build                             |  8 ++-
 meson_mingw.txt                             | 14 +++++
 10 files changed, 108 insertions(+), 17 deletions(-)  create mode 100644 meson_mingw.txt

--
2.25.0

Re: [dpdk-dev] [PATCH 6/6] doc: guide for Windows build using MinGW-w64

[Thomas Monjalon]

05/02/2020 00:57, Dmitry Kozlyuk:
> > I think having the choice between GCC and Clang for Windows is very cool.
> > 
> > When starting the Windows port, I asked the fundamental question of the
> > supported compilers. The answer was MinGW adds "non-standard" DLLs:
> > 	https://mails.dpdk.org/archives/dev/2019-January/124236.html
> 
> Indeed, it does add a pthread implementation DLL, I even documented it. It
> presents no technical or legal trouble though.
> 
> > Now the question is to know which one is the easiest to use?
> 
> The issues are a few (almost all in this message), after they are resolved,
> there is no significant difference. With MinGW, more things work out of the
> box. It also allows for immediate cross-compilation from Linux, which is
> important while Windows drivers are unstable and can crash the system hard.
> 
> > If I understand well, MinGW brings the missing parts we are trying
> > to add in the DPDK repository for compliance with POSIX libraries.
> 
> Yes, having pthread implementation out-of-the-box helps with porting EAL.
> Clang can use external pthread, e. g. winpthreads [0] (not to be confused with
> winpthread without S). MinGW doesn't aim to provide POSIX support, it only
> brings the parts required for GCC toolchain.
> 
> Clang has structure alignment issue, which is a show-stopper for rte_mbuf,
> because its carefully crafted layout is broken and becomes 3 cache-lines
> instead of two:
> 
>     https://bugs.llvm.org/show_bug.cgi?id=24383
> 
> For GCC, there is a way force rte_mbuf layout:
> 
>     https://github.com/PlushBeaver/dpdk/commit/37f052cb18d1d5d425818196d5e1d15a7ada0de0
> 
> GCC, in its turn, has an AVX bug, although it can be worked around by
> force-inlining functions in rte_acl (as I did):
> 
>     https://gcc.gnu.org/bugzilla/show_bug.cgi?id=54412
> 
> [0]:
> https://sourceforge.net/p/mingw-w64/mingw-w64/ci/master/tree/mingw-w64-libraries/winpthreads/

Thanks a lot for all these valuable informations.

I think the strategy should be to progress on both GCC and Clang
at the same time.

Please Dmitry, would you like to review Pallavi's patches
in order to make them coexist with yours?

Re: [dpdk-dev] [EXTERNAL] [PATCH 0/6] MinGW-w64 support

[Dmitry Kozlyuk]

Hi Narcisa,

> I'm having some trouble with cross-compilation:
> 
> python3 meson.py -Dexamples=helloworld ../../dpdk/build ../../dpdk --cross-file ../../dpdk/meson_mingw.txt
> The Meson build system
> Version: 0.53.1
> Source dir: /mnt/d/dpdk
> Build dir: /mnt/d/dpdk/build
> Build type: cross build
> Program cat found: YES (/bin/cat)
> Project name: DPDK
> Project version: 20.02.0-rc1
> C compiler for the build machine: cc (gcc 7.4.0 "cc (Ubuntu 7.4.0-1ubuntu1~18.04.1) 7.4.0")
> C linker for the build machine: cc GNU ld.bfd 2.30
> 
> meson.build:4:0: ERROR: Unable to determine dynamic linker
> 
> Any ideas on how to fix this issue?

Use Meson 0.52, version 0.53 introduced this regression among others:
https://github.com/mesonbuild/meson/issues/6431
DPDK CI also has Meson version pinned to 0.52, see
http://mails.dpdk.org/archives/dev/2020-January/154357.html
I find it handy to switch Meson versions via pip3.

Here's Meson 0.52 output:

> $ meson --cross-file meson_mingw.txt build/cross/mingw
> The Meson build system
> Version: 0.52.0
> Source dir: /home/dmitry/src/dpdk.clean
> Build dir: /home/dmitry/src/dpdk.clean/build/cross/mingw
> Build type: cross build
> Program cat found: YES (/usr/bin/cat)
> Project name: DPDK
> Project version: 20.02.0-rc1
> C compiler for the build machine: cc (gcc 9.2.0 "cc (GCC) 9.2.0")
> C linker for the build machine: GNU ld.bfd 2.33.1
> C compiler for the host machine: /usr/bin/x86_64-w64-mingw32-gcc (gcc 9.2.0
> "x86_64-w64-mingw32-gcc (GCC) 9.2.0") 
> C linker for the host machine: GNU ld.bfd 2.33.1

As you can see, it correctly picks up CC and LD for host machine, unlike 0.53.

-- 
Dmitry Kozlyuk

Re: [dpdk-dev] [EXTERNAL] [PATCH 0/6] MinGW-w64 support

[David Marchand]

On Wed, Feb 5, 2020 at 6:43 AM Dmitry Kozlyuk <dmitry.kozliuk@gmail.com> wrote:
>
> Hi Narcisa,
>
> > I'm having some trouble with cross-compilation:
> >
> > python3 meson.py -Dexamples=helloworld ../../dpdk/build ../../dpdk --cross-file ../../dpdk/meson_mingw.txt
> > The Meson build system
> > Version: 0.53.1
> > Source dir: /mnt/d/dpdk
> > Build dir: /mnt/d/dpdk/build
> > Build type: cross build
> > Program cat found: YES (/bin/cat)
> > Project name: DPDK
> > Project version: 20.02.0-rc1
> > C compiler for the build machine: cc (gcc 7.4.0 "cc (Ubuntu 7.4.0-1ubuntu1~18.04.1) 7.4.0")
> > C linker for the build machine: cc GNU ld.bfd 2.30
> >
> > meson.build:4:0: ERROR: Unable to determine dynamic linker
> >
> > Any ideas on how to fix this issue?
>
> Use Meson 0.52, version 0.53 introduced this regression among others:
> https://github.com/mesonbuild/meson/issues/6431
> DPDK CI also has Meson version pinned to 0.52, see
> http://mails.dpdk.org/archives/dev/2020-January/154357.html
> I find it handy to switch Meson versions via pip3.

Err, we decided to pin to 0.47.1 as it is the documented minimal
supported version.
http://inbox.dpdk.org/dev/20200108110251.20916-1-david.marchand@redhat.com/

So the CI is stuck to the 0.47.1 version, not 0.52.
Do you need something from a newer meson for Windows support? That
would have to be documented.


--
David Marchand

Re: [dpdk-dev] [PATCH 4/6] build: MinGW-w64 support for Meson

[Bruce Richardson]

On Wed, Feb 05, 2020 at 01:41:24AM +0100, Thomas Monjalon wrote:
> 05/02/2020 00:21, Dmitry Kozlyuk:
> > > I really like this patch.
> > > So both GCC (with MinGW) and native clang are supported?
> > 
> > Thanks. Yes, I tested both toolchains.
> > 
> > > [...]
> > > > +# MS linker requires special treatment.
> > > > +# FIXME: use cc.get_linker_id() after upgrading to Meson >=0.53.  
> > > 
> > > What does it mean? It won't work with meson 0.53?
> > 
> > It will work for any Meson version, I meant that Meson 0.54 will introduce a
> > better way to determine linker ID. Can you suggest a better wording?
> 
> FIXME: use cc.get_linker_id() with Meson >= 0.54
> 
> 
> > > > +is_ms_linker = is_windows and (cc.get_id() == 'clang')  
> > > [...]
> > > > +if is_windows
> > > > +	# Require platform SDK for Windows 7 and above.
> > > > +	add_project_arguments('-D_WIN32_WINNT=0x0601', language: 'c')  
> > > 
> > > Please explain. Why Windows 7 is needed? What this define is doing?
> > 
> > Yes, Windows 7 and above is need for already existing code in eal_lcore.c,
> > specifically for GetLogicalProcessorInformation() call.
> > 
> > When including <windows.h>, one must define minimum API version the
> > application is compiled against [0]. MSVC and Clang default to the version of
> > platform SDK (that is, maximum supported). MinGW defaults to Windows XP, so
> > this definition must be either in <rte_os.h> before #include <windows.h> or
> > here. Because other files may include <windows.h>, I'd prefer to have a
> > global definition via compiler command-line.
> > 
> > [0]:
> > https://docs.microsoft.com/en-us/windows/win32/WinProg/using-the-windows-headers
> 
> OK, thanks.
> Please reword the comment with something like
> 	"Minimum supported API is Windows 7."
> 
For this, as an alternative to putting it as a project argument, you can just
add it to dpdk_conf which means it will end up as a define in the global
rte_build_config.h and so be directly included in each compilation unit
ahead of any other headers. (rte_config.h includes rte_build_config.h)

/Bruce

Re: [dpdk-dev] [PATCH 4/6] build: MinGW-w64 support for Meson

[Dmitry Kozlyuk]

> > > > > +if is_windows
> > > > > +	# Require platform SDK for Windows 7 and above.
> > > > > +	add_project_arguments('-D_WIN32_WINNT=0x0601', language: 'c')    
> > > > 
> > > > Please explain. Why Windows 7 is needed? What this define is doing?  
> > > 
> > > Yes, Windows 7 and above is need for already existing code in eal_lcore.c,
> > > specifically for GetLogicalProcessorInformation() call.
> > > 
> > > When including <windows.h>, one must define minimum API version the
> > > application is compiled against [0]. MSVC and Clang default to the version of
> > > platform SDK (that is, maximum supported). MinGW defaults to Windows XP, so
> > > this definition must be either in <rte_os.h> before #include <windows.h> or
> > > here. Because other files may include <windows.h>, I'd prefer to have a
> > > global definition via compiler command-line.
> > > 
> > > [0]:
> > > https://docs.microsoft.com/en-us/windows/win32/WinProg/using-the-windows-headers  
> > 
> > OK, thanks.
> > Please reword the comment with something like
> > 	"Minimum supported API is Windows 7."
> >   
> For this, as an alternative to putting it as a project argument, you can just
> add it to dpdk_conf which means it will end up as a define in the global
> rte_build_config.h and so be directly included in each compilation unit
> ahead of any other headers. (rte_config.h includes rte_build_config.h)

Can you please explain why using dpdk_conf is a better alternative? In
lib/meson.build I can see add_project_arguments('_D_GNU_SOURCE', ...), which
serves a similar purpose on POSIX systems. Compiler option also makes it
impossible to forget or redefine this constant in code by mistake.

-- 
Dmitry Kozlyuk

Re: [dpdk-dev] [EXTERNAL] [PATCH 0/6] MinGW-w64 support

[Dmitry Kozlyuk]

> > Use Meson 0.52, version 0.53 introduced this regression among others:
> > https://github.com/mesonbuild/meson/issues/6431
> > DPDK CI also has Meson version pinned to 0.52, see
> > http://mails.dpdk.org/archives/dev/2020-January/154357.html
> > I find it handy to switch Meson versions via pip3.  
> 
> Err, we decided to pin to 0.47.1 as it is the documented minimal
> supported version.
> http://inbox.dpdk.org/dev/20200108110251.20916-1-david.marchand@redhat.com/
> 
> So the CI is stuck to the 0.47.1 version, not 0.52.
> Do you need something from a newer meson for Windows support? That
> would have to be documented.

My bad, can't tell why I decided that 0.52 was pinned. I verified building
with Meson 0.47.1, both cross and native, and it works well.

Apologies to Narcisa for misguidance.

-- 
Dmitry Kozlyuk

Re: [dpdk-dev] [EXTERNAL] [PATCH 0/6] MinGW-w64 support

[Narcisa Ana Maria Vasile]

On 2/5/2020 12:59 PM, Dmitry Kozlyuk wrote:

Use Meson 0.52, version 0.53 introduced this regression among others:
https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fmesonbuild%2Fmeson%2Fissues%2F6431&data=02%7C01%7CNarcisa.Vasile%40microsoft.com%7C3a156299509a4e20c63108d7aa7e5cf0%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637165332017516796&sdata=eOx4P6FJLWzC7pmpS%2F0xvA2VzjFJ0YCrxRc7ris%2BKhU%3D&reserved=0
DPDK CI also has Meson version pinned to 0.52, see
https://nam06.safelinks.protection.outlook.com/?url=http%3A%2F%2Fmails.dpdk.org%2Farchives%2Fdev%2F2020-January%2F154357.html&data=02%7C01%7CNarcisa.Vasile%40microsoft.com%7C3a156299509a4e20c63108d7aa7e5cf0%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637165332017516796&sdata=Z87jGpDhGqBVdYWXg0b66kTuOr5x0byCPM9TUel5dZ0%3D&reserved=0
I find it handy to switch Meson versions via pip3.



Err, we decided to pin to 0.47.1 as it is the documented minimal
supported version.
https://nam06.safelinks.protection.outlook.com/?url=http%3A%2F%2Finbox.dpdk.org%2Fdev%2F20200108110251.20916-1-david.marchand%40redhat.com%2F&data=02%7C01%7CNarcisa.Vasile%40microsoft.com%7C3a156299509a4e20c63108d7aa7e5cf0%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637165332017516796&sdata=361djaQtaBeKN%2B9Fjg5IjuQ2dhfjCg0eMlHBVZA3aqw%3D&reserved=0

So the CI is stuck to the 0.47.1 version, not 0.52.
Do you need something from a newer meson for Windows support? That
would have to be documented.



My bad, can't tell why I decided that 0.52 was pinned. I verified building
with Meson 0.47.1, both cross and native, and it works well.

Apologies to Narcisa for misguidance.


No worries, I also cross-compiled using both 0.47.1 and 0.52.0. I was able to generate the helloworld executable and ran it successfully. Thanks!

I did have to apply your patch "eal/windows: refine public interface" before applying the "MinGW-w64 support" patch set, to avoid the errors caused by including "Windows.h" in rte_os.h. So, I guess this patch needs to come first and then the patchset for mingw can be applied.

Re: [dpdk-dev] [EXTERNAL] [PATCH 0/6] MinGW-w64 support

[Dmitry Kozlyuk]

> I did have to apply your patch "eal/windows: refine public interface" before applying the "MinGW-w64 support" patch set, to avoid the errors caused by including "Windows.h" in rte_os.h. So, I guess this patch needs to come first and then the patchset for mingw can be applied.

The cause of the error you've encountered is case-dependent Linux filesystem.
Existing code includes <Windows.h> (capital W), but MinGW-w64 on Linux
packages <windows.h> (lowercase W). I missed it because Windows.h is manually
symlinked to windows.h on my system. Same goes for <BaseTsd.h>.

Thanks for pointing this out, I think it's more appropriate to change
filename case in this patchset v2, because the need for this change is caused
by adding MinGW cross-compilation capability.

-- 
Dmitry Kozlyuk

Re: [dpdk-dev] [PATCH 4/6] build: MinGW-w64 support for Meson

[Bruce Richardson]

On Wed, Feb 05, 2020 at 11:41:05PM +0300, Dmitry Kozlyuk wrote:
> > > > > > +if is_windows
> > > > > > +	# Require platform SDK for Windows 7 and above.
> > > > > > +	add_project_arguments('-D_WIN32_WINNT=0x0601', language: 'c')    
> > > > > 
> > > > > Please explain. Why Windows 7 is needed? What this define is doing?  
> > > > 
> > > > Yes, Windows 7 and above is need for already existing code in eal_lcore.c,
> > > > specifically for GetLogicalProcessorInformation() call.
> > > > 
> > > > When including <windows.h>, one must define minimum API version the
> > > > application is compiled against [0]. MSVC and Clang default to the version of
> > > > platform SDK (that is, maximum supported). MinGW defaults to Windows XP, so
> > > > this definition must be either in <rte_os.h> before #include <windows.h> or
> > > > here. Because other files may include <windows.h>, I'd prefer to have a
> > > > global definition via compiler command-line.
> > > > 
> > > > [0]:
> > > > https://docs.microsoft.com/en-us/windows/win32/WinProg/using-the-windows-headers  
> > > 
> > > OK, thanks.
> > > Please reword the comment with something like
> > > 	"Minimum supported API is Windows 7."
> > >   
> > For this, as an alternative to putting it as a project argument, you can just
> > add it to dpdk_conf which means it will end up as a define in the global
> > rte_build_config.h and so be directly included in each compilation unit
> > ahead of any other headers. (rte_config.h includes rte_build_config.h)
> 
> Can you please explain why using dpdk_conf is a better alternative? In
> lib/meson.build I can see add_project_arguments('_D_GNU_SOURCE', ...), which
> serves a similar purpose on POSIX systems. Compiler option also makes it
> impossible to forget or redefine this constant in code by mistake.
> 
I'm not necessarily saying it's better, it's just an alternative to
consider. :-) Having it in rte_config.h makes the define available to any
external apps using DPDK, which may or may not be desirable.

Re: [dpdk-dev] [PATCH 4/6] build: MinGW-w64 support for Meson

[Dmitry Kozlyuk]

> On Wed, Feb 05, 2020 at 11:41:05PM +0300, Dmitry Kozlyuk wrote:
> > > > > > > +if is_windows
> > > > > > > +	# Require platform SDK for Windows 7 and above.
> > > > > > > +	add_project_arguments('-D_WIN32_WINNT=0x0601', language: 'c')      
> > > > > > 
> > > > > > Please explain. Why Windows 7 is needed? What this define is doing?    
> > > > > 
> > > > > Yes, Windows 7 and above is need for already existing code in eal_lcore.c,
> > > > > specifically for GetLogicalProcessorInformation() call.
> > > > > 
> > > > > When including <windows.h>, one must define minimum API version the
> > > > > application is compiled against [0]. MSVC and Clang default to the version of
> > > > > platform SDK (that is, maximum supported). MinGW defaults to Windows XP, so
> > > > > this definition must be either in <rte_os.h> before #include <windows.h> or
> > > > > here. Because other files may include <windows.h>, I'd prefer to have a
> > > > > global definition via compiler command-line.
> > > > > 
> > > > > [0]:
> > > > > https://docs.microsoft.com/en-us/windows/win32/WinProg/using-the-windows-headers    
> > > > 
> > > > OK, thanks.
> > > > Please reword the comment with something like
> > > > 	"Minimum supported API is Windows 7."
> > > >     
> > > For this, as an alternative to putting it as a project argument, you can just
> > > add it to dpdk_conf which means it will end up as a define in the global
> > > rte_build_config.h and so be directly included in each compilation unit
> > > ahead of any other headers. (rte_config.h includes rte_build_config.h)  
> > 
> > Can you please explain why using dpdk_conf is a better alternative? In
> > lib/meson.build I can see add_project_arguments('_D_GNU_SOURCE', ...), which
> > serves a similar purpose on POSIX systems. Compiler option also makes it
> > impossible to forget or redefine this constant in code by mistake.
> >   
> I'm not necessarily saying it's better, it's just an alternative to
> consider. :-) Having it in rte_config.h makes the define available to any
> external apps using DPDK, which may or may not be desirable.

It's certainly *not* desirable, apps usually have this value
consciously and explicitly defined on their own, and it may differ from the
one required to compile DPDK. Thanks for the tip, but as I hope you can see,
it should stay a project argument.

-- 
Dmitry Kozlyuk

Re: [dpdk-dev] [PATCH 6/6] doc: guide for Windows build using MinGW-w64

[Dmitry Kozlyuk]

> I think the strategy should be to progress on both GCC and Clang
> at the same time.

Remembered another issue: thread-local storage (TLS) with shared libraries.
Windows PE doesn't support TLS via special sections, so compilers use TLS
emulation layer. With static libraries, there are no issues described below.

The first aspect is a build-time issue of MinGW. When linking to DPDK shared
libraries, errors occur:

	undefined reference to `__emutls_v.per_lcore__rte_errno'
	undefined reference to `__emutls_v.per_lcore__rte_lcore_id'

DPDK declares per_lcore__XXX in a map file, but GCC places __thread symbols
in __emutls_v section, so the proper name to export becomes __emutls_v.XXX.
This can be worked around by using an additional version script with MinGW,
as I do in my port [0], however, the proper solution would be fixing the bug
on MinGW side [1]. MinGW already converts TLS variable names when generating
DEF files with `-Wl,--output-def` option (not used by DPDK, just a hint).

The second aspect is performance. Per [2], Win32 API TLS functions are ~10%
slower than non-emulated access on Linux, and MinGW emulation layer slows
access by another 20% (30% total). Clang emulation code is similar to
MinGW's [3], although I wasn't able to find any benchmarks. As a DPDK user, I
know that rte_lcore_id() is heavily used on the data-path, so this is severe.
My opinion is, for the first time Windows port should use whatever TLS
implementation the compiler provides, then attempt optimization. It should be
possible, because we don't need a general-purpose TLS, but only TLS for EAL
threads.

[0]: https://github.com/PlushBeaver/dpdk/blob/windows/lib/meson.build#L174
[1]: https://sourceforge.net/p/mingw-w64/bugs/827/
[2]:
https://sourceforge.net/p/mingw-w64/mailman/mingw-w64-public/thread/d72aad95-b6aa-af03-667b-5898456a5a63@gmx.com/
[3]:
https://github.com/llvm-mirror/compiler-rt/blob/master/lib/builtins/emutls.c


> Please Dmitry, would you like to review Pallavi's patches
> in order to make them coexist with yours?

Done. There are no logical conflicts, if code conflicts occur after merging
Pallivi's patchset, I'll resolve them before sending v3.

-- 
Dmitry Kozlyuk

Re: [dpdk-dev] [PATCH 6/6] doc: guide for Windows build using MinGW-w64

[Dmitry Kozlyuk]

> Remembered another issue: thread-local storage (TLS) with shared libraries.
> Windows PE doesn't support TLS via special sections, so compilers use TLS
> emulation layer. With static libraries, there are no issues described below.
> 
> The first aspect is a build-time issue of MinGW. When linking to DPDK shared
> libraries, errors occur:
> 
> 	undefined reference to `__emutls_v.per_lcore__rte_errno'
> 	undefined reference to `__emutls_v.per_lcore__rte_lcore_id'
> 
> DPDK declares per_lcore__XXX in a map file, but GCC places __thread symbols
> in __emutls_v section, so the proper name to export becomes __emutls_v.XXX.
> This can be worked around by using an additional version script with MinGW,
> as I do in my port [0], however, the proper solution would be fixing the bug
> on MinGW side [1]. MinGW already converts TLS variable names when generating
> DEF files with `-Wl,--output-def` option (not used by DPDK, just a hint).

Did some research and AFAICT, there is not effortless solution for
efficient per-lcore variables on Windows. While MinGW-w64 has aforementioned
issues (actually, GCC on Windows does), Clang with default TLS options just
generates wrong results when exporting variables from dynamic libraries.
Demo: https://github.com/PlushBeaver/tlstest

Thread [0] claims this is a fundamental problem with PE-COFF executable
format, but I honestly lack expertise to tell if this is valid. Microsoft
docs [1] suggests that exporting __thread variables won't just work. Can
someone from Microsoft or from UNH Lab comment further?

[0]: https://sourceforge.net/p/mingw-w64/mailman/message/31777672/
[1]:
https://docs.microsoft.com/en-us/windows/win32/dlls/using-thread-local-storage-in-a-dynamic-link-library

-- 
Dmitry Kozlyuk

Re: [dpdk-dev] [PATCH 6/6] doc: guide for Windows build using MinGW-w64

[Thomas Monjalon]

Hi,

We didn't get a confirmation from Microsoft about the TLS problem.
Please could we have a status?

Should we mark shared library linkage as not supported in Windows DPDK?
Is there a difference between clang and MinGW?


17/02/2020 07:27, Dmitry Kozlyuk:
> > Remembered another issue: thread-local storage (TLS) with shared libraries.
> > Windows PE doesn't support TLS via special sections, so compilers use TLS
> > emulation layer. With static libraries, there are no issues described below.
> > 
> > The first aspect is a build-time issue of MinGW. When linking to DPDK shared
> > libraries, errors occur:
> > 
> > 	undefined reference to `__emutls_v.per_lcore__rte_errno'
> > 	undefined reference to `__emutls_v.per_lcore__rte_lcore_id'
> > 
> > DPDK declares per_lcore__XXX in a map file, but GCC places __thread symbols
> > in __emutls_v section, so the proper name to export becomes __emutls_v.XXX.
> > This can be worked around by using an additional version script with MinGW,
> > as I do in my port [0], however, the proper solution would be fixing the bug
> > on MinGW side [1]. MinGW already converts TLS variable names when generating
> > DEF files with `-Wl,--output-def` option (not used by DPDK, just a hint).
> 
> Did some research and AFAICT, there is not effortless solution for
> efficient per-lcore variables on Windows. While MinGW-w64 has aforementioned
> issues (actually, GCC on Windows does), Clang with default TLS options just
> generates wrong results when exporting variables from dynamic libraries.
> Demo: https://github.com/PlushBeaver/tlstest
> 
> Thread [0] claims this is a fundamental problem with PE-COFF executable
> format, but I honestly lack expertise to tell if this is valid. Microsoft
> docs [1] suggests that exporting __thread variables won't just work. Can
> someone from Microsoft or from UNH Lab comment further?
> 
> [0]: https://sourceforge.net/p/mingw-w64/mailman/message/31777672/
> [1]:
> https://docs.microsoft.com/en-us/windows/win32/dlls/using-thread-local-storage-in-a-dynamic-link-library