[dpdk-dev] [PATCH v2 2/3] meson: use threads dependency as provided by meson

[dpdk-dev] [PATCH v2 2/3] meson: use threads dependency as provided by meson

[Gabriel Ganne]

meson guarantees this is portable, so this also allows us to remove a
is_windows switch.

Link: https://mesonbuild.com/howtox.html#enable-threads
Signed-off-by: Gabriel Ganne <gabriel.ganne@6wind.com>
---
 config/meson.build               | 5 +----
 lib/librte_metrics/meson.build   | 2 ++
 lib/librte_telemetry/meson.build | 2 ++
 3 files changed, 5 insertions(+), 4 deletions(-)

diff --git a/config/meson.build b/config/meson.build
index 3cf560b8a3f5..0fb7e1b27a0f 100644
--- a/config/meson.build
+++ b/config/meson.build
@@ -113,10 +113,7 @@ if not is_windows
 endif
 
 # use pthreads if available for the platform
-if not is_windows
-	add_project_link_arguments('-pthread', language: 'c')
-	dpdk_extra_ldflags += '-pthread'
-endif
+threads_dep = dependency('threads')
 
 # on some OS, maths functions are in a separate library
 if cc.find_library('m', required : false).found()
diff --git a/lib/librte_metrics/meson.build b/lib/librte_metrics/meson.build
index d5be6a214530..29d922eded53 100644
--- a/lib/librte_metrics/meson.build
+++ b/lib/librte_metrics/meson.build
@@ -4,6 +4,8 @@
 sources = files('rte_metrics.c', 'rte_metrics_telemetry.c')
 headers = files('rte_metrics.h', 'rte_metrics_telemetry.h')
 
+ext_deps += threads_dep
+
 jansson = dependency('jansson', required: false, method: 'pkg-config')
 if jansson.found()
 	dpdk_conf.set('RTE_HAS_JANSSON', 1)
diff --git a/lib/librte_telemetry/meson.build b/lib/librte_telemetry/meson.build
index 719973ff9240..46ac9829e54b 100644
--- a/lib/librte_telemetry/meson.build
+++ b/lib/librte_telemetry/meson.build
@@ -3,6 +3,8 @@
 
 includes = [global_inc]
 
+ext_deps += threads_dep
+
 sources = files('telemetry.c', 'telemetry_data.c', 'telemetry_legacy.c')
 headers = files('rte_telemetry.h')
 includes += include_directories('../librte_metrics')
-- 
2.29.2

[dpdk-dev] [PATCH v2 3/3] meson: remove unnecessary explicit link to libpcap

[Gabriel Ganne]

libpcap is already found and registered as a dependency by meson, and
the dependency is already correctly used in librte_port. This line is
just unnecessary.

It also has the side effect of messing with the meson link line: dpdk
link will be declared twice: manually and then through pkg-config. If
you configure meson to prefer static linking over dynamic, this will
cause the build to fail on librte_port, since the pcap deps are not yet
seen by the linker.

Signed-off-by: Gabriel Ganne <gabriel.ganne@6wind.com>
---
 config/meson.build | 1 -
 1 file changed, 1 deletion(-)

diff --git a/config/meson.build b/config/meson.build
index 0fb7e1b27a0f..3eb90327dfcc 100644
--- a/config/meson.build
+++ b/config/meson.build
@@ -177,7 +177,6 @@ if not pcap_dep.found()
 endif
 if pcap_dep.found() and cc.has_header('pcap.h', dependencies: pcap_dep)
 	dpdk_conf.set('RTE_PORT_PCAP', 1)
-	dpdk_extra_ldflags += '-lpcap'
 endif
 
 # for clang 32-bit compiles we need libatomic for 64-bit atomic ops
-- 
2.29.2

Re: [dpdk-dev] [PATCH v2 2/3] meson: use threads dependency as provided by meson

[Nick Connolly]

> meson guarantees this is portable, so this also allows us to remove a
> is_windows switch.
Hi Gabriel,

On Windows, there are at least three different compilation options:

  * MinGW-w64 with posix threads
  * MinGW-w64 with win32 threads (different download)
  * Clang without threads support

The Windows EAL library doesn't use a threads package and instead uses 
it's own wrappers around the native Windows system calls. Given that, I 
believe the existing behaviour of not specifying the pthreads package 
for a Windows build is correct and shouldn't be changed.

Regards,
Nick

Re: [dpdk-dev] [PATCH v2 2/3] meson: use threads dependency as provided by meson

[Dmitry Kozlyuk]

2021-03-22 16:04 (UTC+0000), Nick Connolly:
> > meson guarantees this is portable, so this also allows us to remove a
> > is_windows switch.  
> Hi Gabriel,
> 
> On Windows, there are at least three different compilation options:
> 
>   * MinGW-w64 with posix threads
>   * MinGW-w64 with win32 threads (different download)
>   * Clang without threads support
> 
> The Windows EAL library doesn't use a threads package and instead uses 
> it's own wrappers around the native Windows system calls. Given that, I 
> believe the existing behaviour of not specifying the pthreads package 
> for a Windows build is correct and shouldn't be changed.

Hi Nick,

Threads is a dummy dependency for Clang. For MinGW without pthread it still
adds -pthread switch (and it comes to libdpdk.pc), but executable doesn't
require pthread library to run. I can imagine DPDK built by MinGW without
pthread be linked via pkg-config to an app built by MinGW with pthread, then
-pthread switch will be added from libdpdk.pc. Do you think it's an issue?

Re: [dpdk-dev] [PATCH v2 2/3] meson: use threads dependency as provided by meson

[Nick Connolly]

> Threads is a dummy dependency for Clang. For MinGW without pthread it still
> adds -pthread switch (and it comes to libdpdk.pc), but executable doesn't
> require pthread library to run. I can imagine DPDK built by MinGW without
> pthread be linked via pkg-config to an app built by MinGW with pthread, then
> -pthread switch will be added from libdpdk.pc. Do you think it's an issue?

Hi Dmitry,

I've done some experimenting with this change and agree that for Clang, 
-lpthread isn't
used when linking and it doesn't appear in libdpdk.pc.  It is added to 
the compile line,
as -pthread, but this doesn't seem to cause any issues.

Having said that, my test results above are with Meson 0.55.3. When I 
installed 0.49.0
to test it out, Meson was unable to complete the configuration, failing 
with:
> meson -Dexamples=helloworld
> ...
> Compiler for C supports arguments -D_XOPEN_SOURCE=600: YES
> Traceback (most recent call last):
>   File "mesonbuild\mesonmain.py", line 111, in run
> ...
>   File "mesonbuild\mesonlib.py", line 1222, in relpath
>   File "C:\python\lib\ntpath.py", line 560, in relpath
> TypeError: expected str, bytes or os.PathLike object, not NoneType

Unfortunately, I don't have time to pursue this further at the moment 
due to other
commitments, so I've reverted back to 0.55.3.

With MinGW, -pthread is added to compilation which doesn't seem to cause 
any issues.
In addition, -pthread is added to link and -lpthread appears in 
libdpdk.pc. My concerns
with this are:

 1. Adding -pthread to linking is an unnecessary dependency, even though
    it's
    relatively benign.

 2. Adding -lpthread to libdpdk.pc is more problematic. The application
    is almost
    certain to be using a threads library. If this is a 3rd party
    library, then the link
    command will potentially include two libraries defining the same
    exports.
    Which one takes precedence will depend upon the ordering in the
    application's
    build system which seems somewhat arbitrary given that the DPDK
    introduced
    dependency is not required.

I'd still advocate for:

if not is_windows
     threads_dep = dependency('threads')
endif

Regards,
Nick

Re: [dpdk-dev] [PATCH v2 2/3] meson: use threads dependency as provided by meson

[Dmitry Kozlyuk]

2021-03-23 13:35 (UTC+0000), Nick Connolly:
[...]
>  2. Adding -lpthread to libdpdk.pc is more problematic. The application
>     is almost
>     certain to be using a threads library. If this is a 3rd party
>     library, then the link
>     command will potentially include two libraries defining the same
>     exports.
>     Which one takes precedence will depend upon the ordering in the
>     application's
>     build system which seems somewhat arbitrary given that the DPDK
>     introduced
>     dependency is not required.

Agreed. In my tests it's "-pthread", not "-lpthread", but still.

See also comments here:
https://github.com/mesonbuild/meson/issues/553
Summary: dependency('threads') meaning is vague on Windows.

As a exotic case, I'm using NixOS which provides MinGW-w64 built with
mcfgthread (https://github.com/lhmouse/mcfgthread) and it doesn't even
recognize -pthread compiler flag, so this patch breaks cross-build for me.

> I'd still advocate for:
> 
> if not is_windows
>      threads_dep = dependency('threads')
> endif

It's more like, in lib/librte_telemetry/meson.build:

	if not is_windows
		ext_deps += threads_dep
	endif

Gabriel, by the way, why is it needed for librte_metrics?
Isn't librte_telemetry enough since it's a dependency to EAL?

Re: [dpdk-dev] [PATCH v2 2/3] meson: use threads dependency as provided by meson

[Nick Connolly]

> Agreed. In my tests it's "-pthread", not "-lpthread", but still.
Agreed - I must have been distracted by the array of -l: arguments :-)
> It's more like, in lib/librte_telemetry/meson.build:
>
> 	if not is_windows
> 		ext_deps += threads_dep
> 	endif
It's not a big deal, but my thinking was that the library has a genuine
threads dependency, but in the Windows case this is satisfied by EAL
and so threads_dep can perhaps legitimately be empty. It also means
there's only one place where the decision is made regardless of the
number of dependencies.

Regards,
Nick

Re: [dpdk-dev] [PATCH v2 2/3] meson: use threads dependency as provided by meson

[Gabriel Ganne]

Hi Nick, Dmitry,

Thanks for the reviews.

I wrote this patch following the link issue I had with libpcap thinking it
was
"more of the same" ... which clearly it wasn't.

I had a closer look to what dependency('threads') generates and I agree
that it's not that well documented. From what I understand, it will always
the '-pthread' arg in the command line arguments, and then convert the
argument to the targeted environment (which might remove it if unsupported).

For librte_metrics, I am mistaken and the added dep should be removed.

I think that you are right to suggest keeping the "if not is_windows" test.
I will set up more test environments before submitting a new patch.

Best regards,

On Tue, Mar 23, 2021 at 11:17 PM Nick Connolly <nick.connolly@mayadata.io>
wrote:

>
> > Agreed. In my tests it's "-pthread", not "-lpthread", but still.
> Agreed - I must have been distracted by the array of -l: arguments :-)
> > It's more like, in lib/librte_telemetry/meson.build:
> >
> >       if not is_windows
> >               ext_deps += threads_dep
> >       endif
> It's not a big deal, but my thinking was that the library has a genuine
> threads dependency, but in the Windows case this is satisfied by EAL
> and so threads_dep can perhaps legitimately be empty. It also means
> there's only one place where the decision is made regardless of the
> number of dependencies.
>
> Regards,
> Nick
>


-- 
Gabriel Ganne

[dpdk-dev] [PATCH v4] meson: remove unnecessary explicit link to libpcap

[Gabriel Ganne]

libpcap is already found and registered as a dependency by meson, and
the dependency is already correctly used in librte_port. This line is
just unnecessary.

The pcap PMD and the librte_port both declare their dependency to
libpcap with a line "ext_deps += pcap_dep".
"pcap_dep" is declared using the dependency() directive in
config/meson.build meson automatically adds this dependency to the
pkg-config file in the private section for static builds.

Signed-off-by: Gabriel Ganne <gabriel.ganne@6wind.com>
---
 config/meson.build | 1 -
 1 file changed, 1 deletion(-)

diff --git a/config/meson.build b/config/meson.build
index 66a2edcc47f5..95777cf33169 100644
--- a/config/meson.build
+++ b/config/meson.build
@@ -183,7 +183,6 @@ if not pcap_dep.found()
 endif
 if pcap_dep.found() and cc.has_header('pcap.h', dependencies: pcap_dep)
 	dpdk_conf.set('RTE_PORT_PCAP', 1)
-	dpdk_extra_ldflags += '-lpcap'
 endif
 
 # for clang 32-bit compiles we need libatomic for 64-bit atomic ops
-- 
2.29.2

[dpdk-dev] [PATCH v5] build: remove redundant libpcap link

[Thomas Monjalon]

From: Gabriel Ganne <gabriel.ganne@6wind.com>

The pcap PMD and the librte_port both declare their dependency to libpcap
with a line "ext_deps += pcap_dep".
Then meson automatically adds this dependency to the pkg-config file
in the "Requires.private" section for static builds.

The additional update of dpdk_extra_ldflags was adding the dependency
in the "Libs.private" section of the pkg-config, that is unnecessary.

Fixes: efd5d1a8d8dd ("drivers/net: build some vdev PMDs with meson")
Fixes: 268fa581b1ff ("port: fix pcap support with meson")
Cc: stable@dpdk.org

Signed-off-by: Gabriel Ganne <gabriel.ganne@6wind.com>
Signed-off-by: Thomas Monjalon <thomas@monjalon.net>
---
I have a doubt whether this option is really always useless.
In the case of an old pcap (<1.9) without pkg-config support,
and with the minimum meson supported (0.47.1),
are we sure the generated pkg-config file will include -lpcap?
---
 config/meson.build | 1 -
 1 file changed, 1 deletion(-)

diff --git a/config/meson.build b/config/meson.build
index 66a2edcc47..95777cf331 100644
--- a/config/meson.build
+++ b/config/meson.build
@@ -183,7 +183,6 @@ if not pcap_dep.found()
 endif
 if pcap_dep.found() and cc.has_header('pcap.h', dependencies: pcap_dep)
 	dpdk_conf.set('RTE_PORT_PCAP', 1)
-	dpdk_extra_ldflags += '-lpcap'
 endif
 
 # for clang 32-bit compiles we need libatomic for 64-bit atomic ops
-- 
2.31.1

Re: [dpdk-dev] [PATCH v5] build: remove redundant libpcap link

[Thomas Monjalon]

09/04/2021 14:25, Thomas Monjalon:
> From: Gabriel Ganne <gabriel.ganne@6wind.com>
> 
> The pcap PMD and the librte_port both declare their dependency to libpcap
> with a line "ext_deps += pcap_dep".
> Then meson automatically adds this dependency to the pkg-config file
> in the "Requires.private" section for static builds.
> 
> The additional update of dpdk_extra_ldflags was adding the dependency
> in the "Libs.private" section of the pkg-config, that is unnecessary.
> 
> Fixes: efd5d1a8d8dd ("drivers/net: build some vdev PMDs with meson")
> Fixes: 268fa581b1ff ("port: fix pcap support with meson")
> Cc: stable@dpdk.org
> 
> Signed-off-by: Gabriel Ganne <gabriel.ganne@6wind.com>
> Signed-off-by: Thomas Monjalon <thomas@monjalon.net>
> ---
> I have a doubt whether this option is really always useless.
> In the case of an old pcap (<1.9) without pkg-config support,
> and with the minimum meson supported (0.47.1),
> are we sure the generated pkg-config file will include -lpcap?

Any volunteer to test please?

Re: [dpdk-dev] [PATCH v5] build: remove redundant libpcap link

[Dmitry Kozlyuk]

2021-04-14 11:41 (UTC+0200), Thomas Monjalon:
> 09/04/2021 14:25, Thomas Monjalon:
> > From: Gabriel Ganne <gabriel.ganne@6wind.com>
> > 
> > The pcap PMD and the librte_port both declare their dependency to libpcap
> > with a line "ext_deps += pcap_dep".
> > Then meson automatically adds this dependency to the pkg-config file
> > in the "Requires.private" section for static builds.
> > 
> > The additional update of dpdk_extra_ldflags was adding the dependency
> > in the "Libs.private" section of the pkg-config, that is unnecessary.
> > 
> > Fixes: efd5d1a8d8dd ("drivers/net: build some vdev PMDs with meson")
> > Fixes: 268fa581b1ff ("port: fix pcap support with meson")
> > Cc: stable@dpdk.org
> > 
> > Signed-off-by: Gabriel Ganne <gabriel.ganne@6wind.com>
> > Signed-off-by: Thomas Monjalon <thomas@monjalon.net>
> > ---
> > I have a doubt whether this option is really always useless.
> > In the case of an old pcap (<1.9) without pkg-config support,
> > and with the minimum meson supported (0.47.1),
> > are we sure the generated pkg-config file will include -lpcap?  
> 
> Any volunteer to test please?

Ubuntu 16.04, Meson 0.47.1, libpcap 1.7.4-2ubuntu0.1, after the patch
libdpdk.pc contains:

	Libs.private: -lpcap -lpcap -lpcap -lpcap -lpcap -lpcap -lpcap -lpcap
	-lpcap -lpcap [...DPDK libraries...]

Note that -lpcap comes _before_ DPDK libraries that require it.
As a consequence, this doesn't link with unresolved libpcap symbols:

	gcc test.c `pkg-config --static --cflags --libs libdpdk`

Before the patch -lpcap was _after_ DPDK libraries,
link succeeded (there was also _one_ -lpcap before DPDK libraries).

Meson 0.55.1 places -lpcap _after_ DPDK libraries,
link succeeds both before and after the patch.

Conclusion: this patch really breaks .pc file for older meson.
If it can't be merged, dependent patches for net/pcap on Windows
can be easily adjusted to work without it.

Re: [dpdk-dev] [PATCH v5] build: remove redundant libpcap link

[Thomas Monjalon]

14/04/2021 23:02, Dmitry Kozlyuk:
> 2021-04-14 11:41 (UTC+0200), Thomas Monjalon:
> > 09/04/2021 14:25, Thomas Monjalon:
> > > From: Gabriel Ganne <gabriel.ganne@6wind.com>
> > > 
> > > The pcap PMD and the librte_port both declare their dependency to libpcap
> > > with a line "ext_deps += pcap_dep".
> > > Then meson automatically adds this dependency to the pkg-config file
> > > in the "Requires.private" section for static builds.
> > > 
> > > The additional update of dpdk_extra_ldflags was adding the dependency
> > > in the "Libs.private" section of the pkg-config, that is unnecessary.
> > > 
> > > Fixes: efd5d1a8d8dd ("drivers/net: build some vdev PMDs with meson")
> > > Fixes: 268fa581b1ff ("port: fix pcap support with meson")
> > > Cc: stable@dpdk.org
> > > 
> > > Signed-off-by: Gabriel Ganne <gabriel.ganne@6wind.com>
> > > Signed-off-by: Thomas Monjalon <thomas@monjalon.net>
> > > ---
> > > I have a doubt whether this option is really always useless.
> > > In the case of an old pcap (<1.9) without pkg-config support,
> > > and with the minimum meson supported (0.47.1),
> > > are we sure the generated pkg-config file will include -lpcap?  
> > 
> > Any volunteer to test please?
> 
> Ubuntu 16.04, Meson 0.47.1, libpcap 1.7.4-2ubuntu0.1, after the patch
> libdpdk.pc contains:
> 
> 	Libs.private: -lpcap -lpcap -lpcap -lpcap -lpcap -lpcap -lpcap -lpcap
> 	-lpcap -lpcap [...DPDK libraries...]
> 
> Note that -lpcap comes _before_ DPDK libraries that require it.
> As a consequence, this doesn't link with unresolved libpcap symbols:
> 
> 	gcc test.c `pkg-config --static --cflags --libs libdpdk`
> 
> Before the patch -lpcap was _after_ DPDK libraries,
> link succeeded (there was also _one_ -lpcap before DPDK libraries).
> 
> Meson 0.55.1 places -lpcap _after_ DPDK libraries,
> link succeeds both before and after the patch.
> 
> Conclusion: this patch really breaks .pc file for older meson.

Thanks for the test.
I propose to defer this patch.
It could be merged when we upgrade meson requirement.

In the meantime, we could document why this line is required.

> If it can't be merged, dependent patches for net/pcap on Windows
> can be easily adjusted to work without it.

Re: [dpdk-dev] [PATCH v5] build: remove redundant libpcap link

[Stephen Hemminger]

On Wed, 14 Apr 2021 23:10:46 +0200
Thomas Monjalon <thomas@monjalon.net> wrote:

> 14/04/2021 23:02, Dmitry Kozlyuk:
> > 2021-04-14 11:41 (UTC+0200), Thomas Monjalon:  
> > > 09/04/2021 14:25, Thomas Monjalon:  
> > > > From: Gabriel Ganne <gabriel.ganne@6wind.com>
> > > > 
> > > > The pcap PMD and the librte_port both declare their dependency to libpcap
> > > > with a line "ext_deps += pcap_dep".
> > > > Then meson automatically adds this dependency to the pkg-config file
> > > > in the "Requires.private" section for static builds.
> > > > 
> > > > The additional update of dpdk_extra_ldflags was adding the dependency
> > > > in the "Libs.private" section of the pkg-config, that is unnecessary.
> > > > 
> > > > Fixes: efd5d1a8d8dd ("drivers/net: build some vdev PMDs with meson")
> > > > Fixes: 268fa581b1ff ("port: fix pcap support with meson")
> > > > Cc: stable@dpdk.org
> > > > 
> > > > Signed-off-by: Gabriel Ganne <gabriel.ganne@6wind.com>
> > > > Signed-off-by: Thomas Monjalon <thomas@monjalon.net>
> > > > ---
> > > > I have a doubt whether this option is really always useless.
> > > > In the case of an old pcap (<1.9) without pkg-config support,
> > > > and with the minimum meson supported (0.47.1),
> > > > are we sure the generated pkg-config file will include -lpcap?    
> > > 
> > > Any volunteer to test please?  
> > 
> > Ubuntu 16.04, Meson 0.47.1, libpcap 1.7.4-2ubuntu0.1, after the patch
> > libdpdk.pc contains:
> > 
> > 	Libs.private: -lpcap -lpcap -lpcap -lpcap -lpcap -lpcap -lpcap -lpcap
> > 	-lpcap -lpcap [...DPDK libraries...]
> > 
> > Note that -lpcap comes _before_ DPDK libraries that require it.
> > As a consequence, this doesn't link with unresolved libpcap symbols:
> > 
> > 	gcc test.c `pkg-config --static --cflags --libs libdpdk`
> > 
> > Before the patch -lpcap was _after_ DPDK libraries,
> > link succeeded (there was also _one_ -lpcap before DPDK libraries).
> > 
> > Meson 0.55.1 places -lpcap _after_ DPDK libraries,
> > link succeeds both before and after the patch.
> > 
> > Conclusion: this patch really breaks .pc file for older meson.  
> 
> Thanks for the test.
> I propose to defer this patch.
> It could be merged when we upgrade meson requirement.

Current documented meson requirement is:
	Version 0.53.2 or later of meson is required

So this patch should be considered.

Re: [dpdk-dev] [PATCH v4] meson: remove unnecessary explicit link to libpcap

[Stephen Hemminger]

On Fri,  9 Apr 2021 10:39:27 +0200
Gabriel Ganne <gabriel.ganne@6wind.com> wrote:

> libpcap is already found and registered as a dependency by meson, and
> the dependency is already correctly used in librte_port. This line is
> just unnecessary.
> 
> The pcap PMD and the librte_port both declare their dependency to
> libpcap with a line "ext_deps += pcap_dep".
> "pcap_dep" is declared using the dependency() directive in
> config/meson.build meson automatically adds this dependency to the
> pkg-config file in the private section for static builds.
> 
> Signed-off-by: Gabriel Ganne <gabriel.ganne@6wind.com>
> ---
>  config/meson.build | 1 -
>  1 file changed, 1 deletion(-)
> 
> diff --git a/config/meson.build b/config/meson.build
> index 66a2edcc47f5..95777cf33169 100644
> --- a/config/meson.build
> +++ b/config/meson.build
> @@ -183,7 +183,6 @@ if not pcap_dep.found()
>  endif
>  if pcap_dep.found() and cc.has_header('pcap.h', dependencies: pcap_dep)
>  	dpdk_conf.set('RTE_PORT_PCAP', 1)
> -	dpdk_extra_ldflags += '-lpcap'
>  endif
>  

Patch needs to be rebased and retested, now that Windows has pcap support.
Need to make sure that Windows is not depending on this.