[dpdk-dev] Handling missing export functions in MSVC linkage

[dpdk-dev] Handling missing export functions in MSVC linkage

[Tal Shnaiderman]

Hi,

I was wondering if there is a known solution for the following problem:

Whenever building a DPDK library using clang/MSVC there is an linkage issue if the rte_*_version.map names a function that isn't being compiled.

For example, in librte_metrics several files in the lib build depend on the existence of jansson, however, since the functions in those files are stated in rte_metrics_version.map MSVC prints out
errors in the linkage stage in case jansson is missing:

*error LNK2001: unresolved external symbol rte_metrics_tel_.... 

In clang build the .map file is converted into Module-Definition (.Def) File.

https://docs.microsoft.com/en-us/cpp/build/reference/module-definition-dot-def-files?view=vs-2019

Is there a way to instruct the linker to ignore missing functions in the def file?

Thanks,

Tal.

Re: [dpdk-dev] Handling missing export functions in MSVC linkage

[Dmitry Kozlyuk]

On Sun, 7 Jun 2020 12:26:56 +0000
Tal Shnaiderman <talshn@mellanox.com> wrote:

> In clang build the .map file is converted into Module-Definition (.Def) File.

If you create a .def manually, it will override the generation from .map. Of
cause, this adds manual work and ideally all .def files should be generated.


> Is there a way to instruct the linker to ignore missing functions in the def file?

There is /FORCE:UNRESOLVED, but it will ignore *any* unresolved reference,
that's why I'm against using it---this can lead to hard-to-diagnose errors.

https://docs.microsoft.com/en-us/cpp/build/reference/force-force-file-output?view=vs-2019

The only proper way I see is forgenertor script to analyze both names .map and
symbols .lib to filter out undefined functions. Dumpbin.exe should be able to
list symbols and it comes with MS Windows SDK, IIRC.

-- 
Dmitry Kozlyuk

Re: [dpdk-dev] Handling missing export functions in MSVC linkage

[David Marchand]

On Mon, Jun 8, 2020 at 2:09 AM Dmitry Kozlyuk <dmitry.kozliuk@gmail.com> wrote:
>
> On Sun, 7 Jun 2020 12:26:56 +0000
> Tal Shnaiderman <talshn@mellanox.com> wrote:
>
> > In clang build the .map file is converted into Module-Definition (.Def) File.
>
> If you create a .def manually, it will override the generation from .map. Of
> cause, this adds manual work and ideally all .def files should be generated.

On this topic, I just noticed that a patch of mine, that removed
rte_eal_get_configuration() from the stable ABI, missed the
declaration in rte_eal_exports.def.
Probably worth adding a check in devtools/, to avoid further misalignment.


---
David Marchand

Re: [dpdk-dev] Handling missing export functions in MSVC linkage

[Tal Shnaiderman]

> Subject: Re: Handling missing export functions in MSVC linkage
> 
> On Sun, 7 Jun 2020 12:26:56 +0000
> Tal Shnaiderman <talshn@mellanox.com> wrote:
> 
> > In clang build the .map file is converted into Module-Definition (.Def) File.
> 
> If you create a .def manually, it will override the generation from .map. Of
> cause, this adds manual work and ideally all .def files should be generated.
> 

Right, eal uses this ability, however in the metrics lib this will not be sufficient 
since in case jansson is found (and it does have Windows support) we do want
to export the full .map file

> 
> > Is there a way to instruct the linker to ignore missing functions in the def
> file?
> 
> There is /FORCE:UNRESOLVED, but it will ignore *any* unresolved reference,
> that's why I'm against using it---this can lead to hard-to-diagnose errors.
> 
> https://eur03.safelinks.protection.outlook.com/?url=https%3A%2F%2Fdocs.
> microsoft.com%2Fen-us%2Fcpp%2Fbuild%2Freference%2Fforce-force-file-
> output%3Fview%3Dvs-
> 2019&amp;data=02%7C01%7Ctalshn%40mellanox.com%7Cee173c939f094360
> 14f708d80b403a2d%7Ca652971c7d2e4d9ba6a4d149256f461b%7C0%7C0%7C6
> 37271717789602353&amp;sdata=AfIjZd5K2Grztih%2Fu7utJQZimsjpCX7qgxn0
> Z9n0hd0%3D&amp;reserved=0
> 
> The only proper way I see is forgenertor script to analyze both names .map
> and symbols .lib to filter out undefined functions. Dumpbin.exe should be
> able to list symbols and it comes with MS Windows SDK, IIRC.
> 

Looks like a good solution to me, however I'm not familiar with the availability of Dumpbin
We can discuss this solution in the upcoming community call, thanks!

> --
> Dmitry Kozlyuk

Re: [dpdk-dev] Handling missing export functions in MSVC linkage

[Thomas Monjalon]

08/06/2020 10:33, David Marchand:
> On Mon, Jun 8, 2020 at 2:09 AM Dmitry Kozlyuk <dmitry.kozliuk@gmail.com> wrote:
> > On Sun, 7 Jun 2020 12:26:56 +0000
> > If you create a .def manually, it will override the generation from .map. Of
> > cause, this adds manual work and ideally all .def files should be generated.
> 
> On this topic, I just noticed that a patch of mine, that removed
> rte_eal_get_configuration() from the stable ABI, missed the
> declaration in rte_eal_exports.def.
> Probably worth adding a check in devtools/, to avoid further misalignment.

The .def file keeps being out of sync.
We need a script to avoid forgetting the sync,
or/and we must generate this .def from the .map.
Ideally the full EAL API (except VFIO) should be implemented
for Windows.

This situation is waiting for a solution for too much time.
Windows maintainers, please solve it.

Re: [dpdk-dev] [EXTERNAL] Re: Handling missing export functions in MSVC linkage

[Tyler Retzlaff]

Unless I misunderstood the last status of this issue was that it was desired empty stubs that fail at runtime as opposed to conditionally generating exports.

Is this no longer the case?

-----Original Message-----
From: Thomas Monjalon <thomas@monjalon.net> 
Sent: Friday, March 26, 2021 1:40 AM
To: Dmitry Kozlyuk <dmitry.kozliuk@gmail.com>; ranjit.menon <ranjit.menon@intel.com>; Kadam, Pallavi <pallavi.kadam@intel.com>; Harini Ramakrishnan <Harini.Ramakrishnan@microsoft.com>; navasile@linux.microsoft.com; Dmitry Malloy <dmitrym@microsoft.com>
Cc: dev@dpdk.org; talshn <talshn@mellanox.com>; bruce.richardson@intel.com; David Marchand <david.marchand@redhat.com>; Tyler Retzlaff <roretzla@microsoft.com>; Omar Cardona <ocardona@microsoft.com>
Subject: [EXTERNAL] Re: [dpdk-dev] Handling missing export functions in MSVC linkage

08/06/2020 10:33, David Marchand:
> On Mon, Jun 8, 2020 at 2:09 AM Dmitry Kozlyuk <dmitry.kozliuk@gmail.com> wrote:
> > On Sun, 7 Jun 2020 12:26:56 +0000
> > If you create a .def manually, it will override the generation from 
> > .map. Of cause, this adds manual work and ideally all .def files should be generated.
> 
> On this topic, I just noticed that a patch of mine, that removed
> rte_eal_get_configuration() from the stable ABI, missed the 
> declaration in rte_eal_exports.def.
> Probably worth adding a check in devtools/, to avoid further misalignment.

The .def file keeps being out of sync.
We need a script to avoid forgetting the sync, or/and we must generate this .def from the .map.
Ideally the full EAL API (except VFIO) should be implemented for Windows.

This situation is waiting for a solution for too much time.
Windows maintainers, please solve it.