[dpdk-dev] [PATCH] eal/windows: add sys/queue.h.

[dpdk-dev] [PATCH] eal/windows: add sys/queue.h.

[William Tu]

When compiling OVS, lib/dpdk.c, found the missing header.
In file included from ../lib/dpdk.c:27:
C:\temp\dpdk\include\rte_log.h:24:10: fatal error: 'sys/queue.h' file
not found
^~~~~~~~~~~~~
1 warning and 1 error generated.

Signed-off-by: William Tu <u9012063@gmail.com>
---
 lib/eal/windows/include/meson.build     | 4 ++++
 lib/eal/windows/include/sys/meson.build | 6 ++++++
 2 files changed, 10 insertions(+)
 create mode 100644 lib/eal/windows/include/sys/meson.build

diff --git a/lib/eal/windows/include/meson.build b/lib/eal/windows/include/meson.build
index b3534b025f..875cc1cf0d 100644
--- a/lib/eal/windows/include/meson.build
+++ b/lib/eal/windows/include/meson.build
@@ -8,3 +8,7 @@ headers += files(
         'rte_virt2phys.h',
         'rte_windows.h',
 )
+
+sys_headers = []
+subdir('sys')
+install_headers(sys_headers, subdir: 'sys')
diff --git a/lib/eal/windows/include/sys/meson.build b/lib/eal/windows/include/sys/meson.build
new file mode 100644
index 0000000000..6896cbf678
--- /dev/null
+++ b/lib/eal/windows/include/sys/meson.build
@@ -0,0 +1,6 @@
+# SPDX-License-Identifier: BSD-3-Clause
+# Copyright 2021 VMware, Inc
+
+sys_headers += files(
+        'queue.h',
+)
-- 
2.32.0.windows.2

Re: [dpdk-dev] [PATCH] eal/windows: add sys/queue.h.

[Nick Connolly]

> diff --git a/lib/eal/windows/include/meson.build b/lib/eal/windows/include/meson.build
> index b3534b025f..875cc1cf0d 100644
> --- a/lib/eal/windows/include/meson.build
> +++ b/lib/eal/windows/include/meson.build
> @@ -8,3 +8,7 @@ headers += files(
>           'rte_virt2phys.h',
>           'rte_windows.h',
>   )
> +
> +sys_headers = []
> +subdir('sys')
> +install_headers(sys_headers, subdir: 'sys')
> diff --git a/lib/eal/windows/include/sys/meson.build b/lib/eal/windows/include/sys/meson.build
> new file mode 100644
> index 0000000000..6896cbf678
> --- /dev/null
> +++ b/lib/eal/windows/include/sys/meson.build
> @@ -0,0 +1,6 @@
> +# SPDX-License-Identifier: BSD-3-Clause
> +# Copyright 2021 VMware, Inc
> +
> +sys_headers += files(
> +        'queue.h',
> +)

Hi William,

If I've understood this change correctly it will create a sys/queue.h 
header in the build/include folder.
This seems like a reasonable approach to handle missing functionality, 
but unfortunately it is problematic.

An application that links against DPDK is likely to be dealing with 
similar issues with missing functionality
and providing a definition in the DPDK includes can conflict with the 
approach taken by the application
(in this instance there could be two versions of sys/queue.h in the 
include paths leading to ambiguity).

After discussion, the approach adopted by the DPDK community is that:

  * DPDK should depend only on the C library and not require POSIX
    functionality from the underlying
    system (headers, definitions etc). Missing functionality should be
    implemented within the DPDK
    code.
  * DPDK should not export POSIX functionality into the environment
    (symbols, macros, headers etc)
    to avoid these definitions clashing with the application.

In this instance, it seems that rte_log.h depends upon the system having 
sys/queue.h which is not
a standard C dependency. The appropriate response here (based on the 
above approach) is to remove
the sys/queue.h dependency from rte_log.h and ensure that it only 
depends upon rte_* definitions
contained within the DPDK.

Regards,
Nick

Re: [dpdk-dev] [PATCH] eal/windows: add sys/queue.h.

[Dmitry Kozlyuk]

2021-08-10 22:05 (UTC+0100), Nick Connolly:
> > diff --git a/lib/eal/windows/include/meson.build b/lib/eal/windows/include/meson.build
> > index b3534b025f..875cc1cf0d 100644
> > --- a/lib/eal/windows/include/meson.build
> > +++ b/lib/eal/windows/include/meson.build
> > @@ -8,3 +8,7 @@ headers += files(
> >           'rte_virt2phys.h',
> >           'rte_windows.h',
> >   )
> > +
> > +sys_headers = []
> > +subdir('sys')
> > +install_headers(sys_headers, subdir: 'sys')
> > diff --git a/lib/eal/windows/include/sys/meson.build b/lib/eal/windows/include/sys/meson.build
> > new file mode 100644
> > index 0000000000..6896cbf678
> > --- /dev/null
> > +++ b/lib/eal/windows/include/sys/meson.build
> > @@ -0,0 +1,6 @@
> > +# SPDX-License-Identifier: BSD-3-Clause
> > +# Copyright 2021 VMware, Inc
> > +
> > +sys_headers += files(
> > +        'queue.h',
> > +)  
> 
> Hi William,
> 
> If I've understood this change correctly it will create a sys/queue.h 
> header in the build/include folder.
> This seems like a reasonable approach to handle missing functionality, 
> but unfortunately it is problematic.
> 
> An application that links against DPDK is likely to be dealing with 
> similar issues with missing functionality
> and providing a definition in the DPDK includes can conflict with the 
> approach taken by the application
> (in this instance there could be two versions of sys/queue.h in the 
> include paths leading to ambiguity).
> 
> After discussion, the approach adopted by the DPDK community is that:
> 
>   * DPDK should depend only on the C library and not require POSIX
>     functionality from the underlying
>     system (headers, definitions etc). Missing functionality should be
>     implemented within the DPDK
>     code.
>   * DPDK should not export POSIX functionality into the environment
>     (symbols, macros, headers etc)
>     to avoid these definitions clashing with the application.
> 
> In this instance, it seems that rte_log.h depends upon the system having 
> sys/queue.h which is not
> a standard C dependency. The appropriate response here (based on the 
> above approach) is to remove
> the sys/queue.h dependency from rte_log.h and ensure that it only 
> depends upon rte_* definitions
> contained within the DPDK.

Hi William and Nick,

rte_log.h doesn't need sys/queue.h include at all.
Unfortunately, other public headers need it.
Worse, they use TAILQ_ENTRY in structures and TAILQ_FOREACH in macros.
We discussed this offline with Tyler and he approved installing <sys/queue.h>,
but it is right that doing so would be against the policy.

What we can do:

1. Introduce `rte_queue.h` (name can be better) that is env-specific:

   1.1. For Linux and FreeBSD it just includes <sys/queue.h>
        and renames a few macros that are used in headers to RTE_xxx.
   1.2. For Windows it defines the same RTE_xxx macros in a way
        compatible with the <sys/queue.h> version used to build DPDK.

2. Add #include <sys/queue.h> in <rte_os_shim.h>:
   Linux and FreeBSD will include a system header,
   Windows will use the bundled one.

This way application are not exposed to non-RTE symbols,
at the same time RTE_xxx are binary-compatible with what DPDK
implementation expects (and outside of Windows there is no change in fact).

Re: [dpdk-dev] [PATCH] eal/windows: add sys/queue.h.

[Nick Connolly]

> What we can do:
>
> 1. Introduce `rte_queue.h` (name can be better) that is env-specific:
>
>     1.1. For Linux and FreeBSD it just includes <sys/queue.h>
>          and renames a few macros that are used in headers to RTE_xxx.
>     1.2. For Windows it defines the same RTE_xxx macros in a way
>          compatible with the <sys/queue.h> version used to build DPDK.
>
> 2. Add #include <sys/queue.h> in <rte_os_shim.h>:
>     Linux and FreeBSD will include a system header,
>     Windows will use the bundled one.
>
> This way application are not exposed to non-RTE symbols,
> at the same time RTE_xxx are binary-compatible with what DPDK
> implementation expects (and outside of Windows there is no change in fact).

+1
Nick

Re: [dpdk-dev] [PATCH] eal/windows: add sys/queue.h.

[William Tu]

On Wed, Aug 11, 2021 at 1:34 AM Nick Connolly <nick.connolly@mayadata.io> wrote:
>
>
> > What we can do:
> >
> > 1. Introduce `rte_queue.h` (name can be better) that is env-specific:
> >
> >     1.1. For Linux and FreeBSD it just includes <sys/queue.h>
> >          and renames a few macros that are used in headers to RTE_xxx.
> >     1.2. For Windows it defines the same RTE_xxx macros in a way
> >          compatible with the <sys/queue.h> version used to build DPDK.
> >
> > 2. Add #include <sys/queue.h> in <rte_os_shim.h>:
> >     Linux and FreeBSD will include a system header,
> >     Windows will use the bundled one.
> >
> > This way application are not exposed to non-RTE symbols,
> > at the same time RTE_xxx are binary-compatible with what DPDK
> > implementation expects (and outside of Windows there is no change in fact).
>
> +1
> Nick

Hi Nick and Dmitry,
Thanks for such a detailed explanation!
I sent the v2 patch, hopefully I understand you guys correctly.
William