[dpdk-dev] Windows community call: MoM 2021-08-04

[dpdk-dev] Windows community call: MoM 2021-08-04

[Dmitry Kozlyuk]

# About

The meeting takes place in MS Teams every two weeks on Wednesday 15:00 UTC.
Ask Harini Ramakrishnan <Harini.Ramakrishnan@microsoft.com> for invitation.


# Attendees

* Microsoft:
  - Khoa To
  - Narcisa Ana Maria Vasile (Naty)
  - Omar Cordona
  - Tyler Retzlaff
* NVIDIA:
  - Dmitry Kozlyuk (DmitryK)
  - Tal Shnaiderman
* VMWare:
  - Cheng-Chung William Tu (VMWare)
  - Sergey Madaminov (VMWare)
* Mark Cheatham (Boulder Imaging)
* Nick Connolly (Datapath)
* Pallavi Kadam (Intel)
* Yan Vugenfirer (Daynix)


# Agenda

* Patch status review
* Porting OvS build system to meson status report
* Windows 21.11 roadmap planning
* Misc


# Patch review

1. eal: Add EAL API for threading v13

	DmitryK to finish the review and ack.
	Naty to follow up with Thomas if the series can be merged without
	the next one once acked because the code is used in unit tests at
	least.

2. Enable the internal EAL thread API v2

	Naty to send v3.

3. [v2] eal/windows: ensure all enabled CPUs are counted v2

	DmitryK acked, but suggested a shorted working.
	Naty to track the patch (maybe send v3 following suggestion).

4. windows/virt2phys: fix paging issue v2 (DmitryK)

	Tyler to do a security-centered review and/or ping DmitryM.


# Porting OvS build system to meson (William Tu)

Status: OvS compiles with some features disabled, with a lot of warnings.
Issues:

* vhost-user is Linux-specific.
  [Omar] Microsoft is working on functional equivalent.
* rte_version* not exported.
  AI William to send patches.
* rte_open_logstream() implementation relies on Linux-specific fopencookie().
  We need a more generic facility to redirect logs.
  AI William and DmitryK to discuss.
* meson not finding DPDK with pkg-config, maybe meson bug.
  AI William and DmitryK to investigate.

[Omar] What are the use cases for OvS on Windows?
[William]:
1. VMWare NSX on Windows;
2. AF_XDP replacement (fast data path);
   [Omar] Windows now has its own experimental AF_XDP
3. Kubernetes containers scenario.


# Windows 21.11 roadmap

AI DmitryK to send a roadmap patch, below are brief notes.

1. Harini, Omar, and Tyler will work on establishing the process of signing
and publishing netuio and virt2phys. Likely some form of external signing
will be used, i.e. not by Microsoft name. Audit before signing must be
aligned with DPDK releases. CI for signing is also currently missing.
Microsoft will discuss the topic externally and reach more people of needed.

2. DmitryK will work on interrupt support in DPDK and netuio.
   Intel will help with testing on their HW.
   Microsoft will help with code review.

3. Naty will finish the work on threading API.
   DmitryK to track them and review on time.

4. Tyler will work on enabling shared build of DPDK, mostly
solving the issues with thread-local storage. At least patches will be sent
by 21.11, not sure if merged. DmitryK will help with review.

6. DmitryK will replace SetupAPI with cfgmgr32 API in lib/eal and bus/pci.

7. Harini will expedite investigation by Microsoft,
   why netuio doesn't work with vmxnet3 HW in VMWare hypervisor.

8. External issues to track:

8.1) wpcap lacks pkg-config file:
     https://github.com/nmap/npcap/issues/299
8.2) meson 0.58 unable to build DPDK:
     https://github.com/mesonbuild/meson/issues/8981


# Misc

* We need to better automation scripts for things in Windows GSG:
  - to setup development environment;
  - to enable "Lock pages in memory privilege";
  - to install drivers.

* netuio needs tracing (logging), maybe it will be WPP tracing,
  maybe Microsoft will advise a better technology.

Re: [dpdk-dev] Windows community call: MoM 2021-08-04

[William Tu]

On Thu, Aug 5, 2021 at 12:15 PM Dmitry Kozlyuk <dmitry.kozliuk@gmail.com> wrote:
>
snip

> # Porting OvS build system to meson (William Tu)
>
> Status: OvS compiles with some features disabled, with a lot of warnings.
> Issues:
>
> * vhost-user is Linux-specific.
>   [Omar] Microsoft is working on functional equivalent.
> * rte_version* not exported.
>   AI William to send patches.
> * rte_open_logstream() implementation relies on Linux-specific fopencookie().
>   We need a more generic facility to redirect logs.
>   AI William and DmitryK to discuss.
> * meson not finding DPDK with pkg-config, maybe meson bug.

To give more details about this for people who are interested.
At OVS side, we tried to link the DPDK library, by doing below at
meson.build file
    libdpdk = dependency('libdpdk', method: 'pkg-config')" , or give
it a specific path
    libdpdk = cc.find_library('dpdk', dirs: ['C:\\temp\\dpdk\\lib'])

However, it doesn't work, with error below
    Run-time dependency libdpdk found: NO (tried pkgconfig)
    meson.build:45:4: ERROR: Dependency "libdpdk" not found, tried pkgconfig

On the other hand, pkg-config seems OK for locating the dpdk library.
PS C:\k8s-antrea-dpdk-win\ovs> pkg-config --list-all
libdpdk      DPDK - The Data Plane Development Kit (DPDK).
libdpdk-libs dpdk-libs - Internal-only DPDK pkgconfig file. Not for direct use.

We are using meson 0.59.0 and pkg-config-lite 0.28
Currently we can work around it by linking the rte_xxx.lib one-by-one.
    rte_eal_lib = cc.find_library('rte_eal', dirs: ['C:\\temp\\dpdk\\lib'])
    rte_ethdev_lib = cc.find_library('rte_ethdev', dirs:
['C:\\temp\\dpdk\\lib'])

Regards,
William

Re: [dpdk-dev] Windows community call: MoM 2021-08-04

[Bruce Richardson]

On Fri, Aug 06, 2021 at 04:17:32PM -0700, William Tu wrote:
> On Thu, Aug 5, 2021 at 12:15 PM Dmitry Kozlyuk <dmitry.kozliuk@gmail.com> wrote:
> >
> snip
> 
> > # Porting OvS build system to meson (William Tu)
> >
> > Status: OvS compiles with some features disabled, with a lot of warnings.
> > Issues:
> >
> > * vhost-user is Linux-specific.
> >   [Omar] Microsoft is working on functional equivalent.
> > * rte_version* not exported.
> >   AI William to send patches.
> > * rte_open_logstream() implementation relies on Linux-specific fopencookie().
> >   We need a more generic facility to redirect logs.
> >   AI William and DmitryK to discuss.
> > * meson not finding DPDK with pkg-config, maybe meson bug.
> 
> To give more details about this for people who are interested.
> At OVS side, we tried to link the DPDK library, by doing below at
> meson.build file
>     libdpdk = dependency('libdpdk', method: 'pkg-config')" , or give
> it a specific path
>     libdpdk = cc.find_library('dpdk', dirs: ['C:\\temp\\dpdk\\lib'])
> 
> However, it doesn't work, with error below
>     Run-time dependency libdpdk found: NO (tried pkgconfig)
>     meson.build:45:4: ERROR: Dependency "libdpdk" not found, tried pkgconfig
> 

Can you share the meson log file snippet where it tries the pkg-config
call? Does PKG_CONFIG_PATH have to be set to a special value to get the .pc
files found?

For the "find_library" calls, that would not be expected to work since
there is no single "libdpdk.so" file.

By any chance are you looking to investigate meson subproject support in
future? [1] It should provide an easy way to provide DPDK with OVS for
systems without it already installed, but may require some small changes to
DPDK meson files to work neatly. I've always meant to investigate how to
use it for applications using DPDK but never actually got around to it.

/Bruce

[1] https://mesonbuild.com/Subprojects.html

Re: [dpdk-dev] Windows community call: MoM 2021-08-04

[William Tu]

On Tue, Aug 17, 2021 at 7:14 AM Bruce Richardson
<bruce.richardson@intel.com> wrote:
>
> On Fri, Aug 06, 2021 at 04:17:32PM -0700, William Tu wrote:
> > On Thu, Aug 5, 2021 at 12:15 PM Dmitry Kozlyuk <dmitry.kozliuk@gmail.com> wrote:
> > >
> > snip
> >
> > > # Porting OvS build system to meson (William Tu)
> > >
> > > Status: OvS compiles with some features disabled, with a lot of warnings.
> > > Issues:
> > >
> > > * vhost-user is Linux-specific.
> > >   [Omar] Microsoft is working on functional equivalent.
> > > * rte_version* not exported.
> > >   AI William to send patches.
> > > * rte_open_logstream() implementation relies on Linux-specific fopencookie().
> > >   We need a more generic facility to redirect logs.
> > >   AI William and DmitryK to discuss.
> > > * meson not finding DPDK with pkg-config, maybe meson bug.
> >
> > To give more details about this for people who are interested.
> > At OVS side, we tried to link the DPDK library, by doing below at
> > meson.build file
> >     libdpdk = dependency('libdpdk', method: 'pkg-config')" , or give
> > it a specific path
> >     libdpdk = cc.find_library('dpdk', dirs: ['C:\\temp\\dpdk\\lib'])
> >
> > However, it doesn't work, with error below
> >     Run-time dependency libdpdk found: NO (tried pkgconfig)
> >     meson.build:45:4: ERROR: Dependency "libdpdk" not found, tried pkgconfig
> >
>
> Can you share the meson log file snippet where it tries the pkg-config
> call? Does PKG_CONFIG_PATH have to be set to a special value to get the .pc
> files found?

Hi Bruce,
Thanks!
The log below:
---
Target machine cpu: x86_64
Pkg-config binary for MachineChoice.HOST is not cached.
Pkg-config binary missing from cross or native file, or env var undefined.
Trying a default Pkg-config fallback at pkg-config
Found pkg-config:
C:\ProgramData\chocolatey\lib\pkgconfiglite\tools\pkg-config-lite-0
.28-1\bin\pkg-config.EXE (0.28)
Determining dependency 'libdpdk' with pkg-config executable
'C:\\ProgramData\\chocola
tey\\lib\\pkgconfiglite\\tools\\pkg-config-lite-0.28-1\\bin\\pkg-config.EXE'
env[PKG_CONFIG_PATH]:
Called `C:\ProgramData\chocolatey\lib\pkgconfiglite\tools\pkg-config-lite-0.28-1\bin\
pkg-config.EXE --modversion libdpdk` -> 1

Run-time dependency libdpdk found: NO (tried pkgconfig)
---
However, I copy the command and run it on powershell
PS C:\k8s-antrea-dpdk-win\ovs>
C:\ProgramData\chocolatey\lib\pkgconfiglite\tools\pkg-
config-lite-0.28-1\bin\pkg-config.EXE --modversion libdpdk
21.08.0-rc1

Yes, for example:
$env:PKG_CONFIG_PATH = 'C:\temp\DPDK\lib\pkgconfig'

>
> For the "find_library" calls, that would not be expected to work since
> there is no single "libdpdk.so" file.
>
> By any chance are you looking to investigate meson subproject support in
> future? [1] It should provide an easy way to provide DPDK with OVS for
> systems without it already installed, but may require some small changes to
> DPDK meson files to work neatly. I've always meant to investigate how to
> use it for applications using DPDK but never actually got around to it.

Sure, I'm new to Meson, will take a look at subproject support.
Thanks
William

Re: [dpdk-dev] Windows community call: MoM 2021-08-04

[Bruce Richardson]

On Wed, Aug 18, 2021 at 04:40:02PM -0700, William Tu wrote:
> On Tue, Aug 17, 2021 at 7:14 AM Bruce Richardson
> <bruce.richardson@intel.com> wrote:
> >
> > On Fri, Aug 06, 2021 at 04:17:32PM -0700, William Tu wrote:
> > > On Thu, Aug 5, 2021 at 12:15 PM Dmitry Kozlyuk <dmitry.kozliuk@gmail.com> wrote:
> > > >
> > > snip
> > >
> > > > # Porting OvS build system to meson (William Tu)
> > > >
> > > > Status: OvS compiles with some features disabled, with a lot of warnings.
> > > > Issues:
> > > >
> > > > * vhost-user is Linux-specific.
> > > >   [Omar] Microsoft is working on functional equivalent.
> > > > * rte_version* not exported.
> > > >   AI William to send patches.
> > > > * rte_open_logstream() implementation relies on Linux-specific fopencookie().
> > > >   We need a more generic facility to redirect logs.
> > > >   AI William and DmitryK to discuss.
> > > > * meson not finding DPDK with pkg-config, maybe meson bug.
> > >
> > > To give more details about this for people who are interested.
> > > At OVS side, we tried to link the DPDK library, by doing below at
> > > meson.build file
> > >     libdpdk = dependency('libdpdk', method: 'pkg-config')" , or give
> > > it a specific path
> > >     libdpdk = cc.find_library('dpdk', dirs: ['C:\\temp\\dpdk\\lib'])
> > >
> > > However, it doesn't work, with error below
> > >     Run-time dependency libdpdk found: NO (tried pkgconfig)
> > >     meson.build:45:4: ERROR: Dependency "libdpdk" not found, tried pkgconfig
> > >
> >
> > Can you share the meson log file snippet where it tries the pkg-config
> > call? Does PKG_CONFIG_PATH have to be set to a special value to get the .pc
> > files found?
> 
> Hi Bruce,
> Thanks!
> The log below:
> ---
> Target machine cpu: x86_64
> Pkg-config binary for MachineChoice.HOST is not cached.
> Pkg-config binary missing from cross or native file, or env var undefined.
> Trying a default Pkg-config fallback at pkg-config
> Found pkg-config:
> C:\ProgramData\chocolatey\lib\pkgconfiglite\tools\pkg-config-lite-0
> .28-1\bin\pkg-config.EXE (0.28)
> Determining dependency 'libdpdk' with pkg-config executable
> 'C:\\ProgramData\\chocola
> tey\\lib\\pkgconfiglite\\tools\\pkg-config-lite-0.28-1\\bin\\pkg-config.EXE'
> env[PKG_CONFIG_PATH]:
> Called `C:\ProgramData\chocolatey\lib\pkgconfiglite\tools\pkg-config-lite-0.28-1\bin\
> pkg-config.EXE --modversion libdpdk` -> 1
> 
> Run-time dependency libdpdk found: NO (tried pkgconfig)
> ---
> However, I copy the command and run it on powershell
> PS C:\k8s-antrea-dpdk-win\ovs>
> C:\ProgramData\chocolatey\lib\pkgconfiglite\tools\pkg-
> config-lite-0.28-1\bin\pkg-config.EXE --modversion libdpdk
> 21.08.0-rc1
> 
> Yes, for example:
> $env:PKG_CONFIG_PATH = 'C:\temp\DPDK\lib\pkgconfig'
> 

Looking at the meson log, it appears that the PKG_CONFIG_PATH is empty for
the meson call to pkg-config "env[PKG_CONFIG_PATH]:", which may explain why
DPDK is not found. I suspect meson only uses the PKG_CONFIG_PATH from the
environment on first run, but beyond that you can configure a
PKG_CONFIG_PATH using a meson configuration option "-Dpkg_config_path". Can
you try explicitly setting that in your build and see if it fixes it for
you?

/Bruce

Re: [dpdk-dev] Windows community call: MoM 2021-08-04

[William Tu]

[...]
> > Target machine cpu: x86_64
> > Pkg-config binary for MachineChoice.HOST is not cached.
> > Pkg-config binary missing from cross or native file, or env var undefined.
> > Trying a default Pkg-config fallback at pkg-config
> > Found pkg-config:
> > C:\ProgramData\chocolatey\lib\pkgconfiglite\tools\pkg-config-lite-0
> > .28-1\bin\pkg-config.EXE (0.28)
> > Determining dependency 'libdpdk' with pkg-config executable
> > 'C:\\ProgramData\\chocola
> > tey\\lib\\pkgconfiglite\\tools\\pkg-config-lite-0.28-1\\bin\\pkg-config.EXE'
> > env[PKG_CONFIG_PATH]:
> > Called `C:\ProgramData\chocolatey\lib\pkgconfiglite\tools\pkg-config-lite-0.28-1\bin\
> > pkg-config.EXE --modversion libdpdk` -> 1
> >
> > Run-time dependency libdpdk found: NO (tried pkgconfig)
> > ---
> > However, I copy the command and run it on powershell
> > PS C:\k8s-antrea-dpdk-win\ovs>
> > C:\ProgramData\chocolatey\lib\pkgconfiglite\tools\pkg-
> > config-lite-0.28-1\bin\pkg-config.EXE --modversion libdpdk
> > 21.08.0-rc1
> >
> > Yes, for example:
> > $env:PKG_CONFIG_PATH = 'C:\temp\DPDK\lib\pkgconfig'
> >
>
> Looking at the meson log, it appears that the PKG_CONFIG_PATH is empty for
> the meson call to pkg-config "env[PKG_CONFIG_PATH]:", which may explain why
> DPDK is not found. I suspect meson only uses the PKG_CONFIG_PATH from the
> environment on first run, but beyond that you can configure a
> PKG_CONFIG_PATH using a meson configuration option "-Dpkg_config_path". Can
> you try explicitly setting that in your build and see if it fixes it for
> you?

Hi Bruce,

It works, thanks a lot! I did
PS C:\k8s-antrea-dpdk-win\ovs> meson --reconfigure build
-Dpkg_config_path=C:\temp\dpdk\lib\pkgconfig\
Log below, if you're interested
----
The Meson build system
Version: 0.59.1
...
Pkg-config binary for MachineChoice.HOST is not cached.
Pkg-config binary missing from cross or native file, or env var undefined.
Trying a default Pkg-config fallback at pkg-config
Found pkg-config:
C:\ProgramData\chocolatey\lib\pkgconfiglite\tools\pkg-config-lite-0.28-1\bin\pkg-config.EXE
(0.28)
Determining dependency 'libdpdk' with pkg-config executable
'C:\\ProgramData\\chocolatey\\lib\\pkgconfiglit
e\\tools\\pkg-config-lite-0.28-1\\bin\\pkg-config.EXE'
env[PKG_CONFIG_PATH]: C:/temp/dpdk/lib/pkgconfig
Called `C:\ProgramData\chocolatey\lib\pkgconfiglite\tools\pkg-config-lite-0.28-1\bin\pkg-config.EXE
--modversion libdpdk` -> 0
21.08.0-rc1
env[PKG_CONFIG_PATH]: C:/temp/dpdk/lib/pkgconfig
Called `C:\ProgramData\chocolatey\lib\pkgconfiglite\tools\pkg-config-lite-0.28-1\bin\pkg-config.EXE
--cflag
s libdpdk` -> 0
-include rte_config.h -march=native -IC:/temp/dpdk/include
env[PKG_CONFIG_ALLOW_SYSTEM_LIBS]: 1
env[PKG_CONFIG_PATH]: C:/temp/dpdk/lib/pkgconfig
---
William