From: Tal Shnaiderman <talshn@mellanox.com> This patchset implements the EAL functions needed for probing PMDs using vdev on Windows. --- Depends-on: series-10579 ("Windows bus/pci support") --- Tal Shnaiderman (2): eal/windows: Add needed calls to detect vdev PMD bus/vdev: Windows support drivers/bus/vdev/meson.build | 6 ------ lib/librte_eal/common/meson.build | 1 + lib/librte_eal/rte_eal_exports.def | 4 ++++ lib/librte_eal/windows/eal.c | 19 +++++++++++++++++++ 4 files changed, 24 insertions(+), 6 deletions(-) -- 2.16.1.windows.4
From: Tal Shnaiderman <talshn@mellanox.com> Add needed function calls in rte_eal_init to detect vdev PMD. eal_option_device_parse() rte_service_init() rte_bus_probe() Signed-off-by: Tal Shnaiderman <talshn@mellanox.com> --- lib/librte_eal/common/meson.build | 1 + lib/librte_eal/windows/eal.c | 19 +++++++++++++++++++ 2 files changed, 20 insertions(+) diff --git a/lib/librte_eal/common/meson.build b/lib/librte_eal/common/meson.build index dc9b1d2feb..a38b66a2d0 100644 --- a/lib/librte_eal/common/meson.build +++ b/lib/librte_eal/common/meson.build @@ -30,6 +30,7 @@ if is_windows 'malloc_heap.c', 'rte_malloc.c', 'eal_common_timer.c', + 'rte_service.c', ) subdir_done() endif diff --git a/lib/librte_eal/windows/eal.c b/lib/librte_eal/windows/eal.c index d8cfe5cc4d..8e89560aba 100644 --- a/lib/librte_eal/windows/eal.c +++ b/lib/librte_eal/windows/eal.c @@ -270,6 +270,11 @@ rte_eal_init(int argc, char **argv) if (fctret < 0) exit(1); + if (eal_option_device_parse()) { + rte_errno = ENODEV; + return -1; + } + /* Prevent creation of shared memory files. */ if (internal_conf->in_memory == 0) { RTE_LOG(WARNING, EAL, "Multi-process support is requested, " @@ -359,6 +364,20 @@ rte_eal_init(int argc, char **argv) rte_panic("Cannot create thread\n"); } + /* initialize services so vdevs register service during bus_probe. */ + if (rte_service_init()) { + rte_eal_init_alert("rte_service_init() failed"); + rte_errno = ENOEXEC; + return -1; + } + + /* Probe all the buses and devices/drivers on them */ + if (rte_bus_probe()) { + rte_eal_init_alert("Cannot probe devices"); + rte_errno = ENOTSUP; + return -1; + } + /* * Launch a dummy function on all slave lcores, so that master lcore * knows they are all ready when this function returns. -- 2.16.1.windows.4
From: Tal Shnaiderman <talshn@mellanox.com> current support will build vdev with empty MP functions currently unsupported for Windows Signed-off-by: Tal Shnaiderman <talshn@mellanox.com> --- drivers/bus/vdev/meson.build | 6 ------ lib/librte_eal/rte_eal_exports.def | 4 ++++ 2 files changed, 4 insertions(+), 6 deletions(-) diff --git a/drivers/bus/vdev/meson.build b/drivers/bus/vdev/meson.build index abaf36f1dd..967d54e4f8 100644 --- a/drivers/bus/vdev/meson.build +++ b/drivers/bus/vdev/meson.build @@ -1,12 +1,6 @@ # SPDX-License-Identifier: BSD-3-Clause # Copyright(c) 2017 Intel Corporation -if host_machine.system() == 'windows' - build = false - reason = 'not supported on Windows' - subdir_done() -endif - sources = files('vdev.c', 'vdev_params.c') install_headers('rte_bus_vdev.h') diff --git a/lib/librte_eal/rte_eal_exports.def b/lib/librte_eal/rte_eal_exports.def index 374e654264..0714a810d8 100644 --- a/lib/librte_eal/rte_eal_exports.def +++ b/lib/librte_eal/rte_eal_exports.def @@ -5,6 +5,7 @@ EXPORTS rte_calloc_socket rte_bus_register rte_dev_is_probed + rte_devargs_insert rte_devargs_next rte_devargs_remove rte_eal_get_configuration @@ -21,6 +22,8 @@ EXPORTS rte_eal_tailq_register rte_eal_using_phys_addrs rte_free + rte_log_register + rte_log_set_level rte_malloc rte_malloc_dump_stats rte_malloc_get_socket_stats @@ -54,6 +57,7 @@ EXPORTS rte_memzone_walk rte_strerror rte_strsplit + rte_sys_gettid rte_vfio_container_dma_map rte_vfio_container_dma_unmap rte_vlog -- 2.16.1.windows.4
Windows team, I am expecting at least 2 reviews on this patch.
24/06/2020 16:56, talshn@mellanox.com:
> From: Tal Shnaiderman <talshn@mellanox.com>
>
> Add needed function calls in rte_eal_init to detect vdev PMD.
>
> eal_option_device_parse()
> rte_service_init()
> rte_bus_probe()
>
> Signed-off-by: Tal Shnaiderman <talshn@mellanox.com>
> ---
> lib/librte_eal/common/meson.build | 1 +
> lib/librte_eal/windows/eal.c | 19 +++++++++++++++++++
> 2 files changed, 20 insertions(+)
>
> diff --git a/lib/librte_eal/common/meson.build b/lib/librte_eal/common/meson.build
> index dc9b1d2feb..a38b66a2d0 100644
> --- a/lib/librte_eal/common/meson.build
> +++ b/lib/librte_eal/common/meson.build
> @@ -30,6 +30,7 @@ if is_windows
> 'malloc_heap.c',
> 'rte_malloc.c',
> 'eal_common_timer.c',
> + 'rte_service.c',
> )
> subdir_done()
> endif
> diff --git a/lib/librte_eal/windows/eal.c b/lib/librte_eal/windows/eal.c
> index d8cfe5cc4d..8e89560aba 100644
> --- a/lib/librte_eal/windows/eal.c
> +++ b/lib/librte_eal/windows/eal.c
> @@ -270,6 +270,11 @@ rte_eal_init(int argc, char **argv)
> if (fctret < 0)
> exit(1);
>
> + if (eal_option_device_parse()) {
> + rte_errno = ENODEV;
> + return -1;
> + }
> +
> /* Prevent creation of shared memory files. */
> if (internal_conf->in_memory == 0) {
> RTE_LOG(WARNING, EAL, "Multi-process support is requested, "
> @@ -359,6 +364,20 @@ rte_eal_init(int argc, char **argv)
> rte_panic("Cannot create thread\n");
> }
>
> + /* initialize services so vdevs register service during bus_probe. */
> + if (rte_service_init()) {
> + rte_eal_init_alert("rte_service_init() failed");
> + rte_errno = ENOEXEC;
> + return -1;
> + }
> +
> + /* Probe all the buses and devices/drivers on them */
> + if (rte_bus_probe()) {
> + rte_eal_init_alert("Cannot probe devices");
> + rte_errno = ENOTSUP;
> + return -1;
> + }
> +
> /*
> * Launch a dummy function on all slave lcores, so that master lcore
> * knows they are all ready when this function returns.
On Wed, Jun 24, 2020 at 05:56:20PM +0300, talshn@mellanox.com wrote:
> From: Tal Shnaiderman <talshn@mellanox.com>
>
> Add needed function calls in rte_eal_init to detect vdev PMD.
>
> eal_option_device_parse()
> rte_service_init()
> rte_bus_probe()
>
> Signed-off-by: Tal Shnaiderman <talshn@mellanox.com>
> ---
> lib/librte_eal/common/meson.build | 1 +
> lib/librte_eal/windows/eal.c | 19 +++++++++++++++++++
> 2 files changed, 20 insertions(+)
>
> diff --git a/lib/librte_eal/common/meson.build b/lib/librte_eal/common/meson.build
Implicit declaration warning:
../lib/librte_eal/windows/eal.c:371:6: warning: implicit declaration of function 'rte_service_init' is invalid in C99 [-Wimplicit-function-declaration]
if (rte_service_init()) {
^
1 warning generated.
Adding "#include <rte_service_component.h>" in windows/eal.c should solve it.
Getting some linker error:
Creating library drivers\librte_bus_vdev.dll.a and object drivers\librte_bus_vdev.dll.exp
bus_vdev_vdev.c.obj : error LNK2019: unresolved external symbol rte_log_register_type_and_pick_level referenced in function __vdev_logtype_bus
drivers\librte_bus_vdev-0.200.3.dll : fatal error LNK1120: 1 unresolved externals
Probably just a missing export for "rte_log_register_type_and_pick_level" in rte_eal_exports.def?
After adding the missing include and export, compilation and linking are successful.
I see some errors when running the app though:
EAL: error allocating rte services array
EAL: FATAL: rte_service_init() failed
EAL: rte_service_init() failed
PANIC in main():
Cannot init EAL
6: [<unknown> (RtlUserThreadStart+0x21)[0x7FFACF283460]]
5: [<unknown> (BaseThreadInitThunk+0x14)[0x7FFACE7A6DF0]]
4: [<missing_symbols>]
3: [<missing_symbols>]
2: [<missing_symbols>]
1: [<missing_symbols>]
Thank you for the comments, you're right, either master has changed since I sent those patches or I missed out on some exports/includes, anyhow I'll fix and resend v2. The rte_panic you're seeing is unrelated to the new code (BTW, you should build a debug version if you want to see rte_panic's backtrace), it is a result of your setup not being configured with the memory management requirement of "Lock pages" (see [1]). Dmitry, It looks like we got to this stage since hugepage_claim_privilege() cannot actually detect that "Lock pages" isn't granted to the current user, as a result we fail on the first usage of a memory management call [in this case rte_calloc()] without indication to the reason. Is it possible to add an actual check that the current user is in the list of grantees? Alternatively, It would be great to have this privilege added programmatically, I tried the MSDN example in [2] but it didn't work for me while testing, maybe Microsoft team can check if there is a way to do it? [1] https://doc.dpdk.org/guides/windows_gsg/run_apps.html [2] https://docs.microsoft.com/en-us/windows/win32/memory/creating-a-file-mapping-using-large-pages?redirectedfrom=MSDN > -----Original Message----- > From: Narcisa Ana Maria Vasile <navasile@linux.microsoft.com> > Subject: Re: [PATCH 1/2] eal/windows: Add needed calls to detect vdev PMD > > On Wed, Jun 24, 2020 at 05:56:20PM +0300, talshn@mellanox.com wrote: > > From: Tal Shnaiderman <talshn@mellanox.com> > > > > Add needed function calls in rte_eal_init to detect vdev PMD. > > > > eal_option_device_parse() > > rte_service_init() > > rte_bus_probe() > > > > Signed-off-by: Tal Shnaiderman <talshn@mellanox.com> > > --- > > lib/librte_eal/common/meson.build | 1 + > > lib/librte_eal/windows/eal.c | 19 +++++++++++++++++++ > > 2 files changed, 20 insertions(+) > > > > diff --git a/lib/librte_eal/common/meson.build > > b/lib/librte_eal/common/meson.build > > Implicit declaration warning: > ../lib/librte_eal/windows/eal.c:371:6: warning: implicit declaration of function > 'rte_service_init' is invalid in C99 [-Wimplicit-function-declaration] > if (rte_service_init()) { > ^ > 1 warning generated. > > Adding "#include <rte_service_component.h>" in windows/eal.c should > solve it. > > Getting some linker error: > Creating library drivers\librte_bus_vdev.dll.a and object > drivers\librte_bus_vdev.dll.exp bus_vdev_vdev.c.obj : error LNK2019: > unresolved external symbol rte_log_register_type_and_pick_level > referenced in function __vdev_logtype_bus drivers\librte_bus_vdev- > 0.200.3.dll : fatal error LNK1120: 1 unresolved externals > > Probably just a missing export for "rte_log_register_type_and_pick_level" in > rte_eal_exports.def? > > After adding the missing include and export, compilation and linking are > successful. > I see some errors when running the app though: > > EAL: error allocating rte services array > EAL: FATAL: rte_service_init() failed > EAL: rte_service_init() failed > PANIC in main(): > Cannot init EAL > 6: [<unknown> (RtlUserThreadStart+0x21)[0x7FFACF283460]] > 5: [<unknown> (BaseThreadInitThunk+0x14)[0x7FFACE7A6DF0]] > 4: [<missing_symbols>] > 3: [<missing_symbols>] > 2: [<missing_symbols>] > 1: [<missing_symbols>]
On Tue, 7 Jul 2020 08:04:00 +0000, Tal Shnaiderman wrote: > Dmitry, It looks like we got to this stage since hugepage_claim_privilege() cannot actually detect that "Lock pages" isn't granted to the current user, as a result we fail on the first usage of a memory management call [in this case rte_calloc()] without indication to the reason. > > Is it possible to add an actual check that the current user is in the list of grantees? Thanks, I'll look into it. > Alternatively, It would be great to have this privilege added programmatically, I tried the MSDN example in [2] but it didn't work for me while testing, maybe Microsoft team can check if there is a way to do it? I don't think it's a good idea from security perspective if an application grants its user new privileges implicitly. Process with SeLockMemory privilege can affect overall system performance and stability. -- Dmitry Kozlyuk
From: Tal Shnaiderman <talshn@mellanox.com> This patchset implements the EAL functions needed for probing PMDs using vdev on Windows. --- v2: Added missing exports and includes (NarcisaV) --- Tal Shnaiderman (2): eal/windows: add needed calls to detect vdev PMD bus/vdev: build on Windows drivers/bus/vdev/meson.build | 6 ------ lib/librte_eal/common/meson.build | 1 + lib/librte_eal/rte_eal_exports.def | 5 +++++ lib/librte_eal/windows/eal.c | 20 ++++++++++++++++++++ 4 files changed, 26 insertions(+), 6 deletions(-) -- 2.16.1.windows.4
From: Tal Shnaiderman <talshn@mellanox.com> Add needed function calls in rte_eal_init to detect vdev PMD. eal_option_device_parse() rte_service_init() rte_bus_probe() Signed-off-by: Tal Shnaiderman <talshn@mellanox.com> --- lib/librte_eal/common/meson.build | 1 + lib/librte_eal/windows/eal.c | 20 ++++++++++++++++++++ 2 files changed, 21 insertions(+) diff --git a/lib/librte_eal/common/meson.build b/lib/librte_eal/common/meson.build index c1d9f21488..9f32262450 100644 --- a/lib/librte_eal/common/meson.build +++ b/lib/librte_eal/common/meson.build @@ -31,6 +31,7 @@ if is_windows 'malloc_heap.c', 'rte_malloc.c', 'eal_common_timer.c', + 'rte_service.c', ) subdir_done() endif diff --git a/lib/librte_eal/windows/eal.c b/lib/librte_eal/windows/eal.c index eb10b4ef96..cbbf67e253 100644 --- a/lib/librte_eal/windows/eal.c +++ b/lib/librte_eal/windows/eal.c @@ -17,6 +17,7 @@ #include <eal_filesystem.h> #include <eal_options.h> #include <eal_private.h> +#include <rte_service_component.h> #include <rte_trace_point.h> #include <rte_vfio.h> @@ -273,6 +274,11 @@ rte_eal_init(int argc, char **argv) if (fctret < 0) exit(1); + if (eal_option_device_parse()) { + rte_errno = ENODEV; + return -1; + } + /* Prevent creation of shared memory files. */ if (internal_conf->in_memory == 0) { RTE_LOG(WARNING, EAL, "Multi-process support is requested, " @@ -362,6 +368,20 @@ rte_eal_init(int argc, char **argv) rte_panic("Cannot create thread\n"); } + /* initialize services so vdevs register service during bus_probe. */ + if (rte_service_init()) { + rte_eal_init_alert("rte_service_init() failed"); + rte_errno = ENOEXEC; + return -1; + } + + /* Probe all the buses and devices/drivers on them */ + if (rte_bus_probe()) { + rte_eal_init_alert("Cannot probe devices"); + rte_errno = ENOTSUP; + return -1; + } + /* * Launch a dummy function on all slave lcores, so that master lcore * knows they are all ready when this function returns. -- 2.16.1.windows.4
From: Tal Shnaiderman <talshn@mellanox.com> current support will build vdev with empty MP functions currently unsupported for Windows. Signed-off-by: Tal Shnaiderman <talshn@mellanox.com> --- drivers/bus/vdev/meson.build | 6 ------ lib/librte_eal/rte_eal_exports.def | 5 +++++ 2 files changed, 5 insertions(+), 6 deletions(-) diff --git a/drivers/bus/vdev/meson.build b/drivers/bus/vdev/meson.build index 3a662b1075..967d54e4f8 100644 --- a/drivers/bus/vdev/meson.build +++ b/drivers/bus/vdev/meson.build @@ -1,12 +1,6 @@ # SPDX-License-Identifier: BSD-3-Clause # Copyright(c) 2017 Intel Corporation -if is_windows - build = false - reason = 'not supported on Windows' - subdir_done() -endif - sources = files('vdev.c', 'vdev_params.c') install_headers('rte_bus_vdev.h') diff --git a/lib/librte_eal/rte_eal_exports.def b/lib/librte_eal/rte_eal_exports.def index 69204a92c6..f54ed74a58 100644 --- a/lib/librte_eal/rte_eal_exports.def +++ b/lib/librte_eal/rte_eal_exports.def @@ -11,6 +11,7 @@ EXPORTS rte_calloc_socket rte_bus_register rte_dev_is_probed + rte_devargs_insert rte_devargs_next rte_devargs_remove rte_eal_get_configuration @@ -29,6 +30,9 @@ EXPORTS rte_eal_using_phys_addrs rte_free rte_hexdump + rte_log_register + rte_log_register_type_and_pick_level + rte_log_set_level rte_malloc rte_malloc_dump_stats rte_malloc_get_socket_stats @@ -62,6 +66,7 @@ EXPORTS rte_memzone_walk rte_strerror rte_strsplit + rte_sys_gettid rte_vfio_container_dma_map rte_vfio_container_dma_unmap rte_vlog -- 2.16.1.windows.4
On 7/7/2020 1:39 AM, Dmitry Kozlyuk wrote:
> On Tue, 7 Jul 2020 08:04:00 +0000, Tal Shnaiderman wrote:
>> Dmitry, It looks like we got to this stage since hugepage_claim_privilege() cannot actually detect that "Lock pages" isn't granted to the current user, as a result we fail on the first usage of a memory management call [in this case rte_calloc()] without indication to the reason.
>>
>> Is it possible to add an actual check that the current user is in the list of grantees?
> Thanks, I'll look into it.
>
>> Alternatively, It would be great to have this privilege added programmatically, I tried the MSDN example in [2] but it didn't work for me while testing, maybe Microsoft team can check if there is a way to do it?
> I don't think it's a good idea from security perspective if an application
> grants its user new privileges implicitly. Process with SeLockMemory
> privilege can affect overall system performance and stability.
I agree. This is something we forbid, when we do security reviews for
our other products here inside Intel.
Best to have the user explicitly acquire this privilege.
ranjit m.
On Tue, Jul 07, 2020 at 11:48:23AM +0300, talshn@mellanox.com wrote:
> From: Tal Shnaiderman <talshn@mellanox.com>
>
> current support will build vdev with empty MP functions
> currently unsupported for Windows.
>
> Signed-off-by: Tal Shnaiderman <talshn@mellanox.com>
> ---
> drivers/bus/vdev/meson.build | 6 ------
> lib/librte_eal/rte_eal_exports.def | 5 +++++
> 2 files changed, 5 insertions(+), 6 deletions(-)
Tested-by: Narcisa Vasile <navasile@linux.microsoft.com>
Acked-by: Narcisa Vasile <navasile@linux.microsoft.com>
Thanks for the instructions on system configuration, Tal.
Compiled with mingw and clang 9.0.0. VDEV is compiled and registered successfully.
C:\Users\User\hw> .\dpdk-helloworld.exe -l 1
EAL: Registered [vdev] bus.
EAL: Registered [pci] bus.
EAL: Detected 20 lcore(s)
EAL: Detected 2 NUMA nodes
EAL: Multi-process support is requested, but not available.
EAL: Cannot access virt2phys driver, PA will not be available
EAL: WARNING: TSC frequency estimated roughly - clock timings may be less accurate.
hello from core 1
With clang I first got a linker error, that goes away if the symbol is added in the .def file:
bus_vdev_vdev.c.obj : error LNK2019: unresolved external symbol per_lcore__thread_id referenced in function __vdev_logtype_bus
drivers\librte_bus_vdev-20.0.dll : fatal error LNK1120: 1 unresolved externals
On Tue, Jul 07, 2020 at 11:48:22AM +0300, talshn@mellanox.com wrote:
> From: Tal Shnaiderman <talshn@mellanox.com>
>
> Add needed function calls in rte_eal_init to detect vdev PMD.
>
> eal_option_device_parse()
> rte_service_init()
> rte_bus_probe()
>
> Signed-off-by: Tal Shnaiderman <talshn@mellanox.com>
> ---
> lib/librte_eal/common/meson.build | 1 +
> lib/librte_eal/windows/eal.c | 20 ++++++++++++++++++++
> 2 files changed, 21 insertions(+)
Tested-by: Narcisa Vasile <navasile@linux.microsoft.com>
Acked-by: Narcisa Vasile <navasile@linux.microsoft.com>
On 7/7/2020 1:48 AM, talshn@mellanox.com wrote:
> From: Tal Shnaiderman <talshn@mellanox.com>
>
> This patchset implements the EAL functions needed for probing PMDs using vdev on Windows.
>
> ---
> v2: Added missing exports and includes (NarcisaV)
> ---
>
> Tal Shnaiderman (2):
> eal/windows: add needed calls to detect vdev PMD
> bus/vdev: build on Windows
>
> drivers/bus/vdev/meson.build | 6 ------
> lib/librte_eal/common/meson.build | 1 +
> lib/librte_eal/rte_eal_exports.def | 5 +++++
> lib/librte_eal/windows/eal.c | 20 ++++++++++++++++++++
> 4 files changed, 26 insertions(+), 6 deletions(-)
>
Tested-by: Pallavi Kadam <pallavi.kadam@intel.com>
20/07/2020 21:18, Kadam, Pallavi:
>
> On 7/7/2020 1:48 AM, talshn@mellanox.com wrote:
> > From: Tal Shnaiderman <talshn@mellanox.com>
> >
> > This patchset implements the EAL functions needed for probing PMDs using vdev on Windows.
> >
> Tested-by: Pallavi Kadam <pallavi.kadam@intel.com>
Applied, thanks