[dpdk-dev] questions about Windows basic memory management patch

[dpdk-dev] questions about Windows basic memory management patch

[Fady Bader]

Hi,
I have a few questions regarding your memory management patch.
In the "eal/windows: fix rte_page_sizes with Clang on Windows" (http://patches.dpdk.org/patch/67390/) patch I didn't understand the work around that you did and what the problem was.
Regarding rte_mp functions I see you implemented a stub, i.e. empty functions. I don't know why it is needed.
Just to make sure, does the rte_mem_map function that you implemented replaces Linux's mmap function ?
Lastly, in the patch you implemented functions that were common for Linux and FreeBSD and in order to use them in Windows (e.g. eal_file_truncate that replaced ftruncate) and you got a duplicate code for Linux and FreeBSD, how can we solve this duplication ?

Thanks,
Fady

Re: [dpdk-dev] questions about Windows basic memory management patch

[Dmitry Kozlyuk]

Hi,

> In the "eal/windows: fix rte_page_sizes with Clang on Windows" (http://patches.dpdk.org/patch/67390/) patch I didn't understand the work around that you did and what the problem was.

Clang on Windows always uses uint32_t as an underlying type for enums. As a
consequence, rte_hugepage_sizes cannot contain elements from 4GB onwards,
because they will be clamped to 0, resulting in the following error:

[17/41] Compiling C object lib/76b5a35@@rte_eal@sta/librte_eal_common_malloc_heap.c.obj.
FAILED: lib/76b5a35@@rte_eal@sta/librte_eal_common_malloc_heap.c.obj
clang @lib/76b5a35@@rte_eal@sta/librte_eal_common_malloc_heap.c.obj.rsp
../../../lib/librte_eal/common/malloc_heap.c:69:7: error: duplicate case value: 'RTE_PGSIZE_4G' and 'RTE_PGSIZE_16G' both equal '0'
        case RTE_PGSIZE_16G:
             ^
../../../lib/librte_eal/common/malloc_heap.c:66:7: note: previous case defined here
        case RTE_PGSIZE_4G:
             ^
1 error generated.

Maybe there's a better way to explain it in comments and commit message?

After I moved RTE_PGSIZE_4G and RTE_PGSIZE_16G outside of the enum, I had to
add `-fno-strict-enum` so that these values could be passed to where
rte_page_sizes is expected.


> Regarding rte_mp functions I see you implemented a stub, i.e. empty functions. I don't know why it is needed.

They're called from common EAL memory management routines.


> Just to make sure, does the rte_mem_map function that you implemented replaces Linux's mmap function ?

Yes. DPDK libraries have a few places they need memory-mapped files or
anonymous mappings, so I exported it (librte_mempool being the closest one,
libre_bpf also comes to my mind).


> Lastly, in the patch you implemented functions that were common for Linux and FreeBSD and in order to use them in Windows (e.g. eal_file_truncate that replaced ftruncate) and you got a duplicate code for Linux and FreeBSD, how can we solve this duplication ?

In v2 I'm going to create lib/librte_eal/posix subdirectory and move such
code there. I expect more code to end up there eventually, for example,
dynamic library loading. This possibility was among motivations for EAL
directory split.

There're another duplication that worries me: copy & paste from Linux EAL in
eal_malloc.c and eal_memory.c initialization. However, it this can't be
helped, I'd rather leave it be for now and reconsider it when implementing
advanced memory management.

-- 
Dmitry Kozlyuk

Re: [dpdk-dev] questions about Windows basic memory management patch

[Thomas Monjalon]

06/04/2020 14:37, Dmitry Kozlyuk:
> > Lastly, in the patch you implemented functions that were common for Linux and FreeBSD and in order to use them in Windows (e.g. eal_file_truncate that replaced ftruncate) and you got a duplicate code for Linux and FreeBSD, how can we solve this duplication ?
> 
> In v2 I'm going to create lib/librte_eal/posix subdirectory and move such
> code there. I expect more code to end up there eventually, for example,
> dynamic library loading. This possibility was among motivations for EAL
> directory split.

Indeed it was the motivation.
I think /unix/ is a better directory name than /posix/
Maybe I'm wrong but I had the feeling some interfaces are common
between Linux and FreeBSD while not part of POSIX standard.
Comments are welcome to help taking the right decision.

> There're another duplication that worries me: copy & paste from Linux EAL in
> eal_malloc.c and eal_memory.c initialization. However, it this can't be
> helped, I'd rather leave it be for now and reconsider it when implementing
> advanced memory management.

There are a lot of copy/paste in EAL which can be refactored.
Any help here is welcome.