Re: [dpdk-users] Secondary process cannot call functions on hugepages

[Stephen Hemminger <stephen@networkplumber.org>]

On Tue, 20 Jul 2021 17:28:59 +0800
Fengkai Sun <qcloud1014@gmail.com> wrote:

> Hi list,
> 
> Recently I read about the multi-process support of dpdk and found the
> feature of keeping the hugepage mappings between primary and secondary
> processes very interesting.
> Therefore, I made the following experiments:
> 
> 1. I modify the glibc to make dlopen() map shared libraries on hugepages.
> This is done by changing the underlying mapping strategy from mmap to pread.
> 2. I allocate a continuous address by rte_malloc(), and instruct dlopen()
> to map a shared library there, let's call it lib1.so.
> 3. After dlopen() succeeds, I use dlsym() in the primary process to locate
> the global variables and functions in lib1.so, and it turns out they are
> correctly mapped and functions can be called without a problem.
> 4. Because hugepage seems to get away with the effect of ASLR, I record the
> address of those global variables and functions in lib1.so, and verify them
> in the secondary process.
>     The secondary can access the global variables at the same address, but
> when it tries to call the function, a segfault occurs.
> 5. I try to use dlopen() with the same arguments as the primary process in
> the secondary process, but it just gives a segfault.
> 
> Unfortunately, gdb also gives a segfault when the program starts up, so I
> cannot give some useful debug info.
> 
> My question is, does dpdk permit functions to be loaded on hugepages and be
> called by multiple functions? Though the two processes see the exact same
> content on the hugepage, the secondary just cannot call the function on it.


Sounds like no-exec bit.
You might want to look into other libraries that already handle huge
pages and libraries such as hugetlbfs