I did a short program where a primary (--file-prefix=p1) allocates a memzone and generates a file descriptor that is passed to another primary (--file-prefix=p2) .
The process P2 tries to mmap the memory but I get an error (Bad file descriptor):
uint64_t* addr = mmap(NULL, 1024*1024*1024, PROT_READ, flags, mem_fd, 0);
if (addr == -1)
perror("mmap");
2022-04-08 14:31 (UTC+0200), Antonio Di Bacco:
> I know that it is possible to share memory between a primary and secondary
> process using rte_memzone_reserve_aligned to allocate memory in primary
> that is "seen" also by the secondary. If we have two primary processes
> (started with different file-prefix) the same approach is not feasible. I
> wonder how to share a chunk of memory hosted on a hugepage between two
> primaries.
>
> Regards.
Hi Antonio,
Correction: all hugepages allocated by DPDK are shared
between primary and secondary processes, not only memzones.
I assume we're talking about processes within one host,
because your previous similar question was about sharing memory between hosts
(as we have discussed offline), which is out of scope for DPDK.
As for the question directly, you need to map the same part of the same file
in the second primary as the hugepage is mapped from in the first primary.
I don't recommend to work with file paths, because their management
is not straightforward (--single-file-segments, for one) and is undocumented.
There is a way to share DPDK memory segment file descriptors.
Although public, this DPDK API is dangerous in the sense that you must
clearly understand what you're doing and how DPDK works.
Hence the question: what is the task you need this sharing for?
Maybe there is a simpler way.
1. In the first primary:
mz = rte_memzone_reserve()
ms = rte_mem_virt2memseg(mz->addr)
fd = rte_memseg_get_fd(ms)
offset = rte_memseg_get_fd_offset(ms)
2. Use Unix domain sockets with SCM_RIGHTS
to send "fd" and "offset" to the second primary.
3. In the second primary, after receiving "fd" and "offset":
flags = MAP_SHARED | MAP_HUGE | (30 << MAP_HUGE_SHIFT)
addr = mmap(fd, offset, flags)
Note that "mz" may consist of multiple "ms" depending on the sizes
of the zone and hugepages, and on the zone alignment.
Also "addr" may (and probably will) differ from "mz->addr".
It is possible to pass "mz->addr" and try to force it,
like DPDK does for primary/secondary.
Thank you Dmitry, it is really incredible how deep your knowledge is. I will give it a try.