[dpdk-dev] Mmap query

[dpdk-dev] Mmap query

[Brijesh Singh]

Hi there,
We are seeing high mmap time for 1G hugepage during eal memalloc init. Most
time is spent for clearing page, each 4k size..
It's not clear to me why clear huge pages is called? The code mmaps rte_map
file with map_populate and map_shared this shouldn't cause page zeroing.
I am curious to know how zeroing is forced? As an alternative, I would like
to memset from userspace. In my experiment we are noticing memset 1G file
is faster that clear huge pages.

Apreciate your help.

Regards
Brijesh

Re: [dpdk-dev] Mmap query

[Dmitry Kozlyuk]

Hi Brijesh,

2021-10-11 10:25 (UTC-0700), Brijesh Singh:
> Hi there,
> We are seeing high mmap time for 1G hugepage during eal memalloc init. Most
> time is spent for clearing page, each 4k size..
> It's not clear to me why clear huge pages is called? The code mmaps rte_map
> file with map_populate and map_shared this shouldn't cause page zeroing.
> I am curious to know how zeroing is forced?

The kernel clears all anonymous memory before handling it to the user space.
It is done for security: there may be sensitive data left by previous users
of those pages.

There is a pending patchset that aims to solve your issue:
https://mails.dpdk.org/archives/dev/2021-October/223723.html
Please see commit messages for some more explanations.

> As an alternative, I would like to memset from userspace. In my experiment
> we are noticing memset 1G file is faster that clear huge pages.

Can you share the numbers? From my experiments, mmap() is just a little
longer then memset() of the same amount of memory, apparently because the
kernel has some extra work to set up the mapping.

Re: [dpdk-dev] Mmap query

[Brijesh Singh]

Hi Dmitry,

Thank you for your response.  The mystery is that we are not doing mmap
anonymous, and ftrace shows high time for clear page.

I suspect, the file is being unlinked before being opened. But we haven't
been able to trace it yet.


Regards,
Brijesh


On Mon, Oct 11, 2021 at 12:38 PM Dmitry Kozlyuk <dmitry.kozliuk@gmail.com>
wrote:

> Hi Brijesh,
>
> 2021-10-11 10:25 (UTC-0700), Brijesh Singh:
> > Hi there,
> > We are seeing high mmap time for 1G hugepage during eal memalloc init.
> Most
> > time is spent for clearing page, each 4k size..
> > It's not clear to me why clear huge pages is called? The code mmaps
> rte_map
> > file with map_populate and map_shared this shouldn't cause page zeroing.
> > I am curious to know how zeroing is forced?
>
> The kernel clears all anonymous memory before handling it to the user
> space.
> It is done for security: there may be sensitive data left by previous users
> of those pages.
>
> There is a pending patchset that aims to solve your issue:
> https://mails.dpdk.org/archives/dev/2021-October/223723.html
> Please see commit messages for some more explanations.
>
> > As an alternative, I would like to memset from userspace. In my
> experiment
> > we are noticing memset 1G file is faster that clear huge pages.
>
> Can you share the numbers? From my experiments, mmap() is just a little
> longer then memset() of the same amount of memory, apparently because the
> kernel has some extra work to set up the mapping.
>